User Tools

Site Tools


project:stereoscopy

This is an old revision of the document!


Stereoscopic recording using FFmpeg

This is I support material collected for Steroscopic/VR workshops by Radiona. We are using two identical USB cameras with FFmpeg for capturing video or images. FFmpeg is a free software project that produces libraries and programs for handling multimedia data. FFmpeg is available for under most operating systems (Windows, Linux, Mac OS X, Android…)

http://ffmpeg.org/

Accessing USB cameras

Examples here are tested on Windows (DirectShow multimedia framework), but with small changes it should work on other systems. https://trac.ffmpeg.org/wiki/Capture/Webcam. Linux/Debian v4l-utils package.

List dshow input devices (Windows):

ffmpeg -list_devices true -f dshow -i dummy

Linux:

v4l2-ctl --list-devices

Print a list of options from a selected device:

ffmpeg -f dshow -list_options true -i video="Icatch(X) KV Series Null Device"

Linux:

ffmpeg -f v4l2 -list_formats all -i /dev/video0

Camera calibration

Preview/overlay both cameras at the same time for position alignment (there is significant lag but still usable). Adjust for your camera name and supported formats & color modes.

ffmpeg -y -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 0 -i video="Icatch(X) KV Series Null Device" -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 1 -i video="Icatch(X) KV Series Null Device" -filter_complex blend=all_expr='if(eq(mod(X,2),mod(Y,2)),A,B)' -vcodec rawvideo -pix_fmt yuyv422 -f sdl "SDL output"

Video recording

Parallel video recording of two cameras into separate files. Before recording find-out which one is right and left (Usually corresponds to order of plugging them).

ffmpeg -y -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 0 -i video="Icatch(X) KV Series Null Device" LEFT.mp4 | ffmpeg -y -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 1 -i video="Icatch(X) KV Series Null Device" RIGHT.mp4

Record direct mjpeg stream – bigger resolution and without lag.

ffmpeg -y -f dshow -video_size 1280x720 -framerate 30 -vcodec mjpeg -video_device_number 0 -i video="Icatch(X) KV Series Null Device" -vcodec copy RIGHT.mp4 | ffmpeg -y -f dshow -video_size 1280x720 -framerate 30 -vcodec mjpeg -video_device_number 1 -i video="Icatch(X) KV Series Null Device" -vcodec copy LEFT.mp4

Rotating the videos

In case you are recording with rotated cameras, we need to unrotate the video (in this case 90° CW).

ffmpeg -i LEFT.mp4 -codec:v libx264 -profile:v high -preset slower -b:v 1000k -vf "transpose=1" -threads 0 -codec:a aac -b:a 196k LEFT_rot.mp4

Framepacking in stereo video

Pack two different video streams into a stereoscopic video, setting proper metadata on supported codecs. Convert left and right video to one side-by-side (SBS). https://ffmpeg.org/ffmpeg-filters.html#framepack

ffmpeg -i LEFT.mp4 -i RIGHT.mp4 -filter_complex framepack=sbs OUTPUT_SBS.mp4

Convert views into a side-by-side video with the same output resolution as the input (SBS half resolution)

ffmpeg -i LEFT.mp4 -i RIGHT.mp4 -filter_complex [0:v]scale=w=iw/2[left],[1:v]scale=w=iw/2[right],[left][right]framepack=sbs OUTPUT_SBS.mp4

Framepacking might not work with files recorded with direct stream so we need to recompress it forcing the framerate.

ffmpeg -i LEFT.mp4 -r 30 -vcodec libx264 -preset ultrafast -qp 0 LEFT_2.mp4
ffmpeg -i RIGHT.mp4 -r 30 -vcodec libx264 -preset ultrafast -qp 0 RIGHT_2.mp4

Changing stereoscopic format

Convert between different stereoscopic image formats. https://trac.ffmpeg.org/wiki/Stereoscopic https://ffmpeg.org/ffmpeg-filters.html#stereo3d

Converting SBS (full res) to anaglyph (Red-cyan dubois):

ffmpeg -i OUTPUT_SBS.mp4 -vf stereo3d=sbsl:arcd OUTPUT_ANAGLYPH.mp4

Still image capture

Direct stereo SBS image capture from usb cams. There is lattency between left and rignt camera. FIXME

ffmpeg.exe -y -f dshow -video_size 1600x1200 -vcodec mjpeg  -video_device_number 0 -i video="Icatch(X) KV Series Null Device"  -f dshow -video_size 1600x1200 -vcodec mjpeg  -video_device_number 1 -i video="Icatch(X) KV Series Null Device" -filter_complex framepack=sbs -vframes 1 STEREO_SBS.jpeg

SBS image to ANAGLYPH image

ffmpeg -i STEREO_SBS.jpeg -vf stereo3d=sbsl:arcd -vframes 1 STEREO_ANAGLYPH.jpeg
project/stereoscopy.1539695691.txt.gz · Last modified: 2018/10/16 15:14 by dp