Sidebar
This is an old revision of the document!
Table of Contents
Stereoscopic recording using FFmpeg
This is I support material collected for Steroscopic/VR workshops by Radiona. We are using two identical USB cameras with FFmpeg for capturing video or images. FFmpeg is a free software project that produces libraries and programs for handling multimedia data. FFmpeg is available for under most operating systems (Windows, Linux, Mac OS X, Android…)
Accessing USB cameras
Examples here are tested on Windows (DirectShow multimedia framework), but with small changes it should work on other systems. https://trac.ffmpeg.org/wiki/Capture/Webcam. Linux/Debian v4l-utils package.
List dshow input devices (Windows):
ffmpeg -list_devices true -f dshow -i dummy
Linux:
v4l2-ctl --list-devices
Print a list of options from a selected device:
ffmpeg -f dshow -list_options true -i video="Icatch(X) KV Series Null Device"
Linux:
ffmpeg -f v4l2 -list_formats all -i /dev/video0
Camera calibration
Preview/overlay both cameras at the same time for position alignment (there is significant lag but still usable). Adjust for your camera name and supported formats & color modes.
ffmpeg -y -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 0 -i video="Icatch(X) KV Series Null Device" -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 1 -i video="Icatch(X) KV Series Null Device" -filter_complex blend=all_expr='if(eq(mod(X,2),mod(Y,2)),A,B)' -vcodec rawvideo -pix_fmt yuyv422 -f sdl "SDL output"
Video recording
Parallel video recording of two cameras into separate files. Before recording find-out which one is right and left (Usually corresponds to order of plugging them).
ffmpeg -y -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 0 -i video="Icatch(X) KV Series Null Device" LEFT.mp4 | ffmpeg -y -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 1 -i video="Icatch(X) KV Series Null Device" RIGHT.mp4
Record direct mjpeg stream – bigger resolution and without lag.
ffmpeg -y -f dshow -video_size 1280x720 -framerate 30 -vcodec mjpeg -video_device_number 0 -i video="Icatch(X) KV Series Null Device" -vcodec copy RIGHT.mp4 | ffmpeg -y -f dshow -video_size 1280x720 -framerate 30 -vcodec mjpeg -video_device_number 1 -i video="Icatch(X) KV Series Null Device" -vcodec copy LEFT.mp4
Rotating the videos
In case you are recording with rotated cameras, we need to unrotate the video (in this case 90° CW).
ffmpeg -i LEFT.mp4 -codec:v libx264 -profile:v high -preset slower -b:v 1000k -vf "transpose=1" -threads 0 -codec:a aac -b:a 196k LEFT_rot.mp4
Framepacking in stereo video
Pack two different video streams into a stereoscopic video, setting proper metadata on supported codecs. Convert left and right video to one side-by-side (SBS). https://ffmpeg.org/ffmpeg-filters.html#framepack
ffmpeg -i LEFT.mp4 -i RIGHT.mp4 -filter_complex framepack=sbs OUTPUT_SBS.mp4
Convert views into a side-by-side video with the same output resolution as the input (SBS half resolution)
ffmpeg -i LEFT.mp4 -i RIGHT.mp4 -filter_complex [0:v]scale=w=iw/2[left],[1:v]scale=w=iw/2[right],[left][right]framepack=sbs OUTPUT_SBS.mp4
Framepacking might not work with files recorded with direct stream so we need to recompress it forcing the framerate.
ffmpeg -i LEFT.mp4 -r 30 -vcodec libx264 -preset ultrafast -qp 0 LEFT_2.mp4 ffmpeg -i RIGHT.mp4 -r 30 -vcodec libx264 -preset ultrafast -qp 0 RIGHT_2.mp4
Changing stereoscopic format
Convert between different stereoscopic image formats. https://trac.ffmpeg.org/wiki/Stereoscopic https://ffmpeg.org/ffmpeg-filters.html#stereo3d
Converting SBS (full res) to anaglyph (Red-cyan dubois):
ffmpeg -i OUTPUT_SBS.mp4 -vf stereo3d=sbsl:arcd OUTPUT_ANAGLYPH.mp4
Still image capture
Direct stereo SBS image capture from usb cams. There is lattency between left and rignt camera. 
ffmpeg.exe -y -f dshow -video_size 1600x1200 -vcodec mjpeg -video_device_number 0 -i video="Icatch(X) KV Series Null Device" -f dshow -video_size 1600x1200 -vcodec mjpeg -video_device_number 1 -i video="Icatch(X) KV Series Null Device" -filter_complex framepack=sbs -vframes 1 STEREO_SBS.jpeg
SBS image to ANAGLYPH image
ffmpeg -i STEREO_SBS.jpeg -vf stereo3d=sbsl:arcd -vframes 1 STEREO_ANAGLYPH.jpeg
