User Tools

Site Tools


project:stereoscopy

This is an old revision of the document!


Stereoscopic recording using FFmpeg

This is I support material collected for Steroscopic/VR workshops by Radiona. We are using two identical USB cameras with FFmpeg for capturing video or images. FFmpeg is a free software project that produces libraries and programs for handling multimedia data available for most operating systems (Windows, Linux, Mac OS X, Android…)

http://ffmpeg.org/

Accessing USB cameras

Examples here are tested on Windows (DirectShow multimedia framework), but with small changes it should work on other systems. https://trac.ffmpeg.org/wiki/Capture/Webcam. Linux/Debian v4l-utils package.

List dshow input devices (Windows):

ffmpeg -list_devices true -f dshow -i dummy

Linux:

v4l2-ctl --list-devices

Print a list of options from a selected device:

ffmpeg -f dshow -list_options true -i video="Icatch(X) KV Series Null Device"

Linux:

ffmpeg -f v4l2 -list_formats all -i /dev/video0

Camera calibration

Preview/overlay both cameras at the same time for position alignment (there is significant lag but still usable). Adjust for your camera name and supported formats & color modes.

ffmpeg -y -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 0 -i video="Icatch(X) KV Series Null Device" -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 1 -i video="Icatch(X) KV Series Null Device" -filter_complex blend=all_expr='if(eq(mod(X,2),mod(Y,2)),A,B)' -vcodec rawvideo -pix_fmt yuyv422 -f sdl "SDL output"

Video recording

Parallel video recording of two cameras into separate files. Before recording find-out which one is right and left (Usually corresponds to order of plugging them).

ffmpeg -y -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 0 -i video="Icatch(X) KV Series Null Device" LEFT.mp4 | ffmpeg -y -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 1 -i video="Icatch(X) KV Series Null Device" RIGHT.mp4

Record direct mjpeg stream – bigger resolution and without lag.

ffmpeg -y -f dshow -video_size 1280x720 -framerate 30 -vcodec mjpeg -video_device_number 0 -i video="Icatch(X) KV Series Null Device" -vcodec copy RIGHT.mp4 | ffmpeg -y -f dshow -video_size 1280x720 -framerate 30 -vcodec mjpeg -video_device_number 1 -i video="Icatch(X) KV Series Null Device" -vcodec copy LEFT.mp4

Rotating the videos

In case you are recording with rotated cameras, we need to unrotate the video (in this case 90° CW).

ffmpeg -i LEFT.mp4 -codec:v libx264 -profile:v high -preset slower -b:v 1000k -vf "transpose=1" -threads 0 -codec:a aac -b:a 196k LEFT_rot.mp4

Framepacking in stereo video

Pack two different video streams into a stereoscopic video, setting proper metadata on supported codecs. Convert left and right video to one side-by-side (SBS). https://ffmpeg.org/ffmpeg-filters.html#framepack

ffmpeg -i LEFT.mp4 -i RIGHT.mp4 -filter_complex framepack=sbs OUTPUT_SBS.mp4

Convert views into a side-by-side video with the same output resolution as the input (SBS half resolution)

ffmpeg -i LEFT.mp4 -i RIGHT.mp4 -filter_complex [0:v]scale=w=iw/2[left],[1:v]scale=w=iw/2[right],[left][right]framepack=sbs OUTPUT_SBS.mp4

Convert views into a top-above-bottom format:

ffmpeg -i LEFT.mp4 -i RIGHT.mp4 -filter_complex framepack=tab OUTPUT_TAB.mp4

Framepacking might not work with files recorded with direct stream so we need to recompress it forcing the framerate.

ffmpeg -i LEFT.mp4 -r 30 -vcodec libx264 -preset ultrafast -qp 0 LEFT_2.mp4
ffmpeg -i RIGHT.mp4 -r 30 -vcodec libx264 -preset ultrafast -qp 0 RIGHT_2.mp4

Metadata for Youtube video uploads

Unfortunately none of the below recommended metadata work and Youtube is breaking things constantly. It might recognize stereoscopic/3D video in the computer browser (only anaglyph option) but it won't give the cardboard options on Youtube on smartphone. Additionally aspect ratio might be interpreted differently between computer and smartphone. It seems the best way for now is to prerender everything and bypass Youtube's unreliable and crappy 3d support. FIXME

MPEG4 (with re-encoding):

ffmpeg -i input_file.mp4 -vcodec libx264 -x264opts frame-packing=3 output_file.mp4
ffmpeg -i input_file.mkv -vcodec libx264 -x264opts "frame-packing=3:frame-packing-interpret=1:frame-packing-quincunx=0:frame-packing-grid=0,0,0,0" output_file.mp4

Matroska (direct stream copy)

ffmpeg -i input_file.mkv -c copy -metadata:s:v:0 stereo_mode=1 output_file.mkv

There is Google's tool for metadata injection but it also didn't work in this case. https://github.com/google/spatial-media/ This fork is less strict and allows more operations: https://github.com/Vargol/spatial-media

Add left-right (SBS) metadata:

python spatialmedia -i --stereo=left-right INPUT.mp4 OUTPUT.mp4

Print spatial media metadata contained in the file:

python spatialmedia FILE

Changing stereoscopic format

Convert between different stereoscopic image formats. https://trac.ffmpeg.org/wiki/Stereoscopic https://ffmpeg.org/ffmpeg-filters.html#stereo3d

Converting SBS (full res) to anaglyph (Red-cyan dubois):

ffmpeg -i OUTPUT_SBS.mp4 -vf stereo3d=sbsl:arcd OUTPUT_ANAGLYPH.mp4

Still image capture

Direct stereo SBS image capture from usb cams. There is lattency between left and rignt camera. FIXME

ffmpeg.exe -y -f dshow -video_size 1600x1200 -vcodec mjpeg  -video_device_number 0 -i video="Icatch(X) KV Series Null Device"  -f dshow -video_size 1600x1200 -vcodec mjpeg  -video_device_number 1 -i video="Icatch(X) KV Series Null Device" -filter_complex framepack=sbs -vframes 1 STEREO_SBS.jpeg

SBS image to ANAGLYPH image

ffmpeg -i STEREO_SBS.jpeg -vf stereo3d=sbsl:arcd -vframes 1 STEREO_ANAGLYPH.jpeg
project/stereoscopy.1539763212.txt.gz · Last modified: 2018/10/17 10:00 by dp