====== Stereoscopic recording using FFmpeg ======

This is I support material collected for Steroscopic/VR workshops by Radiona. We are using two identical USB cameras with FFmpeg for capturing video or images. FFmpeg is a free software project that produces libraries and programs for handling multimedia data available for most operating systems (Windows, Linux, Mac OS X, Android...)

[[http://ffmpeg.org/]]

===== Accessing USB cameras =====
Examples here are tested on Windows (DirectShow multimedia framework), but with small changes it should work on other systems. [[https://trac.ffmpeg.org/wiki/Capture/Webcam]]. Linux/Debian ''v4l-utils'' package.

List dshow input devices (Windows):
<WRAP prewrap>
<code>
ffmpeg -list_devices true -f dshow -i dummy
</code>
</WRAP>

Linux:
<WRAP prewrap>
<code>
v4l2-ctl --list-devices
</code>
</WRAP>

Print a list of options from a selected device:
<WRAP prewrap>
<code>
ffmpeg -f dshow -list_options true -i video="Icatch(X) KV Series Null Device"
</code>
</WRAP>

Linux:
<WRAP prewrap>
<code>
ffmpeg -f v4l2 -list_formats all -i /dev/video0
</code>
</WRAP>

===== Camera calibration =====
Preview/overlay both cameras at the same time for position alignment (there is significant lag but still usable). Adjust for your camera name and supported formats & color modes.
<WRAP prewrap>
<code>
ffmpeg -y -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 0 -i video="Icatch(X) KV Series Null Device" -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 1 -i video="Icatch(X) KV Series Null Device" -filter_complex blend=all_expr='if(eq(mod(X,2),mod(Y,2)),A,B)' -vcodec rawvideo -pix_fmt yuyv422 -f sdl "SDL output"
</code>
</WRAP>

===== Video recording =====
Parallel video recording of two cameras into separate files. Before recording find-out which one is ''right'' and ''left'' (Usually corresponds to order of plugging them).
<WRAP prewrap>
<code>
ffmpeg -y -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 0 -i video="Icatch(X) KV Series Null Device" LEFT.mp4 | ffmpeg -y -f dshow -video_size 640x480 -framerate 30 -pixel_format yuyv422 -video_device_number 1 -i video="Icatch(X) KV Series Null Device" RIGHT.mp4
</code>
</WRAP>

Record direct ''mjpeg'' stream – bigger resolution and without lag.
<WRAP prewrap>
<code>
ffmpeg -y -f dshow -video_size 1280x720 -framerate 30 -vcodec mjpeg -video_device_number 0 -i video="Icatch(X) KV Series Null Device" -vcodec copy RIGHT.mp4 | ffmpeg -y -f dshow -video_size 1280x720 -framerate 30 -vcodec mjpeg -video_device_number 1 -i video="Icatch(X) KV Series Null Device" -vcodec copy LEFT.mp4
</code>
</WRAP>

===== Rotating the videos =====
In case you are recording with rotated cameras, we need to unrotate the video (in this case 90° CW).
<WRAP prewrap>
<code>
ffmpeg -i LEFT.mp4 -codec:v libx264 -profile:v high -preset slower -b:v 1000k -vf "transpose=1" -threads 0 -codec:a aac -b:a 196k LEFT_rot.mp4
</code>
</WRAP>

===== Framepacking in stereo video =====
Pack two different video streams into a stereoscopic video, setting proper metadata on supported codecs. Convert left and right video to one side-by-side (''SBS''). [[https://ffmpeg.org/ffmpeg-filters.html#framepack]]
<WRAP prewrap>
<code>
ffmpeg -i LEFT.mp4 -i RIGHT.mp4 -filter_complex framepack=sbs OUTPUT_SBS.mp4
</code>
</WRAP>

Convert views into a side-by-side video with the same output resolution as the input (SBS half resolution)
<WRAP prewrap>
<code>
ffmpeg -i LEFT.mp4 -i RIGHT.mp4 -filter_complex [0:v]scale=w=iw/2[left],[1:v]scale=w=iw/2[right],[left][right]framepack=sbs OUTPUT_SBS.mp4
</code>
</WRAP>

Convert views into a top-above-bottom format:
<WRAP prewrap>
<code>
ffmpeg -i LEFT.mp4 -i RIGHT.mp4 -filter_complex framepack=tab OUTPUT_TAB.mp4
</code>
</WRAP>

Framepacking might not work with files recorded with direct stream so we need to recompress it forcing the framerate.
<WRAP prewrap>
<code>
ffmpeg -i LEFT.mp4 -r 30 -vcodec libx264 -preset ultrafast -qp 0 LEFT_2.mp4
ffmpeg -i RIGHT.mp4 -r 30 -vcodec libx264 -preset ultrafast -qp 0 RIGHT_2.mp4
</code>
</WRAP>

===== Metadata for Youtube video uploads =====
Unfortunately none of the below recommended metadata work and Youtube is breaking things constantly. It might recognize stereoscopic/3D video in the computer browser (only anaglyph option) but it won't give the cardboard options on Youtube on smartphone. Additionally aspect ratio might be interpreted differently between computer and smartphone. It seems the best way for now is to prerender everything and bypass Youtube's unreliable/crappy 3d support. FIXME

MPEG4 (with re-encoding):
<WRAP prewrap>
<code>
ffmpeg -i input_file.mp4 -vcodec libx264 -x264opts frame-packing=3 output_file.mp4
</code>
</WRAP>

<WRAP prewrap>
<code>
ffmpeg -i input_file.mkv -vcodec libx264 -x264opts "frame-packing=3:frame-packing-interpret=1:frame-packing-quincunx=0:frame-packing-grid=0,0,0,0" output_file.mp4
</code>
</WRAP>

Matroska (direct stream copy)
<WRAP prewrap>
<code>
ffmpeg -i input_file.mkv -c copy -metadata:s:v:0 stereo_mode=1 output_file.mkv
</code>
</WRAP>

There is Google's tool for metadata injection but it also didn't work in this case.
https://github.com/google/spatial-media/
This fork is less strict and allows more operations: https://github.com/Vargol/spatial-media

Add left-right (SBS) metadata:
<WRAP prewrap>
<code>
python spatialmedia -i --stereo=left-right INPUT.mp4 OUTPUT.mp4
</code>
</WRAP>

Print spatial media metadata contained in the file:
<WRAP prewrap>
<code>
python spatialmedia FILE
</code>
</WRAP>

===== Changing stereoscopic format =====
Convert between different stereoscopic image formats. 
[[https://trac.ffmpeg.org/wiki/Stereoscopic]]
[[https://ffmpeg.org/ffmpeg-filters.html#stereo3d]]

Converting SBS (full res) to anaglyph (Red-cyan dubois):
<WRAP prewrap>
<code>
ffmpeg -i OUTPUT_SBS.mp4 -vf stereo3d=sbsl:arcd OUTPUT_ANAGLYPH.mp4
</code>
</WRAP>

===== Still image capture =====
Direct stereo SBS image capture from usb cams. There is lattency between ''left'' and ''rignt'' camera. FIXME
<WRAP prewrap>
<code>
ffmpeg.exe -y -f dshow -video_size 1600x1200 -vcodec mjpeg  -video_device_number 0 -i video="Icatch(X) KV Series Null Device"  -f dshow -video_size 1600x1200 -vcodec mjpeg  -video_device_number 1 -i video="Icatch(X) KV Series Null Device" -filter_complex framepack=sbs -vframes 1 STEREO_SBS.jpeg
</code>
</WRAP>

SBS image to ANAGLYPH image
<WRAP prewrap>
<code>
ffmpeg -i STEREO_SBS.jpeg -vf stereo3d=sbsl:arcd -vframes 1 STEREO_ANAGLYPH.jpeg
</code>
</WRAP>