User Tools

Site Tools


project:stereoscopy

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
project:stereoscopy [2016/10/25 15:43]
dp [Framepacking]
project:stereoscopy [2018/10/17 10:20] (current)
dp
Line 1: Line 1:
 ====== Stereoscopic recording using FFmpeg ====== ====== Stereoscopic recording using FFmpeg ======
  
-FFmpeg is a free software project that produces libraries and programs for handling multimedia data. FFmpeg is developed mostly on Linux, but can be compiled under most operating systems, including Mac OS X, Microsoft Windows, Android, iOS, as well as AmigaOS and MorphOS+This is I support material collected for Steroscopic/VR workshops by Radiona. We are using two identical USB cameras with FFmpeg for capturing video or images. FFmpeg is a free software project that produces libraries and programs for handling multimedia data available for most operating systems (Windows, Linux, Mac OS X, Android...)
  
 [[http://ffmpeg.org/]] [[http://ffmpeg.org/]]
  
 +===== Accessing USB cameras =====
 +Examples here are tested on Windows (DirectShow multimedia framework), but with small changes it should work on other systems. [[https://trac.ffmpeg.org/wiki/Capture/Webcam]]. Linux/Debian ''v4l-utils'' package.
 +
 +List dshow input devices (Windows):
 <WRAP prewrap> <WRAP prewrap>
 <code> <code>
-ffmpeg -help+ffmpeg -list_devices true -f dshow -i dummy
 </code> </code>
 </WRAP> </WRAP>
  
-===== Accessing USB cameras ===== +Linux:
-Za capturing koristimo dvije identične USB kamere. Predstavljene upute odnose se na Windows operativni sistem (DirectShow multimedia framework). Komande za pristup kamerama na drugim operativnim sistemima su drugačije i nalaze se ovdje: [[https://trac.ffmpeg.org/wiki/Capture/Webcam]]. +
- +
-Example to list dshow input devices:+
 <WRAP prewrap> <WRAP prewrap>
 <code> <code>
-ffmpeg -list_devices true -f dshow -i dummy+v4l2-ctl --list-devices
 </code> </code>
 </WRAP> </WRAP>
  
-Example to print a list of options from a selected device:+Print a list of options from a selected device:
 <WRAP prewrap> <WRAP prewrap>
 <code> <code>
 ffmpeg -f dshow -list_options true -i video="Icatch(X) KV Series Null Device" ffmpeg -f dshow -list_options true -i video="Icatch(X) KV Series Null Device"
 +</code>
 +</WRAP>
 +
 +Linux:
 +<WRAP prewrap>
 +<code>
 +ffmpeg -f v4l2 -list_formats all -i /dev/video0
 </code> </code>
 </WRAP> </WRAP>
  
 ===== Camera calibration ===== ===== Camera calibration =====
-Preview obiju komara (za kalibraciju). Dva cam ulaza i overlay (checkerboard) za provjeru prekpapanja.+Preview/overlay both cameras at the same time for position alignment (there is significant lag but still usable). Adjust for your camera name and supported formats & color modes.
 <WRAP prewrap> <WRAP prewrap>
 <code> <code>
Line 37: Line 45:
  
 ===== Video recording ===== ===== Video recording =====
-Paralelno spremanje dvaju video inputa (kamereu dva file-a. Svakako prije treba utvrditi koja je desna, a koja lijeva kamera.+Parallel video recording of two cameras into separate files. Before recording find-out which one is ''right'' and ''left'' (Usually corresponds to order of plugging them).
 <WRAP prewrap> <WRAP prewrap>
 <code> <code>
Line 44: Line 52:
 </WRAP> </WRAP>
  
-direktan stream, bez rekompresije (veća rezolucija i bez lag-a)+Record direct ''mjpeg'' stream – bigger resolution and without lag.
 <WRAP prewrap> <WRAP prewrap>
 <code> <code>
Line 51: Line 59:
 </WRAP> </WRAP>
  
-===== Framepacking ===== +===== Rotating the videos ===== 
-Pack two different video streams into a stereoscopic video, setting proper metadata on supported codecs. Convert left and right video to one side-by-side. [[https://ffmpeg.org/ffmpeg-filters.html#framepack]]+In case you are recording with rotated cameras, we need to unrotate the video (in this case 90° CW). 
 +<WRAP prewrap> 
 +<code> 
 +ffmpeg -i LEFT.mp4 -codec:v libx264 -profile:v high -preset slower -b:v 1000k -vf "transpose=1" -threads 0 -codec:a aac -b:a 196k LEFT_rot.mp4 
 +</code> 
 +</WRAP> 
 + 
 +===== Framepacking in stereo video ===== 
 +Pack two different video streams into a stereoscopic video, setting proper metadata on supported codecs. Convert left and right video to one side-by-side (''SBS''). [[https://ffmpeg.org/ffmpeg-filters.html#framepack]]
 <WRAP prewrap> <WRAP prewrap>
 <code> <code>
Line 66: Line 82:
 </WRAP> </WRAP>
  
-Framepacking možda neće raditi sa falovima snimljenim s direktnim streamom, pa treba prije rekomprimirati forsirajući framerate.+Convert views into a top-above-bottom format:
 <WRAP prewrap> <WRAP prewrap>
 <code> <code>
-ffmpeg.exe -i LEFT.mp4 -r 30 -vcodec libx264 -preset ultrafast -qp 0 LEFT_2.mp4 +ffmpeg -i LEFT.mp4 -i RIGHT.mp4 -filter_complex framepack=tab OUTPUT_TAB.mp4
-ffmpeg.exe -i RIGHT.mp4 -r 30 -vcodec libx264 -preset ultrafast -qp 0 RIGHT_2.mp4+
 </code> </code>
 </WRAP> </WRAP>
-===== 4 ===== + 
-Konvertanje SBS (full res) u anaglyph red/cyan+Framepacking might not work with files recorded with direct stream so we need to recompress it forcing the framerate.
 <WRAP prewrap> <WRAP prewrap>
 <code> <code>
-ffmpeg -i OUTPUT.mp4 -vf stereo3d=sbsl:arcd OUTPUT_ANAGLYPH.mp4+ffmpeg -i LEFT.mp4 -r 30 -vcodec libx264 -preset ultrafast -qp 0 LEFT_2.mp4 
 +ffmpeg -i RIGHT.mp4 -r 30 -vcodec libx264 -preset ultrafast -qp 0 RIGHT_2.mp4
 </code> </code>
 </WRAP> </WRAP>
  
-===== Still image capture ===== +===== Metadata for Youtube video uploads ===== 
-Stereo SBS image capture from camsPostoji latencija između L i R slike ... FIXME+Unfortunately none of the below recommended metadata work and Youtube is breaking things constantlyIt might recognize stereoscopic/3D video in the computer browser (only anaglyph option) but it won't give the cardboard options on Youtube on smartphoneAdditionally aspect ratio might be interpreted differently between computer and smartphoneIt seems the best way for now is to prerender everything and bypass Youtube's unreliable/crappy 3d support. FIXME
  
 +MPEG4 (with re-encoding):
 +<WRAP prewrap>
 +<code>
 +ffmpeg -i input_file.mp4 -vcodec libx264 -x264opts frame-packing=3 output_file.mp4
 +</code>
 +</WRAP>
 +
 +<WRAP prewrap>
 +<code>
 +ffmpeg -i input_file.mkv -vcodec libx264 -x264opts "frame-packing=3:frame-packing-interpret=1:frame-packing-quincunx=0:frame-packing-grid=0,0,0,0" output_file.mp4
 +</code>
 +</WRAP>
 +
 +Matroska (direct stream copy)
 +<WRAP prewrap>
 +<code>
 +ffmpeg -i input_file.mkv -c copy -metadata:s:v:0 stereo_mode=1 output_file.mkv
 +</code>
 +</WRAP>
 +
 +There is Google's tool for metadata injection but it also didn't work in this case.
 +https://github.com/google/spatial-media/
 +This fork is less strict and allows more operations: https://github.com/Vargol/spatial-media
 +
 +Add left-right (SBS) metadata:
 +<WRAP prewrap>
 +<code>
 +python spatialmedia -i --stereo=left-right INPUT.mp4 OUTPUT.mp4
 +</code>
 +</WRAP>
 +
 +Print spatial media metadata contained in the file:
 +<WRAP prewrap>
 +<code>
 +python spatialmedia FILE
 +</code>
 +</WRAP>
 +
 +===== Changing stereoscopic format =====
 +Convert between different stereoscopic image formats. 
 +[[https://trac.ffmpeg.org/wiki/Stereoscopic]]
 +[[https://ffmpeg.org/ffmpeg-filters.html#stereo3d]]
 +
 +Converting SBS (full res) to anaglyph (Red-cyan dubois):
 +<WRAP prewrap>
 +<code>
 +ffmpeg -i OUTPUT_SBS.mp4 -vf stereo3d=sbsl:arcd OUTPUT_ANAGLYPH.mp4
 +</code>
 +</WRAP>
 +
 +===== Still image capture =====
 +Direct stereo SBS image capture from usb cams. There is lattency between ''left'' and ''rignt'' camera. FIXME
 <WRAP prewrap> <WRAP prewrap>
 <code> <code>
Line 90: Line 158:
 </WRAP> </WRAP>
  
-stereo SBS image to ANAGLYPH image+SBS image to ANAGLYPH image
 <WRAP prewrap> <WRAP prewrap>
 <code> <code>
project/stereoscopy.1477403009.txt.gz · Last modified: 2016/10/25 15:43 by dp