The Insane Way Jon Favreau Directed ‘The Lion King’ on a Virtual Reality Set
The Insane Way Jon Favreau Directed ‘The Lion King’ on a Virtual Reality Set
Increasing Fidelity with Constellation-Tracked Controllers
Increasing Fidelity with Constellation-Tracked Controllers
Tracking Technology Explained: LED Matching
Tracking Technology Explained: LED Matching
DeepFovea: Using deep learning for foveated reconstruction in AR/VR
DeepFovea: Using deep learning for foveated reconstruction in AR/VR
Hands-on with an Alpha build of Google Maps’ Augmented Reality mode
Hands-on with an Alpha build of Google Maps’ Augmented Reality mode

Getting 3D images from 2D Video

Playing with a 3D camera got me thinking about how some 2D movie scenes could contain nearly complete 3D information. A great example is the scene in 12 Angry Men where the judge first instructs the jurors. The camera performs a trucking shot, panning sideways across the jurors, who are sitting relatively still. The film was shot with a 2D camera containing only a single lens, but because the camera is moving sideways you can look a fraction of a second ahead in the film to get a second image in a slightly different location.

In other words, you can view normal 2D film in 3D if:

  • The camera moves horizontally at a somewhat constant speed
  • No object in the scene is moving too fast compared to the speed of the camera

All that’s needed is to stitch together pairs of images that are a few frames apart. You might think there’s some complicated file format for 3D photos and videos, but most viewers will handle standard .jpg or .mp4 with the 2 images concatenated side-by-side.

I used free software to create 3D files: ffmpeg to extract individual frames from video, a few lines of python and montage to create side-by-side images using a fixed number of frames difference, and ffmpeg to stitch the images back into a video. Then I transferred the video to my phone, placed it in a Google cardboard viewer, and voila, I was seeing it in 3D.

Samsung Is Going All In. Samsung is one of the leading companies in the virtual reality space. Years of research into virtual reality are finally paying off for the company. At virtual reality conventions, Samsung's products are often regarded as one of the most popular, based on feedback from attendees. Currently, the Samsung Gear VR is the most popular virtual reality headset on the market. Things in the market might change in a few years, but for now Samsung is in the lead.

The sideways panning shot with 600ms offset side-by-side
I was surprised by the level of detail in the 3rd dimension and by how real it looked. When you watch the 2D film you can still get some sense of depth because objects in the foreground move faster than those in the background, but when each eye gets a different image it’s completely different. For the scene in 12 Angry Men I used a 600ms offset to get the 2nd image (which was actually the left image because the camera was panning left), but I also tried half and double that and didn’t notice a major difference in how it looked.

A tip if you want to view the image above with Google cardboard/daydream: you don’t need any special app — all you really need to do is display the image on your screen, turn off adaptive brightness and the screen timeout, and then stick your phone into the viewer.

It’s time to start caring about “VR cinema,” and SXSW’s stunners are proof
It’s time to start caring about “VR cinema,” and SXSW’s stunners are proof
Intel announces an inside-out tracking camera for robotics and AR/VR hardware
Intel announces an inside-out tracking camera for robotics and AR/VR hardware
Techniques for Improved VR Video w/ John Carmack
Techniques for Improved VR Video w/ John Carmack
What to expect from next-gen HoloLens
What to expect from next-gen HoloLens