The device will then select keyframes from the multiple frames it received. The keyframes are associated with camera poses defined or in relation to a three dimensional (3D) coordinate system.
After that, the device will receive another frame showing the current real-world scene, also found within the camera’s FOV. This frame also includes color values and current depth values for the pixels for the camera’s FOV, all at a current time. The device will determine the current camera pose of the camera in the 3D coordinate system as well.
The start-up company Oculus Rift kickstarted the industry of virtual reality again with the release of a Kickstarter project for their Oculus Rift virtual reality goggles, in the year 2010.
The result is that the device will then be able to provide a virtual or 3D representation of the current frame based on the current camera pose and two or more keyframes. This representation, or model, will be based on the color values and the depth values of two or more keyframes.
RGB-D sensorsRGB-D sensors aren’t new technology. These sensors have been used in the Xbox Kinect as well as the Asus Xtion Pro - two devices that have been available in the market for years. Apple’s new patent, Patently Apple noted, simply shrinks the tech for smaller devices such as a future headset.
Photo: NASA/Charles Babir