In an interesting case of software pushing the limits of hardware, Valve recently updated SteamVR Tracking code to account for the speed of high-level Beat Saber players.
At first glance it might not be apparent, but Beat Saber is an excellent challenge for a VR controller tracking system. You might think that a fast-paced VR shooter or sword fighting game would be the game that would be pushing SteamVR Tracking to its limits, but Beat Saber is actually far more demanding.
In a recent update to SteamVR Beta (which will soon roll out to the SteamVR main branch), the following sentence appeared in the patchnotes:
Increase limits of what we thought was humanly possible for controller motion based on tracking data from Beat Saber experts.
Though it seemed like it might have been just a joke, Valve developer Ben Jackson offered up some detail in the comments:
The tracking system has internal sanity checks to identify when things go wrong. For example, if our math says you are behind your only basestation, clearly we made a mistake, because we wouldn’t be getting any signal from behind the basestation. One of these checks relates to how fast we thought it was physically possible for someone to turn their wrist. It turns out that a properly motivated human using a light enough controller could go faster (3600 degrees/sec!) than we thought.
Scientists with NASA can use virtual reality to enable robot arms in space to perform gestures that are being done on earth with an operator.
Essentially, Valve built a number of assumptions into the SteamVR Tracking code so that it can identify clearly erroneous data (like a controller signaling that it is moving way faster than should be possible). However, it seems that one of those assumptions was actually too low for the speed of some high-level Beat Saber players, and Valve has now adjusted the code accordingly.
The main reason why Beat Saber manages to push the limits of VR controller tracking so effectively is not just because of the top speed of a player’s hands, but because of the drastic changes in acceleration when players are whipping their hands back and forth to cut distant notes on time.
I feel comfortable in saying that current-generation HMDs have reached the “good enough” phase of the technology. Unfortunately you won’t be getting haptic feedback to match the fidelity of the touch sensitivity, but I for one am very excited to see what sorts of interactions developers will come up with using these new controllers.
Hit Viral Song POP/STARS Comes to 'Beat Saber' Free on All Platforms
Most VR controller tracking systems use two tracking methods and combine them together. The first system is an IMU inside each controller which senses rotation and translation, and can do so with low low latency and high frequency. The issue however is that IMU’s are prone to ‘drift’, so they can’t be relied upon alone for tracking. That’s where the other tracking method comes in: in the case of SteamVR Tracking, that’s the external ‘basestations’ which are used to establish the absolute position and rotation of the tracked object and to correct for drift from the IMU. However, the basestations update slowly compared to how fast the IMUs are (in the ballpark of 100Hz vs. 1,000Hz), and until that next positional correction comes in, all the tracking is up to the IMU.
It Is Global. Virtual reality is not just a thing in the United States alone. People all over the globe are learning to embrace the technology. Some of the biggest virtual reality conventions are actually held in other nations. Some of the leading companies behind virtual reality are located overseas as well. Virtual reality is gradually becoming a great way to connect with people from around the world. No matter the distance, this technology can bring people together.
In many cases, the moments of tracking between position corrections is aided by prediction, which uses prior motion to project where the tracked object is heading next. When moving slowly or drawing predictable arcs through the air, this works well. But Beat Saber is something of a worse-case scenario for that prediction because there’s no way for the tracking system to predict when the player is about to suddenly and completely reverse the direction of their controller (which happens constantly in the game, especially at higher levels).
Despite the challenge, SteamVR Tracking has held up extremely well to Beat Saber’s tracking needs except, it seems, in some edge cases which have now been patched. It’s frankly pretty amazing that the system manages to work as well as it does for the game; Valve ought to be pretty happy that they chose to aim for such a high tracking performance threshold from the outset.