Virtual Reality puts users into a simulated environment with the use of technology. It creates a 3D world for the user to interact with, and immerses them into a simulation that uses vision, hearing and touch to interact with this artificial world. It takes people out of the real world, and into one where they can experiment, play, and learn in a virtual world. Applicable industries are vast, including healthcare, automotive, law enforcement, military, construction and fabrication. As VR opportunities increase, common development workflows will be created. User testing is one of those crucial workflows in the creation of VR applications.
The Virtuality Group Arcade Machine Experiences. The 1990s saw huge developments in virtual reality. With the rise of the arcades and arcade games, it was only a matter of time, before developers started coming up with new and exciting concepts and ideas. A company known as The Virtuality Group was at the cutting edge of virtual reality, launching a wide range of arcade games and machines that let either one or a couple of players immerse themselves into amazing 3D visual experiences. This happened in 1991, a year before the movie The Lawnmower Man further introduced the Virtual Reality concept to a wider audience of people.
If you’ve ever done user testing with a desktop or mobile app, you’ll find some overlap of process. First, you’ll define what you’re testing for. Is it the setup of the app? Is it the user interface controls? Is it a specific workflow in the app? This will depend on where you are in the build process of your application, and you can create additional future tests to understand various cases. For this test, your best course of action is testing for one item, as this will help keep your tests focused and short. When testing VR, you need to keep your tests under 20 minutes. Any longer than 20 minutes, and you run the risk of having participants become fatigued and/or nauseated. Compared to traditional application testing, which can run over an hour, 20 minutes might feel short, but have no fear. Even in 20 minutes, you can gather a lot of meaningful data. 20 minutes maximum, understood?
You’ll want to write a testing plan – a series of questions and steps you’ll ask each user. This creates a baseline to help keep tests comparable between users. The questions and steps are influenced by the what you’re testing. This plan helps keep the entire testing team on task, and ensures meaningful content is collected.
Finding participants to test your application can be one of the hardest parts of VR testing. Traditional testing may allow participants to be remote, but VR testing is going to (usually) mandate participants to be on site due to hardware setup. This will limit the pool of potential participants to location. Your goal for recruiting is between 5 and 8 participants. If you have less than 5, you can still test – but your results may not be as trustworthy. If you have more than 8, reserve some participants for the next user test. Higher valued participants will be ones that are potential users of your product – regardless of their VR experience, even rookies. Don’t take participants just because they have VR experience, they may skew the data if they’re not going to be product users. Be prepared to introduce your participants to the VR equipment and bake-in time for walkthroughs and questions they may have. VR doesn’t have heavily-used common patterns yet, and a lot of people don’t have extensive familiarity with it (or any familiarity with it). This is a stark contrast to testing desktop or mobile apps, where users are typically quite comfortable with the interfaces and constraints. Ensure the participant is comfortable, and knows that they can take breaks or end the test at anytime. If recording any data that may be considered sensitive, share what data is collected from the user and why.
The State of VR in the Early 2000s. After so many capable devices on the market and so many let downs that didn’t truly capture the audience they deserved, virtual reality didn’t see much development in the early 2000s. Virtual Reality was at the background in the development of new technology. It took a step back, letting personal devices, such as computers, laptops, iPods, smartphones and tablets take over, which may very well have been the right step. With the development of new technologies, a new door was opened for virtual reality, because now head-tracking and capable displays were cheaper than ever before. However, it wasn’t before one start-up company mentioned the idea, that Virtual Reality truly took off on the consumer’s market.
Unlike traditional testing, there is a bit more physical prep work involved with VR testing. Schedule your test at least 1 week out, so you can prepare a dedicated testing area. You’ll need the machine and equipment to run your app, cleaning supplies for said equipment (or replacement equipment if swapping for each participant), room for the user to move around (if applicable), room for cameras and their stands, and a room that can comfortably fit 4 to 6 people. Keep this space free of clutter and remove anything non-essential to the test. Participants won’t be able to see the real world when they are in your VR app, so remove potential dangers for them – cords, boxes, machines, chairs, desks, etc. Participants can easily trip or bump into these items if they are left out, potentially hurting your participant and your test data. It is advised to task someone with keeping the participant safe and ready for potential nausea.You’ll need a minimum of 2 people available to administer the test. The Protector, as mentioned above, will keep the participant physically safe during the test. The Facilitator, will be the voice that guides the user through the tasks in the test and communicates with the participant. When appropriate and applicable, allowing the Facilitator to enter the VR environment with the user to conduct the test, creates a better cohesive experience and is less jarring. This can be done as audio-only, talking through the VR headset or as an actual avatar or game piece in the VR app itself. Along with these 2 roles, the Recorder is a helpful third and the Tech is a beneficial fourth. The Recorder is in charge of making sure video cameras are working, cameras are on the participant, and recording of what the user is seeing (when possible). The Recorder is also in charge of saving all the video, and organizing it into logical order for future review. The Tech provides support for the VR app when there are technical issues and restarting or resetting the app between participants.
After the test (remember, 20 minutes maximum), you want to ensure the participant is okay and not going to be nauseated or sick. Have them fill out your post-interview questionnaire, answer any remaining questions and do a Simulator Sickness Questionnaire (SSQ). The SSQ is helpful in making sure that participants are leaving in a healthy state. Originally used for airline pilots, it’s been helpful for VR applications due to the similarity of motion sickness awareness. Schedule breaks between tests, with a minimum of 30 mins. Use this time to reset the VR application, clean the VR gear or swap it with already clean equipment, save and compile notes, and mental breaks. Do a maximum of 4 tests per day, to avoid team fatigue.
Post conclusions of the tests, gather the team to review notes and footage. Discuss what went well, what didn’t go well, and what surprised people. Take these edits to your product roadmap, and adjust the application accordingly – knowing that you’re moving in the best direction.
At Cantina, we are dedicated to learning and using the latest Virtual and Augmented Reality technology. Our focused AR/VR working group spends time working with this emerging technology, learning best practices and exploring the latest tools. At our downtown Boston location, we have dedicated space for both developing and testing Virtual Reality and Augmented Reality experiences. If you’re interested in learning how we can help your AR/VR application, please get in touch with us today.
No Single Person Invented Virtual Reality. Virtual reality enthusiasts can’t point to a single person who is responsible for the creation of VR. Instead, many people contributed to the technology’s growth. There are at least five people who can lay significant claim to the title: Morton Heilig, Jaron Lanier, Douglas Engelbart, Ivan Sutherland, and Myron Krueger.