Virtual Reality puts users into a simulated environment with the use of technology. It creates a 3D world for the user to interact with, and immerses them into a simulation that uses vision, hearing and touch to interact with this artificial world. It takes people out of the real world, and into one where they can experiment, play, and learn in a virtual world. Applicable industries are vast, including healthcare, automotive, law enforcement, military, construction and fabrication. As VR opportunities increase, common development workflows will be created. User testing is one of those crucial workflows in the creation of VR applications.
Its not just about the fun and games. Other than providing action-packed, fun-filled entertainment for the entire family, VR plays a big role to help humanity as developers use it to help those in need in ways that were unimaginable 20 years ago. Some use it to cure phantom pain among amputee victims, while others rely on VR to provide therapy for soldiers suffering from post-traumatic stress disorder. VR has also been proven to help children with autism by teaching them social cues and real world lessons. It is being used as a practice board for surgical students before they make the first cut. There is even a VR content targeted at young adults that simulates their physical condition when they get to their 60s, and thus encourages them to save for the future.
If you’ve ever done user testing with a desktop or mobile app, you’ll find some overlap of process. First, you’ll define what you’re testing for. Is it the setup of the app? Is it the user interface controls? Is it a specific workflow in the app? This will depend on where you are in the build process of your application, and you can create additional future tests to understand various cases. For this test, your best course of action is testing for one item, as this will help keep your tests focused and short. When testing VR, you need to keep your tests under 20 minutes. Any longer than 20 minutes, and you run the risk of having participants become fatigued and/or nauseated. Compared to traditional application testing, which can run over an hour, 20 minutes might feel short, but have no fear. Even in 20 minutes, you can gather a lot of meaningful data. 20 minutes maximum, understood?
You’ll want to write a testing plan – a series of questions and steps you’ll ask each user. This creates a baseline to help keep tests comparable between users. The questions and steps are influenced by the what you’re testing. This plan helps keep the entire testing team on task, and ensures meaningful content is collected.
Finding participants to test your application can be one of the hardest parts of VR testing. Traditional testing may allow participants to be remote, but VR testing is going to (usually) mandate participants to be on site due to hardware setup. This will limit the pool of potential participants to location. Your goal for recruiting is between 5 and 8 participants. If you have less than 5, you can still test – but your results may not be as trustworthy. If you have more than 8, reserve some participants for the next user test. Higher valued participants will be ones that are potential users of your product – regardless of their VR experience, even rookies. Don’t take participants just because they have VR experience, they may skew the data if they’re not going to be product users. Be prepared to introduce your participants to the VR equipment and bake-in time for walkthroughs and questions they may have. VR doesn’t have heavily-used common patterns yet, and a lot of people don’t have extensive familiarity with it (or any familiarity with it). This is a stark contrast to testing desktop or mobile apps, where users are typically quite comfortable with the interfaces and constraints. Ensure the participant is comfortable, and knows that they can take breaks or end the test at anytime. If recording any data that may be considered sensitive, share what data is collected from the user and why.
It's Been Around For Decades. As a whole, virtual reality is not as new as people think it is. While the exact origin is still a mystery, some people credit 19th century French playwright Antonin Artaud as the creative force behind the concept of virtual reality. By the 20th century, researchers began diving deep into the different elements of virtual reality. Toys like the View Master are often regarded as a primitive version of what virtual reality would later become.
Unlike traditional testing, there is a bit more physical prep work involved with VR testing. Schedule your test at least 1 week out, so you can prepare a dedicated testing area. You’ll need the machine and equipment to run your app, cleaning supplies for said equipment (or replacement equipment if swapping for each participant), room for the user to move around (if applicable), room for cameras and their stands, and a room that can comfortably fit 4 to 6 people. Keep this space free of clutter and remove anything non-essential to the test. Participants won’t be able to see the real world when they are in your VR app, so remove potential dangers for them – cords, boxes, machines, chairs, desks, etc. Participants can easily trip or bump into these items if they are left out, potentially hurting your participant and your test data. It is advised to task someone with keeping the participant safe and ready for potential nausea.You’ll need a minimum of 2 people available to administer the test. The Protector, as mentioned above, will keep the participant physically safe during the test. The Facilitator, will be the voice that guides the user through the tasks in the test and communicates with the participant. When appropriate and applicable, allowing the Facilitator to enter the VR environment with the user to conduct the test, creates a better cohesive experience and is less jarring. This can be done as audio-only, talking through the VR headset or as an actual avatar or game piece in the VR app itself. Along with these 2 roles, the Recorder is a helpful third and the Tech is a beneficial fourth. The Recorder is in charge of making sure video cameras are working, cameras are on the participant, and recording of what the user is seeing (when possible). The Recorder is also in charge of saving all the video, and organizing it into logical order for future review. The Tech provides support for the VR app when there are technical issues and restarting or resetting the app between participants.
After the test (remember, 20 minutes maximum), you want to ensure the participant is okay and not going to be nauseated or sick. Have them fill out your post-interview questionnaire, answer any remaining questions and do a Simulator Sickness Questionnaire (SSQ). The SSQ is helpful in making sure that participants are leaving in a healthy state. Originally used for airline pilots, it’s been helpful for VR applications due to the similarity of motion sickness awareness. Schedule breaks between tests, with a minimum of 30 mins. Use this time to reset the VR application, clean the VR gear or swap it with already clean equipment, save and compile notes, and mental breaks. Do a maximum of 4 tests per day, to avoid team fatigue.
Post conclusions of the tests, gather the team to review notes and footage. Discuss what went well, what didn’t go well, and what surprised people. Take these edits to your product roadmap, and adjust the application accordingly – knowing that you’re moving in the best direction.
At Cantina, we are dedicated to learning and using the latest Virtual and Augmented Reality technology. Our focused AR/VR working group spends time working with this emerging technology, learning best practices and exploring the latest tools. At our downtown Boston location, we have dedicated space for both developing and testing Virtual Reality and Augmented Reality experiences. If you’re interested in learning how we can help your AR/VR application, please get in touch with us today.
Kids Aren’t the Only People Interested in VR. Both Generation Z and Millennials are interested in trying virtual reality, but Baby Boomers aren’t far behind. According to research by Greenlight VR and Touchstone Research, 64 percent of Baby Boomers have positive feelings about virtual reality.