Robots are a part of our world – and some of them are really smart. Think of Sophia , the AI-powered robot who knows jokes, opinions and can express like and dislike for various things. Of course, Sophia is a pinnacle of technology, a one of a kind example of how advanced robotics can be. But there are countless other less sophisticated robots, tirelessly working in factories to make everyday items from pieces of furniture to cars. What if we these robots could be immersed in a holographic world created by AR and VR technologies?
Pairing Robots with AR and VR Technologies: a Match Made in Heaven
For human beings, the AR and VR environment is exciting and immersive, allowing interactions with holographic items which could not be possible (or would be very difficult) in the real world. For robots, it is a deep immersion in their native environment: one controlled by computers, created, programmed and with instructions given in computer language.
Simply put, using AR and VR technologies to train and monitor robots means talking to them in their native language. This will certainly speed up the learning process and improve the way people and robots work together.
Here are some current and potential applications of AR and VR in robotics:
1. Using Augmented Reality to Improve Human-Robot Communication
Traditionally, people programmed robots and then left them alone to do their job. But since robots evolved beyond the phase of mindless laborers that perform simple and repetitive tasks, it became necessary for people to interact with robots during various work tasks.
Virtual Reality Travel Is Exploding. Who hasn’t wanted to walk down the streets of Venice, or escape to a tropical climate during a particularly rough winter? Some with a travel bug may find that they can partially feed their need for travel through virtual reality. In the fall of 2015, Marriott boasted about its ability to transport clients from London to Maui in 90 seconds, thanks to Oculus. The concept behind the campaign was that people would be inspired to travel, and book with a Marriott hotel. Since then, Hilton, Renaissance, and many others have followed suit.
Having robots as coworkers in the near future is a valid possibility. In order to get to that point, humans and robots have to find a way to communicate with each other in real time. A team of robotic specialists at the University of Boulder, Colorado, have come up with an innovative idea: using augmented reality to predict what a drone will do next.
In the experiment , the human operator wore Microsoft HoloLens mixed reality headset in order to see the field of vision of the drone in the AR window and predict its trajectory according to its movements, existing obstacles and other factors.
2. Virtual Reality Trains Robots for Object Identification
The concept of machine learning is simple: an AI-powered robot is exposed to a series of data and learns how to group it in similar categories, discriminate between different items and recognize new items which look similar to those it has already learned.
In real life, these operations are time consuming and complex. However, a team of researchers at the University of California, Berkeley, managed to train a robot to pick objects indicated by the researchers after being trained to recognize various items in a virtual reality environment.
Using VR has the advantage that robots can be taught anything with minimum cost and effort – the human trainer only needs a 3D VR model of the object, not the object itself. Thus, robots can be taught details of the finest biological structures in the human body, or large scale territories for reconnoiter and exploration purposes.
Some of the Best Applications Have Nothing to Do with Gaming. Typically, when people think of virtual reality, they think of computer games. This is unfortunate because some of the best and most interesting virtual reality experiences out today have nothing to do with gaming. There are virtual reality travel experiences, virtual reality documentaries, and much, much more.
3. Medical Robots Equipped with AR Can Reshape the Future of Disabled Persons
Robots are already used in healthcare. Advanced robotic arms perform delicate surgeries and other robots are present in drug manufacturing facilities. But robots are also capable of providing direct assistance to patients.
A group of students at the Monterrey Institute of Technology and Higher Education, in Mexico, developed an exoskeleton which can assist people with mobility problems to stand, move, sit and balance. Taking one step further, this robotic outer body was equipped with augmented reality capacities, allowing the human operator to view each separate part and choose which one to move.
4. The Future of the Justice System: Virtual Reality Crime Scene Robots
Jurors have a difficult task: they need to give a guilty or not guilty verdict based on spoken, written and photographic evidence. Their decision will determine another person’s future: freedom or imprisonment. Thus, it is of the utmost importance that each member of a jury has a clear and unbiased image of what happened in the respective case.
In the future, they could be assisted in this task by a robot equipped with virtual reality technology that will allow jurors to explore the crime scene as many times as they need, from every angle, and to focus on every minute detail. At least, this is how a PhD researcher in forensic science and criminal investigations, who is working on such a robot, inspired by the Mars Curiosity rover created by NASA, believes robots can be employed in serving justice.
It’s not all going to be plastic. Today, virtually everyone loves everything about VR, which accounts for the magnitude of its success. But the technology continues to evolve at a breakneck speed. One focus of technological advances related to VR is the engineering and design of the headset. Expectedly, there are ultra high-tech and complicatedly designed headsets out there. But some tech wizards have taken it one step further, thereby making it way more accessible to everyone. Now, there are tutorials about making VR headsets out of pieces of cardboard. Not only has this opened a plethora of possibilities for VR, it has gotten people to think in creative ways to upsize their experiences.
5. Programming by Demonstration Taken to the Next Level
Programming by demonstration is similar to employee training: the human operator demonstrates a task to a robot until it is able to replicate it. For simple tasks, there is little need for advanced technology. Things are different when robots need to be trained in very complex or dangerous tasks.
This is where AR and VR technologies step in: the entire demonstration can be created in a virtual environment, or superimposed over the real life environment using augmented reality. For example, OpenAI robots developed by Elon Musk’s company are trained by example in VR using a concept called “Vision Network”. This training concept is also effective for swarm robotics: training a large number of robots that will perform the same tasks.