In July of 2016, Pokémon Go was released. During its first year, it seemed like people everywhere were wandering around attempting to collect Pikachus, Squirtles, and Rattatas — creatures that were visible only to people with the game on and their mobile devices in their hands. While augmented virtual reality had been around for decades, it was the launch of Pokémon Go that introduced millions of people to the technology.
Since then, the technology for delivering realities—virtual, augmented, and mixed—has been steadily improving through advances in optics, displays, artificial intelligence and user interfaces. And when it comes to optics, coatings are a critical component for enabling excellent user experiences in virtual reality, augmented reality, and mixed reality.
What Are the Realities?
There is overlap between virtual reality, augmented reality, and mixed reality, but there are distinctions. The differences lie in whether the user is immersed in a virtual world, sees virtual objects in a real-world experience, or physically interacts with virtual objects that appear in the real world.
Virtual Reality (VR): An Immersive Experience
In virtual reality, the user is isolated from the real world and completely immersed in a virtual world. As Star Trek fans remember, the ultimate virtual reality experience was depicted on The Next Generation as the Holodeck. The Holodeck was a full-body, immersive environment that participants explored, touched, and interacted with.
Current VR devices are not nearly as advanced as the Holodeck. Instead, the user often dons a headset that visually blocks out the real world. The headset is equipped with a display, speakers, sensors to detect the user’s head/body position, and possibly remote controllers. Users are immersed in an environment created by the device, whether an invented alternative world or a recorded real-world location. They can fly through an other-worldly landscape or visit a landmark on the other side of the planet.
Augmented Reality (AR): Seeing Virtual Objects in the Real World
Pokémon Go is an example of augmented reality. In AR mode, players roam around the real world while using a mobile device’s screen to see the virtual creatures (digital elements). They can interact with the object only through controls in the interface, such as flicking a Poké Ball to a Pokémon.
Phone apps utilize AR in a variety of ways. Home decor websites let use the camera to place furniture in a home. Eyeglasses are tried virtually, allowing people to see them on their faces even as they move their heads. While exploring a new city, Yelp Monocle enables people to find restaurants, stores and other services by scanning surrounding buildings. A device called AccuVein uses Near-Infrared (NIR) detection to map a patient’s veins and project them onto their skin to help guide medical professionals in placing IVs.
Mixed Reality (MR): Interacting with Virtual Objects in the Real World
As the name indicates, mixed reality is a combination of features from AR and VR. In this case, the virtual objects can interact with the real world. In MR, users interact with simulated objects as if they are part of the real-world environment. For example, an architectural rendering of a building might be overlayed onto its proposed location. A designer could see the structure and modify the design in MR by interacting with the virtual overlay.
Microsoft’s HoloLens is one of the earliest devices offered in the MR arena. The user wears the HoloLens headset, which projects digital objects onto a screen in the user’s field of vision.
Extended Reality (XR): The Ultimate Future
Extended Reality is mentioned here because it is often discussed alongside VR, AR and MR as a catchall indicating an experience using technology beyond those three. In XR (or sometimes shorten to ER), it is imagined technology can enhance all senses to create a completely interactive experience—perhaps that Holodeck isn’t so farfetched after all.
A Video Introduction to AR/VR/MR Technologies
Most VR, AR and MR applications use either existing devices—computers and mobile phones—or headsets. A VR experience can be as simple as putting a display into a headset. For example, the Google Cardboard uses folded cardboard to create a headset that holds a mobile device. When worn, the headset combined with VR apps allows the user to see and hear a virtual environment.
Unfortunately, simple VR systems have pitfalls. The virtual environment leaks light from the outside environment. The visual experiences aren’t optimized, which can cause optical aberrations like ghosts and reflections – as explained in our earlier blog post Understanding Optical Aberrations. Physically, these headsets can also be heavy and clunky which disrupts the user’s immersion.
For AR and MR systems, ideally, a headset would be unnoticeable. Overlays are projected onto a transparent screen that is also a lens to the real world. The screen/lens needs to be transparent enough to see the real world, but they must support a display for overlays. For Pokémon Go, this isn’t a problem, but for a surgeon practicing a procedure, reflections and ghosts are unacceptable.
Optical Coatings Enabling VR/AR/MR
AR Coatings — The goal in a VR/AR/MR environment is to deliver images to the eye without reflection and other optical aberrations. Optical surfaces need an anti-reflective (AR) coating to reduce visible light reflections (400-700nm wavelengths.) Since the human eye has a wide field of view (~180°) it is also crucial for an AR coating to be effective across a wide angle of incidence.
Systems that use light sources such as lasers to track the movement of a user’s head also rely on AR coatings to eliminate stray light and reduce power requirements.
Metallic Mirror Coatings — Images within the headset are shaped and routed using mirrors. These are often first surface mirrors. Enhanced aluminum or silver metallic mirror coatings offer the high reflectivity needed for VR/AR/MR applications while maintaining good oxidation protection and durability.
Beamsplitters — Dichroic and dielectric beam splitters combine and divide light to deliver images to each eye. The challenge is to prevent image aberrations by designing beamsplitters that can operate over the visible spectrum rather than optimized around a center wavelength (CWL.)
Since the headsets need to be as light as possible, optical surfaces are typically made of plastic or polymer. Most VR/AR/MR environments require curved or irregularly shaped optical surfaces to create the stereoscopic images necessary for a 3D experience. In addition, as a consumer device, many of the surfaces are exposed. These requirements often create challenges for depositing durable coatings consistently and evenly on polymer surfaces – download our whitepaper “Coating Polymer Optics – Perfecting the Science.”
VR, AR, and MR have most notably been used in entertainment and gaming, but these technologies are proving to be just as promising, if not more, for the healthcare, medical, industrial design, defense & aerospace, and engineering industries. For example, in the future, surgeons might train for a tricky procedure in a MR environment. The environment would accurately track the surgeon’s movements, providing real-time feedback all the while they remain immersed in a realistic, 3D mixed reality experience with no optical aberrations. In defense, the U.S. Army will be using Microsoft’s HoloLens technology in high-tech headsets to enhance situational awareness and capabilities for soldiers.