Mixed reality (MR) is an emergent technology that blends virtual reality (VR) and augmented reality (AR). Mixed reality headsets like the Microsoft HoloLens are opening up possibilities in the gaming world for mixed reality experiences. These head-mounted displays include cameras that constantly map the wearer’s environment. When you play games made for these devices, characters can walk around the physical world you’re in and even help themselves to a seat on your couch.
A VR experience consists of entering a computer-generated simulation, with the equipment providing an essential role. A VR headset completely blocks your view of the real world, immersing you in entirely virtual worlds. The headset’s myriad sensors and technologies translate your movements into the virtual world. This information is then used to determine your movements (sometimes embodied in a corresponding avatar) in the virtual world.
AR experiences layer digital objects onto the real world in front of you. AR provides digital content that supplements what is before your eyes and that you see either through special AR glasses or through the camera on your phone, tablet or other device.
VR and AR converge in mixed reality. Researchers Paul Milgram and Fumio Kishino coined the term in 1994 to describe the continuum between totally real and totally virtual environments. These days, mixed reality describes environments in which real and virtual subjects and objects interact in real time — and in which you can interact with both real and virtual components. This requires a headset that has either a transparent lens or a camera, so that you can still see the real world. Note that the Windows Mixed Reality headset is a VR headset with a camera.
With VR headsets like Oculus gaining traction with mainstream audiences, the gaming and cinematic applications of VR are immense. VR games run the gamut, drawing inspiration from blockbuster films, historical settings like medieval Europe or Prohibition-era Chicago and arcade game favourites. The COVID-19 pandemic has led many theatrical directors to use VR as a way to produce dramatic works even as theatres are shuttered, while also taking advantage of the format’s unusual possibilities that allow audiences to become characters or choose their own endings.
AR has broad applications in the entertainment industry as well, including both cinema and gaming. Pokémon Go (2016) is perhaps the best-known example of an AR app that went viral, when millions of people around the globe became engaged in a world populated with cartoon characters where reality — and the user environment—was a little more magical.
With MR being the newest of the three technologies, its entertainment applications are still in their early days. Angry Birds FPS is a spatial gaming experience that uses the Magic Leap headset to superimpose pigs onto your real-world space.
VR and AR are established in a range of industries, including architecture, healthcare and education. Mixed reality technologies have promising applications that are beginning to see adoption.
VR experiences have revolutionised the architecture industry by offering the opportunity to explore conceptual buildings. Whereas clients might previously have had to travel long distances to see a 3D model, anyone with a headset and a “key” to the virtual building — that is, access to an app and the project — can experience the site virtually. AR allows people to make virtual changes to their real living spaces: You can see what new furniture would look like in your sitting room or what different colours of paint might do to your space.
In healthcare, virtual reality allows for aspiring surgeons to see the process through the eyes of more experienced counterparts, while scripted scenarios can prepare surgeons for surprises and allow them to practise their reactions. MR also has broad potential applications in healthcare, allowing surgeons to use MR-powered X-ray vision to see under a patient’s skin to blood vessels and bones during surgery.
All three technologies enable immersive educational experiences, making classroom learning more engaging and creating expansive training opportunities in a variety of settings.
