What is mixed reality?

Mixed reality (MR) is an emergent technology that blends virtual reality (VR) and augmented reality (AR). Mixed reality headsets like the Microsoft HoloLens are opening up possibilities in the gaming world for mixed reality experiences. These head-mounted displays include cameras that constantly map the wearer’s environment. When you play games made for these devices, characters can walk around the physical world you’re in and even help themselves to a seat on your couch.

people using mixed reality goggles to manipulate a virtual 3D rendering of a house

But there’s more to mixed reality than gaming. Artists, designers, and advertisers have been using 3D for years to tell stories and engage people in new ways, and now they can use mixed reality to bring 3D work more fully into the real world.

VR vs. AR vs. MR: What’s the difference?

Virtual reality (VR)

A VR experience consists of entering a computer-generated simulation, with the equipment providing an essential role. A VR headset completely blocks your view of the real world, immersing you in entirely virtual worlds. The headset’s myriad sensors and technologies translate your movements into the virtual world. This information is then used to determine your movements (sometimes embodied in a corresponding avatar) in the virtual world.

Augmented reality (AR)

AR experiences layer digital objects onto the real world in front of you. AR provides digital content that supplements what is before your eyes, and that you see either through special AR glasses or through the camera on your phone, tablet, or other device.

Mixed reality (MR)

VR and AR converge in mixed reality. Researchers Paul Milgram and Fumio Kishino coined the term in 1994 to describe the continuum between totally real and totally virtual environments. These days, mixed reality describes environments in which real and virtual subjects and objects interact in real time — and in which you can interact with both real and virtual components. This requires a headset that has either a transparent lens or a camera, so that you can still see the real world. Note that the Windows Mixed Reality headset is a VR headset with a camera.

Differences in worlds and hardware.

World: In VR, you’re completely closed off from the real world. In AR and MR, the real world can be involved in the experience, and either augmented or mixed with digital elements.
Hardware: You need a device for all three experiences. With VR, you wear a headset that completely blocks the real world. With AR, you might use a device like a smartphone to drive the experience. With MR, you use glasses or a headset such as the Microsoft HoloLens, HTC Vive, or Magic Leap that allows you to continue to view the real world.

Applications of VR, AR, and MR.

Entertainment

With VR headsets like Oculus gaining traction with mainstream audiences, the gaming and cinematic applications of VR are immense. VR games run the gamut, drawing inspiration from blockbuster movies, historical settings like medieval Europe or Prohibition-era Chicago, and arcade game favorites. The COVID-19 pandemic has led many theatrical directors to use VR as a way to produce dramatic works even as theaters are shuttered, while also taking advantage of the format’s unusual possibilities that allow audiences to become characters or choose their own endings.

AR has broad applications in the entertainment industry as well, including both cinema and gaming. Pokémon Go (2016) is perhaps the best-known example of an AR app that went viral, when millions of people around the globe became engaged in a world populated with cartoon characters where reality — and the user environment—was a little more magical.

With MR being the newest of the three technologies, its entertainment applications are still in their early days. Angry Birds FPS is a spatial gaming experience that uses the Magic Leap headset to superimpose pigs onto your real-world space.

Other industries: Healthcare, architecture, education

VR and AR are established in a range of industries, including architecture, healthcare, and education. Mixed reality technologies have promising applications that are beginning to see adoption.

VR experiences have revolutionized the architecture industry by offering the opportunity to explore conceptual buildings. Whereas clients might previously have had to travel long distances to see a 3D model, anyone with a headset and a “key” to the virtual building — that is, access to an app and the project — can experience the site virtually. AR allows people to make virtual changes to their real living spaces: You can see what new furniture would look like in your living room, or what different colors of paint might do to your space.

In healthcare, virtual reality allows for aspiring surgeons to see the process through the eyes of more experienced counterparts, while scripted scenarios can prepare surgeons for surprises and allow them to practice their reactions. MR also has broad potential applications in healthcare, allowing surgeons to use MR-powered X-ray vision to see under a patient’s skin to blood vessels and bones during surgery.

All three technologies enable immersive educational experiences, making classroom learning more engaging and creating expansive training opportunities in a variety of settings.

style

Grid width 8, static links

style