Why you should embrace the extended reality continuum

The emerging technologies of mixed reality (MR) and virtual reality (VR) are full of specialized terminology. In addition to MR and VR, this technology space includes terms such as augmented reality, augmented virtuality, extended reality, spatial computing, wearable computing, ubiquitous computing, and metaverse. If you’re reading this, there may be more.

Any discussion of “digital realities” requires a commitment to define terms and contexts. The plethora of terms can be laboriously exhaustive to understand and erode the excitement and interest of the curious outsiders of innovators and early adopters.

MR and VR are destined to merge into one and are already slowly converging. We need to reflect this when speaking about the technology space by being concise when referring to the general and intentional and when diving into nuances.

Fortunately, we already have a way of taming the jargon and don’t need to expand the already unwieldy lexicon. Extended reality (XR) is the consensus term for “all combined real and virtual environments and human-machine interactions”.

The extended reality continuum

The reality-virtuality continuum describes a digital reality space with reality and virtual reality endpoints. Mixed reality is the spectrum between the endpoints. From a contemporary perspective, there are clear boundaries between reality and MR, and reality and VR. The MR spectrum is naturally and incompatiblely blurry and without clearly defined boundaries. Reality and VR are discrete states, but MR is a non-linear gradient.

Extended reality is the spectrum from MR through VR, or from another perspective, the reality-virtuality continuum excluding reality. We can call this subset of the reality-virtuality continuum the extended reality continuum.

Technological advancements and improved experience design will reduce the significance of the differences between MR and VR, leaving users (even advanced ones) unaware of the differences. Classifications, as defined by the XR continuum, make sense only to designers and developers.

The impact on experience design

It’s interesting to imagine that the distinction between MR and VR might not always be as straightforward as it is today. We’re seeing early examples of immersive digital experiences that are fluid and not clearly MR or VR. This duality raises questions about the influence of both MR and VR on the quality of the user experience.

For the sake of exploration, let’s assume that hardware and software exist to support both high-end MR and VR experiences. The form factor is immaterial, but to aid the imagination, consider Geordi La Forge’s visor (Star Trek: The Next Generation), every helmeted character in it Star Wars (Darth Vader, the Mandalorian, stormtroopers), or an implanted device (contact lens or eye replacements) as in The black mirror “All the History of You” (S1E3).

Several questions immediately arise when considering the impact of multidimensional augmented reality on experience design. What are the pros and cons of a multi-stance experience in the XR continuum? Can a single experience successfully switch back and forth between MR and VR? Can a single experience successfully cover multiple stops in the MR spectrum? How do we design MR experiences that are true augmented reality? Where on the XR continuum is the best experience for my users and their problem area?

Here are some of my early speculative thoughts. Designers need to declare themselves to be MR or VR and stay true to that attitude, but this thinking needs to be challenged and validated. Switching between MR and VR is likely to occur through well-defined modalities if possible, or shift so slowly that the user is not aware of any change. Attempting to be both MR and VR within the same experience is likely a design trap.

For now it is impossible to do more than speculate and experiment. However, we can put forward educated hypotheses using past wisdom from designing interactive experiences.

The impact on hardware

Device hardware will also adopt this dichotomy. The most recent wave of notable devices, such as Oculus, HoloLens, Magic Leap, and smartphones (iPhone and Android), have a strong preference for MR or VR. Each device optimizes to enable one type of experience or the other, but not both, and for good reason. Supporting both creates many technical challenges and can dramatically impact production costs, in addition to the experience design challenges mentioned earlier.

However, this is changing and future iterations of devices will support both MR and VR experiences. Devices like those of Varjo, Lynx and Meta are screen-based, with pass-through camera capabilities. Passthrough means the device uses an external camera to capture what obscures the screen, allowing the user to “see through the screen”. These devices can support high-resolution MR and VR experiences.

The Magic Leap 2 (ML2) can dim the outer lens to create a not quite opaque view of reality. This feature is more about improving the visual quality of displayed content and less about a meaningful attempt to enable VR experiences. MR devices fall significantly short of supporting a high-quality VR experience due to a limited field of view and inability to completely block out the physical environment.

It’s easy to envision (and hope for) a generation of XR devices – any device that can support any experience along the XR continuum – and not devices dedicated solely to MR or VR. Unfortunately, this could take several years and will likely require different form factors than we have now. Still, there will remain a market for dedicated MR and VR devices. As the technologies become commoditized, low-cost or solution-optimized hardware will continue to separately support VR or MR.

Converging points

Embrace XR as the general anchor point for discussing MR and VR technologies as it best fits the common expressions of MR and VR. The term supports discussions with a wide audience and within the maker community. There is a cognitive benefit in simplification for everyone. The extended reality continuum is a basic structure for technical or detailed discourse that designers and developers need. If we settle for a simpler lexicon, we can focus on much more interesting things.

By simplifying and unifying the terminology, we can move beyond the “Can we do this?” stage of an emerging technology and to explore what experiences the technology enables. Designers and technologists need to prepare for the full spectrum of XR experiences. Now is the time to explore the XR continuum and establish the experience design principles that will shape the medium’s future and success.

We’re entering what I think is the most exciting and exuberant phase of all emerging technology.

Jarrett Webb is director of technology at argodesign