Augmented Reality is still evolving but some truths are already emerging. Discover these design insights from the front lines.
The Lumiere brothers helped give birth to the modern cinema. But it’s telling that they called their invention “motion pictures” when they debuted the technology in 1895. Because they couldn’t think beyond the paradigm of photography, they failed to grasp the possibilities of film (and they walked away from their creation in 1905 because they saw movies as nothing more than a novelty).
Augmented reality is at a similar stage of development. Despite a lot of excitement around the technology — and notable successes like Pokemon Go — the true potential of AR remains untapped. What we’ve seen so far is mostly gimmicks that treat AR as a novelty. But that doesn’t mean AR is a gimmick or novelty.
AR has the potential to be a transformative technology that marries the physical and digital worlds in new and astounding ways. As it blurs the line between reality and the artificial, it opens the possibility of a world where almost anything can happen. Because it works on both sides of the screen — the user and their environment are as much a part of the experience as the digital content on the other side — it represents a new paradigm for how we relate with the world around us.
One of the knocks against technology has always been the way it isolates us from one another. VR plays right into this anxiety. But AR holds the potential to bring us out of our screens and into a blended reality that can be social, shareable, and even prompt us to engage the world in new ways.
Through our partnerships with tech leaders and the larger design community, we see certain principles and trends that hint at the shape of things to come. Of course, there are substantial obstacles to overcome. The technology is still in its infancy and the way most of us experience AR — through our smartphones — is less than ideal. But like most technologies today, the speed of development is so rapid that technical limitations will almost certainly disappear rapidly.
The only real obstacle is our imagination. Here you’ll find 8 guiding principles that will both shape your thinking and unlock your assumptions so you can begin to truly explore the potential of this rich technology.
01
Don’t make AR for the sake of AR
Just because your friends do AR doesn’t mean you have to. What makes AR unique is its potential for blending the physical and digital worlds into something new and unprecedented. At its best, AR delivers an experience where the digital enhances the real world and the real world seamlessly integrates the digital. But as with any new technology, there is always a temptation to let the whizzy novelty of the technology overwhelm any other considerations. We have seen AR apps that employ the environment as nothing more than a backdrop. Simply pasting product or characters into a physical realm will feel gimmicky and fails to tap into what makes AR unique.
Take a hard look at your project: If you’re trying to create a user need to fit the technology — rather than create AR to answer a user need, you’re probably forcing a use case — and you should probably look at achieving your goals through another medium rather than invest heavily in AR.
02
Design AR for the real world
Infinite possibilities. Infinite complexity. Digital design deploys responsive design to address a myriad of variables — like form factor, screen size, resolution, device sensors — so that you deliver a consistent experience. AR takes this complexity to a whole new level. You are designing for an infinitely complex canvas — the world — and that creates an exponential increase in variables.
Like a movie, AR asks the user to suspend their disbelief. But unlike the controlled environment of a movie or VR, augmented reality can’t control the world of the user. You can’t account for every variable but our experience reveals there are a number of common issues that you can design for. For example, when a virtual object is placed in a real environment, our brains want to believe it’s actually there. This suspension of disbelief is broken when somebody passes through your object as if it was a ghost. In VR, we can add colliders around objects to make sure it reacts accordingly when other virtual objects collide with it. In AR, how might we detect collision with our virtual and physical objects? Another example is designing for the different lighting conditions and textures in the real world. AR technology detects planes and objects but the qualities of these surfaces are highly complex. You want to design for both ideal conditions — and the more likely scenario where conditions are less than ideal.
Real World Awareness. When a real-world object passes through a virtual object, it breaks the suspension of disbelief. Here is one way to design a virtual Instead of real world objects passing through virtual objects, we illustrated how virtual object to be s might be aware of its “real” surroundings and react to potential collisions.
Another dimension of variability is the variability of device types different users will have. If your AR experience supports multi-user interaction, how would a user with a smartphone and a user with AR glasses interact together? AR experiences that take into account simultaneous users with multiple device types is one way to increase its lifespan.
03
Give users the power to fine tune reality
AR is not about escaping from the world. It’s about creating the world you want to live in. Today, most AR experiences are about adding content to reality. While this fits the definition of “augmented,” we believe there are more interesting opportunities to think of AR as an opportunity to filter or adjust our reality to suit our mood or outlook.
AR has been portrayed as sensory overload — a phantasmagoria of digital content overloading our ability to process all the sensory inputs. But AR doesn’t have to be limited to adding. It’s can also modify or remove layers to idealize our reality.
Take Harman’s concept of in-ear buds that allow the wearer to granularly adjust what they hear around them. This includes dialing up or down or completely replacing sounds around the wearer. Or imagine a visual parallel to this concept in which users with AR glasses or contact lenses control what they see in their surrounding environment. The ability to dial up — or down — the opacity of certain objects or completely replace them presents intriguing possibilities for creating AR that puts all the control in the hands of the user.
MIND & BODY — HOW AR GIVES FORM TO AI
Right now, AI can still feel like the voice in the machine. We believe there’s a compelling case for blending AR with AI to give form and dimensionality to this intelligence. We looked at how AR might add value to a conversation with an AI agent. First, we prototyped an interaction in which the user asks Alexa to show optios of plants that would look good and thrive in his house. The user can then point to a plant and Alexa gives more details about the plant. This integration of AI and AR gives Alexa a presence in the conversation as if you are both looking at the same thing and talking about it.
Greater than the sum of its parts. Use cases like this, in which the physical is seamlessly blended with the virtual, brings together vision, voice and intelligence to create a richer experience.
04
AR is not a solo sport, find ways to share it.
VR is immersive and isolating. AR begs to be shared and socialized. We see our clients designing for AR experiences aimed at the solo user on their smartphone. That ignores people’s built-in desire to share great experiences.
Even if your experience is designed for the single user, think about how it might scale to accommodate multiple users. Make it easy for users to invite others to see what they’re seeing. A family experiencing and interacting with the same AR story together is much more engaging than the isolated user. And accommodating multiple users doesn’t have to be synchronous — you can create AR that reamins hidden, like an Easter egg, until triggered by an event. Imagine users being able to share travel memories with each other by “leaving” their memories at a location. Other invited users would then be able to see those memories when they travel to those locations, perhaps years later, in AR.
Enriching storytime. A solo experience could be enriched if others can share in the experience. Not only can the story be viewed by multiple people, each person can interact with the characters.
05
Augment their reality, don’t disrupt it
The journey to a new world begins with the world they already live in. You’re asking people to make a big leap with AR so don’t make them jump through a bunch of hoops to get there. Nobody blinks when they see someone pulling out their phone to take a photo. However, when someone waves a phone around as they view a virtual object others can’t see, that breaks with established behavior. We find AR works best when you leverage well-established behaviors — like that ubiquitous moment when users pull out their phone to take a photo. What if AR could guide users to take better shots based on others who have taken pictures in the same location.
Better group photos. AR can help us take better group photos by guiding you on how to best position people in the photo for the optimal shot.
In the future, AR will be more ubiquitous and people will be viewing AR through glasses or contact lenses. Until then, most people will be introduced to AR through a smartphone app. Closely coupling your AR experience with existing behaviors will help more people adopt it.
06
Integrate calibration into your story
Calibration doesn’t have to be a drag. Current smartphone AR apps require users to point their cameras to a flat surface in order for the software to calibrate. This stumbling block to enjoying the actual AR experience is usually clunky . As the technology evolves, calibration may no longer be necessary. Until then, we recommend integrating calibration seamlessly into the experience by making the calibration moment the start of your story rather than a barrier to entry that must be overcome.
Here’s one example. We created an augmented visual generated by a smart device in conjunction witha voice agent. The calibration experience starts with users pointing their smartphone to the device. As calibration occurs, a world builds around the device, with the scenery matching the prompt the agent responds to the user with.
Invisible Calibration. In another example of AR meets AI, we show how an assistant might educate the user on the solar system by not only talking about it but showing a visual. This would first require users to first calibrate around the device which acts as the center of the solar system. During the calibration process, flashes of light could appear around the device while the planets slowly take their place in their orbits.
07
AR will be huge, but don’t lose sight of the big picture
Augmented reality. Augmented marketing. AR is only one touchpoint of many your users will have with your brand. Your users may experience the services and/or products your brand has to offer on a range of devices, in various spaces and settings. Before creating your experience and releasing it into the world, ask some key questions. What content are they consuming and engaging with related to your brand? Where and how are they currently consuming it? How are they engaging with it? Once you’ve mapped out your user’s journey, you can strategically determine where AR best fits and adds value.
Conclusion
In the evolution of human-computer interaction, humans have had to adapt to the way computers worked. Graphical user interfaces were a leap forward from command line interfaces but for the most part, GUI’s are at best a mask that hides the code just below the surface.
That’s why AR holds so much promise. While still in its infancy, AR could integrate the digital world into ours in ways that will make the relationship between the two symbiotic and complementary.
However, to create truly transformative experiences we will need to throw out most of our assumptions about the interface between humans and technology. We should resist the temptation of overlaying our familiar GUI paradigms on top of the physical world. How do we push beyond our assumptions and create something truly unique and native to an augmented reality experience? While not exhaustive — or definitive — the principles we’ve outlined here should guide your thinking as you help create the future.
