How does Augmented Reality work?
Augmented Reality (AR) has come a long way since the very early attempts in the late 1960’s to the release of ARKit and ARCore in 2017 that enabled millions of devices to access AR at any time. This has opened a billion dollar market and the promise to transform education, medicine, marketing and eventually our lives. So how does this life-changing technology work?
Computer vision and rendering scenes in real-time
AR adds digital content to a live camera feed and makes it look like being part of the real world. This content can be anything from overlaying information onto the environment like Points-of-Interest or driving directions. Other examples include interactions with 3D objects or 2D content like images projected into your field of vision.
Understanding the real world in 3D, based on 2D images and then adding content to a 2D screen still provides a lot of challenges for AR to overcome. The better the application understands the physical world the more immersive experiences will become.
Two things need to come together to create a realistic experience: computer vision to recognize objects and surfaces to anchor content, and 3D/2D rendering to include the content in the camera feed. To make this a little less abstract, let’s work with an example.
The different types of AR experiences
A common type of AR is using a image to trigger an AR experience. Once this image, let’s say a logo or an image is recognized, the AR content is placed in the scene, using the image as a reference to determine the position of the AR content. Once this point of reference is clear, the scene can be rendered in real-time. That usually happens at a speed of 30 frames per second. As AR is always happening in real-time, only in recent years mobile devices have gotten powerful enough to handle these calculations in on the fly.
Of course, there are other ways to anchor an experience. This can be done based on geolocation, based on the user’s GPS coordinates and viewing direction, or through SLAM (Simultaneous Localization and Mapping) sometimes called imageless AR that is capable of recognizing entire scenes. Alternatively, the software can detect a surface to place the AR content. More information about the different types of AR can be found in this article.
AR-powered devices
At the time of writing this in 2019, AR is still mostly used on smartphones and tablets, but slowly wearables like smartglasses are becoming more and more common for industrial experiences. This is similar to the evolution we have seen for computers, which also started out with very expensive and bulky devices and slowly turned into something mainstream that everybody owns.
So before we all walk around with smartglasses, a few years are certainly going to pass. However, there are dozens of devices available for iOS and Android that support AR already, because all of them count on a web browser so they are ready to launch web AR experiences.
Try Augmented Reality yourself
There is only one thing left to do. Try AR on your own device. With this example that you can find in our Experience library you can give life to a world map to get more info about potential places to visit on your next holidays:
