Augmented reality. It's a term that gets the tech elite excited, yet is often misunderstood and hard to define. While it's bubbled around in the world of business for decades, it's only now starting to make the jump to the consumer world.
Augmented reality smartglasses have been hailed as a potential game-changer for the way we live our lives. But what is AR, actually, and why is it such an exciting buzzword? Why are Apple, Facebook, Google and the rest leaning so hard towards an AR-based future?
Augmented reality isn't exactly easily accessible to try out for yourself, so it can feel a bit obtuse. And even when you do use it, the technology and the ecosystems behind them aren't good or useful enough for most people.
Rest easy, friends. We've put together answers to some of your burning questions. Read on.
Augmented reality: What is it?
Virtual reality promises to immerse you in a simulated reality. It takes you to a space and tries to make you believe you're actually there. Augmented reality is almost exactly the opposite. It wants to keep you in your world, but it also wants to act as a layer between you and your world.
In short, it's technology that augments your world by layering graphs and images and video on top of it. The first AR device, born in the 1960s and called the Sword of Damocles (seriously), set this idea down in stone. While the device itself was huge and unwieldy, it overlaid a geographic grid on top of your eyesight.
So how exactly does augmented reality relate to mixed reality? They're actually quite a bit similar, so much so that Microsoft will insist you call its HoloLens headset a mixed reality device rather than an augmented reality one. Essentially, mixed reality is like a weird in-between of virtual and augmented reality.
Read this: AR v VR - Which is the future?
Think of it like this: Mixed reality is when virtual objects can interact with real objects in the world and vice versa. Imagine putting a virtual table in your room and you could then walk around the tablet. Or a version of Pokemon Go where the Pokemon could move out of the way of random people walking by. That's mixed reality.
The technology itself can take many forms. The most popular form right now is through our smartphones. Snapchat filters can turn you into a cute dog or act as digital makeup. It can also enhance your photos and videos in fun ways, like stretching out your face or placing your head on the body of a dancing elf.
The other two major forms out there are headsets and smartglasses. Smartglasses seem to be the desired form factor for augmented reality, giving us light and easy-to-wear devices. Headsets are a little bulky, but their size is often needed to incorporate all the sensors and technology needed to run.
Oh right, sensors. Augmented reality needs a couple of different things to function. It needs a camera, because it needs to be able to see and interpret the world. It needs motion sensors, so that when you move around your device it understands where it is in relation to the world around it. It needs software that can pull all these things together and, most importantly, it needs a portable processor and battery to keep things powered and running while you're on the go.
Augmented reality: Not just about visuals
Alright, let's back up a second here. Yes, augmented reality typically involves layering a bunch of graphics up against your view to alter the way you see your world. But, well, not always.
It turns out augmented reality is a more nebulous term than we realised. Bose, for instance, has been working on audio-based augmented reality. It's pretty much exactly as it sounds. Instead of showing you layers on top of the current world, it simply tells you about what you're looking at.
This type of AR is obviously better for those of us with impaired vision, but it also serves as a neat stop-gap as visual augmented reality tech still has a long way to go before it's ready for consumers. Visuals need strong processors and large batteries, not to mention advanced projection tech.
Audio doesn't need all that work. It can simply look out into the world with a camera, sense where you are and then tell you about the things you're looking at.
Augmented reality: What's it for?
Great, that's what AR is β but what is it for? Essentially, it's a computing platform. AR has the potential to totally change how we interact with our computers. Mostly because our computers wouldn't be stationary items we sit in front of anymore, they'd be the entire world.
For instance, the most famous example is if you're walking around a city and feel hungry. You need some sustenance, so you tap your smartglasses and ask for good restaurants around. When you look at a restaurant, its Yelp rating could pop up. You could dive in and get some reviews and recommendations, maybe even check out a menu before even going into the place. All without taking out a phone.
If you wanted to go to a place further away and not in your direct viewpoint, AR could line the streets with directions, showing you exactly where to go and how long the journey will take. It could even sync up to a city's systems and display when it's okay to walk across that crossing.
When you do get to the restaurant and order something, you could look at an item and have its nutritional information broken down for you in your gaze. Maybe it could even tell you if this will fit into your diet, or how many minutes of running you need to do before you can fit this pecan pie into your day.
Read this: The best ARKit apps
AR isn't just for objects though. It could also be used to improve interactions with people, as Stanford has been working on with its BodyNet system. You could simply look at someone and, thanks to wearable sensors they're wearing, see whether they're lying to you, uncomfortable with you or happy to see you. This would help decrease awkward dates, upsetting spouses and saying the wrong things. Plus, you know, identifying your second cousin at the wedding before you embarass yourself.
Other uses include general work and play. Magic Leap CEO Rory Abovitz has talked about being able to watch NBA games out in the middle of the air β no TV needed. You could even turn your living room into a game room, playing virtual board games or just shooting weird little ducks without having to set anything up. Plus, it's the fashionable new trend. Don't be stuck in last season.
Augmented reality: The projection question
One technical thing you need to know about visual AR: Not all systems are created the same. See, there are two main methods to creating augmented reality headsets and glasses.
Remember Google Glass? How it was essentially a frame with a tiny little piece of glass in the corner? Glass projected its display onto that tiny piece of glass. The AR in this case is fixed in that position. It can't adjust itself to your vision or anything like that. We've seen other glasses, like the Everysight Razor, adopt similar approaches to AR.
Projection-based AR can also be mapped to sensors and used for public demonstrations of AR. The best example of this is UC Davis' AR Sandbox. It projects a map onto a box of sand, and adjusting the sand adjusts the projection. Public AR hasn't been too much of a hot topic yet, but it's likely to make some waves once companies are better able to find ways to merge physical interaction with mapping techniques.
The other method is based on a technology called light field. This projects the AR images right into your eyes, and its goal is to seamlessly blend AR images with how you see the real world. You may be familiar with light field cameras, which capture the amount of light in any given scene, allowing you to alter focus after-the-fact. Light field tech for AR is similar, in that it allows you naturally focus on those AR images as if they were actually in the world. This is what Avegant and Magic Leap have based their AR tech on.
The effect is basically a subtle normalisation of AR graphics. It makes it feel like it's real, rather than a projector on a piece of glass on a pair of glasses.
Augmented reality: The devices β now and in the future
So now you know what augmented reality is, but how the heck can you use it? Or how will you use it in the future? Well, there are a number of devices out now that'll let you use AR. Most of these are aimed toward businesses and work, but the consumer stuff is just around the corner (2020 seems to be the year everyone is pointing to).
The big daddy. For a long time, HoloLens was seen as the most advanced augmented reality device out there. That advantage has slipped a bit as complaints about its narrow 35-degree field of view have racked up. There's also the promise of consumer AR headsets just around the corner. HoloLens has seen use by companies and developers, and it's also capable of working in the same apps as Windows Mixed Reality headsets for some AR-VR collaboration.
Vuzix Blade AR
The Blade AR specs are the best attempt at mainstream AR glasses yet. They look good, they come with Alexa on board and include features like turn-by-turn navigation, location-based alerts and messages. There's also an 8MP camera with 4GB of storage to fill up. You'll get around 2 to 2.5 hours of battery life. They're not completely consumer (especially that price), but they're awfully close.
Wareable verdict: Vuzix Blade review
Everysight has taken its years of experience building augmented reality devices for the military and put them to use to build a pair of AR glasses for cyclists. The Raptor uses its smartphone-like internals to give you directions and information on your ride. There's also a camera for you to take action cam-like shots of some particularly exciting moments. However, it's also a backdoor product into a wider, more general AR platform.
ODG R Series
Mostly focused on enterprise, ODG's trio of AR smartglasses are almost concurrent. There's the R7, which debuted in 2015 and has 720p lenses, 37 degree field of view, voice recognition, Bluetooth, Wi-Fi and motion sensors. The R8, which will debut later this year, sports a 40-degree field of view, positional tracking, 1080p display and a Snapdragon 835 processor. The R9, shipping in limited quantities, takes all that and bumps the field of view to 50 degrees.
$2,750 (R7), $1,999 (R9), TBD (R8), osterhoutgroup.com
Lenovo Mirage with Star Wars Jedi Challenges
Alright, so this is more of a specific, niche use but it's still augmented reality. Lenovo has partnered up with Disney to use its Mirage AR headset in conjunction with a toy lightsaber and your phone with the Jedi Challenges app. You'll also get a little ball and then, once everything is good to go, you can train with your lightsaber as if you were Luke Skywalker himself. May the force be with you.
Wareable verdict: Star Wars Jedi Challenges review
AR isn't just about allowing you to view the world, it's also about giving you objects you can interact with. That's what the Cube is about, though it's also heavily about giving you an AR-based toy you can play around with. While it needs more app support, it's both cheap and a good deal of fun β especially if you want to get your kids hooked on that augmented life.
Wareable verdict: Merge Cube review
Tim Cook is very excited about augmented reality, so excited that the company is getting going on building out its AR ecosystem before those much-rumoured AR glasses are fully confirmed. Apple has been experimenting and patenting things for a while now, including headsets you can slot your phones into.
Until that technology is ready, though, ARKit is a way to get developers to build AR apps and get an ecosystem ready. In our experience, it's some of the highest quality AR we've seen so far. It even recently got an upgrade with ARKit 1.5, enabling vertical placement of objects and group AR.
Not to be outdone, Google is also looking to build an AR ecosystem with ARCore. Google Glass may have been an epic failure, but that doesn't mean that Google is giving up on augmented reality. Google is interested in AR smartglasses, but it's waiting until the technology is mature enough to become mainstream. So why not hang back and build an ecosystem of apps and developers in the meantime? One thing to keep an eye on: Google recently bought Lytro, a company that specializes in light field technology.
Magic Leap One
After years of secrecy, Magic Leap has unveiled its first product: Magic Leap One, a pair of AR goggles that hooks up to a little disc computer and comes with a remote to control it all. We know it looks to be an all-in-one solution that replaces your computer. We know it'll have apps from the likes of ILM and NBA. We know that its controller has 6DoF. We know that it'll cost about $1,000 and that it's coming this year. And, well, that's about all we know. By most accounts (except for Beyonce's), Magic Leap One could be the AR device we've been waiting for.
Snap's AR Glasses
If there's a hero the world of AR needs, it's Snapchat's dancing hot dog. The company is perhaps the most mainstream pusher of AR out there, and it dipped its toe into wearable cameras with Spectacles. The ultimate push, however, are a pair of AR smartglasses. Snap has both the talent and the patents to do it, we just have to wait and see.
Mark Zuckerberg has literally gone up on stage and told the world that he wants to put AR glasses on our face. When this will happen, and what kind of features it will have, is up in the air. It's likely the basis of Facebook's AR push will be social, because duh, but it will likely not be an easy road for Facebook. The Cambridge Analytica scandal has already pushed back its planned smart speaker, so it's likely Facebook will need to regain some user trust before it ventures into the world of AR and world-sensing glasses.