Snap back to reality: What Snapchat is up to with AR

While the likes of ARKit, Animoji and Instagram have come around to take some of the shine off Snapchat, the app is still one of the big players in augmented reality.

Its addictive filters are still hard to beat, and Snap has doubled down on AR to bring different kinds of augmented experiences to its ephemeral chat app. In fact, a quick peruse around the Snapchat app can reveal a few hints at its long-term AR plans, which we expect will crescendo with smartglasses.

Essential reading: The Snapchat AR glasses investigation

Snap currently has patents for a pair of glasses with AR tech on board. Snap has also hired talent with AR experience under their belts, including those who have worked on HoloLens. Plus, CEO Evan Spiegel has talked about how augmented reality is the next big computing platform. So yeah, it seems like a sure bet.

Snapchat has quietly been building an ecosystem of AR experiences within its messaging app, and it's not hard to see that it will be easy to port them over to its eventual glasses. But what exactly has it been doing? Gather around, folks, as we dive into the AR world of Snapchat.

Snapchat's AR: The experiences

The most popular, and well-known, versions of Snapchat's augmented reality are those face filters. These are little masks and filters that go over your face, whether it's a fun hat, some augmented face paint or – well, you've all seen the dog filter right?

This novelty has gotten an upgrade with Apple's iPhone X. The app now uses the sensors on the front of the phone to deliver higher quality filters that better map to your face. The effect is hard to describe, but the improved tracking is noticeable when you use it. It almost feels like they're painted on your face.

This is all fun and games, however, and Snap has worked hard to turn these things into entertaining ways to interact with your friends. In fact, Snapchat's video chat makes routine use of filters to add some spice to your conversations.

Facebook has attempted to mimic this with filters for both Facebook and Instagram Stories, though at the moment they don't seem to have the same cultural cache or fun factor that Snapchat's filters tend to.

You can easily imagine a world where you've got a pair of Snapchat AR glasses that allow you to place filters on everyone's faces.

The one AR face transformation feature that seems to get anywhere near the same level of buzz is Apple's Animojis, which are going to be a much bigger part of Apple's AR strategy come iOS 12. Apple will soon allow users to create their own Animojis and use them in chats, much like Snapchat is already doing with Bitmoji.

You can easily imagine a world where you've got a pair of Snapchat AR glasses that allow you to place filters on everyone's faces. While Snapchat may have been first, though, Apple has the clout and enough people with iPhones to overshadow Snapchat in this regard. Snap does have one slight advantage in the AR wars here, and it's one of the big AR dreams: Looking at a place and getting all sorts of information on it.

Snapchat kind of – but not really – has this with 'Context Cards'. Say you're watching a snap of some food at a cool restaurant. If someone is using a location filter that identifies that restaurant, you can swipe up to reveal a whole bunch of information about that place. You can see the phone number and address, reviews from TripAdvisor, and even quick links to get an Uber or Lyft to the place.

Not quite augmented reality, but you could imagine how those features could be built into an AR interface (maybe a swipe up on the stem of a pair of smartglasses).

Then there's Snapchat's efforts to create augmented reality advertising, the first of which is called Snap Ad to AR. This basically allow brands to create immersive ad experiences.

For instance, if you are in a designated area then Foot Locker's Snap Ad will pop up. You can play a quick AR experience that has you catch a pair of shoes as they bounce in and out of a Foot Locker box. Similarly, if you were near a BMW dealership then BMW's experience may pop up. That lets you plop a BMW in front of you and walk around it as if it were really there.

It's important to note that these experiences are not Snapchat's traditional World Lenses. These are different experiences that can pop up almost at random, which makes them cheaper for brands to purchase. They're a little reminiscent of Minority Report. For the most part, none of Snap's AR competitors have taken to AR ads as quickly as it has, allowing it to get in on a market that will only grow as augmented reality extends to consumers.

These experiences have also proved fairly popular, with people experiencing them for an average of 45 seconds and garnering over 4 million impressions in limited tests, according to Digiday. In fact, over 100 brands have decided to take part in the new ad program.

Speaking of ads, Snapchat's not stopping there. It's also got new lenses called Shoppable AR. Basically, these are AR experiences inspired by products. For instance, Adidas lens' celebrating its Deerupt shoe places a filter over your face that looks like the material the Deerupt is made out of. There's also a link to the website to go and buy those shoes from within the app.

It's an interesting idea, as it opens up the shopping possibilities of augmented reality, but it also feels half baked. How can you go from a fun piece of AR to a boring old website? It's going to feel like a half measure until Snap can get brands on board to build full augmented reality storefronts within the app.

When it comes to Snapchat and AR, it's hard not to remember that cute little dancing hot dog. Well, now you can use your own Bitmoji and put them out into the world doing different things. You can also invite your friends along, watching your Bitmoji avatars interact in augmented reality even if you can't interact with each other.

Snapchat's core experience is still communication and messaging, so it's no surprise to see Snap try its best to merge augmented reality and messaging in different ways – from video chat filters to letting Bitmojis play with each other in chat. But it also wants to go beyond more passive experiences.

Snapchat's AR: The games

To build a vibrant augmented reality ecosystem you need more than advertisements and social; you need things for people to do – you need games.

Back in March 2018, Snap acquired 3D game engine PlayCanvas and promptly used it to create Snappables. These are small little AR games that can be played in a variety of different ways.

Some of these games allow you to use your facial expressions, movements and voice to, well, play games. Some of the games are solo experiences, while others allow you to invite friends to play with you.

Other games are built for you to experience solo, then send as a snap to another friend, who can then jump in and play the game and send it back as a snap.

In the long term, however, Snap is stretching the abilities of its AR platform.

New Snappables are introduced every week, and some of the more popular ones stay for longer than that. In the short term, Snappables are a way of fighting off Instagram with something unique. It also makes Snapchat a more interesting place, because sometimes there's nothing for you to snap and nothing for you to talk about with your best buds – so why not play some quick and fun games?

In the long term, however, Snap is stretching the abilities of its AR platform. Not only can this platform support things as simple as face lenses and avatars, it can also place things in the world, it can create ad experiences that let people shop and it can allow people to have fun with games.

Snap's biggest competitors here are Apple and Google, who have ARKit and ARCore, respectively, and have a growing number of developers jumping in to create augmented reality games. Thanks to things like ARKit 2.0, developers can also create shared AR experiences. It's one thing to snap AR experiences between friends, it's another to feel like you're having the same experience in the same space at the same time. So in this regard, Snap may have fallen behind (for now).

Speaking of a platform, there's one final thing Snap has worked on this year to begin to build out what it's doing in augmented reality.

Snapchat's AR: The studio and platform

Back at the end of 2017, Snap debuted the Lens Studio. The idea was to let pretty much anybody learn how to create their own lenses for Snapchat. This included 3D figures you could put out into the world, as well as face filters.

You can create your lens in whatever 2D or 3D studio you like, and then import it into Lens Studio. Once it's published you get both a link and snap code that you can use to share your lens with the world. Snapchat will also surface up user-created lenses in a public story.

The idea here is simple: make it as easy as possible for people to create augmented reality content and share it. In the short term, this means that people can make their own events – like weddings or birthdays – a little more special, with unique filters and lenses.

In the long term, it means that Snap already has a platform that people can create content for. If it makes AR smartglasses, and if people buy in, it'll have a small army of people who already know how to create experiences.

And it's not stopping there. Snap is going to go a step further with Snapkit. Snapkit is a full developer platform that will let outside developers utilise Snapchat's tools to make their apps better at augmented reality.

For one, it allows users to sign into apps with their Snapchat account – much like people can do with their Facebook or Google account. For those who are concerned about privacy, this is a potential plus as Snapchat doesn't currently collect the amount of data on you that Facebook or Google does.

This also allows people to bring their Bitmoji with them into third-party apps, instantly giving them a potential avatar in a number of apps – especially augmented reality ones.

Snapkit also allows developers to use Snapchat's camera and AR powers within their apps. This would mean that developers wouldn't have to work as hard to create their own augmented reality technology, or tap into things like ARKit or ARCore. They could just let Snap take care of it.

Speaking of ARKit and ARCore, those are the two big things Snap will have to worry about here. Apple and Google control the two main smartphone operating systems, so plugging into their augmented reality technologies is a no-brainer for developers, even if Snap is offering an easier alternative. The one big thing Snap has going for it here is that it can possibly make it easier to create multi-platform experiences.

In that world, only Facebook poses real competition. Though it also seems like Facebook's AR ecosystem is as far along as Snap's at the moment, Facebook does have a social network with many, many more people on it. Unless privacy is a massive concern for a great number of people, it's hard to see why people would sign into apps with Snapchat over Facebook, a network where a lot of people have hundreds of friends and an established community.

While Spectacles 2 have come and – pretty much – gone, Snap has been in the background building out ways for its AR ecosystem to thrive. It's entirely possible that Snapchat could have one of the most fully fleshed out AR ecosystems by the time the AR smart glass battle begins in 2020. Don't say we didn't warn you.