Apple's ARKit has barely been on the scene five minutes, but it's already blowing minds. In fact, the augmented reality toolkit is only in the hands of developers for now, but come the launch of iOS 11 in just a few weeks time millions of iPhones and iPads are going to transform into tiny AR portals, conjuring all sorts of mixed reality experiences. All expectations are that, eventually, Apple's AR will move to smartglasses, a more natural home for augmented/mixed reality.
Right now Apple's taking the "build it and they'll come" approach, handing developers the necessary tools and letting them do the rest.
And they've been building alright.
Chances are you've seen some of the projects doing the rounds. We've been speaking to some developers who are already using ARKit to get their impressions of how good the platform is, what it has the potential to be, and what it might do for augmented reality as a whole.
Not fussed about any of it? By the end of this you will be, and come September, when iOS 11 hits iPhones, you'll be able to experience all of this for yourself.
Time to experiment…
When Apple revealed ARKit back at WWDC, it was immediately clear that it would mean AR would become as accessible to developers as any other API. And if there's one thread through every chat we've had with developers, it's that ARKit is relatively easy to get to grips with, which has been fruitful for experimentation.
"ARKit is easy to use but still requires some experience in iOS development," said Patrick Balesta, a developer and one of Apple's WWDC 2017 scholars. "All the complicated math is abstracted away from the developer which makes it very easy to get started with."
You're not going to be able to go into it cold, but most developers we spoke to reckon that, with a little coding experience, and the know-how for sourcing 3D animations, ARKit is easy to work with.
"The best surprise with ARKit is that the people don't really understand what they're seeing and how it works but they want it," says digital designer Tomás Garcia. If you've seen the Snapchat Hotdog doing the rounds recently, you know what he's talking about – but the quality of Apple's AR is noticeably more authentic than what we've seen from Snapchat and, arguably, anyone else to date.
Take Garcia's dancing living room robot, shown above. Not only does Apple's SLAM tech (Simultaneous Location And Mapping) and drift correction keep the robot convincingly real, the shadow and lighting effects are spot on.
So impressed were they with the demo after it went viral, Apple reached out an offer of support to Garcia. "A week ago, when my ARKit videos were shared all over the world, the WWDR (World Wide Developer Relations) from Apple contacted me to invite me to a lab in order to give me further assistance for the projects we're starting with ARKit," he tells Wareable. His SpaceX Falcon 9 demo is worth a look too.
We were suitably impressed with a basic demo of virtual coffee cup on a table back at WWDC, but what we've seen being made since have made that look like child's play. Inter-dimensional portals? Check. Lifesize basketball games? You betcha. Or how about a whole new way to order food? Yup, just wait until the #Brands get onboard.
ARKit even has the potential to go beyond the realms of Apple's ecosystem. Developer Normal VR recently shared a video of ARKit being used in conjunction with a HTC Vive. As you can see in the video just above, the ARKit monster is mirroring the Vive user's actions in a Tilt Brush-esque AR experience.
"Unity makes prototyping projects like this very easy," explains Normal VR founder Max Weisel. "The character and brushes you see here are from another one of Normal's Vive projects. All that I needed to do was import Unity's ARKit plugin, rewire some of the interaction for the touchscreen, and that was it."
While this is the only project Normal has shared, Max said that the team have been playing with lots of ideas to mix AR and VR together. "The release of ARKit was so perfectly timed, we just had to try this out. As devices like the Hololens become more prevalent, I personally think the potential for compelling social interactions will be huge."
Real use(ful) cases
Dancing robots and teleporting portals show the potential of AR to wow, but we're also seeing creations with more immediate utility. iOS developer Andrew Hart recently came up with an application for ARKit which overlays map directions onto the real world to product a next-level turn-by-turn navigator.
iPhone app maker Laan Labs also saw its AR measuring tape demo go viral for demonstrating ARKit's precision. "What's really amazing is Apple's sensor fusing," Jason Cyril Laan tells Wareable. "Basically that Apple has been able to combine all the inputs from the phone's various sensors to give developers amazing access to data points in 3D space".
Measuring things just got… interesting? Another of Laan's AR experiences involves writing in the air – again very Tilt Brush, but this one's built solely using ARKit.
Meanwhile Bjarne Lundgren, lead mobile developer at TV 2 Danmark A/S, tells us he's also been toying with 3D mapping and GPS coordinates in ARKit, but you might better know him for his Tic-Tac-Toe game, which has been floating around the web.
"The biggest surprise in using ARKit has been, without a doubt, that it works so well using only phone orientation and a camera," says Lundgren. "It is an amazing feat – and it blows all the competition out of the water."
But Apple is of course not alone in the great AR race. Microsoft, Meta, Avegant, Magic Leap are also busy building, while Google is perhaps the most direct rival with its Project Tango platform. We've been impressed by Tango too, but Google needs to get it running on more smartphones (only the Lenovo Phab 2 Pro and upcoming Asus ZenFone AR have the required sensors).
The quality of Tango, from what we've seen of it so far, feels almost on par with ARKit, but it will be harder for developers (and us) to get as excited until it's in the hands of many more people. Like ARKit, Google's Tango feels ripe for one day transitioning to a headset or a pair of smartglasses.
Where next? (Glasses duh)
In just a few weeks' time, ARKit will be in the hands of millions, and we can't wait to see it pushed to breaking point. It's already capturing people's imaginations, but this is Apple's first play in the AR space. Tim Cook has waxed lyrical about AR and its superiority to virtual reality, laying bare Apple's intentions to be a key player in the space. Talk of AR glasses is providing endless grist for the Cupertino rumour mill, and chances are it will be some while before a pair of Apple Specs land, but the feeling from devs we've spoken to is that if ARKit proves anything, it's edging augmented reality to the point that it will feel too confined by a smartphone or tablet.
Until the glasses come, what more do developers want to see from ARKit?
"It doesn't seem possible to share 3D worlds and 3D space between two devices even if they are looking in the object/floor area," says Bjarne Lundgren. "This is something Apple could – and should – provide better support for in the future, collaborative experiences."
Better depth information is another thing Lundgren wants, and with rumours of 3D sensors on the iPhone 8 that wish might be fulfilled before long. "I think what is missing is a way to get depth information about the Vision detected object so they can be positioned in 3D space," he says.
As for Normal VR's Max Weisel, he says AR glasses are top of his wish list for future developments.
"I think that the most exciting potential applications will be to connect our spacial data with our information and use it for educational, social, practical and storytelling purposes," says designer Tomás Garcia. "We still don't know what tools we are going to create but I think it'll change the way we see and understand information."
"ARKit is in many ways the technology we need to enable truly useful AR glasses," adds Lundgren. "AR experiences are all fine and can be great on an iPhone or iPad screen, but will become truly amazing with glasses – much, much more immersive. I think we will see glasses or lenses from Apple eventually, and I think they have the potential to replace the smartphone."
AR outside Apple