Apple's reality: All the latest details on Apple's AR exploits

Tim Cook ain't playing around
The latest on Apple's AR exploits

The starting pistol on Apple's AR exploits were fired when ARKit launched, and that's only the beginning. Apple is said to be working on a pair of smartglasses for unveiling in 2019, with plans to release them to the mass public in 2020.

So what do we know right know about Apple's plans? Well, we can start with hardware. Nikkei reports that Catcher Technology, which makes metal framing and cases for the iPhone, is going to start making lightweight framing for augmented reality devices. It's not saying it's doing this for Apple, but it being a key Apple partner makes that an easy jump to make.

Read next: Best ARKit apps and games

By the way, "lightweight" framing indicates a form factor more like actual glasses rather than a headset or visor. This would fit in with Apple's obsessiveness with building the most svelte and thin gadgets possible. It would also match with Apple's newfound desire to make wearables that are also fashionable, just like the Apple Watch.

Apple is looking to power this new device with a custom system-on-a-chip, much like it does with the Apple Watch, according to Bloomberg. There's also going to be a display on it.

And then there's the software. It's reportedly going to run a new fork of iOS called rOS, which stands for Reality Operating System. Apple is currently prototyping AR applications for this new headset, so goes the Bloomberg report. These apps are both rebuilt versions of Apple apps, like Maps and Messages, and brand-new apps like virtual meeting rooms and 360-degree video playback. We don't yet know how the interface will look or what kind of things rOS will be capable of.

We are told that rOS will have an App Store where you can download applications built for ARKit. Interestingly, Apple is internally using an HTC Vive and developing a Samsung Gear-like device to test AR applications for the headset.

This matches up with an earlier Bloomberg report, which says Apple had been in talks with potential suppliers for components of a glasses-like device late in 2016, and "has ordered small quantities of near-eye displays from one supplier" for testing purposes. The device would connect to the iPhone and present images over the wearer's vision. It would show images and other information in the wearer's field of vision, and may use augmented reality, the anonymous sources said.

Who's behind it

So who's putting all of this together? That would be the same team behind ARKit, led by former Dolby hardware head Mike Rowell, who has assembled a team consisting of veterans from Oculus, Microsoft's HoloLens team, Amazon's Lumberyard VR platform, Google Earth, and, perhaps unexpectedly, Hollywood special effects studio Weta Digital, which was behind movies like The Lord of the Rings and Avatar.

What's the latest with Apple's AR exploits?

Apple has also moved over people from its camera team to work on the project, which makes sense because the camera is vital toward AR. As AR-focused VC Amitt Mahajan told us, AR not only needs to be good at displaying virtual objects on top of real ones, it needs to be able to see and understand the world around you to do it properly.

So, why 2020? Earlier this year Robert Scoble did say an anonymous Carl Zeiss employee told him the company was working with Apple on a light pair of augmented reality smartglasses that may be announced this year. It turns out that miniaturizing the technology to make AR smartglasses that people actually want to wear is incredibly difficult. Don't take our word for it.

CEO Tim Cook said this to The Independent in an interview. "There are rumors and stuff about companies working on those – we obviously don't talk about what we're working on," Cook said. "But today I can tell you the technology itself doesn't exist to do that in a quality way. The display technology required, as well as putting enough stuff around your face – there's huge challenges with that. The field of view, the quality of the display itself, it's not there yet."

Cook continued: "We don't give a rat's about being first, we want to be the best, and give people a great experience. But now anything you would see on the market any time soon would not be something any of us would be satisfied with. Nor do I think the vast majority of people would be satisfied." He did conclude, however, that "Most technology challenges can be solved" but said it's "a matter of how long."

Must-read: Meet Apple's augmented reality dream team

In the meantime, Apple will be happy to let developers use ARKit, iPhone and iPad to put together some augmented reality so that it has a big old ecosystem of content ready and waiting for those smartglasses. After all, we've seen some amazing stuff coming from developers recently.

Apple will also look to beef up ARKit's abilities in future iOS updates, like iOS 12, so that developers can do more things, like allow apps to remember where you placed virtual objects so that if you close and open them everything is exactly where you left it.

Apple's challenges

In November we reported on word that Apple could launch a mixed reality TV within the next year. It would use the TrueDepth camera system on the iPhone X to be able to do Kinect-like sensing in your living room. Things hints at a larger AR ecosystem built for your home.

However, despite all the apparent fun and games going on, Apple is still trying to figure out a compelling use case for AR moving forward, according to Financial Times and Wired. As it does that, there are multiple prototypes floating around, one of which is as simple as Snap Spectacles, recording video and relying on the iPhone as a display. One thing is for sure: Apple's AR glasses are 100% real, according to AR-focused VC Matthew Miesnieks, who says he's spoken to people that have held them.

While Apple is still looking for a use case, a leak on Reddit from alleged Foxconn insiders reveals reveals there could actually be trouble behind the scenes. Chiefly, the employees noted that there's currently a 65% chance 'Project Mirrorshades' is completely scrapped. At the time, the leak said the smartglasses will be delayed until 2018 or potentially 2019 - if they do see the light of day. However, this timeline is even more aggressive than the reported "aggressive" timeline of revealing Apple's AR glasses in 2019 and releasing them in 2020. And even then, that seems ambitious as we actually think it could be even longer than that before the tech is ready, as per Cook's comments, delays or not.

Plus, according to Bloomberg, Apple is reportedly unsure about how users will actually control the device. It's looking at Siri, touch panels and head gestures to get around the device, but hasn't settled on any of them yet. The Cupertino company doesn't typically "lock" device designs in until a year before it ships, so it's still got most of 2018 to figure all of this out.

While it's clear Apple is still locking down features, it's worth noting that the Reddit leak has been partly corroborated by the Bloomberg report. Thus, it's worth taking a look at how the report said the glasses could supposedly feature a microphone, accelerometer, magnetometer and a capacitive strip on the arm for volume and call functions. Bone conduction tech would be used for audio, while integration with apps could see head movements control the action. Another interaction method could be Siri, according to some updated Siri patents that included references to Siri as a remote for smartglasses.

Resolution for the glasses is allegedly 428 x 240, with crystal (described as clear), champagne and black designs all being entertained in both men and women's sizes. Rough estimates also placed the price tag for a potential set at around $600.

Now, it's no secret that Tim C0ok is excited for AR, and the launched of ARKit proves that Apple is serious about this space. As does the fact that Apple purchased a couple of companies, like Metaio and FlyBy Media, that are well-versed in AR software. More recently, Apple bought SensoMotoric Instruments, which specializes in eye tracking. Wouldn't that be helpful for a pair of AR glasses?

That's what makes the scrapping forecast from Foxconn so interesting. This is a project that has always possessed the potential to be delayed due to the leap forward Apple seemingly wants to make in the space, but a lot is already invested for it to be dropped completely. For now, it appears we'll have to wait for the current moving parts to settle.

If we do see the glasses launched – whatever the case with Foxconn, glasses seem like an eventuality anyway – it would be the second major new hardware category launched under Tim Cook, the first being the Apple Watch. The Apple supremo has also gently nudged away speculation that Apple would be going first to VR, saying that he sees more value in augmented reality. Ol' Timmy Cook: all about the wearables.

Shop for VR headsets on Amazon

Oculus Rift
Oculus Rift
PlayStation VR
PlayStation VR
HTC Vive
HTC Vive
Samsung Gear VR
Samsung Gear VR

Wareable may get a commission


  • Extraneus says:

    iPhone-only smartglasses will never hold my attention; luckily, Laforge are getting their Shima smartglasses out to customers this year, and they'll be compatible with iOS, Android AND Windows 10 Mobile... Can't wait until mine arrive, hopefully just before the summer holiday!

  • SiolioS says:

    There needs not only to be the Specialist's working in this Field, One would be a Fool to ignore others quietly inovating across all Platforms to achieve a more open less proprietary solution. It's a big World with modern communication lessening the distance between Large Corporations that can throw VC & Cashed up Profits at the Subject & the little lone Back Yard tinkerer with Diverse & ever growing Skill sets just shadowing these Large investments.

    To produce a product for release to early is just a stepping stone to the all achievable final sort after solution.

    If one hadn't considered that not only Developers provided with the right environment help bring a product of this nature to masses, then I'd suggest one would only have to review the slow but steady uptake of Corporations & there ever growing arrangements over the last 18 months to work side by side with the Open Source Community. VR, AR/MR & .Ai are vast fields with many obstacles to overcome.( I'll leave this on the note that one in contenplation already knows these fields can not & will not be owned by any one Company/Corporation who decides to pour Endless amounts of liquid Cash into it! - It's just not going to happen. We are in the Age of the Entrepreneur & I do not see that ever reversing itself again. )

What do you think?

Connect with Facebook, Twitter, or just enter your email to sign in and comment.