The starting pistol on Apple's AR exploits was fired when ARKit was announced back in June 2017, and that was only the beginning. Apple is also said to be working on a pair of smartglasses for unveiling in 2019, with plans to release them to the mass public in 2020.
So what do we know right now about Apple's plans? Well, let's start with hardware. Nikkei reports that Catcher Technology, which makes metal framing and cases for the iPhone, is going to start making lightweight framing for augmented reality devices. It's not saying it's doing this for Apple, but the fact that it's a key partner for Apple makes it an easy leap.
Read next: Best ARKit apps and games
By the way, "lightweight" framing indicates a form factor more like actual glasses, rather than a headset or visor. This would fit in with Apple's obsessiveness with building the most svelte gadgets possible. It would also match with Apple's newfound desire to make wearables that are also fashionable, like the Apple Watch.
Apple is looking to power this new device with a custom system-on-a-chip, much like it does with the Apple Watch, according to Bloomberg. There's also going to be a display on it.
That display, according to Digitimes, is going to utilize MicroLED. It's a display technology that uses half the power, is brighter and offers a higher contrast ratio. Plus, it's thinner, which makes it easier to fit into that lighter frame. Apple is working on a secret facility building MicroLED displays, with an aim to debut them in the Apple Watch Series 4 at the end of 2018. Still, it looks like the plan is to expand the tech to its eventual AR glasses, too.
And then there's the software. It's reportedly going to run a new fork of iOS called rOS, which stands for Reality Operating System. Apple is currently prototyping AR applications for this new headset, so goes the Bloomberg report. These apps are both rebuilt versions of Apple apps, like Maps and Messages, and brand-new apps like virtual meeting rooms and 360-degree video playback. We don't yet know how the interface will look or what kind of things rOS will be capable of, though.
We are told that rOS will have an App Store where you can download applications built for ARKit. And interestingly, Apple is internally using an HTC Vive and developing a Samsung Gear-like device to test AR applications for the headset.
This matches up with an earlier Bloomberg report, which says Apple had been in talks with potential suppliers for components of a glasses-like device late in 2016, and "has ordered small quantities of near-eye displays from one supplier" for testing purposes. The device would connect to the iPhone and present images over the wearer's vision. It would show images and other information in the wearer's field of vision, and may use augmented reality, the anonymous sources said.
Who's behind it?
That would be the same team behind ARKit, led by former Dolby hardware head Mike Rowell, who has assembled a team consisting of veterans from Oculus, Microsoft's HoloLens team, Amazon's Lumberyard VR platform, Google Earth, and, perhaps unexpectedly, Hollywood special effects studio Weta Digital, which was behind movies like The Lord Of The Rings trilogy and Avatar.
Apple has also moved over people from its camera team to work on the project, which makes sense because the camera is vital toward AR. As AR-focused VC Amitt Mahajan told us, AR not only needs to be good at displaying virtual objects on top of real ones, it needs to be able to see and understand the world around you to do it properly.
So, why 2020? Earlier last year, Robert Scoble did say an anonymous Carl Zeiss employee told him the company was working with Apple on a light pair of augmented reality smartglasses that may be announced this year. It turns out that miniaturising the technology to make AR smartglasses that people actually want to wear is incredibly difficult. Don't take our word for it.
CEO Tim Cook spoke toThe Independent in an interview. "There are rumours and stuff about companies working on those – we obviously don't talk about what we're working on," Cook said. "But today I can tell you the technology itself doesn't exist to do that in a quality way. The display technology required, as well as putting enough stuff around your face – there's huge challenges with that. The field of view, the quality of the display itself, it's not there yet."
Cook continued: "We don't give a rat's about being first, we want to be the best, and give people a great experience. But now anything you would see on the market any time soon would not be something any of us would be satisfied with. Nor do I think the vast majority of people would be satisfied." He did conclude, however, that "Most technology challenges can be solved" but said it's "a matter of how long."
Must-read: Meet Apple's augmented reality dream team
And it would appear that Apple is looking outwards in order to help solve any technological issues. Back in November, Apple reportedly snapped up AR startup Vrvana, who previously developed a mixed reality headset called the Totem. And while it was given the "mixed" label, it was really more of an AR device.
Though it went unreleased and was instead reserved for demos, the Totem would use a couple of pass-through cameras on the outside to view the world and display it on the OLED displays on the inside of the headset. Then it would be able to take the real world and overlay things on top of it, even seamlessly switching to virtual reality if needed.
We're not suggesting Apple is interested in the VR element of the Totem, but the headset's ability to potentially shift between use for entertainment and business would fit in with Cupertino's form. There's also Vrvana's work with infrared sensors to consider, as this is something Apple has recently explored through both the iPhone X and AirPods.
Of course, the full responsibilities of Vrvana's staff are unknown. But something we know will be happening is Apple allowing developers to use ARKit to lay the groundwork for an ecosystem of content, ready and waiting for those smartglasses. After all, we've seen some amazing stuff coming from developers recently.
Apple will also look to beef up ARKit's abilities in future iOS updates, like iOS 12, so that developers can do more things, like allow apps to remember where you placed virtual objects – so that if you close and open them everything is exactly where you left it.
In November we reported on word that Apple could launch a mixed reality TV within the next year. It would use the TrueDepth camera system on the iPhone X to be able perform Kinect-like sensing in your living room. There's also a hint at a larger AR ecosystem built for your home.
However, despite all the apparent fun and games going on, Apple is still trying to figure out a compelling use case for AR moving forward, according to Financial Times and Wired. As it does that, there are multiple prototypes floating around, one of which is as simple as Snap Spectacles, recording video and relying on the iPhone as a display. One thing is for sure: Apple's AR glasses are 100% real, according to AR-focused VC Matthew Miesnieks, who says he's spoken to people that have held them.
While Apple is still looking for a use case, a leak on Reddit from alleged Foxconn insiders reveals reveals there could actually be trouble behind the scenes. Chiefly, the employees noted that there's currently a 65% chance 'Project Mirrorshades' is completely scrapped. At the time, the leak said the smartglasses will be delayed until 2018 or potentially 2019 – if they do see the light of day. However, this timeline is even more aggressive than the reported "aggressive" timeline of revealing Apple's AR glasses in 2019 and releasing them in 2020. And even then, that seems ambitious as we actually think it could be even longer than that before the tech is ready, as per Cook's comments, delays or not.
With all that said, Bloomberg has reported that Apple was among many industry giants meeting with AR suppliers at CES 2018, which perhaps means that a 2019 announcement isn't completely out of the realms of possibility.
According to Bloomberg, Apple is also reportedly unsure about how users will actually control the device. It's looking at Siri, touch panels and head gestures to get around the device, but hasn't settled on any of them yet. The Cupertino company doesn't typically "lock" device designs in until a year before they ship, so it's still got most of 2018 to figure all of this out.
While it's clear Apple is still locking down features, it's worth noting that the Reddit leak has been partly corroborated by the Bloomberg report. Thus, it's worth taking a look at how the report said the glasses could supposedly feature a microphone, accelerometer, magnetometer and a capacitive strip on the arm for volume and call functions. Bone conduction tech would be used for audio, while integration with apps could see head movements control the action. Another interaction method could be Siri, according to some updated patents that included references to the voice assistant as a remote for smartglasses.
Resolution for the glasses is allegedly 428 x 240, with crystal (described as clear), champagne and black designs all being entertained in both men and women's sizes. Rough estimates also placed the price tag for a potential set at around $600.
- Meet Apple's AR dream teamThe engineers, execs and researchers building Apple's secret AR tech
- The best ARKit apps and games you need to tryGot iOS 11? You need to give these a try
- Apple Watch Series 3 reviewAn LTE smartwatch that doesn't phone it in
- watchOS 4 reviewApple's smartwatch gets more intuitive with the new update
Now, it's no secret that Tim C0ok is excited for AR, and the launch of ARKit proves that Apple is serious about this space. As does the fact that Apple purchased a couple of companies, like Metaio and FlyBy Media, that are well-versed in AR software. More recently, Apple bought SensoMotoric Instruments, which specialises in eye tracking. Wouldn't that be helpful for a pair of AR glasses?
That's what makes the scrapping forecast from Foxconn so interesting. This is a project that has always possessed the potential to be delayed due to the leap forward Apple seemingly wants to make in the space, but a lot is already invested for it to be dropped completely. For now, it appears we'll have to wait for the current moving parts to settle.
If we do see the glasses launched – whatever the case with Foxconn, glasses seem like an eventuality anyway – it would be the third major new hardware category launched under Tim Cook, the first being the Apple Watch, before HomePod recently dropped. The Apple supremo has also gently nudged away speculation that Apple would be going first to VR, saying that he sees more value in augmented reality. Ol' Timmy Cook: all about the wearables.
Hot VR headset deals
Wareable may get a commission