Apple's AR glasses, in some form, are now expected to launch in 2018. AR features for the iPhone should appear either alongside a wearable or perhaps later this year. And over the past few months we've seen more and more leaks around Apple's secret Augmented Reality Group, as it's referred to, the latest being the acquisition of eye-tracking startup Sensomotoric.
We don't know exactly how big the AR team is but it's thought to be made up of hundreds of people, from visual effects experts hired from Weta Digital to talent from Imagination, THX and Amazon. That's alongside in-house movers like Fletcher Rothkopf, who's shifted over from the Apple Watch team to work alongside engineers who've been taken off the iPhone camera.
We thought we'd take a look at some of the most high profile hires in the group, checking out their CVs and seeing just what they have to say about AR and VR. Bear in mind that none of the below people are officially confirmed by Apple to be working on AR – some mentioned aren't even 100% definitely working for Apple.
One thing's for sure, though; if a team like this has Cupertino's resources at its disposal, we can't wait to see what Apple has in store next year.
NASA's AR guy: Jeff Norris
Hired: Early 2017
Previously: NASA
CV: Apple's latest AR hire – according to sources – Dr Jeff Norris joins the company from NASA's Jet Propulsion Lab. In 2015 he led the project Sidekick, which collaborated with SpaceX to send two Microsoft HoloLens helmets to the ISS. In February 2016, astronaut Scott Kelly was able to use the AR software to complete his tasks as well as making a Skype call from the space station to NASA mission control. So Norris more than knows his way around an augmented reality visor.
Quoted: On the HoloLens and SpaceX project: "Our team is excited to be building virtual and mixed reality tools that will make our explorers more efficient and effective. Project Sidekick's goal is to empower astronauts on the ISS with assistance when and where they need it. We want to do that by providing remote expertise and a holographic instruction manual."
The VR veteran: Avi Bar-Zeev
Hired: June 2016
Previously: Amazon
CV: What hasn't Avi Bar-Zeev been involved with? His credits include Microsoft HoloLens – he was in charge of the original concept back in 2010 – as well as co-founding a startup called Keyhole which you might now know as Google Earth. Going even further back, he was involved in pioneering research into VR for Disney in the '90s, way ahead of his time, and his consulting firm RealityPrime has also worked with Linden Lab, which is now making a VR social world.
Just before Apple stole him away from Amazon, he was in charge of UX for a stealth project there which used "novel natural user experiences". Curious. His LinkedIn profile says he works at Apple but doesn't specify the job title. What a tease.
Quoted: "AR, at its best, can help us understand reality. It might help us deconstruct reality better… As people who are designing AR and VR, my suggestion is design your AR and VR experiences in a way that are the most genuine, that tap into the true human experience, that true to root reality by getting at the core of what existence might be."
The forward-thinking exec: Mike Rockwell
Hired: February 2015
Previously: Dolby Labs
CV: Before joining Apple, Rockwell was an executive VP in Dolby Laboratories' Advanced Technology Group. According to some sources, Rockwell was initially brought in to improve the audio and display performance of a whole range of Apple products, but he is now said to be leading Apple's AR efforts. (Now, that's not been confirmed by Tim Cook, for obvious reasons, but multiple Apple sources have told the likes of Bloomberg that he is running the AR group.)
Rockwell has also advised AR glasses maker Meta, so he has experience in dealing with actual augmented reality products and strategy. Plus it looks like that relationship may have lead to another hire: Duncan McRoberts, former director for software development for Meta (who was in charge of its SDK), left in February 2017, hasn't updated his LinkedIn since and is rumoured to be on the team.
The interaction expert: Doug Bowman
Hired: January 2016
Previously: Virginia Tech, where Bowman was director of the Centre for Human-Computer Interaction
CV: Bowman is a computer science professor known as one of the world's top researchers in virtual reality, so everyone got a little bit excited when news of the hire broke in early 2016. He co-wrote a book called 3D User Interfaces: Theory and Practice in 2004 which is still referenced today, and one of his main interests in his research over the past two decades has been 3D interfaces and immersion in VR.
All the signs point to an AR/mixed reality push from Apple, so Bowman is no doubt using his expertise to advise on making AR as convincing and immersive as VR, as users interact with some form of Apple glasses.
Quoted: "I'm interested in computer graphics and the combination of HCI (human-computer interaction) and graphics which is something I do extensively, which is virtual reality. Systems that provide an illusion of being in a place where one is really not and the interfaces that allow people to navigate and interact in those virtual spaces."
Apple's audio chief: Tomlinson Holman
Hired: 2011
Previously: Apple, Lucasfilm
CV: Holman is an audio pioneer who invented the THX sound system during 15 years at Lucasfilm and has been working as audio director at Cupertino since early 2011. His CV also includes working in the sound department on Star Wars and Indiana Jones in the '80s, winning an Academy Award for Technical Achievement and developing the "10:2" surround sound system for increased spatial sound.
As with many others he has reportedly been brought into the AR team, no doubt to lead the development of spatial audio for AR apps for both iPhone and future smartglass AR apps and experiences. Apple recently filed a patent for a next-gen sound system invented by Holman – it controls dialogue separately to music and sound effects and could be relevant to the HomePod smart speaker, any Apple TV plans and the augmented reality mission.
The computer vision engineer: Zeyu Li
Previously: Magic Leap
CV: Zeyu Li's current job title at Apple is senior computer vision algorithm engineer, and it's been reported that he has joined the secret AR team. Before Apple, he worked at the also-secretive Florida startup Magic Leap for just over a year as a principal engineer in computer vision, and before that, a seven month stint at VR studio Jaunt in 2015. Li also holds a couple of patents relating to image capturing devices and multi-plane images.
Another interesting tidbit: Tim Cook reportedly visited Magic Leap's HQ last summer, showing an interest in the tech, just before Li made his own leap to Cupertino in September.
The research scientist: Yury Petrov
Previously: Oculus
CV: Petrov spends his days as a research scientist at Apple, according to his LinkedIn. Now there's a short job title if ever we heard one. He has also been linked to Rockwell's AR group and it's no surprise: Petrov spent two and a half years in the same role at Oculus VR.
A former assistant professor at Northeastern University, at Oculus Petrov focused on the physiological effects of virtual reality and R&D of new optical displays, controllers and 3D audio systems for VR – all ideas and areas Apple will have been very interested in. His past research also includes eye-tracking and EEG brain imaging.
How we test