And what we still - STILL - don't know
The wait for Magic Leap has been a long roller coaster of emotions, but the company has finally shown us what its mixed reality glasses look like and some of the things they’ll be able to do. The glasses are called Magic Leap One, and they’re due to start shipping at some point this year.
Apple’s ARKit and Google’s ARCore have helped push augmented and mixed reality into the mainstream in 2017, making the mystery surrounding Magic Leap all the more intriguing.
We’ve tried a few AR headsets before, but to date nothing has truly blown us away. With Leap, we hope that changes. But stories of chaos in the R&D department and delay after delay have stopped us getting too swept up in the hype. Now that Magic Leap has revealed what it’s been working on, is it time to get excited? Here’s what you need to know.
What is Magic Leap?
Shot through Magic Leap technology
Magic Leap has developed what its CEO, Rony Abovitz, refers to as lightfield technology. Similar to Google Glass and HoloLens, the glasses overlay digital 3D graphics onto your view of the real world via a headset with transparent lenses.
The lightfield part refers to the way Leap accommodates multiple focal points, so you’ll be able to clearly see holograms at various distances. We saw a similar thing in the Avegant headset, and were suitably impressed.
Lightfield can capture not only every beam of light hitting our eyes but also the direction each one is travelling in and the amount of light travelling through every point in a space. This allows VR, AR and MR researchers and engineers to give virtual animations, objects and characters a sense of depth. Abovitz claims that this technology is better for our bodies – eyes, brains etc – than VR as it respects how we see and process the world.
To those outside Magic Leap/Microsoft, the easiest way to describe this is next-gen augmented reality. The companies themselves have been using the term mixed reality (MR) for a while now, to signify that this is AR that interacts with the physical world around it.
In fact, Abovitz leans on its ability to not block out the world as a social feature. “You won’t lose the social aspect of being together,” he said at the February 2018 Recode Code Media conference.
The idea is that the digital world won’t be superimposed onto the real world, but integrated into it and fully interactive. Abovitz used the example of being able to watch things with the family, or going to the kitchen to grab a beer while watching a game and still being able to interact with friends.
Similarly, the device will also be aware of what you’re doing. So if you suddenly get up to go do something else, it’ll pause automatically until you’re done. Once you’re done it’ll resume what it was doing previously. In this way, Abovitz explains, it’s like an amplified version of normal television. But that’s not all it is. Eventually, the goal is to replace the smartphone.
“The number of devices it’s potentially replacing, if you actually add all that up, at some point – we’re not saying for [Magic Leap One] everyone will go down this path – but your phones, your televisions, your laptops, your tablets, that add up to thousands or tens of thousands of dollars, all get virtualized,” he said.
What will it look like?
Magic Leap One looks like a pair of futuristic swimming goggles, attached by a wire to a computer attachment called the Lightpack that can be clipped to the back of your trousers or slipped into a pocket. Its full name is Magic Leap One: Creator Edition, as the company predicts initial interest will come from early adopters and creative types.
The glasses come in two sizes, according to an interview with Rolling Stone, and the nose pieces, temple pads and forehead pad will all be customizable to ensure comfort and a good fit. To adjust the strap, you just pull it and it looks like the back strap is designed to be worn near the crown of your head. You’ll also be able to order prescription lenses.
There are four built-in microphones for spatial sound and six external cameras to track your position and movement. We don’t know the exact field of view, but according to the Rolling Stone piece, “The viewing space is about the size of a VHS tape held in front of you with your arms half extended. It’s much larger than the HoloLens, but it’s still there.”
The glasses didn’t always look like that, as Abovitz says they’ve shrunk a lot since 2014. “If you go to Magic Leap from fall 2014, the light field signal generator is like half of a room, it’s this multi-hundred pound gigantic beast,” he said during his Recode talk. Magic Leap had to shrink that tech down to a nanostructure wafer, and then had to build factories to make those wafers. “It was a bunch of nearly impossible problems.”
We also know that – eventually – Magic Leap will have multiple headsets. The first system is called the Creator Edition and is intended for partners and enthusiasts rather than the mass market. Those headsets will come, and Abovitz imagines multiple tiers with Magic Leap as a multi-product company.
We now know how it will be controlled. Magic Leap One will come with a 6DoF (degrees of freedom) controller that also has haptic feedback. To date mobile VR headsets have typically only offered 3DoF controls, so Magic Leap is stepping things up here.
Golden State Warriors small forward Andre Iguodala told CNET earlier this year that, in a demo of Leap, he was able to control smart home equipment using just his eyes, so gaze tracking will be another way you can control things.
That’s now been confirmed by the SDK which says includes references to eye tracking, head tracking, hand and gesture tracking, spatialised audio, microphone input as well as room scanning and meshing. Magic Leap has also now confirmed (via its monthly web show) that it’s only intended to be used at home on your Wi-Fi network i.e. not out on the go tethered to your phone’s connection.
2018 is the year
While Magic Leap hasn’t given a firm release date for the Creator Edition, it says it will ship in 2018. According to sources and a job advert for a ‘director of store design’, from February 2018, it looks like the company will focus on pop-up shops and kiosks in larger stores, in other words the Snap Specs retail route.
At GDC 2018 in March, Magic Leap launched a technical preview of its LuminSDK and developer portal so that it can give devs all of the tools they need to start building for Magic Leap and grow its ecosystem in time. Its Early Access Program already includes Weta Workshop, ILMxLAB, The Mill and the NBA but now these tools are open to everyone – more on this in the next section.
It’s taken its sweet time to get to this stage. We saw the first concept video for Magic Leap back in 2015, the same year that the software development kit was set to ship. In 2016 rumours from VRWorld suggested that the first planned tech demo was set for the end of 2016 with an official unveil at CES 2017 in Las Vegas. Again, it didn’t happen. So yeah, we’ve been waiting a long time.
On top of that there have been all sorts of stories of troubles in the company, surrounding struggles with the technology and the company culture itself. Magic Leap settled a gender discrimination lawsuit from an ex-employee who said the company was dismissive of ideas put forward by women. And well, Beyonce supposedly tried a demo of a mermaid and wasn’t impressed, so…
All in all, we’re tempering expectations. Even after the big reveal, all we have to go on are mocked-up impressions and videos and it’s hard to get a sense of how good the quality of Magic Leap will be compared to other augmented reality and mixed reality systems we’ve tried on. Just to be clear – we still haven’t had a demo of Magic Leap ourselves.
What apps and games will it run?
Thanks to Magic Leap’s video teasers, we’ve got a good idea of the kinds of things it will do, but we’re still not sure how much better this will be than HoloLens or other AR headsets we’ve tried.
To its credit, Magic Leap realizes that the tech it has is incredibly difficult to explain – and market; the best way to understand Magic Leap is to experience it. Abovitz likens this dilemma to “building a computer where your brain is the monitor and you’re not looking at a monitor.”
So what can you actually do with the thing? We know, for sure, that you’ll be able to watch live NBA games with the goggles. As you’re watching, you’ll have instant access to statistics and multiple angles of the game.
“You can have five or six or seven or eight televisions running different camera angles. So suddenly you could be anywhere, we could be sitting right here and throw up a bunch of TV screens and now you’re watching one game from different perspectives or all your favourite games at the same time,” Abovitz explained.
Elsewhere, the apps we’ve seen details of so far fall into a few different groups – productivity, education, everyday essentials and games – which are highlighted on Magic Leap’s website.
So on the one hand, you’ll be able to open up emails, watch TV and sports, buy shoes and brushing up on your mountain climbing history. Abovitz has also announced a partnership between Magic Leap and Twilio with plans to add its software, which lets app developers tack on phone calls or text messaging, to the Leap system.
And on the other, there are AR games that take over the office and CG animated jellyfish floating around your head. Plus, thanks to Iguodala we know that there’s some kind of digital character you can hold in your hand that acts as an AR personal assistant.
Magic Leap also has some high profile friends in the form of WETA Workshop and Peter Jackson. WETA is building a virtual world for Magic Leap called Dr. Grordborts that features ray guns. And the Lord of the Rings director is working on something for the platform, though we don’t know whether that’s Grordborts or another project.
The WETA project is confirmed but one rumour is that Steven Spielberg’s Ready Player One adaptation could make its way to Magic Leap – that’s seeming less and less likely now the movie is out, complete with HTC Vive tie-ins.
Warner Bros is an early investor so we could look forward to something else from the studio in due course. One more thing detailed in patents is Digital Live Artists and Digital Music Venue apps – beaming musicians into your living room for a private AR gig and placing you (and friends) into a concert environment.
Lucasfilm is another company that’s joined the Magic Leap fold. Specifically, the ILMxLab will be partnering with the company to create Star Wars experiences. We’ve already played with an HTC Vive experience made by xLab and were duly impressed so we can’t wait to see what Magic Leap Star Wars will be like.
Magic Leap has also gone on a bit of a shopping spree for content, purchasing FuzzyCube Software, a game developer founded by former Apple employees and game developers that have worked on games like Halo Wars and Age of Empires. The company also looked into purchasing Moonbot Studios, an Academy Award-winning animation house. While IP issues kept Magic Leap from purchasing the company outright, it did end up hiring about a dozen animators and artists from Moonbot.
From the sound of things, it seems like Magic Leap is working towards creating a healthy content ecosystem with both first-party and third-party entertainment support alongside a more standard AR operating system. We expect that later in 2018 we’ll start hearing more about the specifics of what we can expect to actually do with Magic Leap.
How much will it cost?
We have a much better idea of the answer to this question now, and it’s basically a high-end smartphone or tablet. Abovitz uses “pro-sumerish” to describe it, and points to a high-end smartphone (like the iPhone X) as a floor.
But the plan is to have multiple headsets living alongside each other, with the more premium end priced similarly to a high-end PC. “I think the Magic Leap Creator Edition price point – we will have a product in that price point probably for the company’s history,” he said.
Speaking to Rolling Stone previously, Abovitz said, “Pre-order and pricing will come together. I would say we are more of a premium computing system. We are more of a premium artisanal computer.”
Of course, as components become cheaper and the rest of the industry continues to experiment, release new products and competition thrives, prices will (hopefully) eventually climb down. Don’t forget, the heavily rumoured Apple VR/AR headset, said to be wireless and powered by ARKit, is expected to launch in 2020.
Who is behind it?
You might have seen Rony Abovitz’ incredible, bonkers TEDx Talk from way back in 2013. Before he founded Magic Leap, which is now based in Dania Beach, Florida, Abovitz co-founded a company called MAKO Surgical Corp which made surgical robotic arms and was sold for $1.65 billion in 2013. He is the best kind of eccentric entrepreneur, at least in terms of his public persona.
He blogs. Plays in an indie rock band. Drew quirky cartoons in his university newspaper. And describes his technology in poetic language like “a living river of light sculpture” and “the part without atoms” instead of releasing specs and features.
Then there are the investors. Magic Leap raised around half a billion dollars in 2014 initially then added a huge $794 million round to that total to make $1.4 billion. The latest round, of $461 million from Saudi Arabia’s Public Investment Fund, was in March 2018 and brings the total so far to a humongous $2.3 billion. That means it has a valuation of over $6 billion. Hence all the interest. It’s not just the amounts either – investors so far include big names like Google, Qualcomm, Warner Bros, Alibaba and JP Morgan.
Are you excited about Magic Leap? Or are you just jaded now? Let us know in the comments.