Magic Leap: Everything we know so far about the mysterious tech

And what we still don't know
Magic Leap: Everything we know so far

Magic Leap is the most exciting thing in AR right now and we still don't know exactly how it works. Or when it's coming. Or what it looks like.

And yes, more exciting than Microsoft HoloLens (with its small field of view), more exciting than Google Glass 2 (with its enterprise focus). Now that Oculus and Vive have arrived, and the hype around VR is turning into the more mundane fare of people actually using the headsets, the attention around Magic Leap is only set to increase throughout 2016.

Read this: Awesome upcoming HoloLens AR apps and games

Here's everything that Magic Leap has confirmed so far with a few juicy rumours thrown in since the Florida startup is still being pretty damn secretive.

What is Magic Leap?

Shot through Magic Leap technology

Magic Leap, the secretive startup, is developing what its CEO, Rony Abovitz, refers to as Mixed Reality Lightfield. Like Google Glass and HoloLens, it overlays digital 3D graphics onto your view of the real world via some kind of headset with transparent lenses or smartglasses.

To those outside Magic Leap/Microsoft, the easiest way to describe this is next gen augmented reality. The companies themselves are using the term mixed reality (MR) to signify that this tech combines AR and VR and places 3D virtual elements in real environments. We'll see which one really catches on before coming down either side.

We now know that it does this using its 'photonic lightfield chip' which Abovitz doesn't want people to refer to as a lens though it seems this is what users will see through. As far as we know so far, it projects virtual images onto the user's retinas which has been described as a "digital lightfield signal" by the company's CEO. We don't know exactly how light is beamed around the device and into the wearer's eyes and how this will trick our brains - this, for now, is Magic Leap's secret sauce.

A lightfield can capture not only every beam of light hitting our eyes but also the direction each one is travelling in and the amount of light travelling through every point in a space. This allows VR, AR and MR researchers and engineers to give virtual animations, objects and characters a sense of depth. Abovitz claims that its technology is better for our bodies - eyes, brains - than VR as it respects how we see and process the world.

As for what the device(s) will be called, Magic Leap filed for a trademark for 'Sensoryware' in March 2013, a name which definitely fits with Abovitz's language so far.

What will it look like?

Shot through Magic Leap technology

This is one thing we really don't know yet. Few journalists have tried Magic Leap and none have been allowed to take photos of them wearing prototype kit. The wearable itself has been referred to as a "headset", "spectacles" and "goggles".

We also don't know how it will be controlled yet - voice, controllers, buttons - though the videos show that gaze is used to hover over, if not select, menu items and it looks like the tech can recognise the player's hands. Will it be wired or wireless? Also too soon to tell.

Google is one of Magic Leap's investors so there's a chance that Google wants future Glass editions to use Magic Leap's lightfield chip/lens. Or we could be looking at more of a HoloLens-style helmet with, hopefully, a larger field of view for the virtual elements than Microsoft's first gen device.

What apps and games can we use with it?

With WETA Workshop effects

Thanks to Magic Leap's video teasers, we've got a good idea of the kinds of things it wants us to use its tech for in the future. Abovitz has called it "an operating system for reality". So far, the apps fall into a couple of groups - productivity/education/everyday essentials and games.

So on the one hand, the vids show users opening up emails, buying shoes and brushing up on their mountain climbing history. And on the other, there's AR games that take over the office and CG animated jellyfish floating above the wearer's head.

Magic Leap also has some high profile friends in the form of WETA Workshop and Peter Jackson. WETA is building a virtual world for Magic Leap. It's called Dr. Grordborts and features ray guns. And the Lord of the Rings director is working on something for the platform, though we don't know whether that's Grordborts or another project.

The WETA project is confirmed but one tantalising rumour is that Steven Spielberg's Ready Player One adaptation will, in some way, include something for Magic Leap.

Warner Bros is an early investor so we could look forward to something from the studio in due course. One more thing detailed in patents is Digital Live Artists and Digital Music Venue apps - beaming musicians into your living room for a private AR gig and placing you (and friends) into a concert environment.

Lucasfilm is another company that's joined the Magic Leap fold. Specifically, the ILMxLab will be partnering with the company to create Star Wars experiences. We've already played with an HTC Vive experience made by xLab and were duly impressed so we can't wait to see what Magic Leap Star Wars will be like.

Who is behind it?

You might have seen Rony Abovitz' incredible, bonkers TEDx Talk from way back in 2013. Before he founded Magic Leap, which is now based in Dania Beach, Florida, Abovitz co-founded a company called MAKO Surgical Corp which made surgical robotic arms and was sold for $1.65 billion in 2013. He is the best kind of eccentric entrepreneur, at least in terms of his public persona.

He blogs. Plays in an indie rock band. Drew quirky cartoons in his university newspaper. And describes his technology in poetic language like "a living river of light sculpture" and "the part without atoms" instead of releasing specs and features. His Twitter bio simply says: "This is me - friend of people, animals and robots."

Then there's the investors. Magic Leap raised around half a billion dollars in 2014 initially then added $794 million to that total to make $1.4 billion. Investors so far include Google, Qualcomm, Warner Bros and Alibaba.

Abovitz recently announced a partnership between Magic Leap and Twilio with plans to add its software, which lets app developers tack on phone calls or text messaging, to the Leap system.

In keeping with the usual air of mystery around Magic Leap, there's also a new program from the company letting 10 outside developers work with Magic Leap devices but the timeframe was undisclosed and details were sparse. "We'll build a 'Hello world, this is how you do stuff,'" said Abovitz, "and then you guys build something with us and be amongst the first people in the world to get your hands on what we're doing and make really cool things."

When is Magic Leap actually launching?

Your guess is as good as ours but the latest rumour comes from VRWorld which says it has 'discovered' that the first planned tech demo is set for the end of 2016 with an official unveil at CES 2017 in Las Vegas.

Now, take that info with a pinch of salt of course. But that timeline does seem realistic considering that Magic Leap let Wired around its HQ and is stepping up its demo video game.

Plus back in February 2016, Abovitz posted this update: "We are setting up supply chain operations, manufacturing - many whirligigs and test machines and gizmos abound these days. Engineers move about our spaces with a sense of urgency. Intense debates about every form of science and art you can imagine float about. Plans have been made. Program and production managers track progress. Coders are coding."

Most recently, at a Fortune conference in Aspen, Colorado, Abovitz said the team is currently debugging the production line. While it sounds promising he was still cagey about an actual release.

That still leaves CES as a potential date to try out the final tech if it's ready for January 2017.

How much will it cost?

No clue. We're crossing our fingers for a sub-HoloLens price as the dev kit of that - very impressive - AR helmet costs $3,000.

Are you excited about Magic Leap? What are you most itching to find out - price? Release date? How it works? Let us know in the comments.

8 Comments

  • neonray8 says:

    Well, the Title says, everything we know about MagicLeap.

    If u still calling it as Augmented Reality (AR) instead of Mixed Reality (MR) then u dont know anything at all!

    • nostrop45 says:

      They specifically addressed that:

      "To those outside Magic Leap/Microsoft, the easiest way to describe this is next gen augmented reality. The companies themselves are using the term mixed reality (MR) to signify that this tech combines AR and VR and places 3D virtual elements in real environments. We'll see which one really catches on before coming down either side."

  • neonray8 says:

    Magic Leap: Everything You know so far is WRONG!

    coz its not augmented reality (AR) tech but Mixed Reality (MR) tech.

    Good Luck!

    • lucky012345 says:

      Mixed Reality is an augmented reality where both real and virtual objects are in interaction. So Mixed reality is augmented reality of today (and not old augmented reality from 10 years ago) !

      An autonomous car is a car that can interact with it's environment, but it's still a car !

  • NifanNern says:

    I am actually interested in this for a more personal reason.

    I have congenital nystagmus. Giving me a sight of only 15% (my eyes move up, down, left and right uncontrollably).

    This technology might solve my problem given a little bit of time and high hopes, no?

  • davea0511a says:

    It will not hit the shelves until 2020 at the earliest and at that point cost $10k.  The clarity of the display won't be great ... it will be like wearing sun glasses. It will drop to $3k in the next couple years ... and eventually down to $500 ... which will be pretty cheap because the bottom will have fallen out of the economy and we'll all be wearing paper bags...  which cost $10 each.  Which will be all that we can afford.  Eventually they'll get the through-light transmission down to some acceptable level, at which time we'll all be living lives through VR anyway, being fed through a tube, since real life will suck so much.  That's the upbeat version.

  • davea0511a says:

    It will not hit the shelves until 2020 at the earliest and at that point cost $10k.  The clarity of the display won't be great ... it will be like wearing sun glasses. It will drop to $3k in the next couple years ... and eventually down to $500 ... which will be pretty cheap because the bottom will have fallen out of the economy and we'll all be wearing paper bags...  which cost $10 each.  Which will be all that we can afford.  Eventually they'll get the display to be water-clear at which time we'll all be living lives through VR anyway, being fed through a tube, since real life will suck so much.  Can't wait.

  • Jwieczerzak says:

    You gotta wonder if Magicleap's technology will be worth the wait. Lots of hype but no meat yet. It could be genius or it could be too little too late. Fortunately for them they are privately funded. 

    This will be an excellent study in corporate strategy - regardless of the outcome.  I am anxiously awaiting the next reveal and the long term results.

What do you think?

Connect with Facebook, Twitter, or just enter your email to sign in and comment.