Inside-out v Outside-in: How VR tracking works, and how it's going to change

Positional tracking is one of VR's most interesting challenges
VR tracking: Inside-out v Outside-in

Mobile VR has reached the point of delivering experiences that don't make us feel nauseous or short changed. No, you're not going to get the same level of quality as the Oculus Rift of PS VR - yet - but both technology and games are rapidly improving. It also allows us to cut the pesky cords, but the boom of mobile VR has also opened up new ways of thinking about six-degree-of-freedom (6DoF) in virtual worlds.

The ability to roam free in a (limited) space is what makes the HTC Vive the most compelling system on the market right now, but the way it uses positional tracking isn't necessarily best for everyone else. Oculus has a similar approach, but it's already announced plans for an upgrade that relies on inside-out tracking, without the wires, to afford much greater mobility. In fact, we had a play with the Santa Cruz prototype late last year.

You might have heard about inside-out and outside-in tracking, and you might already know what these mean. But what are the merits of each? And which is the one we'll be using in 10 years time?

Outside-in

Inside-out v Outside-in: How VR tracking works, and how it's going to change

If you've used one of the 'big three' systems - Oculus Rift, HTC Vive, PS VR - then you'll have seen outside-in tracking in action. This is where the headset and accessories are tracked by an external device. In the case of the Vive it's the Lighthouse sensors that are placed around the room. For the Oculus Rift, it's the positional tracker that sits pointed at the headset, likely by your computer display, and it's a similar setup for the PS VR camera.

In all of these cases, the camera(s) is placed in the room and determines the position of the headset in relation to its environment. While it's something we're seeing more in higher-end system, some mobile VR is also experimenting with outside-in solutions.

Pros

  • Outside-in solutions remain fixed and are best for accuracy right now. You can also add more trackers around the room to increase accuracy.
  • Latency is better for outside-in too right now, making it less likely you'll feel nauseous. Until we can get this level of power into lower cost hardware, outside-in will continue to be the most powerful form of tracking - that's why there's still benefits of outside-in for mobile VR.

Cons

  • One of the main limitations is a problem called occlusion, which is the technical term for "stuff getting in the way". Should you suddenly vanish behind a couch or tall plant, and out of sight of the cameras, they'll have a harder time tracking you. It doesn't just have to be other objects: the sensors need to be able to track you effectively in 360 degrees, else they'll lose your placement the moment you turn around.
  • The other big problem is the limitation of the cameras. Unlike inside-out, you're only tracked as long as you're in its field of view, and as soon as you step outside of that space the immersion dies. This can be particularly problematic if you have limited space for your VR setup.

Inside-out

Inside-out v Outside-in: How VR tracking works, and how it's going to change

In this scenario, the camera is placed on the device being tracked and looks out to determine how its position is changing in relation to the external environment. As the headset moves, the sensor re-coordinates its place in the room, giving the impression that you're moving through the virtual environment in real time. Inside-out tracking can be done with or without markers around the environment. While headsets like the Pico Neo CV offer the freedom to use them where you want, you may also recall that picture of a Valve demo room, where fudicial markers covered the walls in order to aid the inside-out tracking.

Read next: The race to mixed reality is on

We're starting to see companies offer compelling inside-out solutions, including Qualcomm's 835 reference headset, designed to help other manufacturers build their own systems. Eonite also showed us its inside-out solution, a sensor attached to a HTC Vive that, while still wired, allowed us to roam freely in the virtual world without any sensors around the room. The big one though is Microsoft, which is using inside-out in HoloLens as well as its upcoming Holographic headsets, including the Acer 'mixed reality' HMD.

Pros

  • Freedom! With inside-out you're not restricted to a play space, so mobility is increased and the virtual feels a bit more… real.
  • For augmented and mixed reality, inside-out tracking will become essential as we demand more mobility from the systems.

Cons

  • Accuracy and latency. Inside-out tracking requires good computer vision, and this technology is a little further behind outside-in solutions, but it's getting there.
  • All computational work must be done by the headset itself.

So where is all this going?

Inside-out v Outside-in: How VR tracking works, and how it's going to change

Perfectly accurate inside-out tracking is the holy grail of all the "realities". After all, who wants to be leashed to a computer like a misbehaved dog when exploring far-flung universes in VR?

"I suspect we may see outside in continue to exist only in the high-end desktop PC part of the market (Vive) as the absolute best solution and will likely be the last to move towards inside out," says Anshel Sag, a VR analyst at Moore Insights & Strategy.

"I believe we may also see some new tracking solutions come out that could accelerate the move towards inside out, but those are yet to exist. Ultimately, technologies like VR and AR tend to gravitate towards integration for a multitude of reasons and there are plenty of reasons why integrated tracking is beneficial, especially in AR. Many of those reasons have to do with portability and power consumption, both of which are pain points for many VR solutions today without inside out tracking."

Perhaps one future solution won't require making a leap in technology, but exploiting a human foible. There's an interesting locomotion technique called 'redirected walking' that takes advantage of our inability to walk in a straight line. Yeah that's right - try putting a blindfold on someone and get them to walk forward, you'll see what we mean.

This means it's possible to manipulate someone's trajectory slightly in VR and have them believe they're still following a straight line. While a flaw in the real world, in the virtual one this could prove advantageous when it comes to positional tracking.

Here's an example of redirected walking in action, where a circular arc with a radius of at least 22m is used to trick people walking round and round into thinking they're going in a straight line. The problem here is that this requires a large play space to work, but the "unlimited corridor" is one other interesting technique in positional tracking being explored.

While inside-out tracking has started creeping in, we're yet to see it hit headsets like Daydream VR and Samsung Gear VR. But give it time. It seems logical that this is the way forward for spatial tracking, especially with all the promises and demands of AR. Back in 2014, John Carmack tweeted that he was working on inside-out positional tracking but that "everyone is skeptical that it will work out, so don't hold your breath". Now, it feels like more of an inevitability.


2 Comments

  • RomaJoy says:

    I think depth sensors have just one plus: no markers, but many minuses, for example, it doesn't work in big spaces, doesn't work in one-color zones, very difficult to create multiplayer experience or game controllers because large stream of date, it's very cool technology for AR but not for VR market

  • ggtftdt says:

    what

What do you think?

Connect with Facebook, Twitter, or just enter your email to sign in and comment.