The smartphone in your pocket is a box of many tricks, but there's one in particular we all take for granted. Its camera mimics the human eye by automatically focusing on objects at different distances, which is an impressive feat when you really think about it, but not one that has lent itself to virtual reality. Oculus VR's scientists may have found a solution, though, and we could see it in our VR headsets before too long.
If you've tried any VR headset, then it might be something you've noticed yourself. Just recently I gave a friend his first taste of virtual reality with the PS VR, but his immediate reaction was "this is weird and blurry". After making some adjustments I realized that there was nothing wrong with the headset. He just couldn't understand why some objects in the scene were sharp and others remained out of focus. This is because virtual reality uses fixed-focus displays, and this leads to something called a vergence-accommodation conflict. Essentially our eyes can't quite focus on things in the way they do in the real world.
The team working on Oculus's research team now think they've gone some way to cracking this problem with a technology that more accurately mimics the way our eyes work. That new technology is called a focal surface display, Oculus research scientist Douglas Lanman explains to me. Lanman was brought onto the team by industry veteran and head Oculus scientist Mike Abrash to tackle the problems of the future of VR. Chief among these is making the virtual look a bit more like reality.
It's going to take many years to make science fiction into science fact
For a quick display science 101, it's all about how light from objects hits our eyes. Normally when you look at an object, the lens of each eye bends to focus the light on your retina. As you look to different objects of different distances, your eye adjusts (or "accommodates") to ensure the light is still meeting the retina correctly, giving you a sharp look at it. At the same time, your eyeballs are moving closer to each other or further away depending on the distance of what you're looking at. It's something that comes naturally to us, so we never need to think about it, but with VR the screen position remains at a fixed distance on our faces, so while our eyes converge to focus on objects at different lengths on the screen, the light from all of these objects enter our retina at the same angle - so there's a disconnect. Things can look blurry, and sometimes you end up feeling uncomfortable.
Oculus's new idea is what it calls a focal surface display, and this essentially "bends" the light from different objects in the scene you're looking at, meaning your eye follows the contours of the scene more naturally, and object details appear more sharply. A little like this...
"That makes the focus of the headset this sort of stretchy rubber surface that lets us do really convenient things," explains Lanman. "This is something that no standard camera would do. It takes a special optical element, in this case a spatial light modulator."
A spatial light modular isn't quite as USS Enterprise as it sounds - it's just the tech that bends the light rays in different parts of the scene so that you get a clearer image beamed to your retinas. This could also mean that people who wear corrective lenses could use VR without glasses. "[The spatial light modulator] hasn't been quite applied in this way before," says Lanman. "It means you have a more powerful ability to focus, rather than a faster or more accurate way to focus."
Lanman is referring to the long list of other means of solving the focusing problem that have been explored for over a decade, and Oculus knows its new breakthrough isn't the perfect solution to the problem. "We need to figure out how to run this algorithm in real time, we need to figure out how to make the packaging of the device more compact and elegant," he says.
In July, Oculus Research will present a paper on its findings at the SIGGRAPH conference: "I'm really excited that both Oculus and other institutions are sharing this information publicly, and we're essentially having this public discourse in the industry on how are we going to solve focusing in a headset."
Oculus is going public with the details of its research because it wants the whole field of VR to work on this together and crack the problem, just like the camera industry did.
"That industry figured out how to solve it," says Lanman. "No reporter would be talking to me if I figured out how to focus a camera better, but working out how to focus a headset is a really interesting problem right now."
On the horizon
While Oculus has managed to get a rudimentary version of its focal surface display tech working on a Rift headset, there are no guarantees we'll see this cool new tech any time soon. When I asked whether it could surface in any upcoming hardware, an Oculus spokesperson told me: "We don't have anything specific to talk about how this is going to be available in today's products or if it will be integrated into these products... It's something we'll internally talk about, and look at how it's possible to leverage for a future product".
Lanman adds: "Now that VR is a vibrant industry, we're going to pursue all these things in parallel, and hopefully out of that will come the equivalent of an autofocus camera, something that solves the problem reliably and delivers that next plateau of experience."
Oculus won't reveal much else about what's going on behind closed doors in that secretive research lab, but Lanman and the rest of the team are working on a lot of moonshot technology that may not trickle down to our VR headsets for many years to come. "I'm fortunate that Mark Zuckerberg, the management of Oculus and my own manager Michael, know this is a long road, and it's going to take many years to make science fiction into science fact," says Lanman. "It's why I became a researcher - to tackle the hard problems, not the near term problems."
One other thing Lanman is intrigued by is the current quest to categorize virtual, augmented and mixed reality. "For me these terms are sort of segmenting things that's more than are necessary," he says. "They're somewhat separated by the optical systems used to create the display device, but with things like focal surface displays, you start looking down at the horizon line and you see these things are going to start blurring together. You're going to start seeing VR and AR devices being the same device.
"These terms are necessary now to help the public understand what type of eyewear it is, but I think down the road, decades ahead, it'll be just one term: wearable computing."
How we test