London-based journalist Alex Lee has a rare genetic disease that robbed him of his vision; whenever he plays video games, it's mostly a blur. But when he recently put on a VR headset to try out Rockstar's detective game L.A. Noire, he saw more clearly than he had in five years. "I was met with a surprise," he said, "I could see more in VR than I could in real life."
According to University of Sydney ophthalmology professor Dr Michael Crossland, VR can help the visually-impaired because the screens are very close to the eyes, incorporating big images, bright lights and bold text, which helps people with low vision see better and up close.
Read this: Building the bionic body
But beyond video games, a handful of companies are currently working on specific ways to use VR and AR to help the visually impaired and partially sighted to see. In fact, New York-based RaayonNova is already building what many considered the "end game" to all of this: smart contact lenses. "There are a number of issues with the existing technologies; they are not discreet," says Aleksandr Shtukater, the founder of RaayonNova. "They're visible to others and the system's control is driven by gestures."
Each lens has an embedded display directly over the eye's cornea, and would include features to help the visually impaired, such an embedded display that uses colour to direct the wearer, or to help magnify street signs or warn the person when they're near danger.
"We believe that the final frontier [is] AR and VR applications in the long run will replace smartphones," says Shtukater. "Our solution is to plug into the sense of sight in an intimate fashion and project visual information directly into the retina of the eye in an enriching and enabling fashion."
RaayonNova is working towards launching in 2019, which would make it a "first to market" if it can pull it off. "A mom working from an office can monitor her infant in a crib back home and can make the camera, at home, change direction or perhaps even initiate a conversation with the child," says Shtukater. "Even a lazy or tired individual working late and planning to go home can order from a local Chinese restaurant online through an AR smart contact lens."
Size matters
But even the sighted need help. According to Hans Streng, the CEO of Luxexcel, a Dutch company which makes 3D-printed contact lenses, 60% of Americans need eye correction. Luxexcel recently partnered with Vuzix to make 3D-printed prescription lenses for the Vuzix Blade AR headset, which overlays information like directions, maps and weather forecasts onto the world.
Another company working to help the blind is the Fraunhofer Institute in Germany, which too hopes to reduce the size of clunky VR goggles. For the past decade, Uwe Vogel, the head of microdisplays and sensors, has been advancing OLED microdisplays for VR and AR. "VR glasses are increasingly popular, but they have usually been heavy and oversized, until now," says Vogel. "Large-area microdisplays are expected to change that, because they make it possible to produce ergonomic and lightweight VR glasses; the new OLED displays now reach very high frame rates and achieve extremely high resolutions with extended full HD."
What will digital screens look like in the future? "The trick isn't just to raise the resolution and frame rate, but also to keep power consumption to a minimum at the same time," he adds. "It has worked out well, thanks to a cleverly designed system concept and modern design methodology, not to mention the more than 10 years of experience here designing the microdisplays."
Fraunhofer has been working on this decade-long project with LOMID 2020, a EU-funded project aiming to make microdisplays compact, affordable and outperforming others on the market (they also claim to have a better resolution than smartphone displays). "One application we are testing is in prosthetics for people with impaired vision, often legally classed as blind," says Vogel. "The new displays use OLED-on-CMOS technology for bright, high contrast images, which are larger, clearer and have more pixels than existing microdisplays."
While these new microdisplays are well suited to help the visually impaired, there is still a burgeoning market for them in smartglasses, AR and VR headsets, says Vogel, "especially when used with compact optics and integrated with a camera in smartglasses. In time, with training, blind people will be able to safely navigate and interact with their surroundings."
How we test