Microsoft Research's latest project will come as welcome news to Oculus Rift and HTC Vive owners looking for a touch more immersion. Handpose is a futuristic, precise 3D hand-tracking technology which uses Microsoft's Kinect sensor and computer vision. It's designed to let you use your hands to naturally interact with computers and yes, VR headsets.
Accuracy, including for finger tracking, has been improved by the team with the work based on algorithms dating back to the 1940s and research carried out since then. The aim is to get to the point where an avatar hand feels like your hand.
Read this: Oculus Rift review
The virtual controls could also be so impressive that if you touch your fingers together you could get the sensation of touching something hard or else something soft and pliant. That's all thanks to haptics, which will become more important as VR gets more sophisticated.
According to Microsoft Research's blog post, Hrvoje Benko, a senior researcher in the natural interaction group at the Redmond lab, thinks it will become harder and harder to trick our body into immersing itself in a virtual experience without anything to touch.
Haptics could be combined with limited real world objects and accessories such as one building block to be used over and over in a building game or a couple of knobs and buttons for a flight sim. The project is called haptic retargeting and it could really open up VR controls.
"A single physical object can now simulate multiple instances in the virtual world," said Eyal Ofek, another senior researcher.
The system, presented at SIGGRAPH 2016, is still a research project within Microsoft Research for now. But it's easy to see how this could be applied to its augmented reality device, the HoloLens, as well as VR headsets. Watch the video above for more details.
How we test