Face tracking is coming to mobile VR

Let's take eyebrow waggling virtual
Face tracking is coming to mobile VR

We could conduct a whole conversation in virtual reality and I could shake my avatar head or turn it towards you to show I'm listening. By the end of the year, though, I'll be winking, smiling and frowning, with the whole gamut of emoji faces at my disposal.

Specifically, if I'm wearing Veeso, a new mobile VR headset for Android and iOS that's raising funds on Kickstarter now. Starting at $70 for early bird units, and $80 for dev kits, Veeso adds two infrared cameras to a fairly standard device.

Read this: Where VR tourism is heading next

One camera tracks your mouth, lips and jaw and the other tracks your eyelids, eyebrows and pupils, more like the experiments in eye tracking that we've seen before. These are then recreated in real time by your 3D digital avatar, in multiplayer VR apps, games and social networks.

It is designed to work with Cardboard apps - not sadly (generally better) Gear VR ones - for now, though we presume Veeso is keeping one eye on Daydream. Plus, Veeso plans to release 10 apps and games of its own, over six months.

The headset is currently a pretty ridiculous-looking prototype in the demo videos but from the artwork, the New York based startup reckons it can ship a polished product by the end of the year. If the campaign doesn't reach its $80,000 target - it's still sub $1,000 - the tech itself might get snapped up.

Feel my face

Veeso isn't the only company interested in trying to make face tracking a reality, of course. FOVE has been working on eye tracking in mobile VR - both for controls and the ability to 'look' at other avatars in VR.

In PC-based VR, last year Oculus partnered with researchers at the University of Southern California to track facial expressions. They did this using a camera pointed down at the mouth and strain gauges for the upper part of the face which are added to the headset's foam padding. Hao Lin, who led the project, said the experiment could be put into production quickly.

Social VR apps like Altspace VR and Oculus Social are the obvious case for this type of tracking. Ebbe Altberg, CEO of Linden Labs, which is building a virtual world called Project Sansar, has also alluded to the potential for realising facial expressions in multi-player VR.

"We are working a lot on facial tracking to make it very natural; mouth and lip syncing so that it's a very natural feeling when you interact with other people," he said at Web Summit last year.

If you can't wait and fancy pledging for a Veeso headset on Kickstarter, the team aims to ship in December 2016.


Shop for VR headsets on Amazon

Oculus Rift
Oculus Rift
$599.99
PlayStation VR
PlayStation VR
$399
HTC Vive
HTC Vive
$799
Samsung Gear VR
Samsung Gear VR
$97.98

Wareable may get a commission


What do you think?

Connect with Facebook, Twitter, or just enter your email to sign in and comment.


 Most Recent News