Humans are about 72% accurate at reading emotions from facial expressions, computer vision is already up to 82%. That's just the start. What's coming next are wearables that read exactly what is going on in your emotional health, not just physical, and align it with what's happening in your life.
We've already explored some of the biofeedback tech looking to minimise stress along with the now and next of tracking health and happiness. One new way we can think about what's in store for emotion sensing is to break it down into input (tracking), analysis and algorithms, and output in the form of apps and exercises.
How to track emotions
Sure, there are innovations like temporary tech tattoos that read your facial expressions, but for now we're interested in practical, mainstream emotion sensing devices. And it seems the demand is there.
A few months ago CCS Insight asked 2,000 people in the US and UK what they are most interested in tracking automatically throughout the day and night and stress levels came second only to 'how well I sleep' with 45% of the vote.
"It has been suggested that by using galvanic skin response (GSR) technology, a user's stress levels can be determined," says George Jijiashvili, an analyst at CCS Insight who covers wearable technology.
"Jawbone and Basis have previously used GSR technology in their wearables to determine perspiration levels and heart rate, but I believe that its potential hasn't been fully explored yet. I continue to believe that next year Fitbit and other major players in the wearables space will start expanding the capabilities of their device by adding additional sensors."
For now, we can look to the rise of the mobile app Headspace which has added four million users in the past six months alone, as well as manual mood-tracking Apple Watch apps like Moodnotes from ustwo and Thriveport.
Pebble is another example of a company which asks users to manually input their mood levels over the course of the day via its Happiness app. But the fate of the Pebble Happiness experiment is now in doubt with the Fitbit buyout. Fitbit is said to be interested in its software, in particular, though, so - you never know - we could see some version of this pop up for future Fitbit trackers and watches.
Making sense of everything
The most controversial part of emotion sensing is the secret sauce in the middle of biometric sensors and manual mood diaries, and the insights given around breathing and changing your lifestyle habits. Any company in this area will be keeping their algorithms close to their chest but a couple of startups such as Vinaya and its upcoming Zenta, as well as the makers of the Feel wristband, have shared the basics of their science.
Take Zenta. It's a biometric bracelet which measures galvanic skin response, as mentioned above, as well as heart rate and heart rate variability and combine this with your digital life - calendar, social media etc - to build a picture of your emotional life.
This isn't just a determination of whether you're stressed or not but a rich, actionable profile that grows over time and can be compared to the rest of the data Vinaya is capturing. Kate Unsworth, Vinaya's co-founder, told us she wants Zenta to take the place of an AI "superhuman therapist".
Vinaya's algorithms are essentially matching physiological signals to emotions like anger, sadness, affection etc based on an academic model. If you're angry, certain signals relating to HRV should decrease for instance and your skin conductivity should increase too - that's what galvanic skin response measures. You can read more examples here.
"What technology can enable us to do today is truly amazing. But as we let our devices and virtual realities distract us from the present and negatively impact our wellbeing, we should recognise that this is an unbalanced relationship," says Unsworth. " We've built a lab in London, where our team conducts research and experiments into things like stress, anxiety, sleep, happiness, peace and fulfilment."
Emotion sensing wearables are new territory for the science so when these devices launch next year, the proof will be in the correlation between how our devices think we feel and how we think we feel (which could, of course, be somewhat inaccurate itself). As with Moodnotes and Pebble, some element of correcting the system to help it learn over time will be useful as with health and sleep tracking.
How to deal
So we are stressed or sad or angry - what now? We can split this part up in two: visualising our emotions and offering practical actions. In the first instance, we are seeing some bog-standard graph screens but also interesting experiments in how to present the data.
Bellabeat's new stress tracking feature on its Leaf range of trackers has an in-app graphic that shows how you're doing compared to yesterday - showing any more than a day's difference would probably be overwhelming. It also changes shapes depending on whether sleep, exercise or a woman's menstrual cycle is positively or negatively affecting your readiness to deal with stress. The women's tech startup thinks this is more useful than simply telling you that you are stressed in the moment.
If you want high art as you're checking your heartbeat, turn to the collaboration between Intel and British-Cyrpiot fashion designer Hussein Chalayan. They used brainwave, heart rate and breathing tracking smartglasses to track data on emotions like nerves, stress and attraction before analysing them and turning them into videos of a series of videos. In each case, the visualisation changes as your respiration or heart rate changes in real time, as a form of biofeedback. It's not designed to be commercial, more a demo of what Intel's tech can do - if you're in London, you can see it in the Design Museum until next April.
On the second part, breathing exercises, seems to be key. It's an old tradition but Silicon Valley seems to be sticking to it with both Apple and Fitbit now offering haptic help with meditation and breathing via the Watch and fitness trackers. Fitbit personalises the sessions based on your heart rate variability too and says the feedback from Fitbit users so far has been positive.
"In today's world, we're seeing people address healthier living by not only looking at their activity, but also at their overall wellbeing and mental health," Benoit Raimbault, Fitbit's marketing director, told us. "Relax is our first feature that is designed to support people in reaching their wellness health goals… Our guided breathing sessions, based on PurePulse heart rate technology, help reduce stress by finding moments of calm each day."
All told, 2017 is set to be a big year for wearable tech that cares as much about our mental health as our physical health. People are already starting to move beyond tracking basic metrics towards more substantial health monitoring, as well as trying to prevent diseases occurring in the first place. Paying attention to our emotional wellbeing is going to be just as vital.
How we test