Knowing the deepest workings of your emotions has been the holy grail of neuroscience for decades. A future in which machines can understand and adapt to the mental state of their user, like Spike Jonze's masterpiece Her, could transform the world from robotics and entertainment, to healthcare and law enforcement.
Ever since research into 'Affective Computing' was started at MIT by Rosalind Picard in 1995, with the aim of simulating empathy in machines, the greatest tech minds have plotted to get hardware and software to understand our basic emotions.
But now a handful of wearable concepts are emerging that claim to be able to read our minds. The aim is to sense our thoughts and turn our emotions into everything from futuristic fashion pieces to crowd-sourced experiences at gigs – perfectly pushing your buttons for the ultimate live experience.
But some of the technology may not be up to the task...
Facebook hacked my emotions
Allowing machines to read your innermost thoughts is an ethical and practical minefield. Facebook recently took heat over allowing researchers to detect and manipulate the emotions of almost 700,000 users by varying what they saw in their news feeds. And Tesco has been criticised over plans to install facial recognition devices that serve up more relevant advertising to customers while they stand at the tills, despite promises that personal information is not collected.
Meanwhile, the uncanny valley looms large - an inaccurate device could cause far more problems than it solves. Today's emotion sensors are limited in their ability to differentiate nuanced expression. An angry person gives off very similar signals to someone who's turned on – and you wouldn't want a robot to mistake the two.
Lady Gaga’s arousal band
Enter Studio XO, a wearable tech company that's built an 'emotional technology platform' called XOX that purports to allow its customers to track the emotional states of large groups of people during an event. It’s created a wristband that collects data that can then be gathered en-masse into a central server and used in different ways to tailor the proceedings to the audience.
Studio XO was set up in 2011 by mechanical engineer Benjamin Males and fashion designer Nancy Tilbury, and the pair have since worked with Lady Gaga and The Black Eyed Peas. Tilbury, in particular, has a deep background in wearable technology - working with Picard's group at MIT on the Galvactivator - a glove that uses skin conductance response to measure arousal levels. "I'm a bit of a veteran, if you like," she says.
Bookmark Wareable: Latest wearable technology news and reviews
Skin conductance response is a phenomenon where the skin sweats a little when the body is psychologically or physiologically aroused – which in this case means excited, or happy. This moisture conducts electricity better, which can be detected by a weak current passing over the skin. It's the same technology used by the Church of Scientology's E-meter, and is also one component (alongside blood pressure, pulse and respiration) of the supposedly lie-detecting polygraph test.
Studio XO's wristband uses this same technology, but Tilbury refused point-blank to discuss how her system works.
It's reading skin resistance...we don't want to get into the finer details
"It's reading skin resistance," she said. "We don't want to get into the finer details. It's authentically reading your body's signals and translating those emotional signals into a data algorithm."
That data is displayed on the wristband as a series of colours, depending on how strong the signal is, then sent to a server, which combines it with readings from the other wristbands in the crowd and spews it out as a stream of numbers that can be interpreted in many ways.
Organisers at a gig could observe how arousal levels change when different things happen to make curation decisions, or artists could visualise the data in real-time.
XO near yet XO far
But there are significant questions over whether the data being collected by the wristband has anything to do with the emotions of the person wearing it. "Based on the environment you're in - particularly when you geolocate it and timestamp it, you can tell the difference between happiness and sadness," claims Tilbury. But Jim Kalat, a psychologist from South Carolina State University and author of several textbooks on psychology and emotion, disagrees.
"You can detect differences in arousal, but that doesn't distinguish one emotion from another," he says. "It doesn't tell you whether you're happy or angry or sad or frightened." The only circumstance you can do that, he adds, is if you already know what emotion is likely at a given moment. If you're expecting happiness and see a response, you can only assume it signifies happiness.
And that's under perfect conditions. In the middle of a concert or performance, it may have trouble even detecting arousal accurately. "If you're starting to [exert yourself], that's going to increase all those measurements quite independently of emotion," says Kalat. "Or if it's getting hot in the room, you'll breathe differently and sweat differently." Averaged over a crowd at a real-world situation like a gig, the wristband's data becomes more of a thermometer than emotion-meter.
See also: LG G Watch v Samsung Gear Live: what are the differences?
MIT freely admits the shortcomings of its Galvactivator. "When the body is significantly overheated and there is a lot of perspiration, the overall level of skin conductance will indeed climb," reads the official FAQ for the device. "Everyone's electrodermal response is different. Some may be highly variable in response, while someone else may generate such a slow response that it is hard to visualise. These individual differences simply mean that you can't really compare yourself to your neighbour."
Mind readers
But that doesn't mean that neuroscientists aren't getting closer to genuinely reading our emotions with computers.
A host of fashion designers have built smart headwear that use the same technology to change the colours of clothing depending on your mood. The Neurotiq headdress (pictured above) is the brain child of Kristin Neidlinger, a San Francisco fashion designer who uses EEGs, heart rate monitors and skin detecting sensors to blend technology and fashion in ways that seem lifted from the pages of The Hunger Games.
It's not just Stateside that fashion innovation is happening, with London based design studio The Unseen using lab grown Swarovski crystals, which are dipped in a special ink so that they light up with your brain patterns.
The Unseen are using lab grown Swarovski crystals dipped in a special ink to light up with your brain patterns
Carl Heath, a researcher at the Interactive Institute of Swedish ICT, and a team at the Collaboratory in Gothenburg have also found applications for brain data harvested from the scalp. They're working on a robot named OBO, designed for use in theatre that collects electrical activity data from the scalp of an actor on stage nearby to read the emotions that he's projecting. "We let the narrative of the play shift and twist depending on the biometric data from the actor," says Heath, "Which gives new cues and response patterns to OBO."
Meanwhile, in a research setting, Kalat says that the most valuable tool for emotion sensing today is functional magnetic resonance imaging (fMRI). "There's a procedure called optogenetics developed in the last few years which enables us to trace activity from one neuron to another," he says, "You can tell that if you have a lot of activity in the amygdala, for example, that's an important area for processing emotional information - that would indicate that someone is either having an emotional experience or interpreting someone else's emotional experience."
What is certain is that future wearable technology will be able to read the body's signals and use that to improve our interactions with computers. "This type of technology" Tilbury says, "will be a really, really important factor in the next five to ten years."
How we test