O Caption, My Caption: The National Theatre rolls out accessible smartglasses

As I prepare to take my seat in London's Olivier Theatre, I have a pair of Epson’s Moverio BT-350 smartglasses in one hand and a small controller in the other. That controller is my means of navigating items that are displayed on the smart eyewear I’m about to put on my face.

As I don those techy AR specs and look around me, it feels like I could be at a Glassholes convention in 2012. But I’m not. I’m here to test out new smart caption glasses that the National Theatre is launching as a service for those who suffer from hearing loss.

Epson is providing the hardware, while the National Theatre and Accenture worked on the tech which should make the experience of going to the theatre a significantly better and, crucially, more enjoyable one for people with hearing loss.

Seeing it all play out

So here’s how it works. The glasses form part of a service that uses something called speech following technology, which can follow the phonetics of a performance and also look at cues like lighting, sound or video effects. It combines that information to provide an automated service to help those with hearing loss understand what’s happening on stage.

That information is transmitted over Wi-Fi to the glasses, letting you follow the action on stage as it happens. The text you see displayed on the glasses has been prepared by captioners, so the tech essentially listens to dialogue that’s been spoken on stage or sound effects and then broadcasts it to the glasses, giving you a synchronised transcript of dialogue and sound from the production. Unlike services currently on offer, these glasses allow you to sit anywhere in the theatre and still enjoy the show.

Essential reading: Hearables are breathing new life into hearing aids

We wanted people with hearing loss to come with their families and sit in any seat just like everyone else

The smartglass-centric service has been in the works since 2014 when the National Theatre’s technical team started to evaluate whether it was feasible to use speech following technology. After a year of testing the glasses and getting feedback from users (including adding glasses-style nose pads), the smart caption glasses are finally ready for audiences to try.

They’re free to use and 90 pairs are set to be available across the three auditoria at the NT for use at performances of War Horse and Hadestown. You’ll need to pre-book to be able to try them initially, but the hope is that in the future that booking part of the process will no longer be required. That's according to Jonathan Suffolk, technical director at the National Theatre. These glasses will not replace existing services already on offer either, but will simply be an additional option for people to try.

So why did the National Theatre feel like the time was now right to offer another solution for theatre-lovers who suffer from hearing loss? Lisa Burger, executive director at the National Theatre explains.

“A huge priority is opening the National to more people,” says Burger. “One of the areas we were very much aware of that we hadn’t made progress in was for people with hearing loss. We are proud that we provide live captioning, but it’s only available for two or three performances. We wanted people with hearing loss to come with their families and sit in any seat just like everyone else.

“It was Accenture who came on board who gave us the impetus to really got the ball rolling on this. They introduced us to Epson who provide the glasses and we finally felt we were in a place to offer something. 12 months ago we said we were really going to do this and now we have.”

Trying out smart caption glasses

I don’t suffer from hearing loss (well, I don’t think I do), but I had the opportunity to see these smart caption glasses in action. I’m treated to a short excerpt from Eugène Ionesco’s tragic-comedy Exit The King to get a feel of exactly what future audience members who book out the specs can expect. And crucially, to find out whether they actually work.

With the glasses on, a short tutorial displayed explains that I don’t need to worry about the top half of the controller that I’m holding in my hand. The lower half is where I can slide my fingers across to scroll through the options on the display. After displaying the name of the play and the cast, it’s time to see the tech in action.

As there is movement on the stage in front of me the first caption appears to indicate an audio effect, in this case the sound of crashing thunder. But then that caption remains on screen for about 30 seconds as dialogue begins. It then kicks back into gear and what is being said on stage is being replicated in caption form on the glasses.

It’s clear, bright and easy to read. Crucially, there’s no lag for the rest of the short performance. I can play around with the settings via the controller, making the caption text bigger and smaller, moving the text to the top or the bottom of the display, switching from scrolling captions or having dialogue delivered line by line. While the glasses might feel a little on the clunky side, the delivery of the dialogue, sound and lighting effects isn’t – aside from those initial 30 seconds. You’re firmly fixed on what’s happening on stage and wherever you look, the caption follows.

Testing the specs

David Finch, a member of the testing group for the caption glasses, lost his hearing 7-8 years ago. He stopped going to theatre as a result. But then he started to look at the options that were available that might allow him to enjoy the theatre again. “I looked at the infrared induction loops that are in most theatres,” Finch explains. “That was a bit hit and miss. You have to sit in specific seats, you had to point the receiver directly at the transmitter. Quite often it didn’t work and when it did work, it was essentially an amplified version of the mish-mash of noise you were trying to get away from.

“Then I progressed to caption performances. They are very good but you are looking at screens. You’re adjusting your focal length and in the process missing the movement on the stage, the facial expression and gestures of the performers. As soon as you put those smartglasses on, they are major advancements on what I’ve been used to in the past. You can focus forward. What that means is you can sit there in the audience and align the text below or above the performers. That means I can still lip read, I can still see the expressions and the gestures.

Read this: Fitbit's next chapter – Fashion, data and taking on Apple

"I will never get my hearing back, but glasses are the next best thing to giving you that sense of involvement. The biggest advantage of all of this is the fact that without these glasses I wouldn’t be in the theatre. I’d be sitting at home not at the theatre.”

Eventually what people would like to see is something comparable to 3D glasses

Jesal Vishnuram is technology manager of the UK charity Action on Hearing Loss and was also present at the launch of the National Theatre's new smart caption glasses. The charity has had a lot of people try the service out during the year it’s been trialled, as well as few of its staff members. She believes this is an exciting development for the space, but also feels there’s one aspect that clearly needs to be addressed.

“A glasses solution is a preferred solution because it’s overlapping what you’re looking at already,” she told us. “But it’s not the choice for everyone. Some people still want the StageText for instance. But I think it’s a good choice in addition to what is already on offer.

“I think it has been fed back that the current glasses are bit heavier than what people want. People prefer something similar to 3D glasses. Something that’s very light and more importantly, won’t have to rely on a staff member giving it to them to make sure it’s working. There’s also a bit of technical know-how to get them work. Eventually what people would like to see is something comparable to 3D glasses, but until then it’s a really good step to getting to that point.”

Smarter glasses

So what’s next for the smart caption glasses? According to the NT's executive editor Lisa Burger it will now explore all the uses it can with the tech. It’s working with a number of venues on its tour of Macbeth where five of those venues will have the glasses available for audiences. It also thinks it can make a service for those with visual impairments work too and also feels that translation could be another feature that would give these smartglasses a permanent residency at the theatre.

If you want to try out the new smart caption glasses, they’re available to book now for the National Theatre’s new season, with performances beginning in January 2019.