There's a side to augmented reality that we don't usually get to see as wearable tech 'civilians'. AR is on the road to changing healthcare, and one of its pioneers is Indiana surgeon Paul Szotek.
Szoetk's introduction to augmented reality was familiar, though. After searching for some new glasses for his wife back in 2012, Google's ad algorithm worked its magic and started firing adverts his way for the recently announced Project Glass.
Most of us deliberately ignore these ads, but this one ended up changing the direction of his career.
"I clicked on it and wrote one of these stories, If I had Glass this is what I'd do," says Szotek. "The idea was to put it on the emergency medical services so you'd get real-time update information on the scene, and send the information back to the medical centre and have a two-way interaction between the field provider as well as the in-hospital providers."
The inspiration makes a bit more sense when you hear Szotek wasn't just a doctor in a GP surgery. "I happened to be one of the trauma guys for the Indianapolis speedway at the time," he says. That's the home of the Indy 500, where cars race at upwards of 200mph.
He commissioned custom Glass software that enabled video streaming between headsets on the scene and the local medical centre. "I worked with them to do a pilot study where we got four pairs of Glass and put them on the four corners of the track, and built the software to be able to stream the live event from the scene to the medical centre as well as the hospital."
It proved an interesting test ground, not just because of the "speed, fire and crashes" that can happen at a race track, but the connectivity problems of competing with the 200,000 people in the stands.
AR in the surgery
Szotek now works as a hernia repair surgeon in Indiana, and has brought augmented reality hardware into a more familiar setting. He uses Xpert Eye, a platform made by AMA, and a mix of Google Glass and other AR headsets. Xpert Eye is designed for both industry and the medical field, and lets others 'log into' the feed of the headset wearer.
Essential reading: Meet the people making AR magic with Apple ARKit
In February 2017, Szotek used Xpert Eye and a pair of ODG augmented reality glasses to livestream a surgery to Dr. Shirin Towfigh in Beverley Hills, California, and a second test between Indiana and Cancun in Mexico, letting her see through his eyes on a laptop screen. She is a surgical specialist in the same field as Szotek, abdominal wall surgery.
Now that everything from birthdays to traffic accidents are streamed over Facebook Live, this alone may not impress.
However, there's more to it. "It gives you the ability to take pictures and send telestrations back," says Szotek. The remote viewer can consult on the operation in real time, sending annotated images back as well as talking to the surgeon as they operate.
ODG's top-end R-9 glasses can record video at 4K resolution, for an alarmingly clear, if grisly, view of an operation, and put a 1080p 50-degree wide floating screen in front of your eyes. Before this technology, a surgeon would have to call up a colleague to call up a colleague for a second opinion, something Szotek says happens "all the time". That's several steps removed from the action.
Training with AR
The impression we get from medical TV dramas is a little different, of course, one of large hospitals with countless surgeons and doctors poking their heads in on an operation. What this doesn't account for is specialisation: there are countless 'flavours' of doctors and surgeons. "There is a limited number of surgeons specialising in these repairs worldwide," Szotek tells us.
Surgeons have been wearing wearable tech for hundreds of years
"I'm using it in a lot in the operating room to train other surgeons. I do very specialised abdominal wall reconstruction procedures, and so very commonly a surgeon from somewhere in the US will log in while I'm streaming just to learn how to do a new mesh or a new technique."
AR is being used to dissolve the geographic barriers between practitioners, and that can only be a good thing. However, he has found "there's a lot of regulatory things slowing it. The bigger problem in the US is everyone's got attorneys making them scared of things."
There are also regulations regarding practising medicine over state lines, which is a rather stark problem when it's a core part of the appeal of using AR in this manner.
Szotek sees these wearables as a completely natural fit, though. "Surgeons were the first people in the world to use wearable tech. If you've ever seen a surgeon wearing 'loops', the little magnifying glasses that they've been wearing for hundreds of years, that's wearable tech."
Get your next AR fix
I asked Szotek why he is so doggedly invested in this tech when there are so many barriers and he is not gaining too much personally from his efforts. He isn't paid by AMA to talk about this tech. "You sound like my wife: What are you doing, you're nuts?" he replied.
However, he has a vision of this tech far more advanced than what is currently available today, of an AR headset that will combine a patient's stats and the view of a laparoscopic camera for a real next-generation approach to surgery.
His conclusion is that the tech isn't quite there yet, with experiments merging the laparoscopic camera (with a company other than AMA) proving not reliable enough to real-world medical use. But digital health is booming: you can bet we'll get there in the end.