Today, at the Royal London Hospital, Dr. Shafi Ahmed livestreamed a three hour operation on a patient with colon cancer, in 360 degrees to smartphones, Google Cardboards and Samsung Gear VRs around the world.
He did it with the help of London based VR experts Mativision who handled the capturing, processing and broadcasting of the operation. Two 360-degree cameras, one test run at the hospital to make sure they had enough bandwidth, and one for Dr. Ahmed in charge of every decision behind the trial.
"The technology allows us to do many things, much more than we are doing at the moment," said Anthony Karydis, Mativision's founder and CEO, speaking to Wareable before the event. "The use of a second camera was the furthest in complexity Dr. Shafi was willing to go at this stage. It is still the first time something like this has happened and quite naturally and logically he wanted us to keep it simple."
An all-access pass to surgery
Barts Health and Medical Realities, the AR and VR startup which Dr. Ahmed founded in late 2015, plans to expand this first livestream into a program of training modules, both live and on demand, for medical students and surgical trainees.
It's an extra resource for students at medical schools such as Barts in London but could also transform medical education in developing countries where access to top class operating theatres and, indeed, surgeons is limited. The viewpoint of one of the cameras is actually next to the surgeon, closer than even trainees are allowed to get to the procedure.
After keeping it simple for this first test run, which successfully livestreamed via the VRinOR app, Mativision is excited about what it is capable of adding in the future.
"The aim we have is to produce training modules that will have lots of layers on top of the video itself," said Karydis. "These levels will be on demand, they will not show all the time.
"For instance, in the future someone is watching this operation and wants to get more information on the status of the patient. Through the intelligent interface we are integrating, they could call up specific information from the monitors and equipment or even from the medical file of the patient. We will overlay both computer generated graphics and text and videos on the 360 image itself."
Cardboard for med students
Mativision has been livestreaming events including Muse gigs and American Idol shows for VR consumption since 2010. It has broadcasted to audience figures in the millions and powered experiences that received big marketing campaigns from the likes of Coach, Samsung and Vodafone. Plus, it has big plans for its own VR livestreaming platform with channels for sports, music and education.
But the success of this particular livestream won't be measured in thousands of app downloads or viewing figures but by the response of the participants, medical students and doctors who aren't an early gadget adopter like Dr. Ahmed.
Before the surgery, Dr. Ahmed gave out bespoke Cardboard headsets to both students based in London and students who had visited the hospital and since returned to other countries. Time will tell if it's a useful and efficient way to educate in medicine but Mativision are ready to roll out a library of on demand VR videos to the VRinOR app if that's what Medical Realities wants.
"I think VR operating will go beyond being a gimmick and find itself within the e-learning syllabus of many medical schools and training programs in future," Danyal Awal, a Barts medical student, told us.
"It allows you to feel like you really are in the room watching the operation. This could be beneficial for all medical trainees, whether you're an A-level student interested in pursuing medicine as a career, or a surgical Consultant wishing to learn a new technique from a colleague around the other side of the world."
Training through HoloLens
When Dr. Ahmed experimented with Google Glass livestreaming in surgery in 2014, he allowed students to ask him questions which would pop up in the side of his view throughout the operation. Mativision is already looking to the future of this kind of interactive instruction via a partnership with Microsoft to develop applications for HoloLens.
"What we plan to do is to work closely with Microsoft and see how we can implement their HoloLens technology to make the immersiveness a kind of two way activity," Karydis told us.
"We are discussing the potential of using HoloLens as a part of all this exercise," he said. "If we add equipment like HoloLens, we may introduce a whole other level of interactivity and exchange of live communications between the doctor delivering the training and the people that watch."
Medical Realities isn't involved in this project yet but no doubt this is only a matter of time. And the timeline for Mativision's HoloLens project? "We have absolutely no idea. We have to wait for Microsoft."
How we test