Microsoft HoloLens is finding its way into the operating theatre

With the hype around augmented reality ramping up since Apple and Google both introduced their smartphone-based platforms last year, it can sometimes be easy to forget that another tech giant, Microsoft, already has its own hardware maturing in the open world.

We've already witnessed the Redmond company take steps in education, bringing both AR and VR experiences to the classroom and giving a peek into what the future of the area could look like, but that's not the only area it's exploring with partners.

Read this: Everything you need to now about augmented reality

Alongside Imperial College London, Microsoft's AR technology is being used in operating theatres to help those needing reconstructive and plastic surgery – and we went hands on with the technology to see what HoloLens is providing and where the medical future of the technology lies.

Fixing problems through HoloLens

In the field of reconstructive and plastic surgery, the key area of focus for surgeons in the pre-operative phase is marking up the patient's blood vessels. This involves CT scans to pinpoint potential vessels that can keep skin flaps oxygenated, and traditionally relies on ultrasound technology to find 'good' vessels to work from.

However, as you might expect, that's not an entirely straightforward process. And the current technology in place isn't always a reliable source of finding these blood vessels, which is where HoloLens is coming into its own.

"I'd been looking at the applications of HoloLens for a while, and myself and my colleague Dr Dimitri Amiras realised one day that it could be the answer to some of the same problems we faced in surgery," says Dr Philip Pratt, senior research fellow at Imperial College London.

In-depth: All the essential details on Microsoft HoloLens

"We always start with a scan – say, it's a person's leg – then we would usually interpret that scan and pick out all the key structures. But with the HoloLens, we're now doing the same kind of thing and then processing the data captured to make it suitable to look at. That means we end up with a silhouette of a limb, the location of the injury and the course of the vessels through the area, as opposed to this grayscale image of a scan and a bit more guesswork."

The problem with the current method of finding perforated vessels, Dr Pratt notes, is that it's tough to ascertain the depth of vessels and know where to make the first incision. And that's the problem Microsoft's headset is fixing, with graphics like the one above. It's essentially like giving surgeons a cheat-sheet or a preview before the real thing in the operating theatre.

AR in the operating theatre

The team at Imperial has been getting to grips with the technology for the best part of the last year, with a number of real-life use cases and a paper from Dr Pratt and Dr Amiras detailing the work also now published.

But it all comes back to making sure the technology is improving the efficiency of the healthcare, or is at least on the road to doing so. And as Matthew Ives, a reconstructive and plastic surgeon at Imperial, describes, HoloLens is already able to bring useful insight to the pre-operative process.

"It's started to change how we'd approach certain areas of reconstructive surgery," he says. "We can do the CT scans and angiograms to get an idea, but where HoloLens has made a difference is not only making the mark-up with a pen more accurate, but also with the context of location around the fracture.

"When you're in the operating theatre, you already have a better idea of where to go because you've had this visualisation. That's crucial, because people's bodies aren't all like they are in textbooks or how you're taught in school."

This doesn't mean that the technology is perfect in its current format, though. And while it's on the way to changing how these surgeons approach procedures, this is still really the first chapter in a bigger tale.

For example, while HoloLens is currently increasing accuracy in this particular area, it's not yet able to cut down the time of pre-operative process. As Dr Pratt tells us, readying data for the HoloLens still takes roughly 30 minutes, the same amount of time it would with the traditional ultrasound tech. And that's because scans are high-resolution, meaning transferring from A to B isn't always straightforward.

There are also steps to take in order to bring the HoloLens into the theatre during procedures, and not limit its applications to just marking up patients.

"One of the problems we have with it at the moment is that the overlay doesn't snap straight to the leg," Ives continued. "If you wanted to operate with the headset on, you can't currently because the mesh doesn't move with it. It's already pretty user-friendly in its current state, but with a feature like that it would become even more interesting to use."

What does the future hold?

Bringing HoloLens to this point has had its challenges, but the early signs indicate that it has a bright future in the hospital environment. The challenge now becomes figuring out how it progresses from being an interesting tool to an essential one. And, as Dr Pratt told us, that reality may not be so far away.

"I'd like to think that within two or three years this isn't technology that's limited to just two or three hospitals in the UK, but actually something we could help scale up by a factor of ten or 100," he said.

Doing so would mean a step further in the technology from Microsoft's end, to provide features that can help improve the accuracy and efficiency of medical procedures, but this is something we're told is already in the works.

Read: Joining Mixed Reality with HoloLens could be Microsoft's end-game

And, Pratt says, more fields are interested in leveraging the technology in the same way Imperial College has. This is limited somewhat due to the ethical approval needed for the device to be used within individual NHS Trusts, but as the technology gains more trust in the eyes of the medical community, the doors will likely continue to open for Microsoft.

As with its punt in education, all the focus now switches to how this grows from an intriguing start. And done right, we have a feeling it could be even more revolutionary than what we've already seen from the company.