Covid-19 and Big Tech are driving augmented reality into healthcare

In those wards there’s an obvious risk that Covid stalks the corridors, but older stalwarts such as MRSA remain equally infectious. As such, Alder Hey, the world’s first hospital to test penicillin and the UK’s first to ever establish a neo-natal unit, wanted to use immersive, augmented and mixed reality (MR) technology to reduce the amount of physical contact between hospital staff and the public.

Working with Insight’s FastStart service to find technological solutions, the hospital primarily needed a way to communicate and share images and real-life scenarios with other practitioners, regardless of location. This had to be done through video and audio in real time 24/7, not only for remote patient care, but also virtual ward rounds and staff training.

Wearing a HoloLens 2 device running Microsoft’s remote assist software Dynamics 365, Guerrero could perform his rounds alone. Other specialists joined in using the Teams app, seeing exactly what the clinician saw, along with three-dimensional (3D) echocardiography.

Alongside this augmented reality including real-life video, there was the ability for 2-way audio/video communication and sharing other visual content. Other important tasks  – such as updating patient records  – could also be redistributed among the team to save time as Guerrero worked.

“Using HoloLens, we have already demonstrated the potential of mixed-reality healthcare applications to minimise the number of staff required to be in contact with patients; share expertise in the shortest possible time and from any location; and, ultimately, increase the quality of care,” Guerrero tells Verdict.

Guerrero believes these applications would allow Alder Hey to support clinicians and patients in other hospitals and in the community, without the need for travel in either direction, thus helping efficiency and expertise-sharing in the NHS.

“While Covid-19 has acted as a catalyst, mixed reality devices that can support collaboration, mobility and remote learning need to be part of the technology that we use to drive a healthcare revolution,” he says.

GlobalData forecasts that the global AR market will be worth $76bn by 2030, up from $4bn in 2018. As GlobalData analysts write in a recent report on AR in the health industry, “the healthcare sector is undergoing a digital transformation, fuelled by changing healthcare payor and provider need… AR is gradually making its way into the healthcare sector.”

This revolution is coming about for various reasons in more than one area of healthcare. Another application of AR is aiding visualisation during minimally invasive surgery (MIS).

Startups such as Medivis and Proprio are selling MR solutions in the surgical space, with the latter recently talking to Verdict about its AR and artificial intelligence-combined solution. Surgeons can also benefit from the augmented reality “x-ray vision” granted by US brand Augmedix, with an AR navigation tech that allows them to see a patient’s anatomy through skin and tissue whilst operating.

AR is a useful surgeon’s aid for the visualisation of complex biological processes. EchoPixel’s True 3D software can help healthcare professionals visualise and interact with 3D images that depict human tissue and organs in open space as if they were real objects. Novarad’s OpenSight solution renders 2D, 3D and 4D images of patients interactively, while accurately overlaying them directly onto the patient’s body. Both solutions allow for more accurate surgical planning.

The Alder Hey case study, though, shows how AR can be useful when integrated into telemedicine to help healthcare specialists work remotely. The same can apply to medical students: Imperial College London pre-empted Alder Hey somewhat in early 2020 by creating the world’s first virtual ward round for medical students, who watched as a HoloLens-equipped doctor examined a patient.

In times of social distancing, medical students can improve their knowledge and skills by accessing human body models created by AR. Solutions like those from EchoPixel allow medical students to familiarize themselves with the human anatomy and practice surgeries on virtual patients.

Real-life flesh and blood patients can also benefit from an augmented reality when distanced from doctors. A recent GlobalData report on digital health solutions in neurology suggests that virtual reality (VR) and AR are “well placed as a therapy product for neuropsychological conditions because virtual simulations of the outside world are highly realistic and immersive, as well as being non-invasive and non-pharmacological.”

Simulations offer less time-consuming options in comparison to some pharmacological interventions due to their immersive attributes, making for highly engaging forms of therapy.

Indeed, augmented reality technology appears in therapies targeting everything from chronic pain management, mental health and addiction, to behavioural conditions and neurorehabilitation. AR is also used effectively in exposure therapy, overlaying the hands of insect-phobic patients in live video with digital cockroaches and spiders.

Cognoa’s Superpower Glass wearable meanwhile, a device based on Google Glass, helps autistic children better understand facial expressions through the use of emojis overlaid on the lens.

The rise of AR in health comes as VR/AR headsets and smart glasses become cheaper, and more AR apps appear on smartphones. Some brands like XRHealth are even sending out headsets to customers to treat ADHD using extended reality (XR).

With FDA approval and products covered by insurance for over 50 million people in the US, XRHealth was able to trial a service which also offers a personalised care plan, in-app messaging and video calls with a dedicated clinician. While headsets may be an alien concept in the common household, especially so with regards to healthcare, teleconferencing with a medical professional is perhaps more agreeable to consumers after the pandemic’s video boom.

Medical professionals themselves are also succumbing to AR’s charms. A 2021 poll shared in GlobalData’s report on digital health in neurology found that 18% of 109 respondents in the industry thought AR/VR solutions would be most suitable to treat mental and behavioural health conditions such as depression, anxiety and substance abuse compared to other technologies.

Meanwhile 15% of 110 respondents believed AR/VR technologies would be most suitable for neurological conditions such as Alzheimer’s and Parkinson’s.

Besides the approval of the medical community, AR will also need Big Tech’s money to fully push it through the hospital doors. The adoption of HoloLens devices by Alder Hey and other hospitals shows that Microsoft already has one foot in the door. The tech company’s recent $19.7bn purchase of Nuance Communications will also help.

UK-based Nuance, a speech-recognition company best known for having provided Siri’s speech recognition engine, offers enterprise AI tools that transcribe doctors’ notes and visits during a working day. It isn’t much of a stretch to see that tech mixed with smart glasses: Healthcare professionals regularly require access to information systems, whilst often needing their hands free.

Here AR can reduce the burnout physicians face as it provides easy and quick access to real-time information, thus easing administrative burden. Data from the US suggests that more than 50% of the physician workforce is burned out, with excessive admin a major contributory factor; such burnout can affect safety and the quality of care in hospitals, and also increase healthcare costs.

With smart glasses augmenting reality, staff can work and take notes without the added weight of a physical notepad. This also allows for more accuracy as notes are taken in real-time, rather than the staff trying to recall information after patient interactions.

For Guerrero, this easier way of dealing with information was one reason why the HoloLens experiment was a successful one in Alder Hey.

“Ultimately, the success of any procedure is down to the skills of the surgeon and the team,” he says. “Although new technology is used as a tool or aid for the surgeon to use, there are other ways of accessing the same information. It just takes longer and is not as accessible.”

In general, AR in medical devices is led by smaller vendors specialising in software and apps, according to GlobalData analyst Aliyah Farouk. But big tech players are investing in healthcare and working with the industry’s giants. Roche, for example, partnered with Samsung’s AR/VR subsidiary Harman in 2020.

“It’s difficult to say the extent of investment or impact in augmented healthcare as this is still a largely untapped market,” says Farouk. “You can look at areas where AR is making a big difference in tech, e.g. smart glasses and whether these devices are being used for healthcare purposes. For example, Google relaunched Google Glass Enterprise in 2017 targeting healthcare companies.

“Harman’s collaboration with Roche and Microsoft’s recent purchase of Nuance is reflective of this trend where healthcare companies are increasing their connectivity/digitalisation and tech companies are capitalising on this.”