Augmented Reality in Medicine

Since Robert Mann introduced the first virtual system in medicine in 1965, Augmented Reality has forever changed the way Physicians and practitioners go about caring for their patients. A revolutionary advancement in the realm of medicine, Augmented Reality, also known simply as “AR”, provides learners with an interactive experience of a real-world environment where the objects are enhanced by computer-generated perceptual information. This is all regardless of the physical setting, whether that be an operating room, clinic or classroom.

Benefits of Augmented Reality

The objects can be generated across multiple sensory modalities, including visual, auditory, haptic, somatosensory and olfactory. This all occurs amidst a virtual space where physiological data from multiple sources can merge in real time. This maturing technology has since made possible imaging 3-D anatomical information actually overlaid on top of the patient.


Sponsored Content:


Oftentimes the objects are generated using displays that are worn directly in front of the eyes, and that overlay information onto the world of the viewer. A single AR display that integrates all imaging and patient data and allows doctors to keep their eyes on the patient has the potential to improve quality, safety and reduce cost by decreasing procedure-related complications.

Another benefit of the implementation of Augmented Reality in medicine is that the technology will likely reduce costs caused by safer procedures and by eliminating the need for redundant screens. Currently ultrasound, endoscopy, and bronchoscopy all require hospitals to buy costly systems, each with a separate display. Alternatively, Augmented Reality can provide a shared display, reducing the need for a dedicated monitor for each aspect of a patient’s data over time.

Although this technology has not been readily integrated into common societal use, Augmented Reality is expanding rapidly in professional context. This is because the potential benefits stemming from the use of Augmented Reality for educational and training purposes are immeasurable.

For example, Augmented Reality has been shown to have great purpose across surgery and surgical training. According to the Harvard Business Review article “How Augmented Reality Will Make Surgery Safer,” Augmented Reality’s potential ability to concurrently display imaging data and other patient information could save lives and decrease medical errors. Augmented Reality could also potentially transform Electronic Medical Records by providing a means by which Physicians and Practitioners do not need to turn away from a patient to look at a monitor.


Sponsored Content:


However, the article’s authors, Sarah Murthi, MD and Amitabh Varshney clearly establish that the technology is still in relatively early stages. Considering Augmented Reality is in such early phases, there are a number of challenges that will need to be overcome to expand the technology’s use across the medical community.

One challenge of Augmented Reality is that hardware needs to be advanced to fit securely and comfortably on a practitioner’s head. The images also need quality advancements that will enable them to be as opaque as possible, while for others they will need to be more translucent. Another technical challenge impacting Augmented Reality use is that, if the projected images are being used for operative guidance, they need to be positioned with extreme accuracy.

Despite these challenges, the future of Augmented Reality being used across medicine for training and education is bright. From brain surgery to reconnecting blood vessels research teams across the world are working closely with Augmented Reality technology companies. Today, teams at Stanford, Duke and Johns Hopkins are all working to merge and project visual data, while simultaneously creating Augmented Reality displays ideal for patient care, the Harvard Business Review reports.

Augmented Reality Products

One type of relatively well-known Augmented Reality that could be used in medical simulation product is Google Glass, a brand of smart glasses with an optical head-mounted display. The display is designed in the shape of a pair of eyeglasses so that Google Glass can exhibit information in a smartphone-like, hands-free format. Pairing with smart glasses, a few AR software companies have developed an ultrasound scanner, which with the help of products like Google Glass work as a traditional one.

HealthySimulation.com is dedicated to providing the latest Augmented Reality news and #MedSim resources from around the world. To follow along, sign up for our free medical simulation email newsletter, follow @HealthySim on Twitter and @HealthySim on Facebook, or join our HealthySim LinkedIn Group!

Another Augmented Reality option, a team of researchers at Imperial College London at St. Mary’s Hospital in London, England have demonstrated how Mixed Reality and Microsoft HoloLens headsets can be used while operating on patients undergoing reconstructive lower limb surgery. According to the Imperial Team, the technology can help surgeons locate and reconnect key blood vessels during reconstructive surgery.



This technique could help to improve outcomes for patients. Using HoloLens, Practitioners and learners can overlay scan images onto the patient during the operation. This is so they can see elements like bones, the course of the blood vessels and identify exactly where the targets are located.

Also on the market, an Augmented Reality system called ProjectDR, created by computing science graduate students Ian Watts and Michael Fiest at the University of Alberta in Canada, allows medical images such as CT scans and MRI data to be displayed directly on a patient’s body. Then, the image moves as the patient moves thanks to custom software, giving the practitioner a full view of the patient’s internal anatomy.

Click Here to Connect to Leading Medical Simulation AR Vendors

According to the developers, ProjectDR also has the capacity to present segmented images. This means the software is capable of showing only the lungs or the blood vessels. The exact body part needing imaging will depend on what a practitioner is required to see, leaving out any other distraction and achieving more focus in the targeted area.

In the coming years, U.S.-based Magic Leap and Munich-based Brainlab are also hoping to add an innovative dimension to surgical procedures and medical imaging in an operating system through Augmented Reality for craniotomy planning. Their operating system combines medical technology company Brainlab’s data management, cloud computing, visualization, and data pre-processing software with Augmented Reality technology company Magic Leap’s spatial computing and experiential platforms.

According to the companies, the first release of the product promises to enable surgical planning and simulation in an office setting. The companies also says this will quickly be expanded to include a next-generation mix of virtual and physical worlds for the operating room, radiotherapy treatment room, intensive care unit, and radiology suite.

AR in Medicine Latest News

Explore XR Summit 2023

Xplore Conference to Gather Asia Pacific XR Champions in Australia this November

The Xplore XR Summit 2023 promises to be the ultimate gathering for medical innovators, visionaries, and inquisitive minds. This summit will allow participants to embark on an exclusive journey into ...
Digital Twins in Healthcare Education

How Digital Twins Will Change Healthcare Education

Do Digital Twins have applications in healthcare simulation? Digital Twins are virtual models of physical objects, processes, or complex systems that are created to accurately mimic their real-world counterparts. These ...
surgical simulation

Use of XR in Spinal Surgery and Incorporation of Surgical Theater Tool

The Hoag Health System recently reported that “the future of spine surgery is here.” This is because the first augmented reality (AR) spine surgeries in Orange County, California were recently ...
Cae Healthcare Vimedix

Honor Medical Ultrasound Awareness Month with CAE Healthcare Ultrasound Simulation Solutions

Ultrasound technology’s ability to visualize soft tissues, such as organs and muscles, has vastly impacted the field of medical imaging. This diagnostic tool is able to help ultrasound technicians and ...
Matt Merino EMS CEO Interview

Exclusive: Education Management Solutions New CEO Matthew Merino Shares Vision for Healthcare Simulation Industry

Education Management Solutions’ (EMS) new CEO Matt Merino is a force to be reckoned with. As a former Team Leader and member of the U.S. Army Special Operations Community, Mr ...
top healthcare simulation articles summer 2022

Top 10 Healthcare Simulation Articles of Summer 2022

HealthySimulation.com is dedicated to bringing the latest healthcare simulation news, updates, and information to the clinical simulation community across the world. During the summer of 2022, the articles published included ...
VR AR simulation

Augmented Reality vs Virtual Reality in Healthcare Simulation

Have you ever wondered about the differences between Augmented Reality vs Virtual Reality? There is no doubt that the very nature of medicine has been forever changed by the COVID-19 ...
xr simulation

Introduction to XR Technologies in Healthcare Simulation

The field of healthcare simulation has greatly benefited from advancing technologies, especially regarding the use of extended reality (XR). This term includes all immersive technologies that “extend” the traditional learning ...
INACSL 2022 Sim Media Center HealthySimulation.com Interviews - Part I

INACSL 2022 Sim Media Center HealthySimulation.com Interviews – Part I

HealthySimulation.com was in attendance at the 2022 International Nursing Association of Clinical and Simulation Learning (INACSL) annual conference in mid-June in Milwaukee, Wisconsin. As part of this year’s event, themed ...
Augmented Reality Mobile Learning

Translating Healthcare Simulation to Augmented Reality Mobile Learning

Basic competency in patient safety has been a goal of nearly all healthcare systems and healthcare education leaders since the turn of the century. An app-based approach can create a ...
GIGXR

Mixed Reality in Medical Education: Transforming Access to Healthcare Simulation Training

Healthcare professionals require the highest quality training to ensure outcomes are optimized. While they themselves have put their lives at risk to treat patients, the COVID-19 pandemic has added to ...
Lucid Reality Labs

How Lucid Reality Labs Supports Medical Device Manufacturers with VR Training Tools

Virtual reality has become an important element of healthcare simulation education and training. As VR technologies continue to advance, an increasing number of tools are becoming readily available and easily ...

Subscribe for the Latest AR in Medicine News!


Sponsored Content: