Extended reality

AI is a very broad discipline that has led to the development of new devices, such as connected watches, smart glasses and virtual/augmented reality head-mounted displays. These new devices belong to the category of the “wearable technology”, referring to connected or intelligent objects that can be worn. While they have been developed and are already used in various fields, such as sports or the gaming industry, there is a growing interest in their application in medicine. Our group has been working on the use of virtual/augmented reality head-mounted displays for the management of vascular diseases.

What is extended reality?

Extended reality (XR) is a generic term that refers to all combined real and virtual environments that are used to simulate or enhance the physical world. It includes the three main categories:

• Virtual reality: This is a set of technologies that allows the user to be immersed in a completely artificial world, called “virtual”. This world can be totally fantastic, imaginary or mimic and reproduce the world we know.
• Augmented reality: consists in blending virtual objects into the real world. The operator visualizes virtual objects that are projected into the real world thourgh holograms. The user is thus able to visualize the virtual objects, while keeping a view of the real world.
• Mixed reality (MR) corresponds to the merging of real and digital worlds, where digital (virtual) and physical (real) objects co-exist in space and can interact in real time.

Extended reality for patients

By offering the possibility of immersing oneself in a virtual environment, virtual reality has been evaluated in various hospitals in France for the treatment of pain, by helping to induce a state of semi-consciousness or even hypnosis. Various studies are also being carried out on the use of headsets to reduce stress and anxiety in patients and prepare them for a therapeutic procedure or intervention. Similarly, they are attracting growing interest by potentially offering new tools in various medical specialties such as rehabilitation or psychiatry. If the results prove conclusive, it is possible that the use of these helmets for patients will become widespread in the coming years.

Extended reality for health professionals

Extended reality represents a major advance that can be used for the education and training of health professionals. It facilitates the 3D visualization of complex structures and allows interactive learning, which is useful for different disciplines, such as anatomy teaching for example. It can also be used for training in procedures and technical gestures, particularly in simulation centers where the operator can train on mannequins or virtual patients while reproducing real conditions (see dedicated chapter).

In addition to training and education, extended reality brings new tools for health professionals. Various studies have shown the interest in using this technology to allow a precise 3-dimensional visualization of the patient’s anatomy from the imaging examinations carried out, which could be useful to improve pre-surgical planning.

Applications of AI to improve the visualization and analysis of medical imaging are expanding rapidly. Pilot studies have recently showed the potential interest of augmented reality headsets for Image-guided surgery. Image-guided surgery corresponds to any surgical intervention where the surgeon uses pre-operative or peri-operative images in order to guide the procedures. Operating theatres equipped with X-ray tables and visualization screens are required for these procedures. However, the surgeons have to face several limitations. Depending on the patient’s position and the surgeon’s position during the intervention, the visualization on the screen can be difficult. It requires effective coordination between the surgeon and the teams in the room, as well as the need for the surgeon to perform a procedure while alternating between the operating field and the viewing screen. Using augmented reality head-mounted displays, the technology allows the operator to project patients’ imaging in the form of holograms directly into his field of vision and allows to maintain a constant focus on the surgical field. The helmets are voice-activated, allowing the operator to use them autonomously while keeping his hands mobile and respecting the sterile conditions required for surgery.

Although still in the pilot study stage, this type of technology could be very useful in improving ergonomics in the operating room by eliminating the need for visualization screens while accurately guiding therapeutic gestures.