Spatial computing, sometimes called the metaverse, will revolutionize healthcare by seamlessly merging the digital and physical worlds. Leveraging technologies such as augmented reality (AR) and virtual reality (VR), it offers exciting transformative possibilities, from enhanced surgical training and improved diagnostics to personalized treatment plans and remote care solutions. As the technology matures, it presents both opportunities and challenges for technology solution providers.

AR, VR, and Extended Reality solutions offer an immersive healthcare experience

Spatial computing uses technologies such as virtual, augmented, and mixed reality to interact with digital information in 3D space, fundamentally changing how healthcare is delivered. It is gradually finding applications across the industry. For example, surgeons can practice complex procedures in realistic, risk-free VR environments, improving their skills and preparedness. Doctors can visualize medical images in 3D for more accurate diagnoses.

Patients also become more engaged in their care through interactive tools that explain their conditions and treatment options. Holographic images expand the scope of remote care, providing patients with a stronger sense of connection to their doctors. VR can even be used for immersive therapy sessions, helping patients manage anxiety or PTSD in a safe, controlled setting.

The potential applications are vast. AR solutions can overlay medical imaging data (CT scans, MRIs) onto a patient’s body for improved diagnosis. Virtual patient models can simulate treatment scenarios for optimized treatment planning. Tools such as EnVisio provide continuous 3D surgical awareness, enabling surgeons to plan approaches from any angle for increased precision.

Virtual clinics promise to reshape care delivery models, while live surgeries can also benefit from this technology. Within six months of Apple Vision Pro’s launch in February 2024, surgeons at UC San Diego Health successfully performed minimally invasive operations using Apple Vision Pro headsets, which offered significant cost advantages over traditional surgical monitoring systems.

Surgery-specific applications and AI integration drive innovation

The field is attracting significant innovation. Companies such as Medivis and Osso VR are pioneering the development of spatial computing solutions for surgical planning and medical training. There are several solutions (such as Medivis’s Surgical AR and Surgical Theater) available that use real-time camera images to project 3D models in a surgeon’s headset for both surgical planning and rehearsal before the procedure. Other solutions include AR and VR to help surgeons and health professionals in their medical training.

Since the launch of Apple Vision Pro, several large solution providers and startups started developing software solutions that leverage Apple’s advanced capabilities, specifically for applications in the healthcare sector. Siemens Healthineers developed its ‘Cinematic Reality’ app in March 2024, which offers advanced imaging and visualization solutions, including 3D reconstruction and VR tools for surgical planning.

Several large surgical players are also developing solutions that help optimize the processes, making surgeries more efficient and cost-effective. For example, Stryker‘s Mako SmartRobotics app for Apple Vision Pro (launched in March 2024) enables surgeons to review and visualize patients’ surgical plans. Zeiss‘s Surgery Optimizer (launched in April 2024), an AI-powered app for cataract surgery preparation using the Artevo 850 microscope and Apple Vision Pro, is another example of this trend.

Epic‘s Spatial Computing Concept for Apple Vision Pro aims to streamline charting, lab review, and secure communication for physicians. This also enables real-time updating of patient information on the hospital’s EHR systems. Integration of AI with spatial computing is a likely next step, promising optimized procedures with spatial computing overlays. AI is also likely to aid in better processing and analysis of spatial data.

Growth in spatial computing is also likely to bring in further investments from venture capital and existing healthcare giants. Spatial computing startup XRHealth raised US$6 million in funding in January 2024, while Medivis raised US$20 million in Series A funding that would enable it to develop advanced surgical solutions that integrate AR technology.

Apple Vision Pro is a key enabler for startups to develop healthcare solutions

The competitive landscape is dynamic, with various players vying for market share. Established medical technology giants such as Siemens Healthineers, GE Healthcare, and Philips are integrating spatial computing into their existing platforms. Meanwhile, startups such as Osso VR, Surgical Theater, and Medivis are disrupting the market with innovative solutions.

Software and hardware specialists, including Microsoft (HoloLens) and Apple (Vision Pro), are crucial enablers. While Apple Vision Pro is currently prominent, other hardware and platform developers are likely to emerge.

We can anticipate mergers and acquisitions as larger companies acquire promising startups. There is an increasing focus on user experience and integration with existing healthcare systems. Apple’s launch of the Vision Pro has spurred interest in spatial computing, with startups developing for the mixed reality environment offered by AR/VR headsets such as the Vision Pro and Meta Quest.

Dependence on limited hardware ecosystems is a key challenge for developers

Solution developers face several key challenges. Dependence on hardware companies, including Apple and Meta, creates vulnerabilities. These companies have their own ecosystems for developing supported applications that limit the possibilities and options for startups.

Data privacy and security, particularly compliance with regulations such as HIPAA and GDPR, are paramount, especially as AI integration becomes more prevalent. Compatibility with legacy systems, including imaging systems and EHRs, is also essential to be successful.

Maintaining cost-effectiveness is crucial, as the high prices of devices like the Vision Pro and HoloLens pose a barrier. Funding, particularly for startups, remains a challenge, although recent successful funding rounds offer encouragement.

EOS Perspective

The spatial computing market in healthcare is poised for substantial growth, driven by the increasing demand for immersive technologies, the rise of AR/VR in medical training, and the growing need for remote healthcare solutions.

Spatial computing advancements are transforming areas such as clinical education, surgical planning, training, medical imaging, and behavioral health. We can expect even more applications to emerge, improving care delivery and surgical outcomes. Imaging data will become more interactive, detailed, and accessible. Increased deployment in surgical settings will drive further growth in the development of supporting software and hardware. We may also witness an increase in hardware companies developing spatial computing systems suitable for application in the healthcare sector.

As the technology matures and costs decrease, spatial computing is expected to become an integral part of the healthcare ecosystem, transforming patient care and revolutionizing how medical professionals work.