Augmented Reality helps surgeons and engineers.

Before certain operations, three-dimensional sectional images (e.g., computer tomography, magnetic resonance tomography) are produced in order to plan them exactly. In the future, surgeons will be supported by an augmented reality solution developed in the VizARd project. This image data can then be visualized in an intuitive and user-friendly manner directly on the patient‘s skin at the site of the incision.

Soft tissue and position recognition as a challenge

The challenge: Although there are many valid data on humans and their “inner life,” humans are anything but a rigid “object” – the position of skin surface, soft tissue and organs changes with the body position. In order to tackle these research challenges, the project partner uses technologies that have been developed in recent years for assistance systems, such as image-based position recognition based on deep learning technologies. What previously served the digitalization of the human being, his tools and his workpieces in the assembly hall is now being transferred to the human being in the operating theater.

Another central aspect is the deformation of soft tissue, which must be taken into account when visualizing image data. The Medical Informatics Research Department of RISC Software GmbH in Hagenberg has already gained experience in biomechanical modeling in the medical field through previous projects. The visualization of medical images directly on the 3D patient – in contrast to a 2D screen – forms the basis for trend-setting research projects and the improvement of existing applications in the medical field.

Cooperation with MedUni Graz

The medical project partner is MedUni Graz with the Clinical Department of Plastic, Aesthetic and Reconstructive Surgery and the University Hospital of Radiology, which are involved in the practical implementation and provide the application scenarios.

User-friendliness: A topic for surgeons

Another challenge is usability. The digital agency Netural is responsible for the design of the software’s user interface. The company has been conducting intensive research in the field of e-health and medical technology for many years. Especially the user-friendly application of mixed-reality applications is in the focus of the developments.