Experts from zMed, HUN-REN SZTAKI and PTE develop medical device using extended reality

The Pécs-based zMed team, partnering with outstanding professionals from HUN-REN SZTAKI and PTE has in the recent years been diligently striving to expedite the introduction of cutting-edge medical advancements into domestic hospitals and healthcare institutions. The project's goal from the outset was to create innovative 3D and AR images from traditional CT and MR scans, making the daily work of doctors and medical students easier and facilitating collaborative work, whether it's for diagnostics or surgical planning. Their unique development, the zMed Editor, now enables image processing and display that is unparalleled in the market. Doctors can simultaneously work with the images, consult with their colleagues, and share the images with their patients. Additionally, the device plays a prominent role in surgical planning and preparation.

At the beginning of the development that started six years ago, the team thoroughly explored the related and available technologies. To validate their usability, feasibility prototypes were also created, thus supporting the work of researchers, developers, and testers. In parallel, they mapped out how to integrate the system into medical protocols, examined how the device could be used for educational purposes, and identified the selected organs and diseases for research. Alongside these efforts, they also defined the research and system plan.

Following this, they developed the key components of the project: the static space reconstruction and dynamic space recording systems, processing algorithms and procedures, as well as the so-called motion capture systems and the associated procedures. Upon completion of the project, an augmented reality solution was created, allowing users to navigate around CT and MR scans and examine various organs from different angles.

 

In addition, a motion capture suit that tracks the movement of the entire human body was created to monitor the progress of patients in need of rehabilitation. Finally, an augmented reality solution was also developed for the examination of hand and finger movements, allowing for the modelling of certain hand and finger-related diseases, as well as their healing processes.

 

The innovative results can be applied in three main areas:

• Provides future medical professionals with the opportunity to further enhance their education through technological advancements.
• Strengthening the doctor-patient relationship and communication by providing visual assistance.
• Enhances the efficiency of surgical preparation, making doctors' work more precise, time-effective, and less risky.

The project titled "The development of an innovative healthcare tool (zMed) that incorporates augmented reality-based 3D medical images and integrates reality to support the doctor-patient relationship and education”, under project number GINOP-2.2.1-15-2017-00083, was implemented through the effective collaboration of zMed Zrt., HUN-REN SZTAKI and PTE.