During surgery, doctors must identify vital anatomical structures (nerves and lymph nodes) to prevent unnecessary healthy tissue damage. Correct intraoperative identification by normal eyesight remains enormously challenging due to natural variation. Therefore, surgeons require high-tech intra-operative imaging for reliable high-resolution visual discrimination of critical anatomical structures.
The H3D-VISIOnAiR project, under the ATTRACT umbrella, is a breakthrough innovation based on infrared imaging and the recognition of tissues in the human body. The idea is to make them visible in real-time and on 3D images as an Augmented Reality (AR) overlay for the surgeon. It consists of two multi-spectral cameras, combining visual range with NIR (Near Infrared) visualization, a belt computer with real-time data processing, and a high-end stereoscopic head-mounted display with wireless connection to the operating room infrastructure.
Get to know more about it through this interview with Jaap Heukelom, coordinator of the H3D-VISIOnAiR project, and Gabrielle Tuijthof, professor at the University of Twente, who explain what this project is about.
Can you describe which are your roles at the H3D-VISIOnAiR project? My name is Jaap Heukelom, I’m the coordinator of the H3D-VISIOnAiR project and I’m also the co-founder of i-Med Technology based in the Netherlands. My name is Gabrielle Tuijthof, a professor at the University of Twente and I have two roles: I facilitate the bridge between the hardcore engineers and the clinicians, and I also coach the students who are involved in this project and take care of the dissemination.
What is the project about? And which partners are involved? The project is a breakthrough very innovative system, a head-worn 3D microscope with two hyperspectral cameras that can make invisible tissues of the human body visible in real-time for the surgeon. The partners involved in the project are i-Med Technology; Quest Medical Imaging, which has skills in hyperspectral imaging. Also, the Amsterdam University Medical Centre with a lot of knowledge on high perspective imaging and algorithms; the Maastricht University Medical Centre is involved, they also have a lot of knowledge on recognizing tissues. The Twente University is involved, with a lot of knowledge about AI and machine learning.
What challenges have you faced so far? The challenges we faced so far are finding the right camera that can do the job: the hyperspectral camera, which we solved. Secondly, is very important to have the right lighting conditions, especially in the infrared area. We have overcome that, and now it’s important to get sufficient data sets of live human tissues.
How would you explain the potential implications of this breakthrough to a non-scientific audience? Imagine that you would need surgery and I think many of you have maybe seen one on television and you see that it’s pretty bloody. The surgeon needs to identify and see which structures are critical, for instance, the nerve. We developed a new 3D camera that allows the surgeon to get superhero eyes, which means the surgeon can see more than the normal eyes can see. This way we improve the safety of all the operations.
During surgery, doctors must identify vital anatomical structures (nerves and lymph nodes) to prevent unnecessary healthy tissue damage. Correct intraoperative identification by normal eyesight remains enormously challenging due to natural variation. Therefore, surgeons require high-tech intra-operative imaging for reliable high-resolution visual discrimination of critical anatomical structures. The H3D-VISIOnAiR project, under the ATTRACT umbrella, is a […]
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.