Neurosurgery simulation just saw a revolution in realism and application, with a real-time brain simulator which can help neurosurgeons train for and perform better during clinical practice. Out of the Igarashi Laboratory in the Graduate School of Information Science and Technology at The University of Tokyo, Dr. Seo and team have developed a cutting edge application for the very complicated task of looking at brain anatomy during an actual surgical intervention. Utilizing the Unreal Engine, this project is supported by Epic Games who we also shared about recently with their support ofย Precision OS: Orthopedic Surgery Training in VR. Let’s take a deeper look at Sรฉbastien Lozรฉ’s recent article on this innovative brain surgery simulator!
Brain surgery remains one of the most intricate and challenging tasks a surgeon might have to perform, and requires extensive training and practice. While the medical field has long used CGI to visualize the human body inside and out, many applications are limited to opaque shaders on rigid objects, which donโt accurately depict the many translucent, thin, or flexible parts of the brainโs structure.
In their 2018 paper Enhancement Techniques for Human Anatomy Visualization, Hirofumi Seo and Takeo Igarashi state that โHuman anatomy is so complex that just visualizing it in traditional ways is insufficient for easy understandingโฆโ To address this problem, Seo has proposed a practical approach to brain surgery using real-time rendering with Unreal Engine.
As part of the User Interface Research Group, Igarashi Laboratory, Graduate School of Information Science and Technology at The University of Tokyo, Seo and his team are working on a real-time visualization and training application for brain surgery that more accurately portrays the brainโs structure and how it deforms during surgery. The software prototype, made possible with a grant (Grant Number JP18he1602001) from Japan Agency for Medical Research and Development (AMED), helps surgeons visualize a patientโs unique brain structure before, during, and after an operation.
โNeurosurgeons all over the world performing aneurysm surgery want some kind of pre-surgical simulation, practice, or check, because the actual surgical view is very limited and the surgery itself is very difficult,โ says Seo. โThey also know that the brief blood-vessel-branch dominant region of each blood vessel is easily predictable if they can see the whole brain and the blood vessels. So many neurosurgeons have wanted to use 3D CG for a long time, but they donโt know how to implement it.โ
With a real-time 3D visualization, the surgeon can load a model of a patientโs brain from the individualโs MRI and 3D Rotational Angiography (3DRA) data, look at it from any angle, pull apart the CG sulcus to see inside, and even make the lobes invisible to better see the blood vessels. The user controls everything via simple mouse cursor movements or via multi-touch, making the app easy and accessible for surgeons without technical experience.
In developing the application, Seoโs team chose Unreal Engine as the underlying real-time technology because of its graphics and programming tools. โUnreal Engine has powerful mathematical C++ APIs such as FVector, FMath, and UKismetMathLibrary, so we find it to be a suitable platform for research on 3D CG geometry,โ says Seo.