This
research project introduce the development of robot assisted surgery tools.
Including virtual reality, 3D graphics, force feedback and control algorithms.
Master and slave system interacting with objects in a VR-environment generating
haptic feedback. The research focus is on implementation and 3D modeling
of a skull taken from CT-data and also of finding a relevant force feedback.
The PhD student position is funded by the Centre
for Technology and Health care (CTV), which is a co-operation between
KTH and Karolinska
University Hospital. This research project is a part of the skullbase
project at the Mechatronics Lab at KTH.
An illustration of the complete surgery simulation system.
For
removal of cancer tumors in certain locations of a human head, the surgeon
has not only to open up a hole in the skull bone, a path along the inside
of the skull bone must also be made in certain cases.
Today, the surgeon mills this path very carefully with a small hand held
mill such that the tumor can be reached without affecting the brain more
than necessary and not to damage other vital parts of the head located
close to the tumor. Typically, this path is located in a region where
the skull bone is geometrically complicated and is surrounding neurons,
brain tissue and critical parts of the nervous system. Hence, the milling
phase of an operation of this type is difficult, safety critical and very
time consuming. Reduction of operation time by only a few percent would
in the long run save society large expenses.
The
surgical situation today
In order to reduce operation time and to provide surgeons with an invaluable
practicing environment, this project is a start up for introduction of
a VR-simulator system in both surgeon curriculum and in close connection
to the actual operations. Today VR and haptic simulation is used to educate
surgeons most on soft tissue interaction, such as laparascopy simulation.
For more details, visit Center
for Advanced Medical Simulation at Karolinska University Hospital.
Simulation
of a gallbladder dissection
In earlier research, a prototype master slave system for introduction
of telerobotic surgery in the described task is developed (see the
skullbase project). This project is an extension to that system in
terms of development of a simulator system based on a virtual reality
representation of the human skull from which both haptic and visual feedback
to the surgeon is generated.
Implementation
of the 3D graphics of the skull bone and implemantation of the haptic
models are considered as the two major research steps needed before
the overall goal with this research project is achieved; a well suited
virtual reality system with haptics capable of training and education
of surgeons practicing bone milling.
The working process consists of different parts, which is illustrated
in the figure below and these parts will help us to reach the over all
goal with this project.
3D
visualization of the bone removal process.
|
Implementation
of force models for realistic haptic feedback.
|
Make
the simulator "user friendly" and evaluate it together
with surgeons.
|
A
bone milling surgery simulator (can also be used for e.g. dental
training or free form design).
The
research focus is on finding a well working VR-system for realistic
representation of the skull itself and vital phases in the operation.
A challenging problem is to find an efficient method to represent the
CT-data of the skull such that the haptic rendering can run at a 1000
Hz in update frequency. This is a problem of general concern since computational
workload is much larger when rendering a deformable object in real-time
compared to a non-deformable.
In the SenseGraphics
H3D API is our developed algorithms implemented for graphical rendering
of the milling process of the skull bone. The H3D API also deals with
the communication to Sensables
OpenHaptics HD/HL API to handle control of the master unit, a Phantom
Omni.
The VR-representation of the 3D drill/bone interaction process consists
of the following key elements:
1. Manage the discrete data taken from CT or MRI.
2. Rendering of the skull bone.
3. Calculation of the normals used for light reflection from the skull
bone surface.
4. Visualization of bone material removal.
Visualization
of the haptic milling process of temporal bone surgery
Haptic simualtion
of tooth milling
The simulator
can also be used for free form design with haptic feedback
A
second challenging problem is modeling of the forces involved when milling
in human bone. This step includes a lot of identification of force characteristics.
The contact forces that occurs during the milling process depends on
mill rotational speed, cutting speed and depth, material data etc.
The dynamics involved in the force build up is assumed to be much more
complex than the traditional spring-damper models commonly used in haptic
simulation.
The image below is a description of the haptic rendering procedure of
voxel-based volume data. Click on the image to open it a pdf-file for
easier reading.
A description of the haptic rendering procedure of voxel-based
volume data
Here
are two films illustrating the haptic and visual simulation of milling.
The algorithms used are the ones described above.
Film
1.
Film
2.
For more detailed information of the status today, please e-mail
me or read the latest papers under "publications".
USING
MATLAB/SIMULINK FOR HAPTICS
Another research issue I have been focusing on is to make a haptic interface
using Matlab/Simulink. Please, see the two films below how it works
and also read an abstract under "publications".
The two films below show Haptic Milling in the Augmented Reality Set
Up developed at IIP KTH. Real 3D visualization and haptic milling without
any stereo glasses!
MULTIPLE
POINT COLLISION DETECTION AND 6-DOF HAPTIC FEEDBACK IN A MILLING SURGERY
SIMULATOR
The two films below show results from the research going on with 6-DOF
haptic feedback. Please, view the films and send me an email if you
have further interest in this topic. Two different algorithms has been
developed and verified, papers will be presentaed later.
Here are three Haptic Milling demos available to download. The zip-files
include all necessary files. The earlier problem with haptic fall-through
has now been solved(!). And the milling sound is implemented(!).
None of the demos require H3DAPI installed (all necessary files are
in the zip-folders). Just downlad the folders. (Keep the Omni demo
separeted from the Falcon demo).
The Omni demo requires a Phantom haptic device installed (of course)
and the Falcon demos require Novint Falcon haptic device.
- Download folder.
- Start Demo by double click on corresponding .bat-file (omni_tooth256.bat,
falcon_box.bat or falcon_tooth256.bat).
- By pressing the main button of each device you mill and remove material.
The milling sound is activated when pressing the button and is proportional
to the haptic force.
- The object (tooth or cube) can be rotated by using the mouse or
the keyboard.
DEMO #1 "OMNI TOOTH MILLING"
DEMO
#2 "FALCON TOOTH MILLING and FREEFORM SCULPTING"
