Augmented reality and haptic interfaces for robot-assisted surgery

作者：Yamamoto Tomonori^*; Abolhassani Niki; Jung Sung; Okamura Allison M; Judkins Timothy N

来源：International Journal of Medical Robotics and Computer Assisted Surgery, 2012, 8(1): 45-56.

DOI：10.1002/rcs.421

• robotics
• teleoperation
• stereo-vision
• robot-assisted minimally invasive surgery
• haptics
• augmented reality

Background Current teleoperated robot-assisted minimally invasive surgical systems do not take full advantage of the potential performance enhancements offered by various forms of haptic feedback to the surgeon. Direct and graphical haptic feedback systems can be integrated with vision and robot control systems in order to provide haptic feedback to improve safety and tissue mechanical property identification.
Methods An interoperable interface for teleoperated robot-assisted minimally invasive surgery was developed to provide haptic feedback and augmented visual feedback using three-dimensional (3D) graphical overlays. The software framework consists of control and command software, robot plug-ins, image processing plug-ins and 3D surface reconstructions.
Results The feasibility of the interface was demonstrated in two tasks performed with artificial tissue: palpation to detect hard lumps and surface tracing, using vision-based forbidden-region virtual fixtures to prevent the patient-side manipulator from entering unwanted regions of the workspace.
Conclusions The interoperable interface enables fast development and successful implementation of effective haptic feedback methods in teleoperation.

• 出版日期2012-3