Figure 1: The two Major Thrusts of this NSF CAREER proposal

Abstract:

Current medical robotics systems use non-intelligent surgical manipulators that place the entire burden on surgeons for safeguarding against damage to the anatomy. The emergence of new surgical paradigms, such as Natural Orifice Endoscopic Trans-luminal Surgery (NOTES), requires surgical robots that are capable of supporting safe interaction with the anatomy while accessing deep surgical sites through often long natural access pathways. This requires new types of robots capable of safeguarding against damage to the anatomy by acting as intelligent intervention and information gathering tools for assisting surgeons during increasingly complex procedures. The objective of this research is to provide the theoretical foundation for modeling and control of flexible robots for intelligent and safe interaction with the anatomy. Intelligence refers to the ability of these robots to gauge their force interaction with the anatomy, gather information about the anatomy, and act based on this information. Screw theory and stochastic estimation methods are used for modeling the ability of these robots to estimate their wrench interaction with the anatomy by using intrinsic and extrinsic sources of information. These performance measures are used in hybrid force control algorithms that allow characterizing shape, stiffness, and anatomical constraints governing safe maneuvering of suspended organs. The outcomes of this research will allow the development of radically new technologies for newly emerging surgical paradigms (e.g. NOTES). This research will also advance the field robotics by addressing control and resolution of multi-point contact problems along flexible robots for compliant insertion control and bracing against soft environments.

Funding:

This research is funded by NSF grant #NSF Career grant #IIS-0844969.

Scope:

Figure 1 shows the two main thrusts of this reserach project. Thrust 1 focuses on three sub-areas of modeling, control, and interaction with flexible environments. Thrust 2 focuses on developing curriculum and educational outreach for the commmunity. On the right, the figure shows the areas of medical robotics that are likely to be advanced as a result of our ongoing research. Figure 2 shows samples of the robot architectures considered in our research. These robots have been previously demonstrated for clinical applications requiring small size and dexterity. Our aim in this NSF career award is to answer questions about design, control, and models for force sensing and intelligent interaction with the environment while focusing on derivatives of this design and designs of parallel robots.

Some of the Ongoing Research Activity

A) Modeling and Optimal Design

One aspect of our reserach is finding design formalisms and modeling frameworks for continuum robots and flexible parallel robots. Figure 3 shows sample simulation of the reachable workspace of a three-segment snake robot where the end effector is maintained at a cosntant orientation. Questions of optimal dimensional anslysis, singularity, force sensing, kinematic attributes and perforamcne measures are being explored as part of this research.

B) Intrinsic Force Sensing

We previously investigated the capabilities of multi-backbone continuum robots to also act as sensors. Figure 4 shows a single segment snake robot palpating a silicone strip with embedded steel balls. It was shown in this work that a) there is a way to interpret the force sensing limitations of single segment snake robots using screw theory b) these robots can be very good sensors, b) we can use these robots to probe flexible media unknown environments for stiffness information and to localize tumors. Figure 1 shows some representative results. Our group has been extending the proof-of concept results in figure 3 to provide full capabilities of estimating the forces and moments of interaction with the environment by using more dexterous snakes that have two or more segments. Movie 1 demonstrates some of the work being done by our group on enabling intrinsic force sensing using continuum robots for exploration of stiffness of organs. The movie shows a three-segment snake robot "probing" an anatomical training model of the prostate and the resulting stiffness image of the process shows the hard nodule inside the prostate as shown in red.

Figure 4: Investigation of the intrinsic force sensing capabilities of multi-backbone continuum robots: (A) an interpretation of the sensible wrenches using a single segment snake robot. (B) A single segment robot probing a silicone strip with embedded steel balls. (C) A map of resulting sensed stiffness allows the robot to localize the mockup tumors (steel balls).

Movie 1 : A demonstration of a multi-segment snake palpating the surface of a training anatomical model of the prostate

C) Exploration

Our eventual aim is to enable exploration and safe interaction with flexible unknown anatomy. We are investigating new algorithms for exploring both shape and stiffness of flexible organs. This is a first stage of enabling intelligence by enabling robots to autonomously gather infomration about their environment. Figure 5 shows such an experimental setup where a Cartesian robot equipped with a 6-axis load cell is probing a mockup silicone model for infomation about stiffness. Our eventual goal is to enable this operation for snakes as in figure 4 and movie 1.

Figure 5: the two Major Thrusts of this NSF CAREER proposal

D) Control

Our research on control of these robots is focused on making strides in improving the performance of flexible robots and in supporting modalities of shared control with the surgeons whereby both the surgeon and an intelligent slave robot cooperate in performing surgical tasks. We would like to investigate new control strategies that use the intrinsic force sensing for imnproved accuracy and safety of interaction and for simplified telemanipulation.

Figure 6: A xPC Target real-time implementation of a telemanipulation framework for multi-segment continuum robots.

Figure 7: Block diagram of the proposed tiered mixed feedback controller architecture

E) Collision Detection and Estimation of Contact Location

Current algorithms do not address voluntary or involuntary contact of continuum robots with organs and tissues along the whole robotic structure. Sensing forces at the operational point is not enough when the workspace of the surgical robot is surrounded by delicate tissues and vital organs. Ideally, the entire robotic structure should safely move without damage to the anatomy via both automovements and safe interaction of any portion of the continuum robot with the anatomical environment. This basic observation motivates our investigation of new methods for contact detection along sinuous surgical slaves. The collision can be detected using two methods: Joint Force Deviation (JFD) and Fixed Centrode Deviation (FCD). The former uses the actuation forces and a static model of the robot to detect the collision. The latter compares the centrode of motion obtained by the kinematic model of the robot with the one obtained by a magnetic sensor placed at the robot's end-effector. The location of the contact can be estimated by finding the arc-length that minimizes the FCD error using a modified kinematic model that takes into account the constraint. Movie 2 shows a demonstration of collision detection using the FCD method. The continuum segment bends in a fixed plane.

E) Education

Our mission is to be able to translate the results of our reserach into new curriculum and new activities that engage various segments of the community as shown in Figure 1. Scientists often forget about the value of seeing the shine in the curious eyes of a young high school kid who is fascinated by what science and engineering means and how it may affect their life in so many ways he never appreciated before. We are actively seeking collaborations with local high schools for mentoring high-school students during the school year and during the summer. We also mentor science teachers on how to translate some of our reserach into high-school curriculum.

Figure 6: Sample outreach activity providing local NY middle school kids with hands-on
experience in using a telemanipulation system with a paralle robot for surgery. Activity organized with local organizatio HEAF.

Publications

• Bajo, A. & Simaan, N (2010). Finding Lost Wrenches: Using Continuum Robots for Contact Detection and Estimation of Contact Location. In 2010 IEEE International Conference on Robotics and Automation, pages 3666-3673. Anchorage, Alaska.
• Bajo, A. & Simaan, N. (2012). Kinematics-Based Detection and Localization of Constraints Along Continuum Robots. IEEE Transactions on Robotics, 28(2), 291-301.
• Bajo, A., Dharamsi, L., Netterville, J. L., Garrett, G. C. & Simaan, N (2013). Robotic-Assisted Micro-Surgery of Throat: the Trans-Nasal Approach. In Appected for publication in IEEE International Conference on Robotics and Automation (ICRA'2013).
• Bajo, A., Goldman, R. E. & Simaan, N (2011). Configuration and Joint Space Feedback for Improved Accuracy of Continuum Robots. In IEEE International Conference on Robotics and Automation, pages 2905-2912. Shanghai, China.
• Bajo, A., Goldman, R. E., Wang, L., Fowler, D. & Simaan, N (2012). Integration and Preliminary Evaluation of an Insertable Robotic Effectors Platform for Single Port Access Surgery. In International Conference on Robotics and Automation (ICRA’2012), pages 3381-3387.
• Bajo, A., Pickens, R. B., Herrell, D. S. & Simaan, N (2013). Constrained Motion Control of Multisegment Continuum Robots for Transurethral Bladder Resection and Surveillance. In Accepted for publication in IEEE International Conference on Robotics and Automation (ICRA'2013).
• Bhattacharyya, S. & Simaan, N (2013). Characterization of Constraints in Flexible Unknown Environments. In Accepted for publication in IEEE International Conference on Robotics and Automation (ICRA'2013).
• Ding, J., Xu, K., Goldman, R. E., Allen, P., Fowler, D. & Simaan, N (2010). Design, Simulation and Evaluation of Kinematic Alternatives for Insertable Robotic Effectors Platforms in Single Port Access Surgery. In Robotics, . & Automation (editors), pages 1053-1058. Anchorage, AK.
• Ding, J., Goldman, R. E., Xu, K., Fowler, D. & Simaan, N. (2012). Design and Coordination Kinematics of an Insertable Robotic Effectors Platform (IREP) for Single Port Access Surgery. IEEE/ASME Transactions on Mechatronics.
• Goldman, R. E., Bajo, A. & Simaan, N (2011). Compliant Motion Control for Continuum Robots with Intrinsic Actuation Sensing. In 2011 IEEE International Conference on Robotics and Automation, pages 1126-1132. Shanghai, China.
• Goldman, R. E., Bajo, A. & Simaan, N. (2013). Algorithms for Autonomous Exploration and Estimation in Compliant Environments. Robotica, 31(1), 71-88.
• Goldman, R. E., Bajo, A. & Simaan, N. (2014). Compliant Motion Control for Multisegment Continuum Robots With Actuation Force Sensing. IEEE Transactions on Robotics, 1-13.
• Pile, J., Cheung, M.-Y., Zhang, J. & Simaan, N (2011). Algorithms and Design Considerations for Robot Assisted Insertion of Perimodiolar Electrode Arrays. In 2011 IEEE International Conference on Robotics and Automation. Shanghai, China.
• Reiter, A., Goldman, R. E., Bajo, A., Iliopoulous, K., Simaan, N. & Allen, P (2011). A Learning Algorithm for Visual Pose Estimation of Continuum Robots. In 2011 IEEE International Conference on Intelligent Robots and Systems, pages 2390-2396. San Francisco, CA.
• Simaan, N., Bajo, A., Reiter, A., Long, W., Allen, P. & Fowler, D. (2013). Lessons learned using the insertable robotic effector platform (IREP) for single port access surgery. Journal of Robotic Surgery.
• Wang, L. & Simaan, N (2014). Investigation of Error Propagation in Multi-Backbone Continuum Robots. In Advances in Robot Kinematics, pages In press.
• Wei, W. & Simaan, N. (2012). Modeling, Force Sensing and Control of Flexible Cannulas for Micro-Stent Delivery. ASME Journal of Dynamic Systems, Measurement and Control, 134(4), 041004.
• Xu, K. & Simaan, N. (2008). An Investigation of the Intrinsic Force Sensing of Continuum Robots. IEEE Transaction on Robotics, 23(3).
• Xu, K. & Simaan, N. (2010a). Analytic Formulation for Kinematics, Statics and Shape Restoration of Multi-Backbone Continuum Robots via Elliptic Integrals. ASME Journal of Mechanisms and Robotics, 2(1), 011006-1,011006-.
• Xu, K. & Simaan, N. (2010b). Intrinsic Wrench Estimation and its Performance Index for Multi Segment Continuum Robots. IEEE Transaction on Robotics, 26(3), 555-561.
• Xu, K., Goldman, R. E., Ding, J., Allen, P., Fowler, D. & Simaan, N (2009). System Design of an Insertable Robotic Effector Platform for Single Port Access (SPA) Surgery. In 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5546-5552. St. Louis, USA.