Login

If you are a member or an alumnus of A.R.M.A. you may login to submit articles and contribute to the website.

Complementary Situational Awareness for Human-Robot Partnerships

Project Title: NRI Large: Collaborative Research: Complementary Situational Awareness for Human-Robot Partnerships

NSF award number: IIS-1327566

 

NRI

Proposed framework for robot-human
complimentary situational awareness (CSA)

This is a new 5-year collaborative project involving three collborative teams at Vanderbilt, Carnegie Mellon, and Johns Hopkins University. The Principal investigators on this grant are Dr. Nabil Simaan (Vanderbilt), Dr. Howie Choset (CMU) and Dr. Russell H. Taylor (JHU).

 

The grant consists of three partner institutions contributing to laying the foundations to a new concept in robotics that we call Complementary Situational Awareness. Robots have been primarily used to augment human skill during manipulation tasks (e.g. for surgical applications, telemanipulation in hazardous environments) and in some cases to augment sensory presence (e.g. by providing force feedback to surgeons in cases where forces are below humanly perceptible thresholds). In our new approach robots will augment the human user not only in manipulation but also in understanding of the task and in action planning and execution. The idea is that the robots in some cases can sense things beyond human perception and this information may be used by the robot controller to create a model of the environment shape and the interaction characteristics. This robot situational awareness is then used to augment user/surgeon skill and situational awareness for carrying out complex tasks.

 

More details about this research will be disseminated through our shared project website. Additional details are available on NSF website.

 

In the coming years we will be updating this page with a list of publications and excerpts of research progress. Please check this page again in the future.

Poster of the planned reserach was presented in 2013 NRI Awardee Conference. A copy of the PDF poster is available here. 

 

 

Publications

 

  1. N. Zevallos, R. A. Srivatsan, H. Salman, L. Li, J. Qian, S. Saxena, M. Xu, K. Patath and H. Choset “A Real-time Augmented Reality Surgical System for Overlaying Stiffness Information” in Proceedings of Robotics: Science and Systems, June 2018.
  2. N. Zevallos, R. A. Srivatsan, H. Salman, L. Li, J. Qian, S. Saxena, M. Xu, K. Patath and H. Choset, “A surgical system for automatic registration, stiffness mapping and dynamic image overlay,” International Symposium on Medical Robotics (ISMR), Georgia Tech, Atlanta, March 2018.
  3. Rangaprasad Arun Srivatsan, Mengyun Xu, Nicolas Zevallos, and Howie Choset, Probabilistic pose estimation using a Bingham distribution-based linear filter, The International Journal of Robotics Research. First Published June 25, 2018. https://doi.org/10.1177/0278364918778353
  4. Ayvali, H. Salman, and H. Choset, “Trajectory-Optimized Sensing for Active Search of Tissue Abnormalities in Robotic Surgery,” International Conference on Robotics and Automation (ICRA), pp. 5356-5363, May 2018.
  5. R. A. Srivatsan, P. Vagdargi and H. Choset “Sparse Point Registration,” The 18th International Symposium on Robotics Research Dec.11-14, 2017, Puerto Varas, Chile.
  6. R. A. Srivatsan, “Probabilistic Approaches for Pose Estimation,” CMU-RI-TR-18-24, PhD Thesis, Carnegie Mellon University 2018
  7. Simaan, N., Yasin, R., Wang, L., "Medical Technologies and Challenges of Robot-Assisted Minimally Invasive Intervention and Diagnostics", Annual Review of Control, Robotics, and Autonomous Systems, Vol. 1, No. 1, pp. 465-490, doi: 10.1146/annurev-control-060117-104956, 2018.
  8. Abah, C., Orekhov, A., Simaan, N., "Design Considerations and Redundancy Resolution for Variable Geometry Continuum Robots", IEEE International Conference on Robotics & Automation (ICRA), 2018.
  9. Orekhov, A., Abah, C., Simaan, N., "Snake-Like Robots for Minimally Invasive, Single Port, and Intraluminal Surgeries", Encyclopedia of Medical Robotics, Jaydev Desai & Rajni Patel Eds., World Scientific Publishing, 2018.
  10. Yasin, R. M., Wang, L., Abah, C., Simaan, N., "Using Continuum Robots for Force-controlled Semi-autonomous Organ Exploration and Registration", IEEE International Symposium on Medical Robotics (ISMR), March 1-3, Atlanta, GA, doi: 10.1109/ISMR.2018.8333309,2018. (finalist for best student paper award).
  11. Chalasani, R. M. Yasin, L. Wang, N. Simaan, P. Kazanzides and R.H. Taylor “Constrained Semi-autonomous Telemanipulated Palpation with Assistive Virtual Fixtures”, Hamlyn Symposium on Medical Robotics, London, UK, 2017.
  12. Preetham Chalasani, Anton Deguet, Peter Kazanzides, and Russell H Taylor. A computational framework for complementary situational awareness (csa) in surgical assistant robots. In 2018 Second IEEE International Conference on Robotic Computing (IRC), pages 9–16. IEEE, 2018.
  13. Preetham Chalasani, Long Wang, Rashid Yasin, Nabil Simaan, and Russell H Taylor. Preliminary evaluation of an online estimation method for organ geometry and tissue stiffness. IEEE Robotics and Automation Letters, 3(3):1816–1823, 2018.
  14. Wang L, Chen Z, Chalasani P, et al. Force-Controlled Exploration for Updating Virtual Fixture Geometry in Model-Mediated Telemanipulation. J Mech Robot. 2017;9(2):021010. doi:10.1115/1.4035684.
  15. Roy R., Wang L., Simaan N. Modeling and Estimation of Friction, Extension, and Coupling Effects in Multisegment Continuum Robots. IEEE/ASME Trans Mechatronics. 2017;22(2):909-920. doi:10.1109/TMECH.2016.2643640.
  16. Rangaprasad Arun Srivatsan and Prasad Vagdargi and Howie Choset, “Sparse Point Registration”, The International Symposium on Robotics Research (ISRR), 2017
  17. Li, B. Yu, C. Yang, P. Vagdargi, R. A. Srivatsan and H. Choset, “Development of an Inexpensive Bi-axial Force Sensor for Minimally Invasive Surgery”, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, Canada, September 24–28, pp. 906-913, 2017
  18. Ayvali, A. Ansari, L. Wang, N. Simaan and H. Choset, "Utility-Guided Palpation for Locating Tissue Abnormalities," in IEEE Robotics and Automation Letters, vol. 2, no. 2, pp. 864-871, April 2017. doi: 10.1109/LRA.2017.2655619.
  19. Rangaprasad Arun Srivatsan and Prasad Vagdargi and Nicolas Zevallos and Howie Choset, “Multimodal Registration Using Stereo Imaging and Contact Sensing”, Robotics: Science and Systems, workshop on 'Revisiting Contact - Turning a Problem into a Solution, 2017.
  20. Ayvali, H. Salman and H. Choset, "Ergodic coverage in constrained environments using stochastic trajectory optimization," 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Vancouver, BC, 2017, pp. 5204-5210. doi: 10.1109/IROS.2017.8206410
  21. Ayvali, E., Srivatsan, R. A., Wang, L., Roy, R., Simaan, N. & Choset, H (2016). Using Bayesian Optimization to Guide Probing of a Flexible Environment for Simultaneous Registration and Stiffness Mapping. In International Conference on Robotics and Automation (ICRA’2016), pages 931 - 936.
  22. Bajo, A. & Simaan, N. (2016). Hybrid Motion/Force Control of Multi-Backbone Continuum Robots. International Journal of Robotics Research, 35(4), 422-434.
  23. Chalasani, P., Wang, L., Roy, R., Simaan, N. & Taylor, R. H (2016). Concurrent Nonparametric Estimation of Organ Geometry and Tissue Stiffness Using Continuous Adaptive Palpation. In International Conference on Robotics and Automation (ICRA’2016), pages 4164-4171.
  24. Zihan Chen ; Anand Malpani ; Preetham Chalasani ; Anton Deguet ; S. Swaroop Vedula, Peter Kazanzides, Russell H. Taylor, "Virtual fixture assistance for needle passing and knot tying," 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Daejeon, 2016, pp. 2343-2350. doi: 10.1109/IROS.2016.7759365.
  25. Roy, R., Wang, L. & Simaan, N (2016b). Investigation of effects of dynamics on intrinsic wrench sensing in continuum robots. In International Conference on Robotics and Automation (ICRA’2016), pages 2052-2059.
  26. Srivatsan, R. A. & Choset, H. (2016). Multiple Start Branch and Prune Filtering Algorithm for Nonconvex Optimization The 12th International Workshop on The Algorithmic Foundations of Robotics.
  27. R. A. Srivatsan, E. Ayvali, L. Wang, R. Roy, N. Simaan and H. Choset, "Complementary model update: A method for simultaneous registration and stiffness mapping in flexible environments," 2016 IEEE International Conference on Robotics and Automation (ICRA), Stockholm, 2016, pp. 924-930. doi: 10.1109/ICRA.2016.7487224.
  28. Srivatsan, R. A., Rosen, G. T., Ismail, F. N. & Choset, H. (2016). Estimating SE(3) elements using a dual quaternion based linear Kalman filter. Robotics: Science and Systems, July, 2016.
  29. Srivatsan, R. A., Wang, L., Ayvali, E., Simaan, N. & Choset, H. (2016). Simultaneous Registration and Stiffness mapping of a Flexible Environment using Stiffness and Geometric Prior. The Hamlyn Symposium on Medical Robotics (2016), July, 2016.
  30. Wang, L., Chen, Z., Chalasani, P., Pile, J., Kazanzides, P., Taylor, R. H. et al (2016a). Updating Virtual Fixtures From Exploration Data in Force-controlled Model-based Telemanipulation. In ASME 2016 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference, pages V05AT07A031-V05AT07A. Charlotte, NC, USA.
  31. Simaan, N., Taylor, R. H. & Choset, H. (2015). Intelligent Surgical Robots with Situational Awareness: from Good to Great Surgeons. ASME Dynamic Systems Magazine (3), 2.
  32. Wang, L. & Simaan, N (2014). Investigation of Error Propagation in Multi-Backbone Continuum Robots. In Advances in Robot Kinematics, pages 385-394. Springer International Publishing.
  33. Srivatsan, R. A. & Choset, H. (2014). Using Lie algebra for shape estimation of medical snake robots. IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014), 2014, 3483-3488.