Objective Assessment of Robotic Surgical Technical Skill: A Systematic Review.

PURPOSE: Robotic surgeries, especially in urology, have grown exponentially during the last decade. Various skills assessment tools have been developed. We reviewed the current status, the current challenges and the future needs of robotic evaluations with a focus on urological applications.

MATERIALS AND METHODS: According to PRISMA (Preferred Reporting Items for Systematic Review and Meta-Analysis) criteria 2 paired investigators screened the PubMed®, Scopus® and Web of Science® databases for all full text, English language articles published between 2006 and 2018 using the query (evaluation OR assessment) AND (robot-assisted surgery OR robotic surgery) AND (surgical performance OR surgical skill) AND training. The research design, validity and reliability of each study were ascertained and analyzed.

RESULTS: A total of 259 studies were identified, of which 109 were included in the final analysis. We grouped the studies into 2 categories, including manual and automated assessments. Manual evaluation included global skill, procedure specific and error based assessments. For automated assessment we summarized evaluations derived from robotic instrument kinematic tracking data, systems events and surgical video data, and we explored those associations with various domains by manual evaluation. We further reviewed the current progress in automated surgical segmentation and skill evaluation with machine learning and deep learning. Concerns remain regarding efficient and effective surgeon training and credentialing.

CONCLUSIONS: No universally accepted robotic skills assessment currently exists. The purpose of assessment (training or credentialing) may dictate whether manual or automated surgeon assessment is more suitable. Moving forward, assessment tools must be objective and efficient to facilitate the training and credentialing of competent surgeons.