Development and Validation of an Objective Scoring Tool for Robot-Assisted Radical Prostatectomy: Prostatectomy Assessment and Competency Evaluation.

PURPOSE: Comprehensive training and skill acquisition by urological surgeons are vital to optimize surgical outcomes and patient safety. We sought to develop and validate PACE (Prostatectomy Assessment and Competence Evaluation), an objective and procedure specific tool to assess the quality of robot-assisted radical prostatectomy.

MATERIALS AND METHODS: Development and content validation of PACE was performed by deconstructing robot-assisted radical prostatectomy into 7 key domains utilizing the Delphi methodology. Reliability and construct validation were then assessed using de-identified videos performed by practicing surgeons and fellows. Consensus for each domain was defined as achieving a content validity index of 0.75 or greater. Reliability was assessed by the intraclass correlation and construct validation using a mixed linear model accounting for multiple ratings on the same video.

RESULTS: After 3 rounds consensus was reached on wording, relevance of the skills assessed and concordance between the score assigned and the skill assessed. An intraclass correlation of 0.4 or greater was achieved for all domains. The expert group outperformed trainees in all domains but reached statistical significance in bladder drop (4.5 vs 3.4, p = 0.002), preparation of the prostate (4.4 vs 3.2, p <0.0001), seminal vesicle and posterior plane dissection (8.3 vs 6.8, p = 0.03), and neurovascular bundle preservation (4.1 vs 2.4, p <0.0001). Limitations included the lack of assessment of other key skills such as communication and decision making.

CONCLUSIONS: PACE is a structured, procedure specific and reliable tool that objectively measures surgical performance during robot-assisted radical prostatectomy. It can differentiate different levels of expertise and provide structured feedback to customize training and surgical quality improvement.