JOURNAL ARTICLE
RESEARCH SUPPORT, NON-U.S. GOV'T
Add like
Add dislike
Add to saved papers

How Many Observations are Needed to Assess a Surgical Trainee's State of Operative Competency?

Annals of Surgery 2019 Februrary
OBJECTIVE: To establish the number of operative performance observations needed for reproducible assessments of operative competency.

BACKGROUND: Surgical training is transitioning from a time-based to a competency-based approach, but the number of assessments needed to reliably establish operative competency remains unknown.

METHODS: Using a smart phone based operative evaluation application (SIMPL), residents from 13 general surgery training programs were evaluated performing common surgical procedures. Two competency metrics were investigated separately: autonomy and overall performance. Analyses were performed for laparoscopic cholecystectomy performances alone and for all operative procedures combined. Variance component analyses determined operative performance score variance attributable to resident operative competency and measurement error. Generalizability and decision studies determined number of assessments needed to achieve desired reliability (0.80 or greater) and determine standard errors of measurement.

RESULTS: For laparoscopic cholecystectomy, 23 ratings are needed to achieve reproducible autonomy ratings and 17 ratings are needed to achieve reproducible overall operative performance ratings. For the undifferentiated mix of procedures, 60 ratings are needed to achieve reproducible autonomy ratings and 40 are needed for reproducible overall operative performance ratings.

CONCLUSION: The number of observations needed to achieve reproducible assessments of operative competency far exceeds current certification requirements, yet remains an important and achievable goal. Attention should also be paid to the mix of cases and raters in order to assure fair judgments about operative competency and fair comparisons of trainees.

Full text links

We have located links that may give you full text access.
Can't access the paper?
Try logging in through your university/institutional subscription. For a smoother one-click institutional access experience, please use our mobile app.

For the best experience, use the Read mobile app

Mobile app image

Get seemless 1-tap access through your institution/university

For the best experience, use the Read mobile app

All material on this website is protected by copyright, Copyright © 1994-2024 by WebMD LLC.
This website also contains material copyrighted by 3rd parties.

By using this service, you agree to our terms of use and privacy policy.

Your Privacy Choices Toggle icon

You can now claim free CME credits for this literature searchClaim now

Get seemless 1-tap access through your institution/university

For the best experience, use the Read mobile app