We have located links that may give you full text access.
Establishing Validity Evidence for an Operative Performance Rating System for Plastic Surgery Residents.
Journal of Surgical Education 2018 September 23
OBJECTIVE: The aim of this study was to describe an operative performance rating system for plastic surgery residents and provide validity evidence for the instrument.
METHODS: Three plastic surgery residents (PGY levels 1, 5, and 6) from Southern Illinois University School of Medicine (SIUSOM) performed a carpal tunnel release with audio video recording. The 3 videos were reviewed by 8 expert hand surgeons and 3 SIUSOM faculty using the operative performance rating system instrument to assess resident operative performance. Validity evidence including content, internal structure, and relationship to other variables was collected.
RESULTS: Inter-rater reliability was consistently fair to moderate (weighted Cohen's Kappa 0.44-0.84 for experts, 0.24-0.55 for SIUSOM raters), and all assessment items were highly correlated (Cronbach's alpha of 0.9867). Local SIUSOM faculty routinely demonstrated higher overall scores for PGY 1 and PGY 6 residents compared to expert raters.
CONCLUSIONS: Although limited by small numbers, this pilot study suggests that potential bias based upon PGY year, identity, and performance history may exist and independent assessment by unbiased raters or comparison to national operative norms may be valuable. Our study provides baseline validity evidence for a resident operative performance assessment tool that can be integrated into practice in plastic surgery training programs.
ACGME COMPETENCIES: Patient Care, Practice-Based Learning and Improvement.
METHODS: Three plastic surgery residents (PGY levels 1, 5, and 6) from Southern Illinois University School of Medicine (SIUSOM) performed a carpal tunnel release with audio video recording. The 3 videos were reviewed by 8 expert hand surgeons and 3 SIUSOM faculty using the operative performance rating system instrument to assess resident operative performance. Validity evidence including content, internal structure, and relationship to other variables was collected.
RESULTS: Inter-rater reliability was consistently fair to moderate (weighted Cohen's Kappa 0.44-0.84 for experts, 0.24-0.55 for SIUSOM raters), and all assessment items were highly correlated (Cronbach's alpha of 0.9867). Local SIUSOM faculty routinely demonstrated higher overall scores for PGY 1 and PGY 6 residents compared to expert raters.
CONCLUSIONS: Although limited by small numbers, this pilot study suggests that potential bias based upon PGY year, identity, and performance history may exist and independent assessment by unbiased raters or comparison to national operative norms may be valuable. Our study provides baseline validity evidence for a resident operative performance assessment tool that can be integrated into practice in plastic surgery training programs.
ACGME COMPETENCIES: Patient Care, Practice-Based Learning and Improvement.
Full text links
Related Resources
Get seemless 1-tap access through your institution/university
For the best experience, use the Read mobile app
All material on this website is protected by copyright, Copyright © 1994-2024 by WebMD LLC.
This website also contains material copyrighted by 3rd parties.
By using this service, you agree to our terms of use and privacy policy.
Your Privacy Choices
You can now claim free CME credits for this literature searchClaim now
Get seemless 1-tap access through your institution/university
For the best experience, use the Read mobile app