We have located links that may give you full text access.
Effects and Unforeseen Consequences of Accessing References on a Maintenance of Certification Examination: Findings From a National Study.
Academic Medicine 2017 October 13
PURPOSE: Increasing criticism of maintenance of certification (MOC) examinations has prompted certifying boards to explore alternative assessment formats. The purpose of this study was to examine the effect of allowing test takers to access reference material while completing their MOC Part III standardized examination.
METHOD: Item response data were obtained from 546 physicians who completed a medical subspecialty MOC examination between 2013 and 2016. To investigate if accessing references was related to better performance, an analysis of covariance was conducted on the MOC examination scores with references (access or no access) as the between-groups factor and scores from the physicians' initial certification examination as a covariate. Descriptive analyses were conducted to investigate how the new feature of accessing references influenced time management within the test day.
RESULTS: Physicians scored significantly higher when references were allowed (mean = 534.44, standard error = 6.83) compared to when they were not (mean = 472.75, standard error = 4.87), F(1, 543) = 60.18, P < .001, ω = 0.09. However, accessing references affected pacing behavior; physicians were 13.47 times more likely to finish with less than a minute of test time remaining per section when reference material was accessible.
CONCLUSIONS: Permitting references caused an increase in performance, but also a decrease in the perception that the test has sufficient time limits. Implications for allowing references are discussed, including physician time management, impact on the construct assessed by the test, and the importance of providing validity evidence for all test design decisions.
METHOD: Item response data were obtained from 546 physicians who completed a medical subspecialty MOC examination between 2013 and 2016. To investigate if accessing references was related to better performance, an analysis of covariance was conducted on the MOC examination scores with references (access or no access) as the between-groups factor and scores from the physicians' initial certification examination as a covariate. Descriptive analyses were conducted to investigate how the new feature of accessing references influenced time management within the test day.
RESULTS: Physicians scored significantly higher when references were allowed (mean = 534.44, standard error = 6.83) compared to when they were not (mean = 472.75, standard error = 4.87), F(1, 543) = 60.18, P < .001, ω = 0.09. However, accessing references affected pacing behavior; physicians were 13.47 times more likely to finish with less than a minute of test time remaining per section when reference material was accessible.
CONCLUSIONS: Permitting references caused an increase in performance, but also a decrease in the perception that the test has sufficient time limits. Implications for allowing references are discussed, including physician time management, impact on the construct assessed by the test, and the importance of providing validity evidence for all test design decisions.
Full text links
Related Resources
Trending Papers
Review article: Recent advances in ascites and acute kidney injury management in cirrhosis.Alimentary Pharmacology & Therapeutics 2024 March 26
Get seemless 1-tap access through your institution/university
For the best experience, use the Read mobile app
All material on this website is protected by copyright, Copyright © 1994-2024 by WebMD LLC.
This website also contains material copyrighted by 3rd parties.
By using this service, you agree to our terms of use and privacy policy.
Your Privacy Choices
You can now claim free CME credits for this literature searchClaim now
Get seemless 1-tap access through your institution/university
For the best experience, use the Read mobile app