Standard Setting for Clinical Performance of Basic Perioperative Transesophageal Echocardiography: Moving beyond the Written Test.

BACKGROUND: Credible methods for assessing competency in basic perioperative transesophageal echocardiography examinations have not been reported. The authors' objective was to demonstrate the collection of real-world basic perioperative transesophageal examination performance data and establish passing scores for each component of the basic perioperative transesophageal examination, as well as a global passing score for clinical performance of the basic perioperative transesophageal examination using the Angoff method.

METHODS: National Board of Echocardiography (Raleigh, North Carolina) advanced perioperative transesophageal echocardiography-certified anesthesiologists (n = 7) served as subject matter experts for two Angoff standard-setting sessions. The first session was held before data analysis, and the second session for calibration of passing scores was held 9 months later. The performance of 12 anesthesiology residents was assessed via the new passing score grading system.

RESULTS: The first standard-setting procedure resulted in a global passing score of 63 ± 13% on a basic perioperative transesophageal examination. The global passing score from the second standard-setting session was 73 ± 9%. Three hundred seventy-one basic perioperative transesophageal examinations from 12 anesthesiology residents were included in the analysis and used to guide the second standard-setting session. All residents scored higher than the global passing score from both standard-setting sessions.

CONCLUSIONS: To the authors' knowledge, this is the first demonstration that the collection of real-world anesthesia resident basic perioperative transesophageal examination clinical performance data is possible and that automated grading for competency assessment is feasible. The authors' findings demonstrate at least minimal basic perioperative transesophageal examination clinical competency of the 12 residents.