A Training Program Using an Audience Response System to Calibrate Dental Faculty Members Assessing Student Clinical Competence.

The aim of this study was to assess the effectiveness of calibration training of departmental faculty and competency graders using an audience response system on operative dentistry concepts across 12 months. The training sessions were designed to further solidify the process and equilibration of clinical opinions among faculty members and provide a more calibrated grading assessment during patient care for student performance feedback. Four (quarterly) calibration sessions occurred over 12 months in 2015. The first session was considered the baseline (control value) for this study. Pre- and post-calibration interrater agreement was assessed. Additionally, a pre and post assessment with ten Likert-scale questions was used to measure students' perceptions of instructional consistency. The results showed that a statistically significant increase in conceptual knowledge scores occurred for both departmental faculty members and competency graders across each of the four sessions (one-factor ANOVA; p<0.05). Interrater reliability agreement also significantly improved for both department faculty members and competency graders' clinical assessments over 12 months of implementation (Cohen's Kappa; p<0.05). There was a statistically significant increase in positive student perceptions on all ten questions (dependent t-test; p<0.05). Implementation of an audience response system for departmental and competency graders was found to be effective in facilitating a discussion forum, calibrating clinical assessments, and improving student perceptions. The positive results from this study support the value of dental schools' introducing faculty development programs to ensure consistent instruction for assessing dental student competence.