Standard setting of objective structured practical examination by modified Angoff method: A pilot study.

BACKGROUND: The undergraduate curriculum at our institution is divided system-wise into four blocks, each block ending with theory and objective structured practical examination (OSPE). The OSPE in Physiology consists of 12 stations, and a conventional minimum score to qualify is 50%. We aimed to incorporate standard setting using the modified Angoff method in OSPE to differentiate the competent from the non-competent student and to explore the possibility of introducing standard setting in Physiology OSPE at our institution.

METHODS: Experts rated the OSPE using the modified Angoff method to obtain the standard set cut-off in two of the four blocks. We assessed the OSPE marks of 110 first year medical students. Chi-square test was used to compare the number of students who scored less than standard set cut-off and conventional cut-off; correlation coefficient was used to assess the relation between OSPE and theory marks in both blocks. Feedback was obtained from the experts.

RESULTS: The standard set was 62% and 67% for blocks II and III, respectively. The use of standard set cut-off resulted in 16.3% (n=18) and 22.7% (n=25) students being declared unsuccessful in blocks II and III, respectively. Comparison between the number, who scored less than standard set and conventional cut-off was statistically significant (p=0.001). The correlation coefficient was 0.65 (p=0.003) and 0.52 (p<0.001) in blocks II and III, respectively. The experts welcomed the idea of standard setting.

CONCLUSION: Standard setting helped in differentiating the competent from the non-competent student, indicating that standard setting enhances the quality of OSPE as an assessment tool.