Demonstrating Construct Validity of the American Board of Physical Medicine and Rehabilitation Part I Examination: An Analysis of Dimensionality.

BACKGROUND: Ideally, high-stakes examinations assess 1 dimension of medical knowledge to produce precise estimates of a candidate's performance. It has not been reported whether the American Board of Physical Medicine and Rehabilitation Part 1 Certification Examination (ABPMR-CE-1) is unidimensional or not.

OBJECTIVE: To examine the ABPMR-CE-1 to measure how many dimensions it assesses.

DESIGN: Retrospective observational study.

SETTING: We assessed examination results from the 2015 ABPMR-CE-1.

PARTICIPANTS: A total of 489 deidentified candidates taking the 2015 ABPMR-CE-1.

METHODS: A 1-parameter Item Response Theory (IRT) measurement model was utilized. A Principal Components Analysis (PCA) of standardized residual correlations was used to detect multidimensionality.

MAIN OUTCOME MEASURE: Number of primary dimensions reflected in the 325 test questions.

RESULTS: The results of the dimensionality analysis indicated that the ABPMR-CE-1 examination is highly unidimensional from a psychometric perspective. Expert content review of the substantive content of small contrasting clusters of questions provided additional assurance of the unidimensional nature of the examination.

CONCLUSIONS: The ABPMR-CE-1 appears indeed to measure a single construct, which suggests a sound structure of the examination. It closely approximates the assumption of statistical unidimensionality.

LEVEL OF EVIDENCE: Not applicable.