Pilot study of a new comprehensive radiology report categorization (RADCAT) system in the emergency department.

PURPOSE: The purpose of this study was to describe a new, broadly applicable radiology report categorization (RADCAT) system that was developed collaboratively between radiologists and emergency department (ED) physicians, and to establish its usability and performance by interobserver variation.

METHODS: In collaboration with our ED colleagues, we developed the RADCAT system for all imaging studies performed in our level-1 trauma center, including five categories that span the spectrum of normal through emergent life-threatening findings. During a pilot phase, four radiologists used the system real-time to categorize a minimum of 400 reports in the ED. From this pool of categorized studies, 58 reports were then selected semi-randomly, de-identified, stripped of their original categorization, and recategorized based on the narrative radiology report by 12 individual reviewers (6 radiologists, and 6 ED physicians). Interobserver variation between all reviewers, radiologists only, and ED physicians only was calculated using Cohen's Kappa statistic and Kendall's coefficient of concordance.

RESULTS: Altogether, agreement among radiologists and ED physicians was substantial (κ = 0.73, p < 0.0001) and agreement for each category was substantial (all κ > 0.60, p < 0.0001). The lowest agreement was observed with RADCAT-3 (κ > 0.61, p < 0.0001) and the highest agreement with RADCAT-1 (κ > 0.85, p < 0.0001). A high trend in agreement was observed for radiologists and ED physicians and their combination (all W > 0.90, p < 0.0001).

CONCLUSIONS: Our RADCAT system is understandable between radiologists and ED physicians for categorizing a wide range of imaging studies, and warrants further assessment and validation. Based upon these pilot results, we plan to adopt this RADCAT scheme and further assess its performance.