Objective Comparison Using Guideline-based Query of Conventional Radiological Reports and Structured Reports.

BACKGROUND: This feasibility study of text-mining-based scoring algorithm provides an objective comparison of structured reports (SR) and conventional free-text reports (cFTR) by means of guideline-based key terms. Furthermore, an open-source online version of this ranking algorithm was provided with multilingual text-retrieval pipeline, customizable query and real-time-scoring.

MATERIALS AND METHODS: Twenty-five patients with suspected stroke and magnetic resonance imaging were re-assessed by two independent/blinded readers [inexperienced: 3 years; experienced >6 years/Board-certified). SR and cFTR were compared with guideline-query using the cosine similarity score (CSS) and Wilcoxon signed-rank test.

RESULTS: All pathological findings (18/18) were identified by SR and cFTR. The impressions section of the SRs of the inexperienced reader had the highest median (0.145) and maximal (0.214) CSS and were rated significantly higher (p=2.21×10-5 and p=1.4×10-4 , respectively) than cFTR (median=0.102). CSS was robust to variations of query.

CONCLUSION: Objective guideline-based comparison of SRs and cFTRs using the CSS is feasible and provides a scalable quality measure that can facilitate the adoption of structured reports in all fields of radiology.