Interobserver agreement in describing the ultrasound appearance of adnexal masses and in calculating the risk of malignancy using logistic regression models.

PURPOSE: To estimate interobserver agreement with regard to describing adnexal masses using the International Ovarian Tumor Analysis (IOTA) terminology and the risk of malignancy calculated using IOTA logistic regression models LR1 and LR2, and to elucidate what explained the largest interobserver differences in calculated risk of malignancy.

EXPERIMENTAL DESIGN: One hundred and seventeen women with adnexal masses were examined with transvaginal gray scale and power Doppler ultrasound by two independent experienced sonologists who described the masses using IOTA terminology. The risk of malignancy was calculated using LR1 and LR2. A predetermined risk of malignancy cutoff of 10% indicated malignancy.

RESULTS: There were 94 benign, four borderline, and 19 invasively malignant tumors. There was substantial variability between the two sonologists in measurement results and some variability in assessment of categorical variables (agreement 40%-98%, Kappa 0.30-0.91). Interobserver agreement when classifying tumors as benign or malignant was 84% (98/117), Kappa 0.68 for LR1, and for LR2 85% (99/117), Kappa 0.68. When using LR1 and LR2, the interobserver difference in calculated risk was ≥ 25 percentage units in 9% (11/117) and 12% (14/117) of tumors, respectively. Differences in assessment of wall irregularity, acoustic shadowing, color score, and color flow in papillary projections explained most of these largest differences.

CONCLUSIONS: Interobserver agreement in classifying tumors as benign or malignant using the risk of malignancy cutoff of 10% for LR1 and LR2 was good. However, because risk estimates may differ substantially between sonologists, one should be cautious with using the risk value for counseling patients about their individual risk.