Comparison of Automated Posttonsillectomy Bleed Capture With Self-report.

Importance: Tonsillectomy is one of the most common procedures performed by otolaryngologists and is associated with postoperative bleeding. Bleed rates are usually monitored by self-report.

Objective: To evaluate whether using automated capture and reporting of pediatric posttonsillectomy bleeding is feasible and accurate compared with traditional self-reporting by the surgical team.

Design, Setting, and Participants: An automated complication-reporting algorithm was designed to query the local health information exchange and then tested against self-reported tonsillectomy complication data collected from January 1, 2014, through December 31, 2015, at a tertiary pediatric hospital. The algorithm identified patients undergoing tonsillectomy and searched their postoperative encounters for a hand-selected set of diagnosis codes from the International Classification of Diseases, Ninth Revision and International Statistical Classification of Diseases and Related Health Problems, Tenth Revision and free-text words to identify complication events. Five months of the 2014-2015 data set were used to help design the algorithm. Data from the remaining 19 months were compared with self-reported complications.

Main Outcomes and Measures: Automated system findings compared with self-reported bleeding events.

Results: During the 19-month period, 1017 tonsillectomies were performed. We compared the algorithm's effectiveness in finding tonsillectomy and adenotonsillectomy procedures for the evaluated surgeons with the hand-reviewed master tonsillectomy list. The algorithm reported 51 false-positive (5.01% missed) and 74 false-negative (7.28% misidentified) procedures. The algorithm agreed with self-report for 986 tonsillectomies and disagreed on 31 cases (3.05%) (κ = 0.69; 95% CI, 0.66-0.73). The algorithm was found to be sensitive to correctly identifying 60.53% (95% CI, 48.63%-71.34%) of tonsillectomies as having bleeding complications, with a specificity of 98.30% (95% CI, 97.19%-98.99%).

Conclusions and Relevance: Capture of posttonsillectomy bleeding is possible through an automatic search of the medical record, although the algorithm will require continued refinement. Leveraging health information exchange data increases the possibilities of capturing complications at hospitals outside the local health system. Use of these algorithms will allow repeatable automated feedback to be provided to surgeons on a cyclical basis.