Statistical controversies in clinical research: publication bias evaluations are not routinely conducted in clinical oncology systematic reviews.

Background: Publication bias is an over-representation of statistically significant results in the published literature and may exaggerate summary effect estimates in oncology systematic reviews. Omitting non-significant results in systematic reviews may therefore affect clinical decision-making. We investigate ways that systematic reviewers attempted to limit publication bias during the search process as well as the statistical methods used to evaluate it. For a subset of reviews not reporting publication bias evaluations, we carried out our own assessments for publication bias to determine its likelihood among these reviews.

Design: We examined systematic reviews from the top five highest impact factor oncology journals published between 2007 and 2015. Systematic reviews were screened for eligibility and qualifying reviews (n = 182) were coded for relevant publication bias study characteristics by two authors. A re-analysis of reviews not initially evaluating for publication bias was carried out using Egger's regression, trim-and-fill, and selection models.

Results: Of the 182 systematic reviews, roughly half carried out a hand search to locate additional studies. Conference abstracts were the most commonly reported form of gray literature, followed by clinical trials registries. Fifty-one reviews reported publication bias evaluations. The most common method was the funnel plot (80%, 41/51) followed by Egger's regression (59%, 30/51) and Begg's test (43%, 22/51). Our publication bias evaluations on non-reporting reviews suggest that the degree of publication bias depends on the method employed.

Conclusion: Our study shows publication bias assessments are not frequently used in oncology systematic reviews. Furthermore, evidence of publication bias was found in a subset of non-reporting reviews. Systematic reviewers in oncology are encouraged to conduct such analyses when appropriate and to employ more robust methods for both mitigating and evaluating publication bias.