Application of Benford's law: a valuable tool for detecting scientific papers with fabricated data? : A case study using proven falsified articles against a comparison group.

BACKGROUND: In naturally occurring numbers the frequencies of digits 1-9 in the leading position are counterintuitively distributed because the frequencies of occurrence are unequal. Benford-Newcomb's law describes the expected distribution of these frequencies. It was previously shown that known fraudulent articles consistently violated this law.

OBJECTIVE: To compare the features of 12 known fraudulent articles from a single Japanese author to the features of 13 articles in the same research field from other Japanese authors, published during the same time period and identified with a Medline database search.

RESULTS: All 25 articles were assessed to determine whether the data violated the law. Formulas provided by the law were used to determine the frequencies of occurrence for the first two leading digits in manually extracted numbers. It was found that all the known fraudulent papers violated the law and 6 of the 13 articles used for comparison followed the law. Assuming that the articles in the comparison group were not falsified or fabricated, the sensitivity of assessing articles with Benford-Newcomb's law was 100% (95% confidence interval CI: 73.54-100%) but the specificity was only 46.15% (95% CI: 19.22-74.87%) and the positive predictive value was 63.16% (95% CI: 38.36-83.71%).

CONCLUSION: All 12 of the known falsified articles violated Benford-Newcomb's law, which indicated that this analysis had a high sensitivity. The low specificity of the assessment may be explained by the assumptions made about the articles identified for comparison. Violations of Benford-Newcomb's law about the frequencies of the leading digits cannot serve as proof of falsification but they may provide a basis for deeper discussions between the editor and author about a submitted work.