Challenges to generating evidence-informed policy and the role of systematic reviews and (perceived) conflicts of interest.

BACKGROUND: Multiple efforts to generate evidence-informed policy have attempted to teach policymakers how to understand and apply scientific research findings in their decision-making. These efforts have had limited success, because policymakers generally do not understand scientific methods.

OBJECTIVE: We piloted efforts to teach policy intermediaries - specifically consumer advocacy groups - how to understand and apply health research, anticipating that they might offer such evidence to policymakers in more accessible forms.

DESIGN: Four workshops focusing on research design and methods were conducted with consumer advocacy groups in 2010. We coded and analyzed participant responses regarding their confidence in interpreting research findings and assessments of research credibility, and the extent to which their knowledge about research findings changed after completing the workshops.

RESULTS: Our findings suggest that although participants expressed strong interest in understanding scientific research, their ability to develop confidence about scientific research methods was limited. However, like policymakers, consumer advocacy group members intuited that financial conflicts of interest could compromise scientific findings, although they initially underestimated their effects on research results. After training, consumer advocates also saw the value of using systematic reviews rather than individual studies.

DISCUSSION AND CONCLUSIONS: Our findings suggest that although advocates may not feel confident in their understanding of scientific research, they found it easier to understand the value of systematic reviews and the risks of conflicts of interest than other statistical concepts and terminology. Focusing on making these types of information available may offer a useful way for policymakers and consumer advocates to interpret the validity of policy-relevant scientific research.