Accounting for potential conflicts of interest and biases that might be held by panel members and selecting criteria that will determine the research to be assessed are among the main challenges to improving the quality of work performed by independent scientific reviews that form the basis of many regulatory decisions.
Those tasks are addressed in a new report from a research integrity roundtable convened by The Keystone Center. Members of the roundtable included multiple representatives from the chemical industry, the EPA, and the National Institutes of Health. Representatives from the Food and Drug Administration, the Department of Interior, the Union of Concerned Scientists, and the Natural Resources Defense Council also participated. The members represented their own views and not necessarily those of the organizations with which they are associated, according to the report.
Key questions
The roundtable’s report takes as its starting point the work of the Bipartisan Policy Center’s (BPC) 2009 report, Science for Policy Project: Improving the Use of Science in Regulatory Policy, and seeks to advance some of the BPC report’s findings by focusing on questions that included:
- How should panels be composed and the qualifications of prospective advisory panelists vetted?
- How should concerns about biases and conflicts of interest of advisory panelists be handled?
- Which studies should agencies review when examining the scientific literature related to a regulatory policy issue?
- How should contending views regarding the relevance of particular scientific results to a regulatory issue and the credibility of those results be addressed?
Conflict waivers
According to the report, conflicts of interest should not automatically disqualify an individual who can make a substantial contribution to the work of the panel. For example, a person associated with an entity that would benefit from a regulatory decision that might result from the work of the panel can still contribute if that entity is part of a class of entities that could equally benefit by the panel’s decision. Panels would need to develop criteria for issuing conflict-of-interest waivers in such situations.
The report acknowledges that all potential panelists will have conscious and unconscious biases. However, panelists should disqualify themselves if they believe their biases will render them ineffective at fairly weighing the facts and opinions of others who hold alternative views on the matters under discussion.
Credible/noncredible
Criteria to use in establishing the credibility of studies to be reviewed include a determination of whether the study is reproducible, the results have already been duplicated, and the work conforms to good laboratory practices, where applicable.
Examples of nonscientific credibility include nondisclosure of sources that funded the study and control of the design and publication of the study by the sponsor or funder, as opposed to appropriate control by the principal investigator. However, the latter situation would not apply in cases where the design of the study is determined in advance by explicit regulatory agency direction. For example, the Federal Insecticide, Fungicide, and Rodenticide Act and Toxic Substances Control Act require adherence to test guidelines that prescribe experimental study design elements; the Organization for Economic Co-operation and Development imposes similar requirements.
Information on the report is at http://www.keystone.org/policy-initiatives-center-for-science-a-public-policy/health/research-integrity-roundtable.html.