Bioethics Forum Essay
The Need for Open and High Quality Preclinical Science
An investigative report The BMJ published recently about a failed tuberculosis vaccine trial conducted with infants in South Africa underscores several issues in translational science that are gaining increased attention: low standards in the rigor, reporting, and transparency of preclinical research.
Conducted with nearly 2,800 healthy infants aged 4-to-6 months who had already received the Bacille Calmette-Buerin vaccine, the study aimed to determine the safety and efficacy of the MVA85A vaccine as a protective booster. The pediatric vaccine trial joins the ranks of several recent mid- and late-stage clinical trials, including research on possible therapies for Alzheimer’s disease, that failed to show the intervention worked as intended in humans even though efficacy was reported from preclinical animal studies.
The BMJ report raises concerns about whether the vaccine researchers selectively reported to the funders of the pediatric study, the research ethics committees that reviewed and approved it, and the parents or legal guardians who provided informed consent for their child’s participation, positive findings from animal studies of the safety and efficacy of the MVA85A vaccine while downplaying negative results. Deborah Cohen, the associate editor of The BMJ and author of the report, contends that to allay some of these concerns access to the protocol for one of the key animal studies is warranted. Yet The BMJ’s requests for access have been denied.
Several commentators, including James A. Anderson and Jonathan Kimmelman in Canada and Robert Califf, the former commissioner of the U.S. Food and Drug Administration, have called for preregistration of preclinical research protocols or, alternatively, the key elements of the experimental design and how the data will be analyzed. These types of initiatives and others are already underway to foster transparency in the design and conduct of preclinical research, as well as to improve the publication of positive and negative results. For example, in November 2017, the TACTICS (Transnational Alliance for Regenerative Therapies in Cardiovascular Syndromes) project supported the launch of a website for the registration of preclinical animal studies. The initial goal was to support an international registry of studies testing regenerative medicine therapies for cardiovascular syndromes, but the registry is now open to all preclinical animal research. Although the actual protocols that funders, as well as animal research and human research ethics committees, review are not accessible from the registry, key information from the protocols is provided, including the experimental design of the studies registered and measures researchers plan to take to reduce experimental bias.
Other initiatives are designed to link the registration of preclinical animal studies to the publication of positive and negative results of those studies. In November 2017, the National Institutes of Health launched a publicly accessible database that includes information about preclinical Alzheimer’s drug studies, such as study designs, the animal models used, the therapeutic agents and targets, and efficacy data from published and unpublished studies. Importantly, the database provides details about the experimental design of studies that the researchers did or did not report in their published articles, including the power/sample size of the study, whether the animals were randomized into groups, and whether the study was blinded for treatment and outcome measures.
According to the Center for Open Science, 97 journals that publish results from preclinical trials, human trials, or both use all or some features of the format for a registered report, a model pioneered by the journal Cortex. With this model, a researcher’s experimental methods and proposed analyses are pre-registered with a journal and undergo peer review before the collection of data. If this information passes pre-study peer review, the journal gives the researcher an “in-principle acceptance” of publication of the study’s results regardless of the study’s outcome. Revocation of the in-principle acceptance will occur if the research is not conducted in accord with the registered protocol, quality assurance problems with the study and reporting of results are identified, or if problems with the clarity or style of reporting results cannot be resolved.
Several complementary initiatives have been implemented that help researchers improve the rigor and reproducibility of preclinical research and ensure thorough reporting of both negative and positive results. For example, the PREPARE (Planning Research and Experimental Procedures on Animals: Recommendations for Excellence) guidelines contain a checklist of elements for designing rigorous and reproducible preclinical animal studies.
The widely endorsed ARRIVE (Animal Research: Reporting of In Vivo Experiments) guidelines recommend that, in their published reports, researchers provide details about the experimental design and procedures of the study, the animals used, the sample size, how animals were assigned to experimental groups, and the statistical methods for the analyses conducted. Yet two years after the guidelines were issued, an analysis of papers published in selected journals revealed that many researchers failed to include key details about their animal studies.
Although registries of preclinical studies will not fix the problems related to poor experimental designs or selective reporting of data, they represent one of the many steps toward establishing an open ecosystem of preclinical animal research within and across disciplinary boundaries and disease categories. Moreover, when concerns are raised like those about the pediatric TB vaccine trial, protocols for the relevant animal studies should be available for public scrutiny – with proprietary information redacted when warranted.
Knowing that preclinical animal studies were conducted as prescribed in the research protocol and that positive and negative results were obtained from scientifically rigorous experimental designs does not mean that beneficial outcomes observed in animal models will always translate into similar outcomes in human clinical trials. But the documented low rate of translation from animal research to beneficial interventions for patients has implications for the willingness of pharmaceutical companies to invest in drug and vaccine discovery. Moreover, proceeding to human clinical trials on the basis of unreliable safety or efficacy data from animal studies raises ethical issues about exposing research participants to risks of harm from the interventions being tested and to other burdens of participating in clinical trials. Responsible, ethical, and successful translational science requires open and scientifically rigorous preclinical research.
Karen J. Maschke is a research scholar at The Hastings Center.
Having received BCG in 1964 as a medical student during my second year, my TST was 8-9 mm induration before going to a Straight Medicine internship at Harlem Hospital, NYC, in 1969. A year later the TST was 15 mm induration. Subsequent chest XR have all been negative and the TST has been unchanged, it all became suspect after an elderly patient of mine was hospitalized with cavitary TB two years before my retirement. And now 4 years later and 6 months of INH, the Chest XR is still negative. So, did the BCG prevent two episodes of TB or not for this 75 y/o retired Primary Physician? The BCG in 1964 was questionable since Nebraska is a State with a very low TB rate, except with our Indian Reservations. My choice for Post-graduate education was still over the horizon at that time.
So did the BCG given to the infants in this study before randomization into the vaccine or placebo groups just mask the benefit of the new vaccine or NOT. AND, does the study involve a 5 year follow-up study chest XR for both the placebo and study vaccine groups to determine which one worked better?
My patient was a retired driver for a national transportation company. TB tests on two occasions during routine checkups with chest XRs were negative during 30 years of f/u.