- IRB: ETHICS & HUMAN RESEARCH
Implementing Single IRB Review of Multisite Research: Lessons Learned from the National Children’s Study
Abstract: In keeping with recent regulatory and policy requirements, a large proportion of multisite studies will be using a single institutional review board (IRB) of record. Further research is needed to better understand how to effectively establish reliance relationships and to understand the pros and cons of different models for implementing a single IRB. We describe a case study in the adoption and implementation of a single IRB model, the federated model of IRB review, by seven institutions that participated in the initial stage of the National Children’s Study Vanguard Study—a planned large-scale epidemiological cohort study. We highlight lessons learned in the process of establishing a single IRB model and facilitating adoption of the model by academic medical centers. These lessons include the need to plan sufficient time to obtain support from key stakeholders in institutions considering adoption of the model and the importance of standardizing policies and mechanisms related to review of local context issues.
Keywords: human subjects research, cooperative research, multisite studies, human subjects research, single institutional review board, reliance agreements, local research context
With the publication of the final revisions to the Federal Policy for the Protection of Human Subjects (the Common Rule), mandating the use of a single institutional review board (IRB) for multisite research, and of the National Institutes of Health’s (NIH’s) “Policy on the Use of a Single Institutional Review Board for Multi-Site Research,” a large proportion of multisite studies will soon be required to use a single IRB (often referred to as “sIRB”).1 Investigators and their institutions therefore need a better understanding of the pros and cons of different approaches to using a single IRB. In this essay, we describe the lessons we learned from enrolling the first seven institutions of the National Children’s Study (NCS) Vanguard Study in a single IRB model, and we consider how these lessons might inform current and future efforts to implement use of single IRBs. Two of us (ND, SL) were investigators from two of the seven institutions, one of us was the IRB chair (MS) from another of the institutions, and one of us (JS) was the point of contact for the NIH’s federated model of IRB review. To inform this case study, we queried principal investigators from the other four institutions to gain their personal and institutional perspectives.
The NCS Main Study, which was never initiated, was planned to be a minimal risk, epidemiological cohort study “which would have followed 100,000 children from before birth to age 21 and involved multiple study visits during the initial years of the study.”2 As a pilot study to test methods and procedures planned for use in the larger main study, the NCS Vanguard Study ran from 2009 to 2014 and enrolled about 6000 children and their caregivers, initially from 7 and then from 40 study sites around the country.3 At the time of implementation of the federated model of single IRB review, over 100 IRBs were estimated to have reviewed some or all of the NCS Vanguard protocol. The protocol was expanded over time to include additional age-appropriate assessments (as the cohort of children grew older), leading to multiple cycles of parallel IRB reviews of amendments and continuing review submissions across study sites. This resulted in delays for fielding time-sensitive study assessments and required significant human and financial resources for obtaining duplicative IRB reviews with no perceptible benefit to the welfare of study subjects, precisely the types of concerns that influenced mandates for single IRB review.4 To address these challenges, we offered the use of a single IRB as one option in a streamlined approach to multiple IRB review that we have described as the federated model of IRB review. We have previously published a detailed description of the structure of the federated approach.5 Briefly, the federated model allows each participating institution to select one of three tiers of participation:
Tier 1 (single IRB): reliance on a single IRB, in this case the National Institute of Child Health and Human Development (NICHD) intramural IRB, as the only reviewing IRB and the IRB of record;
Tier 2: a joint review by the NICHD IRB and a local institutional IRB, with coordination of local context review, reporting of adverse events and unanticipated problems, and human subject protections review by an IRB local to the study site; and
Tier 3: review by the IRB at the study site as the IRB of record, with sharing of determinations and other materials with the NIH and other institutions participating in the federated model.
The successful implementation of any single IRB model hinges on acceptance of and participation in the model by potential relying institutions. However, a high level of distrust has been reported on the part of IRB members toward single IRBs.6 Given this distrust and the absence of a single IRB mandate at the time, the federated model included multiple options for IRB review of multisite studies. All sites were encouraged to choose the single IRB option, as the preferred tier of participation in the federated model, and those that chose other tiers of the program were encouraged to reconsider this option in the future.
Methods for Implementing the Single IRB Review Component of the Federated Model
To foster consideration and acceptance of the federated model by stakeholders with varying knowledge of single IRB mechanisms, we adopted a two-pronged approach for introducing the federated model and promoting the benefit of the single IRB model option to the seven institutions that were the first to begin fielding the NCS Vanguard Study. Table 1 lists these institutions as well as the counties where the Vanguard Study was initially implemented.
For the first approach, we disseminated a tool kit of federated IRB materials for review by all institutions, and, second, we identified “advocates” from within each institution. The tool kit of materials included 1) frequently asked questions about the federated IRB model; 2) a slide deck describing the NCS Vanguard Study, the structure of the federated IRB model, and benefits to participation in the model at each level for the relying institution; 3) a memorandum of understanding that served as the IRB authorization agreement and specified the relying and reviewing institutions’ duties, rights, and division of responsibilities; and 4) a compact that articulated the core principles of the federated model (which include transparency across all institutions, compliance with regulations, and commitment to protection of human subjects) and described metrics that could be used to capture performance of the federated model and the single IRB option in particular.
The NCS Vanguard Study principal investigators at the seven initial study site institutions were asked to serve as advocates for the pilot of the federated IRB model and to encourage the use of the single IRB option. These investigators were selected to serve as advocates both because of their knowledge of the study and their knowledge of the clinical research and human research protections program (HRPP) infrastructure and leadership within their institutions. To prepare for their role as advocates for the single IRB pilot, the investigators received training on the materials in the tool kit from the NCS regulatory affairs study staff via group webinars and conference calls. In August 2010, the investigator-advocates disseminated the federated IRB tool kit to key stakeholders at their institutions. To facilitate dissemination of information about the federated IRB model, NCS staff members contacted key individuals within each institution (institutional officials, IRB chairs, institutional counsels, deans for research, grants and contract officers, compliance officers) to describe the advantages and disadvantages of the single IRB review process. The investigator-advocates at the seven study site institutions worked to engage key stakeholders at their home institutions over the next three to six months. Additionally, they contacted partnering institutions, including any subcontracting institutions (for example, scientific or research partners at other academic institutions, contract research organizations, and medical facilities where research activities took place) to describe the federated IRB model and gauge potential interest.
The number of institutions contacted during this stage of the outreach process varied across the seven study sites, with some sites reporting no subcontractor institutions and few collaborating medical institutions, and some sites coordinating with multiple collaborating institutions. There was no requirement in the federated model for all relevant parties to participate in the model at the same level. However, subcontractors and hospitals were invited to participate on the condition that they chose the single IRB option (reliance on the NICHD IRB) to simplify the administrative burden for NCS federated IRB support staff members during the piloting of the model.
To support and reinforce the investigator-advocates’ outreach efforts, the NCS staff also engaged with many institutional representatives to further describe the federated model and answer technical questions. Typically, multiple conversations about the implementation of the single IRB model between institutional leadership and NCS investigators-advocates and regulatory staff were required prior to adoption of the federated IRB model.
Executing the IRB authorization agreement and recruiting institutions to the federated model was time consuming. Table 2 shows the average time from dissemination of the federated IRB reliance agreement and supporting materials to execution of a reliance agreement—the beginning of participation in the federated model of IRB review. The average time for execution of a reliance agreement was 88.6 days across all tiers. The three institutions choosing the single IRB option had the quickest average turnaround time, 76.3 days. The two institutions choosing the shared review model had the slowest time to execution of the agreement, averaging 113.5 days.
Lessons Learned and Recommendations
The lessons we learned in the initial years of the federated IRB model can be helpful to other institutions as they implement the single IRB approach for multisite research, consider participation in single IRB arrangements, and seek institutional resources for serving as or relying on a single IRB.
Execution and negotiation of reliance agreements. Unless an established reliance agreement template, such as the SMART IRB reliance agreement,7 is used or institutions have established agreements together previously, the dissemination and review of the agreement should take place as soon as the use of a single IRB is identified. While it is difficult to generalize from a small sample, we believe that the shorter times for the execution of reliance agreements from two of the three institutions using the single IRB option had to do with the experience of those institutions in reviewing such agreements as well as their willingness to rely on external IRBs.
To reduce the time needed for executing reliance agreements, institutions should develop standard operating procedures for their HRPP for reviewing, negotiating, and executing reliance agreements and should establish corresponding workflows. The Association for the Accreditation of Human Research Protections Programs (AAHRPP) requires documentation of oversight of shared research in a recent standard (I.9) released in October of 2017.8 Institutions should also consider what level and type of staffing resources are needed to support the maintenance of reliance agreements as well as for supporting other administrative needs related to participating in single IRB arrangements. The NCS supported 1.5 additional full-time equivalent staff members who were responsible for tracking and helping to negotiate the reliance agreements and for providing support with developing standard operating procedures and managing amendment submissions and materials for relying institutions. Other recent pilot efforts to set up single IRB models have also called out the importance of adequate staffing for development of policies and management of IRB submission materials.9
Challenges faced by study sites in implementing single IRB review. Investigators from the seven study site institutions participating in the federated IRB model identified several factors that made implementation of the model challenging and affected which tier they chose to use. Several investigators reported that a large amount of their time and effort was required to inform institutional and IRB leadership about the federated model and to reach a decision about which tier of participation to select. Investigators in institutions experienced with the use of a single IRB—such as the Central IRB (CIRB) of the National Cancer Institute (NCI) or an independent IRB—or with experience relying on another institution’s IRB reported that their institutions were receptive to and appreciative of the possible benefits to participating in the federated IRB pilot and were more comfortable considering and selecting the single IRB option of the federated model.
One institution reported that, although its study investigator was interested in relying fully on the NICHD’s IRB as the single IRB of record for review of the protocol, their institutional policies would not allow this because the NIH’s HRPP was not yet accredited by the AAHRPP at the time of initial implementation of the federated IRB model in 2010 (the NIH received full AAHRPP accreditation in 2014). Interestingly, other sites did not have a similar requirement but requested copies of the NIH HRPP standard operating procedures to see if they were comparable to their own procedures. Of the seven study sites, the two participating in tier 2 of the federated model—where responsibilities for local context review and other functions are shared between the study site and the institution maintaining the single IRB—found the standard operating procedures and the division of labor confusing and burdensome. In fact, one tier 2 institution ultimately chose to do a full local review of all materials, even though this was not required by the terms of the reliance agreement, thereby reducing the efficiency gained by a shared model of review across two institutions.
The five institutions participating in either the single IRB option or the shared review option (tier 2), developed processes and procedures for sharing information about local context (state laws, institutional policies, etc.) with the single IRB at the NIH. This required an ongoing commitment of IRB members and IRB office staff members as well as additional investigator time and effort. The institutions designated either an IRB member or a team including the principal investigator and an IRB member to provide feedback to the single IRB about local context. Local context considerations specific to a given submission to the NICHD IRB by a tier 1 or tier 2 institution were provided on an ongoing basis, whenever the institution would request NICHD IRB review of local recruitment materials. Although this approach was effective in providing the single IRB with specific and detailed knowledge of local context issues particular to a given protocol submission, it was time consuming for the study site as well as the single IRB. A streamlined approach to communicating local context would be to require an initial report of local context considerations with periodic updates on an annual or semiannual basis. The NCI’s CIRB model uses this kind of strategy.10
Enlisting the support of thought leaders at each institution. In our experience, finding one or more advocates among key leaders within an institution who are familiar with or at least open to the idea of relying on another IRB is helpful and facilitates implementation of the single IRB process. Ideally, the advocate should be a thought leader within the institution and be familiar with the approach taken by the institutional official, the general counsel, and the IRB leadership in regard to relying on outside IRBs for human subject protections review. For new multicenter projects, making the decision to implement a single IRB model should take place before finalizing the protocol to achieve the greatest efficiencies and costs savings.
Starting the process early. Our overarching advice for those implementing or participating in a single IRB model for multisite research is to allow sufficient time and staff resources for engaging study sites and executing IRB authorization agreements and in developing supporting policies, procedures, and staffing models. In our experience, multiple conversations across several levels of institutional leadership were necessary to fully explain the federated model and present the three levels of participation. Communication with institutional officials and IRB chairs, in particular, was critical for many of the discussions with the institutions initially utilizing the federated model of IRB review. While a simpler model might have required less explanation, until there is standardization in IRB authorization agreements, working through and negotiating IRB authorization agreements requires a significant investment of time, and investigators should factor this into their planning for study start dates.
Making connections across institutional leadership. Institutions participating in other single IRB initiatives or institutions that have relied on independent IRBs were more comfortable with the idea of relying on an outside IRB and were able to execute the reliance agreements more quickly, but we do not have sufficient data to explore this relationship in a more systematic way. Some institutions benefited from knowing which tier institutions similar to theirs had chosen. Participation in the NCS federated IRB may have familiarized some institutions with this model, resulting in greater willingness and faster approval in the future.
Improving procedures for local context review. In the context of the NCS federated IRB model, we required each relying institution to propose and implement an approach to communicating key information about local context to the single IRB. Our rationale was that allowing each institution to craft a customized approach would make the process easier and more efficient for local institutions. The feedback we received from institutions participating in tier 2 of the federated model (local context review conducted by the relying institution) indicated that they would have preferred a process where local context review was conducted entirely by the single IRB. Single IRB models need to account for sufficient staff time and resources to conduct such review.
Finally, some of the institutions taking part in the federated IRB pilot indicated that they would have preferred that NCS provide a single, defined process for implementation of local context review across all sites. The NIH’s Office for Human Research Protections offers some flexibility with regard to the conduct of local context review, and there are multiple models for how such a review can be organized in the context of a single IRB. Overall, a systematic approach across all institutions would have made the implementation of the model more straightforward and efficient, from the perspectives of the single IRB as well as the relying institutions.
At this time, with the exception of the NCI CIRB, other nationally vetted reliance agreement templates do not yet have associated standardized local context forms or information requirements, though the SMART IRB initiative is developing such materials.11 Accordingly, individual institutions need to identify which state, local, and institutional requirements they need to convey to reviewing institutions as well as to identify the information that their local IRB requires when serving as the IRB of record.
Choosing the right approach for a reliance agreement. There are multiple successful operational models from which to choose for using a single IRB for review of multisite research. Each model involves different distributions of responsibilities between reviewing and relying IRBs and institutions, arrangements for designating the reviewing IRB, and formats for reliance agreements. The memorandum of understanding the NCS used was a reliance agreement limited to the review of a single protocol. However, institutions that intend to work together on multiple projects or as part of a research network should consider developing a master reliance agreement with likely partner institutions. This approach has been successfully implemented by networks such as NeuroNEXT, the NCI CIRB, and PEDSnet.12 While it may take longer to negotiate a master reliance agreement than a single-study agreement, once it is in place, the start-up time for new studies should be substantially reduced. For institutions or projects that are newly establishing a single IRB review process, the choice of model and selection of reliance agreement is key to successfully and efficiently leveraging human subject protections review by a single IRB. The SMART IRB initiative represents an important effort to harmonize these agreements across institutions and to reduce resources spent on negotiating agreements.
A significant limitation of our initial experience with implementing a single IRB initiative is the small sample of institutions as well as the lack of variation among these institutions. Each of the institutions in our sample is an academic research center with significant experience participating in multisite studies, if not in single IRB arrangements. Additionally, we implemented the model after the NCS study protocol was established and had an initial review by the local IRB at each of these institutions. Thus, the study was already known by some institutional stakeholders. Another limitation is the level of support resources available for this initiative. We worked with motivated principal investigators who were both knowledgeable about the single IRB approach and plugged into institutional HRPP infrastructure, and who were supported by dedicated single IRB staff members at the NIH who were available to provide education through webinars and consultation. This level of support and engagement may not be available in all single IRB collaborations. It is notable that, despite these resources, the execution of the reliance agreement was still very time consuming. A final limitation is that we did not have a mandate for institutions to participate in the single IRB option. With the current NIH policy and HHS regulatory initiatives in place, in many instances, institutions will no longer have a choice about opting out of taking part in single IRB arrangements.
The often-cited barriers to participation in single IRB models include lack of trust in the ability of another institution’s IRB to provide a quality review attentive to local context considerations, concerns about liability, and lack of empirical evidence that there are gains to be made from the use of single IRBs. While regulatory and policy mandates will require that institutions overcome these barriers, our experience provides valuable lessons for facilitating negotiations and more efficiently implementing reliance relationships.
We found that gaining acceptance from institutions to participate in a single IRB model required significant lead time (an average of 76.3 days). Preparation of comprehensive materials, general education presentations and conversations with multiple stakeholders, and identification of an advocate were all strategies that facilitated participation in a single IRB model.
Once a single IRB model is established, careful thought must be given to how mechanisms for sharing local context considerations can best be simplified and standardized and how the relying institution and the institution supporting the single IRB can most effectively and efficiently coordinate their designated responsibilities and communicate about shared concerns. As the use of single IRBs becomes more prevalent, we anticipate that a range of best practices will emerge and that the initial stages of enrolling institutions to single IRB models will become more efficient.
Julia Slutsman, PhD, is the director of research regulatory affairs at the Children’s National Medical Center and an assistant professor in the Department of Pediatrics at George Washington University School of Medicine and Health Sciences (and at the time of writing this manuscript, she was affiliated with the Office of Human Subject Protections and the Department of Bioethics at the National Institutes of Health); Nancy Dole, PhD, is the deputy director of the Carolina Population Center at the University of North Carolina at Chapel Hill; Steven Leuthner, MD, MA, is a professor of pediatrics in the Division of Neonatology and a professor of bioethics at the Center of Bioethics and Medical Humanities; and Mark S. Schreiner, MD, is an associate professor of anesthesia and critical care at The Children’s Hospital of Philadelphia.
This manuscript, a primary National Children’s Study publication, was developed by a writing team assembled by the NCS Publications Committee for the purpose of timely sharing of centrally collected NCS data. This analysis was conducted as part of the National Children’s Study, supported by the Eunice Kennedy Shriver National Institute of Child Health and Human Development, and funded, through its appropriation, by the Office of the Director of the National Institutes of Health.
Dr. Dole was supported in part by NICHD contracts HHSN275200503413C and HHSN267200700049C; Dr. Leuthner was supported in part by NICHD contract HHSN275200503396C; and Dr. Schreiner was supported in part by NICHD contract HHSN275200503414C.
Dr. Slutsman would like to thank the peer reviewers for their perceptive comments, which improved the quality and organization of this manuscript, and to thank Matthew Hoffman for his insightful review and edits.
The views expressed in this article are the responsibility of the authors and do not necessarily represent the position of the National Children’s Study, the National Institutes of Health, or the U.S. Department of Health and Human Services.
- Federal Policy for the Protection of Human Subjects, § ____.104(d)(2). 82 Fed. Reg. 7149 January 19, 2017; National Institutes of Health. Single IRB policy for multi-site research. https://grants.nih.gov/policy/clinical-trials/single-irb-policy-multi-site-research.htm.
- National Institutes of Health, Eunice Kennedy Shriver National Institute of Child Health and Human Development. National Children’s Study (NCS). https://www.nichd.nih.gov/research/supported/NCS.
- See ref. 3, National Institutes of Health, Eunice Kennedy Shriver National Institute of Child Health and Human Development.
- Menikoff J. The paradoxical problem with multiple-IRB review. NEJM 2010;363(17):1591-1593; Greene SM, Geiger AM. A review finds that multicenter studies face substantial challenges but strategies exist to achieve institutional review board approval. Journal of Clinical Epidemiology 2006;59:784-790.
- Slutsman J, Hirschfeld S. A federated model of IRB review for multisite studies: A report on the National Children’s Study Federated IRB Initiative. IRB: Ethics & Human Research 2014;36(6):1-6.
- Flynn E et al. Using central IRBs for multicenter clinical trials in the United States. PLOS One January 30, 2013. https://doi.org/10.1371/journal.pone.0054999; Klitzman R. How local IRBs view central IRBs in the US. BMC Medical Ethics 2011;12:13-14.
- SMART IRB. https://smartirb.org/.
- Association for the Accreditation of Human Research Protections Programs, Inc. Standard I-9: The organization has written policies and procedures to ensure that, when sharing oversight of research with another organization, the rights and welfare of research participants are protected. Available as a PDF at http://www.aahrpp.org/apply/process-overview/standards.
- Splinter K et al., Members of the Undiagnosed Diseases Network. Implementing the single institutional review board model: Lessons from the Undiagnosed Diseases Network. Clinical and Translational Science 2018;11(1):28-31.
- National Cancer Institute. Welcome to the CIRB. https://www.ncicirb.org/.
- Personal communication with Nichelle Cobb, director of health sciences IRBs, University of Wisconsin. November 5, 2017.
- See https://www.neuronext.org/, https://www.ncicirb.org/, and http://www.pcori.org/research-results/2013/national-pediatric-learning-health-system-pedsnet.