IRB: Ethics & Human Research
Incident Reports and Corrective Actions Received by OHRP
Abstract:
The Office for Human Research Protections (OHRP) is responsible for overseeing compliance with the Department of Health and Human Services’ regulations governing research with human subjects. This article describes OHRP’s review of 6,511 incident reports from institutions conducting research over which OHRP has jurisdiction that were received between January 1, 2008, and December 31, 2014. We provide an analysis of unanticipated problems, suspensions or terminations of institutional review board approval, and noncompliance that institutions reported to OHRP. A larger number of incident reports were received by OHRP in 2009 than other years studied. The type of incident most commonly reported during this time was serious noncompliance; the subcategory of serious noncompliance most reported during this time related to changes to research without IRB approval. We also analyzed five major categories of corrective actions that institutions reported to OHRP. Protocol- or case-specific corrective actions comprised the largest category of corrective actions (5,905 instances) reported. The most common corrective action subcategory was adding or revising standard operating procedures for research (2,551 instances).
Key words: human subjects research, regulatory compliance, regulatory noncompliance, incident reports, corrective actions, Office for Human Research Protections (OHRP)
_____________________________________________________________________________
In this article, we describe our review of incident reports and the corrective actions included in those reports that the Office for Human Research Protections (OHRP) received between January 1, 2008, and December 31, 2014. OHRP is the division of the Department of Health and Human Services (HHS) responsible for overseeing compliance with the HHS regulations governing research with human subjects.1 The regulations require institutions that conduct research under an OHRP-approved assurance to have written procedures to ensure that incidents related to regulatory requirements are promptly reported to OHRP.2 These incidents include 1) any unanticipated problems involving risks to subjects or others, 2) any serious or continuing noncompliance with this policy or the requirements or determinations of the institutional review board (IRB), and 3) any suspension or termination of IRB approval. After receiving, processing, and reviewing an incident report, OHRP responds to the institution in writing, typically by email, generally confirming that the report was adequate, seeking additional information about the reported incident, or recommending or requiring that the institution enact additional corrective actions.
When reviewing an incident report, OHRP assesses most closely the adequacy of the actions taken by the institution to address the incident. Specifically, OHRP assesses whether the corrective actions will help ensure that the incident will not happen again with the investigator or protocol in question, with any other investigator or protocol, or with the IRB. Therefore, OHRP recommends that, when appropriate, corrective actions be applied to the entire institution. Corrective actions, as defined by OHRP, are those activities that an institution implements when it is trying to resolve an issue that resulted in regulatory noncompliance in its human research protections program or to address other problems in research. These corrective actions can be either self-imposed by the institution or mandated by OHRP. Corrective actions have been key in rectifying the systematic problems that institutions encounter related to protecting human subjects in research and complying with the HHS regulations.
This review of incident reports and corrective actions as stated in those incident reports from 2008 to 2014 will examine the most prevalent incidents reported and corrective actions implemented by institutions and discuss the limitations of the data and best practices for institutions. The overwhelming majority of the institutions at which reported incidents occurred were domestic (97.8%, or 818 out of 836 institutions). The institutions that submitted the reports include domestic and international public and private universities and colleges, private research institutions, IRBs, medical schools, academic medical centers, community hospitals, and federal health care and research facilities.
Key Findings
Between January 1, 2008, and December 31, 2014, OHRP processed 6,511 incident reports from 780 institutions. The numbers of incident reports received by OHRP each year were as follows: 827 in 2008, 1,105 in 2009, 955 in 2010, 913 in 2011, 885 in 2012, 827 in 2013, and 999 in 2014. Many of the incident reports received and processed contained more than one incident; in total, then, these 6,511 incident reports documented 8,570 incidents.
Figure 1 shows the types of incidents reported to OHRP between January 1, 2008, and December 31, 2014. These include 1,506 adverse events (including unanticipated ones), 582 risks to or breaches of confidentiality, 1,556 other unanticipated problems, 734 complete suspensions of IRB approval, 1,030 suspensions of enrollment of new subjects, 274 terminations of IRB approval, 344 other IRB actions, 2,943 instances of serious noncompliance, and 583 instances of continuing noncompliance. Examples of other unanticipated problems included events such as research participants’ receiving the wrong dose of a study drug or the wrong drug but having no ill effects, a study’s enrolling ineligible subjects, and subjects’ threatening research staff. “Other IRB actions” include notifying a sponsor or regulatory authorities of the event. Note that a single event may be counted as two (or three) different incidents: for example, noncompliance could also represent an unanticipated problem, which could lead to a suspension of IRB approval. There is some fluctuation between years in the numbers of each type of incident being reported. OHRP received the most incident reports in 2009.
Figure 2 (available via the IRB: Ethics & Human Research web page) shows the 10 most common areas of serious and continuing noncompliance that were reported to OHRP from 2008 to 2014. Serious noncompliance was the most common type of incident, with 2,943 instances (data not shown). Approximately 51% of that noncompliance related to protocol changes without IRB review and approval (1,515 instances). Another category of serious noncompliance involves informed consent issues; 970 such incidents were reported to OHRP during this time period (33% of the instances of serious noncompliance). These were the top two categories of serious noncompliance and continuing noncompliance. Protocol changes without IRB review and approval that were reported to OHRP included study interventions not administered as required by protocol, compensating subjects more than allowed in the protocol, and failure to follow inclusion or exclusion criteria. Noncompliance related to informed consent included failure to obtain informed consent prior to inclusion in research, failure of the informed consent document to include all the risks of the research, and failure of the subject to sign the consent form prior to participation in research.
Between January 1, 2008, and December 31, 2014, OHRP received reports of a total of 12,326 corrective actions distributed in six major categories from 836 institutions. The categories include 52 individual subcategories of corrective actions. Figure 3 shows the total number of corrective actions reported to OHRP between January 1, 2008, and December 31, 2014, in each of the five major categories: changes in IRB or institutional structure; addition or revision of IRB policies and procedures; education of investigators, research staff members, IRB members or staff members; protocol- or case-specific actions; and other corrective actions. A sixth major category, revisions of IRB application forms, was not included in the chart due to the low number of corrective actions in this category (13 between 2008 and 2014).
Protocol- or case-specific corrective actions comprise the largest category of corrective actions and are those that involve an action related to a particular study or studies (with 5,905 instances reported). Some of the subcategories in this category include re-review of protocol(s) or grants by an IRB, submitting subpart C certifications to OHRP (for research involving prisoners, certifying that the IRB has made certain findings regarding the research), suspension of the investigator, termination or replacement of the principal investigator or another research staff member, monitoring or auditing of the investigator, training of the principal investigator on specific issues, requiring the principal investigator to submit amendments to the protocol, requiring the principal investigator to revise consent forms for specific studies, requiring the principal investigator to obtain additional research staff, terminating the protocol, suspending or revoking the investigators’ privileges to conduct human subjects research, disallowing the use of data or attaching conditions to its use, and requiring reconsent or notification of subjects.
Instances of adding or revising policies and procedures comprise the second-largest category of corrective actions (with 3,098 instances reported). They include revision or addition of the following: initial review procedures; continuing review procedures; procedures regarding reporting unanticipated problems involving risks to subjects or others, serious or continuing noncompliance, and suspending or terminating IRB approval; contingent approval procedures; IRB reviewer checklists (informed consent; expedited reviewer; subpart B, C, or D; waiver of informed consent; and waiver of documentation of informed consent); expedited review procedures, procedures for review of proposed changes to research, minute recording, and documentation of IRB findings or actions; electronic tracking of protocols or development of the electronic IRB record; procedures for determining exemptions; subpart reviews; auditing programs; procedures for conducting investigations; and research standard operating procedures.
Educational corrective actions involve the education of investigators and research staff members or all investigators at the institution, IRB members or staff members, or institutional officials. OHRP received reports of 2,114 instances of corrective actions involving education from 2008 to 2014.
“Other” corrective actions are those that did not fit into the five other major categories (with 1,091 instances reported). Some of the subcategories under other corrective actions include disallowing use of data, mentoring or supervision of a researcher, and other case-by-case specific corrective actions (e.g., investigation by a safety monitoring board, providing a subject with treatment for an adverse event, or obtaining a Certificate of Confidentiality for the study). Corrective actions involving revisions of IRB application forms include soliciting information on informed consent, soliciting information on subpart D of the regulations (research involving children) or other subparts, soliciting information on other vulnerable populations, and soliciting information on criteria of approval of research.3 There were only 13 such corrective actions reported to OHRP from 2008 to 2014.
IRB or institutional structure corrective actions included restructuring the IRB, adding staff members or making staffing changes, adding an IRB(s), changing the signatory official, changing the official or office to which the IRB reports, and adding a research compliance officer or office.
Figure 4 (available via the IRB: Ethics & Human Research web page) shows the top 15 subcategories of corrective actions reported to OHRP between January 1, 2008, and December 31, 2014, and the percentage of the total. A total of 12,326 corrective actions were reported to OHRP from 2008 to 2014. The most common corrective action subcategory was addition or revision of research standard operating procedures, which falls under the category of policies and procedures (2,551 instances). Examples of this include the implementation of encryption protocols for institution research laptops and flash drives, the use of checklists to confirm conduct of study procedures, and the revision of a mechanism to notify the research team of lab data. Education of investigators or research staff members was the second most common, at 1,779. The third most common corrective action was to have subjects reconsent or to notify them.
The category “addition or revision of policies and procedures” also included (but is not limited to) corrective actions revising policies and procedures involving implementation of auditing program(s) (196) and electronic tracking of protocols or development of the electronic IRB record (61). In the “protocol or case-specific changes” category, the following corrective actions are also included: reconsent or notify subjects (1,206), require the principal investigator (PI) to submit an amendment or to amend the protocol (1,071), require the PI to revise consent forms for specific studies or to amend consent (760), use of data disallowed or conditions attached (439), training of the PI on specific issues (289), re-review of protocol(s) or grants by the IRB (242), and the PI’s obtaining additional research staff (227). Corrective actions related to training or educating also include education of investigators and research staff members (1,779), education of all investigators at an institution (188), and education of IRB members or staff members (131).
Each year, the numbers for the corrective actions reported to OHRP in incident reports remained relatively the same for each category. From 2008 to 2014, the protocol or case-specific changes category led in numbers, while revision of IRB application forms consistently had the fewest instances reported, with an average of 3 reported per year (data not shown).
To estimate the percentage of incident reports for which OHRP requests additional information, we analyzed the number of times OHRP did so in 2013 and 2014. OHRP received 1,826 reports in 2013 and 2014 and requested additional information 69 times. Therefore, only approximately 3.8% of the time did OHRP request additional information. The information requested was usually a final report on the matter. OHRP may request that any corrective actions taken will be ones that help prevent a particular noncompliance problem from reoccurring at that institution.
Discussion
The data presented here have certain limitations. For example, the number of incident reports processed from 2008 to 2014 is greater than what is presented. Multiple reports were sometimes logged in one entry in the Compliance Activity Tracking System. Reasons for this include that 1) multiple reports are sent from one institution at one time, 2) the principal investigator is the same for several protocols, 3) the reports received from an institution include multiple incidents that are in the same category of noncompliance, and 4) the reports received from an institution include multiple incidents that are for the same protocol. This means that the actual number of incidents reported is also greater since multiple reports are processed in one entry.
As noted above, more incidents were reported in 2009 than any other year described in this analysis, particularly, adverse events, suspensions of new enrollments, and serious noncompliance. It is not clear why there were more such incidents reported that year than other years. We note that OHRP issued several final and draft guidance documents during this time (including guidance on continuing review of research and on IRB “Approval of Research with Conditions”) but none dealing with reporting of incidents to OHRP. OHRP issued “Guidance on Reviewing and Reporting Unanticipated Problems Involving Risks to Subjects or Others and Adverse Events” in 20074; it is conceivable that institutions took two years to implement this guidance, but it seems unlikely. OHRP issued guidance on compliance oversight procedures in 2009,5 but this guidance does not describe incident reporting. Another possibility is the increased number of quality assurance workshops that OHRP conducted in the previous year (10 in 2008, compared to an average of 6 in other years). These workshops often include training on reporting to OHRP. We also do not know if there were more actual events in 2009 than in other years or whether the rate of reporting was higher.
Another limitation is that the corrective actions examined represent only quantified data for each action taken by institutions that is reported to OHRP. As indicated above, OHRP received a total of 12,326 corrective actions distributed in six major categories between 2008 and 2014. The data do not show the fact that institutions often implement multiple corrective actions per incident. In addition, no follow-up data are available to assess the effectiveness of the actions to remedy systemic problems. When institutions report incidents to OHRP, each incident may have multiple corrective actions, or a single corrective action may address multiple incidents. Thus, there is not a one-to-one correlation between incidents and corrective actions, and OHRP does not track correlation between these. This should be kept in mind when reviewing the data.
Another limitation of the data presented here is that institutions might implement corrective actions but not report them to OHRP. In addition, the kinds of activities represented by corrective actions may be undertaken independently of any incident at all. For example, an annual review of policies and procedures might lead the institution to revise several policies and procedures, and institutions may educate investigators and staff members as part of a regular annual program.
Our analysis of reported corrective actions demonstrates that the largest numbers of corrective actions relate to the conduct of research, rather than IRB processes. This is seen in the top three subcategories of corrective actions reported: addition or revision of research standard operating procedures, education of investigators and research staff members, and monitoring or auditing of a principal investigator or a research study or studies.
This analysis of incident reports may help institutions in identifying major areas of problems that need to be reported to OHRP. Institutions may wish to audit their records to ensure that such events are being adequately reported. This data analysis can also prove useful in identifying the types of noncompliance that are most frequent at reporting institutions. Awareness of this can help in targeting the root cause of the incidents and what steps should be taken to prevent further incidents. Institutions often fear that reporting incidents to OHRP will be a “red flag” of concern for OHRP. Indeed, the opposite is the case: OHRP is more concerned about institutions that do not report, as we recognize that incidents do occur, despite the best intentions of all involved. For example, low numbers of reports to OHRP are sometimes a factor in OHRP’s decision to conduct a not-for-cause evaluation.
The conduct of human subjects research can be complicated and not necessarily under the control of the IRB. During the conduct of research, it is not uncommon to find that, for one reason or another, the research must be suspended or terminated or the approved protocol has not been followed—that there is noncompliance or unanticipated problems that may need to be reported to the IRB, institutional officials, sponsors, and federal agencies. These occurrences or deviations can have a range of possible impacts depending on multiple factors such as the overall risk of the study and the nature and extent of the incident(s). Once the causes of the problem are discovered, they must be assessed, and action must be taken to correct them and prevent such occurrences in the future.
Observers of enforcement of the human subjects protections system sometimes focus their attention on findings of noncompliance in cases of investigations by the Division of Compliance Oversight in OHRP. From 2008 to 2014, there were 129 such findings made by the division. The incidents reported here indicate that there is another enforcement function that is actively in operation in the current system, one that depends on the self-governing activity of research institutions and IRBs. Of course, the two functions may be interrelated insofar as one of the incentives for self-reporting of incidents is to avoid the possibility of a compliance case investigation by OHRP. However, it is also true that many institutions may self-report incidents because they appreciate that this is a legitimate feature of the current system. In any case, looking at incident reporting as well as compliance investigations provides a more complete picture of the regulatory enforcement in the current system. OHRP plans to conduct another analysis of determinations of noncompliance found in OHRP investigations in the near future.
The authors hope that this evaluation of corrective actions reported to our office in incident reports will help institutions formulate appropriate corrective action plans when addressing noncompliance, unanticipated problems, and suspensions and terminations that occur in research conducted by investigators. This report should also highlight the areas in which problems are most likely to occur so that institutions and federal agencies can better target educational programs to help prevent those problems. For example, since protocol changes and informed consent issues have been the most frequent types of serious noncompliance, it might make sense to focus some educational efforts in these two areas. We also note that educational efforts are a common form of corrective action, and so institutions should be prepared to design and conduct educational activities in response to anticipated incidents.
Figures
Figures 2 and 4 are available via the IRB: Ethics & Human Research web page, part of The Hastings Center website.
- Kemnique Ramnath, MHA, is an Oak Ridge Institute for Science and Education fellow in the Division of Compliance Oversight in the Office for Human Research Protections; Sarita Cheaves, MHA, is a public health advisor in the Division of Compliance Oversight in the Office for Human Research Protections; Lisa Buchanan, MAOM, is a public health analyst in the Division of Compliance Oversight in the Office for Human Research Protections; Kristina Borror, PhD, is the director of the Division of Compliance Oversight in the Office for Human Research Protections; and Marinna Banks-Shields, PhD, is the branch chief for the Western region of the Division of Community HIV/AIDS Programs in the HIV/AIDS Bureau in the Health Resources and Services Administration.
Disclaimer
The views represented here are those of the authors and are not intended to represent those of the Department of Health and Human Services.
Acknowledgment
We wish to thank Ivor Pritchard, PhD, senior advisor to the director of OHRP, for review of and suggestions for this manuscript.
References
- Department of Health and Human Services. Protection of Human Subjects. 45 CFR 46.
- Department of Health and Human Services, Office for Human Research Protections. Unanticipated problems involving risks & adverse events guidance (2007). January 15, 2007. https://www.hhs.gov/ohrp/policy/advevntguid.html.
- 45 CFR 46.111.
- See ref. 2.
- Department of Health and Human Services, Office for Human Research Protections. Compliance oversight procedures for evaluating institutions (2009). October 14, 2009. https://www.hhs.gov/ohrp/compliance-and-reporting/evaluating-institutions/index.html.