IRB: Ethics & Human Research
OHRP Compliance Oversight Letters: An Update
In this article, we describe our review of 235 compliance oversight determination letters that the Office for Human Research Protections (OHRP) issued to 146 institutions between August 1, 2002, and August 31, 2007. Our previous review of 269 determination letters issued to 155 institutions between October 1, 1998, and June 20, 2002, appeared in the September-October 2003 issue of this journal.1
As a component of the Department of Health and Human Services (HHS), OHRP is responsible for overseeing compliance with the HHS regulations governing research with human subjects. An institution engaged in nonexempt human subjects research conducted or supported by HHS must submit to OHRP written assurance that it will comply with the HHS regulations. This is now called a Federalwide Assurance (FWA). OHRP may also have jurisdiction over research not conducted or supported by HHS if an institution elects to extend its FWA to all nonexempt human subjects research at that institution, regardless of the source of funding.
As part of its oversight responsibilities, OHRP evaluates all written substantive allegations or indications of noncompliance with the HHS regulations. Sources of allegations or indications of noncompliance include research subjects or their loved ones, the institution itself, internal institutional whistleblowers, patient/subject advocates, and OHRP staff members. OHRP employees may raise concerns based on published accounts of research in the scientific literature or the lay media.
If OHRP has jurisdiction over the human subjects research that is allegedly out of compliance with the HHS regulations, it notifies the relevant institution of the allegations and asks the institution to conduct an investigation.2 The institution is asked to provide OHRP with a written report on the outcome of the investigation accompanied by pertinent institutional review board (IRB) records and any other materials relevant to the inquiry. After reviewing these materials—and in certain cases conducting telephone interviews with key individuals at the institution or on-site evaluations of an institution’s program for protecting human subjects—OHRP issues written determinations regarding whether research was reviewed and conducted in accordance with the HHS regulations. OHRP may require the institution to take corrective actions.
OHRP also has expanded its not-for-cause compliance oversight evaluation program. In this program institutions are chosen for evaluation in the absence of substantive allegations of noncompliance. OHRP can and does make determinations of noncompliance in the course of not-for-cause compliance oversight evaluations, and such determinations were included in our analysis reported here.
The compliance oversight determination letters examined for this article include those in which OHRP made a definitive citation of noncompliance with the HHS regulations and/or expressed concern about apparent regulatory or other deficiencies that resulted in the institution taking action. In addition, we examined letters in which the only determination was that no noncompliance occurred. The institutions in the sample include both domestic and international public and private universities, private research institutions, medical schools, academic medical centers, community hospitals, and federal health care and research facilities. The majority of the institutions were domestic (145 of 146). We also discuss trends that have evolved since our 2003 report in this journal.
Key Findings
Between August 1, 2002, and August 31, 2007, OHRP issued 235 determination letters to 146 institutions. These letters contained 762 citations of noncompliance and deficiencies. In our previous analysis, we examined 269 letters with 1,120 citations to 155 institutions.
Of the 146 institutions to which letters were issued, 133 had at least one citation of noncompliance or deficiency. Thirteen institutions (approximately 9%) did not receive a citation of noncompliance; in our previous report, the figure was approximately 8%.Table 1 identifies the number and percentage of the 146 institutions cited by OHRP for each major area of noncompliance or deficiency. It shows that institutions were most frequently cited for noncompliance and deficiencies related to the initial IRB review process (56% of institutions); informed consent documents and/or informed consent process approved by the IRB (51% of institutions); IRB continuing review process (22% of institutions); and IRB policies and procedures (20% of institutions). In our last review, these same categories of noncompliance represented the top four categories, but in a slightly different order: the initial review process (55%); IRB policies and procedures (55%); informed consent documents and/or informed consent process approved by the IRB (51%); and IRB continuing review process (45%).
Table 2 provides a distribution of the 762 citations for each major area of noncompliance or deficiency without regard to any particular institution. It should be noted that the total number of citations or deficiencies within a particular major area of noncompliance may be greater than the total number of institutions cited by OHRP—e.g., total number of citations relating to IRB approval of informed consent documents/process = 260 vs. total number of institutions = 146. This table was included, in part, to highlight that some institutions have been cited multiple times for major areas of noncompliance or deficiencies in subsequent, unrelated determination letters. The data inTable 2 show that the most common areas of noncompliance and deficiency involved informed consent documents and procedures (34%) and the process for IRB initial review of research protocols (20%). These were also the top two categories of citations in our previous review (27% and 25%, respectively). Fifteen percent of the citations of noncompliance and deficiencies involved written IRB procedures, which is the third most common category of citation. In our last review, the third most common category of citation was continuing review.
OHRP made 40 citations (5%) of noncompliance or deficiencies in which protocol changes were implemented without prior IRB review and approval, which was the fourth most common category of citation; this was the ninth most common category of citation in our previous review. Thirty-one (4%) of the citations of noncompliance or deficiencies were related to deficiency in IRB records, including minutes of IRB meetings. Of these citations, 14 (45%) related to noncompliance or deficiencies involving IRB records pertaining to poorly maintained IRB files, while another 12 (39%) involved IRB records pertaining to inadequate minutes of IRB meetings.
The “other miscellaneous” category inTable 2 involved 1) failure of investigators to ensure the protections at 45 CFR 46.111 (see description below); 2) inappropriate application of exempt categories of research; 3) failure to satisfy certain regulatory provisions pertaining to research with prisoners;3 4)failure to ensure that all collaborating institutions had an OHRP-approved assurance; 5) approval by an institutional official of research that had not been approved by the IRB; and 6) a signatory official’s interference, lack of authorization to act on behalf of the institution, or failure to assume obligations imposed by the HHS regulations. These last two findings were not included in our previous analysis, the former because it had not been made prior to 2004, and the latter because it was an isolated event that we did not feel merited discussion beyond issuing the determination letter.
Table 3 provides a breakdown by subject category of the 153 citations of noncompliance or deficiencies involving IRB initial review of research protocols. Almost two-thirds of the citations pertained to IRB failure to approve research protocols in accordance with HHS regulations at 45 CFR 46.111, a regulatory provision requiring IRBs to determine, among other things, that 1) risks to subjects are minimized and are reasonable in relation to anticipated benefits; 2) selection of subjects is equitable; 3) when appropriate, privacy of subjects and confidentiality of data is protected; and 4) appropriate safeguards are in place to protect vulnerable subjects. This represents substantially more violations in this category than we found in our last review, which showed about one-third of the citations in this category (153 citations in the 2004 study vs. 277 citations in the current one). The 27 (18%) citations related to research with children involved failure of the IRB to make the findings required under the HHS regulations for additional protections for research involving children.4
Table 4 indicates that nearly a third of the citations of noncompliance or deficiencies involved matters related to informed consent, which is similar to our previous analysis. The most common citations were that consent documents failed to adequately describe the purpose, procedures, and duration of the research (23%) and the risks and discomforts of the research (23%). Sixteen percent of the informed consent determinations involved complex informed consent language.
Although only 5% of the overall citations of noncompliance or deficiencies involved research conducted without IRB review or informed consent (Table 2 ), they represent the most serious forms of noncompliance and have occurred at multiple institutions. These percentages are similar to what we found in our previous analysis. Research conducted without IRB review and approval included open-label uncontrolled clinical trials, randomized controlled clinical trials, research involving tissue and data repositories, and research involving retrospective chart reviews. Some of the studies conducted without IRB review and approval utilized experimental interventions that involved greater-than-minimal risk. Of particular note, OHRP made 19 determinations of noncompliance involving research conducted without the legally effective informed consent of the subjects or their legally authorized representative and without the IRB appropriately waiving the requirements for obtaining informed consent.
Discussion
The data presented have certain limitations. For example, the letters examined only represent institutions that were involved with allegations or indications of noncompliance or underwent a not-for-cause evaluation (n = 146); the data do not represent a random sample of institutions that hold an OHRP-approved assurance (n > 10,000). The vast majority of cases were for-cause evaluations by OHRP during the time period noted above. In addition, for many evaluations the scope of research assessed was small, focusing on a single complaint and involving a limited sample of an institution’s human subjects research portfolio. Although OHRP usually evaluates systemic protections by reviewing an institution’s IRB policies and procedures and minutes of IRB meetings, this primary focus on a particular complaint should be kept in mind. Of note, while compliance with the HHS human research regulations is a necessary first step in the protection of human subjects, the regulatory protections represent a floor rather than a ceiling and do not in and of themselves guarantee the ethical conduct of research. Moreover, the actual conduct of the research is not necessarily under the control of the IRB.
The time period covered in this analysis is greater than that in our previous analysis (60 vs. 44 months). However, we note that OHRP made fewer citations of noncompliance and deficiencies during this time period (762 vs. 1,120), fewer determination letters were issued (235 vs. 269), and fewer institutions were cited for noncompliance or deficiencies (133 vs. 142). We also note that OHRP opened fewer for-cause compliance oversight evaluations during this time period than during the time period covered in the previous analysis (131 vs. 258). There are several possible reasons for this. First, OHRP has partially changed its policy on opening an evaluation when there is no HHS support for the research, but the institution has voluntarily extended its FWA to all research regardless of support. In these circumstances, if the research is regulated by the U.S. Food and Drug Administration (FDA), the case is typically referred to the FDA, and OHRP usually does not conduct its own evaluation, as was the case in the past. Second, more institutions have decided not to extend their FWA to research not supported by HHS, which means that OHRP has jurisdiction over fewer research studies than in the past. Based on an informal review of a sample of institutions, approximately 74% of domestic institutions currently holding an FWA that formerly held a Multiple Project Assurance (MPA) apply either subpart A or subparts A, B, C, and D of 45 CFR 46 to all research regardless of support. On the other hand, greater than 90% of those same institutions applied 45 CFR 46 to all research regardless of support when they held MPAs. Finally, issues involving human subjects and research are often reported in the press, highlighting the general importance of human subject protections, and a growing human protection industry provides education, certification, professional review, and record-keeping systems for IRBs. Thus, it’s possible that more institutions have robust human protection programs now than in the past, which could account for fewer incidents of noncompliance. However, there are no empirical data to indicate whether this is the case.
Our current analysis also demonstrates that since our 2003 report, there has been an increase in the percentage of institutions with citations of noncompliance and deficiencies regarding changes in research without prior IRB review and approval (25% in our previous analysis and 35% in our current analysis). The types of changes that were implemented without prior IRB review and approval included: enrollment of ineligible subjects into protocols; surpassing IRB-approved subject accrual limits; performing procedures not listed in the protocol (such as sending letters to prospective subjects, implementing a subject incentive program, substituting a study drug, and performing additional tests); failing to conduct protocol specified procedures; changing IRB-approved protocol procedures (such as subject compensation, ventilator settings, changing a multiarm study to a single-arm study, or changing a double-blind study to a single-blind study). We note that these citations of noncompliance and deficiencies generally indicated investigator noncompliance rather than IRB noncompliance.
As many readers are aware, HHS regulations at 45 CFR 46.103(b)(4)(iii) require that during the period for which IRB approval has been given, prior to initiating changes in the protocol, the investigator must obtain IRB approval for the proposed changes except when changes are necessary to eliminate apparent immediate hazards to the research participants. Although the percentage difference in citations regarding approval of changes is not statistically significant, this category of noncompliance and deficiency currently is the fourth most cited, whereas previously it ranked ninth. This difference in rank order is due, in part, to a decrease in the number and percentage of other categories of citation of noncompliance and deficiency since our last analysis. For example, in our previous analysis, the number of citations in the category of IRB continuing review process was 109 (third in rank order), and in our current analysis it was 37 (fifth). This decrease is statistically significant (37/762 vs. 109/1,120; chi-square = 15.1; p = 0.00001). Conversely, in our previous analysis, the number of citations of noncompliance and deficiency in the category of written IRB policies and procedures was 88 (fourth in rank order), and in our current analysis it was 113 (third). This increase is also statistically significant (113/762 vs. 88/1,120; chi-square = 23.1; p < 0.00001).
The complex and unique nature of many research protocols requires careful thought on how to apply the HHS regulations. Resources are available to assist investigators in developing protocols that maximize human subject protections.5 Investigators should communicate frequently with the IRB and be familiar with the policies and procedures at their institutions for obtaining approval for research involving human subjects. Additionally, investigators, IRB members, and institutional officials are encouraged to contact regulatory agencies, such as OHRP and the FDA, to obtain guidance on issues that relate to the protection of human subjects and how these regulations affect particular protocols. Further, OHRP’s Division of Education and Development supports education of institutional officials, IRB members, investigators, and the general public about human subject protections by distributing educational materials, conducting workshops, and presenting seminars.
The analysis of these data provides OHRP with a reference point by which it may develop targeted guidance and educational programs to help strengthen protections for human subjects. Awareness of the most frequent problem areas identified by OHRP may allow institutions to take proactive measures before noncompliance occurs.
Disclaimer
The views represented here are those of the authors and are not intended to represent those of the Department of Health and Human Services.
Carol Weil, JD,is Public Health Analyst, Division of Compliance Oversight, Office for Human Research Protections, Rockville, MD;Lisa Rooney, JD,is Public Health Analyst, Division of Compliance Oversight, Office for Human Research Protections, Rockville, MD;Patrick McNeilly, PhD,is Health Scientist Administrator, Office of Extramural Research, Education, and Priority Populations, Agency for Healthcare Research and Quality, Rockville, MD;Karena Cooper, JD, MSW,is Regulatory Counsel, Food and Drug Administration, Center for Drug Evaluation and Research, Office of Compliance, Silver Spring, MD,Kristina Borror, PhD,is Director, Division of Compliance Oversight, Office for Human Research Protections, Rockville, MD; andPaul Andreason, MD,is Staff Psychiatrist, Walter Reed Army Medical Center, Washington, DC.
References
1. Borror K, Carome M, McNeilly P, and Weil C. A review of OHRP compliance oversight letters.IRB: Ethics &Human Research 2003;25(5):1-4.
2. OHRP’s Compliance Oversight Procedures for Evaluating Institutions, October 14, 2009.https://www.hhs.gov/ohrp/compliance/ohrpcomp.pdf .
3. Department of Health and Human Services, Additional DHHS Protections Pertaining to Biomedical and Behavioral Research Involving Prisoners as Subjects. 45 CFR 46, Subpart C.
4. Department of Health and Human Services. Additional DHHS Protections for Children Involved as Subjects in Research, 45 CFR part 46 Subpart D.
5. For example, see Office for Human Research Protectionshttps://www.hhs.gov/ohrp/ ; Public Responsibility in Medicine and Research,https://www.primr.org .
Carol Weil, Lisa Rooney, Patrick McNeilly, Karena Cooper, Kristina Borror, and Paul Andreason, “OHRP Compliance Oversight Letters: An Update,”IRB: Ethics & Human Research 32, no. 2 (2010): 1-6.