From Bioethics Briefings
Quality Improvement Methods in Health Care
- The American health care system has serious problems with quality and safety that can be reduced through quality improvement (QI) activities.
- QI uses data-based methods to bring about immediate improvements in health care delivery.
- QI has encountered an unexpected obstacle in federal regulations governing the ethical conduct of human subjects research.
- There is substantial uncertainty about the relationship between QI and the federal regulations for human subjects research.
- Because QI uses methods like those used in research and often involves interactions with patients and health care workers, some activities could be classified as both QI and human subjects research.
- While all agree that QI should be conducted ethically, which QI activities must meet the ethical standards embodied in the federal regulations is not clear.
- It is essential to clarify the ethical requirements for QI and other data-guided management and research activities, bring the regulatory system into harmony with them, and minimize the effort and resources needed to ensure ethical conduct of these activities.
Framing the Issue
The American health care system has serious problems with quality and safety. One effective way to attack these problems is through the methods of quality improvement (QI). The term QI refers to activities that use data-based methods—some developed in manufacturing industries—to bring about immediate improvements in health care delivery. Change has always been an intrinsic part of medical practice, as clinicians and managers adapt to new medical knowledge, new technology, and new patterns of disease. QI methods enable them to make change in a systematic way, measuring and assessing the effects of a change, feeding the information back into the clinical setting, and making adjustments until they are satisfied with the results.
Although QI has achieved notable successes, it has run into an unexpected obstacle in the form of the federal regulations governing research with human subjects. These regulations were created in response to situations in which people were harmed by being made the subjects of dangerous research without their knowledge and consent, as in the infamous Tuskegee study of untreated syphilis in black men. In research, an investigator exposes subjects to risks in order to benefit others, not the subjects themselves. The regulatory system is intended to respect subjects’ rights by ensuring that research risks are not excessive, that the confidentiality of personal information is protected, and that potential subjects are informed about risks and voluntarily agree to participate. It specifies rules for the ethical conduct of research and requires federally funded projects to receive review and approval from committees called institutional review boards (IRBs) before being carried out. Although developed for federally funded research, the rules are considered basic ethical requirements. The government encourages organizations receiving federal funds to apply them to all their research with human subjects, and many journals make IRB approval a condition for publication of research results.
There is substantial uncertainty about the relationship between QI and this framework of protection. One might ask why the framework is even relevant to QI, since QI is an extension of clinical practice—an activity designed to make the local system of care work better, rather than to develop knowledge. The answer lies in the rule that an activity that has any research elements and involves human subjects must be treated as human subjects research. Since QI uses methods like those used in research, often involves interactions with patients and health care workers, and may produce results that contribute to general understanding of the improvement process, some QI activities could be classified as both QI and human subjects research. The problem is that while all agree that QI should be conducted ethically, which QI activities must meet the specific ethical standards embodied in the regulations is not clear. Following the research rules when they don’t apply increases the cost and complexity of doing QI and may discourage useful improvement work. Not following them when they do apply may endanger subjects and result in sanctions from the federal Office for Human Research Protections.
The Nature of the Problem
This problem has been simmering for some time in the QI and research communities. In 2007, it came to the attention of policymakers and the public because of a QI activity that dramatically reduced infection rates in intensive care units in 67 Michigan hospitals. (See “Avoiding Deadly Infections—and Red Tape.”) An anonymous report to OHRP alleged that the project was human subjects research and violated the federal regulations. OHRP agreed, and in its initial determination letter said that investigators should have obtained local IRB approval for the project at each hospital, as well as informed consent from both the health care workers and the ICU patients involved. The ruling was controversial and attracted significant media attention. A resolution was ultimately achieved in this case, but for QI practitioners, it has not resolved the uncertainty about how to interpret the regulations in the future.
People who use QI methods have three major issues with the human research protection system. They find it difficult to determine whether a QI activity is also human subjects research. When it is, they find it difficult to determine what the regulations require. And what is required often seems overly burdensome and unrelated to the stated goal of protecting people.
Is this QI activity human subjects research?
Only activities that meet the OHRP definition of research and that involve people who meet the definition of human subject come under the regulations. Unfortunately, the definitions are ambiguous (see box, “Research Glossary”). Depending on the interpretation of specific words in the research definition–especially “generalizable”–nearly all QI activities could also be considered research, or only a few. The definition of human subject requires interpretation of terms such as “intervention” and “identifiable.” OHRP has issued guidance stating that QI activities are important to the health care system and that most QI activities are not also human subject research; however, it has not provided clear guidance on how to identify those QI activities that are.
If an activity is both QI and human subjects research, what is required?
The rules are both hard to understand and unpredictable. Eleven complex decision charts summarize the regulations. The key questions are: Does the activity require IRB review? Will the IRB find the project’s design ethically acceptable and approve it? Will the IRB require specific informed consent? The first step is to determine whether the activity falls into one of six exempt categories of human subjects research; if so, the regulations do not apply. OHRP has issued guidance stating that researchers shouldn’t be allowed to decide on their own that their projects are exempt. In response, many organizations require the research to be reviewed by the IRB chair or the IRB itself to establish that it doesn’t require IRB review.
If the activity is not exempt, it might be expeditable—eligible for review by a single IRB member, rather than the full IRB. The activity also might be eligible for a waiver or an alteration of the standard requirement that all human subjects give their informed consent. Making these determinations requires knowledge of details about the precise timing of the collection of data and the type of interaction with the subjects, along with interpretation of regulatory language such as “minimal risk.” Often a small difference in a detail or an interpretation makes a large difference in the regulatory burden, yet seems unrelated to any difference in subject protection.
The IRB serving the research site interprets and applies the regulations to specific projects. The results are unpredictable because IRB interpretations vary significantly, and the local IRB’s decision prevails, except in the rare instance in which OHRP chooses to intervene. When research is to be conducted at multiple institutions, the regulations allow one IRB to review and approve the project, but each institution must have an authorization agreement with that IRB’s institution and document that fact to OHRP. Otherwise, each institution’s IRB reviews the project, which is time-consuming and can lead to different answers to the key questions.
Is all this really worth it? Does it appropriately protect people?
The regulatory system was developed to protect the human subjects of risky research. To many people, it seems unsuited to data-guided management activities designed to improve care in particular health care facilities, whether the activities are considered QI alone or a combination of QI and human subjects research. All agree that these activities need clearly articulated ethical requirements and a process for ensuring that the requirements are met. In the opinion of many, the requirements are not the same as for clinical research projects, and the IRB process is not designed to provide the appropriate kind of supervision.
QI and research differ in their relationship to the health care system. In research, the person responsible for the project’s design and management (the investigator) is seeking knowledge, not the immediate benefit of the research subjects. The IRB was designed to provide independent, prospective ethical review to protect subjects from exploitation by researchers focused on the success of their research. In contrast, the clinicians and managers who engage in QI are directly responsible for patients’ well-being, and the existence of a robust QI program benefits the patients they serve. Patients are receiving care in a setting that is prone to errors and inefficiencies and that experiences constant change. It is in their interest to have their health care providers use data to manage care processes, identify best practices, and learn how to put these practices in place.
Because of their different relationships to the health care system, human subjects research and QI have different ethical requirements for informed consent. In research, all subjects must give informed consent. Even when the risk is minimal, the IRB must explicitly waive the requirement and can only do so if it finds that without the waiver, the research is “not practicable” (meaning it cannot be done). In the quality improvement context, a requirement for specific informed consent to every activity seems excessive. Instead, when patients seek care from an organization, they should understand that using data to manage and improve care (with confidentiality protected) is part of normal health care operations. Their informed consent to a specific activity should be required only when the activity exposes them to significant additional risk compared to the risk inherent in receiving standard care. Similarly, the informed consent of workers to inclusion in QI shouldn’t be an automatic ethical requirement. Health professionals have an ethical duty to improve their performance, and health care organizations have an ethical duty to make sure that their workers are competent. Worker consent should be required if an activity imposes risks beyond those inherent in the normal work situation: for example, exposure to a toxic chemical, or collecting information employers are not entitled to have (such as on alcohol use outside the workplace). Consent should not be an ethical requirement if the only risk is from the collection of information related to job performance, since the economic risk associated with being found to be incompetent or unnecessary is part of basic workplace risk.
The IRB process is not the best way to ensure that QI and other data-guided managerial activities are ethically conducted. Quality-conscious organizations should be constantly engaged in these activities. Most are routine and carry minimal risk. Prospective review of each one by an IRB would be slow and costly and could discourage engaging in them. Moreover, the IRB process is not designed to provide the kind of supervision QI needs. A research project has a fixed protocol with fixed goal, methodology, population, and time period, whereas QI activities entail frequent adjustments in the intervention, the measurement, and even the goal over time as experience accumulates. It would be more effective to integrate ethical oversight of QI—including QI that is also minimal-risk human subjects research—into an organization’s normal management and supervision structure and to hold QI practitioners responsible for obeying ethical rules through the system of accountability for the quality of clinical care. Organizations might decide to establish an IRB-like committee to perform prospective review of some activities but could reserve it for QI and other data-guided activities that involve significant additional risk.
Learning Health Care Systems
The first step to solving this problem is to put it into a broader context. Around the time of the infection control project controversy, the Institute of Medicine (renamed the National Academy of Medicine) was calling on health care leaders to transform their health care delivery systems into “learning health care systems” capable of studying and continuously improving their practices. Such systems would routinely conduct a wide variety of data-guided investigations, including activities using QI methods but also clinical effectiveness studies, health services research, quality improvement and cost containment research, and other management and research activities. The recommendation was the end result of the Institute’s years-long, in-depth study of patient safety and quality problems in American health care.
In learning health care systems, the traditional distinction between treatment and research that forms the foundation of the human subjects protection framework disappears, since by definition, treatment and research are linked. Therefore, we need a new framework for ensuring the ethical conduct of their activities, one that depends on the level of risk and burden and not on whether an activity is seen as treatment or research. First, we must clarify the ethical requirements for all the various data-based activities that learning health care systems engage in, not just QI and QI-related human subjects research. Then we must bring the regulatory system into harmony with those requirements and minimize the effort and resources needed to ensure that the requirements are met.
In short, a proposed solution should:
- Be based on a clear, common sense ethical framework and have rules that are easy to apply in specific cases.
- Encourage data-guided management of organizational change, health services research on processes and outcomes, and collaboration across organizations to share best practices and improve the standard of care.
- Appropriately protect the confidentiality of patient and worker information.
- Minimize the use of prospective IRB review and, instead, integrate ethical oversight of these activities into the system of accountability for the conduct of clinical care.
- Link the intensity of scrutiny of a specific activity and the requirement for specific informed consent from patients or workers to the level of additional risk the activity imposes on them, compared to the risk inherent in receiving or providing standard health care.
- Be cost-conscious. Most of the resources for these activities come directly from patient care resources; thus, imposing significant costs on their conduct in the name of patient protection is in itself a harm to patients if the protective measures are ineffective or redundant.
Avoiding Deadly Infections – and Red Tape
No QI effort has gained more notoriety in recent years than one involving a simple checklist to reduce catheter-related bloodstream infections in intensive care units, which cause as many as 28,000 deaths in the United States each year. Following a checklist of five proven procedures, such as washing hands and disinfecting the patient’s skin before inserting the catheter, can prevent infections and save lives. An initiative involving the checklist underscores the confusion that even experts experience in determining how to apply the human subjects protection regulations to quality improvement work.
Peter Pronovost and colleagues developed the checklist and successfully implemented it at their institution, Johns Hopkins University Hospital. The checklist intervention then became part of a quality improvement initiative undertaken by the Michigan Health and Hospital Association (MHA) in collaboration with Johns Hopkins, with funding from a federal agency. Pronovost and colleagues helped team leaders in 67 local hospitals introduce the checklist in 103 ICUs. The hospitals collected data on catheter-related bloodstream infections (most had already been collecting it before the initiative). Local ICU teams were given monthly feedback on infection rates in their units and comparisons with aggregate data from other ICUs.
The Johns Hopkins investigators used the ICU data to evaluate the project and in 2006 reported in the New England Journal of Medicine that the program was highly successful, reducing catheter-related infections by up to 66% over the course of 18 months. Before beginning the project, the investigators had submitted it to a Johns Hopkins institutional review board. The IRB classified it as research that was exempt from the regulations because the data provided to the investigators did not identify individuals and the research represented minimal risk.
After publication, an anonymous complaint was made to the Office for Human Research Protections (OHRP) stating that the program did not qualify for the exemption and that the investigators were violating the regulations. OHRP agreed. In 2007, OHRP determined that the investigators should have obtained IRB approval for the project at each hospital and informed consent from the health care workers and the ICU patients involved. It directed Johns Hopkins and the MHA to take corrective action. This determination led the MHA to suspend the ongoing effort to promote and evaluate the checklist.
A public outcry followed. Especially influential was an op-ed essay in the New York Times in late December 2007 by Atul Gawande, a surgeon at Brigham and Women’s Hospital in Boston and staff writer at the New Yorker. “The government’s decision was bizarre and dangerous,” Gawande wrote.
In February 2008, OHRP retreated from its original position and reached a compromise with Johns Hopkins and the MHA on the corrective actions required. Dr. Ivor Pritchard, OHRP’s acting director, emphasized that the checklist initiative could continue, saying, “We do not want to stand in the way of quality improvement activities that pose minimal risks to subjects.”
Research – A systematic investigation, including research development, testing, and evaluation, that is designed to develop or contribute to generalizable knowledge.
Human subject – A living individual about whom an investigator (whether professional or student) conducting research obtains (1) data through intervention or interaction with the individual, or (2) identifiable private information.
Mary Ann Baily, PhD, is a Hastings Center Fellow.
45% of our work is supported by individual donors like you. Support our work.
- Office for Human Research Protections
- Ethical Oversight of Learning Health Care Systems, Hastings Center Report special report, 43, no.1 (2013).
- Mary Ann Baily, “Harming Through Protection?” New England Journal of Medicine 358 (2008): 768-9.
- Joanne Lynn et al., “The Ethics of Using Quality Improvement Methods in Health Care,” Annals of Internal Medicine 146, no. 9 (2007): 666-73.
- Mary Ann Baily, Melissa Bottrell, Joanne Lynn, and Bruce Jennings, The Ethics of Using QI Methods to Improve Health Care Quality and Safety, Hastings Center Report special report July-August 2006.
- Mary Ann Baily, PhD
- Mildred Z. Solomon, EdD