- BIOETHICS FORUM ESSAY
Spin Doctors and Torture Doctors: Inconvenient Truths About Complex Systems
Here are some of the things we know, and keep forgetting, about complex systems:
Complex systems are continuously changing. The individuals who are part of these systems are continuously adapting to changing conditions. Their adaptations further change but do not “fix” the system.
Complex systems are not intrinsically safe or unsafe. Studies of workers in complex systems such as health care, energy, and transportation reveal that while these workers create safety continuously through their adaptations to ever-changing circumstances, these adaptations can also lead to failures, as when efforts to ramp up production direct attention away from hazards associated with these efforts. Thus, in complex systems safety is in play at all times.
Complex systems cannot be controlled by a single individual. However, we act as if they can be.
Complex systems are not fully describable. We cannot stop the ever-changing systems known as “health care” or “the Internet” so that we can get a good look at them and write down what we see. If we want to understand a complex system, we should expect to go after, collect, and analyze multiple stories from different perspectives.
System theorists distinguish between “complex” and “complicated.” A patient care system is complex. A medical device is complicated if it has lots of parts that can fail. How this device is used and how these uses can succeed or fail are features of the system’s complexity. Relationships between individuals in a complex system can have lots of parts, too. (As we often say of human relationships, it’s complicated.)
These insights are useful in getting a grip on the slippery situation in the Gulf of Mexico, including its ethical dimensions and our own moral judgments. We’re frustrated that the oil keeps spilling and that there’s no quick fix in sight. We’re angry about the revelations of safety violations, even though we know about hindsight bias, and that this system looked good enough until it blew up.
Even if we think it’s odd to demand that our chief executive shows anger while solving a problem, we still retain some hope that he can solve this problem. We’re spooked by those underwater videos and by images of plumes drifting toward the coast, distressed by those photos of oil-soaked birds, and are not sure what to make of the flow of data and analysis we’re receiving: it’s a catastrophe, but unless we’re on that coast, it may not feel like one yet. How we decide which story to follow in this complex system may be a product of our complicated personal histories and values, as well as what systems theorist Erik Hollnagel terms ETTO: efficiency-thoroughness trade-offs.
Here’s how ETTO works: I visited the Web site of one environmental organization after I recognized the name of its oil spill expert in a news article. I justified choosing this organization’s information flow, and then getting on with my own work, as follows: Because I remembered that this expert once worked with my mother on a Congressional campaign staff, I presumed that we shared relevant values, and I was inclined to trust his organization’s story (efficiency) even though I knew it was not the whole story, and that I should pay attention to other stories (thoroughness). And I fully intend to do this – when I find some spare time, and before my next ETTO arises.
Complexity theory helps us get a grip on another slippery story in the news, one with even greater relevance to bioethics. The report from Physicians for Human Rights, alleging that the torture of detainees in U.S. custody incorporated research methods – including the systematic collection of data and the use of data to guide practice – follows a leaked report from the International Committee of the Red Cross that, like the PHR report, focused on the role of medical professionals as “monitors.” The allegations in these reports reveal a looking-glass-land version of a legitimate health care system, in which goals such as safety and effectiveness were applied to illegitimate activities, as if torture could be considered safe as long as those being tortured did not die, and as if effective torture methods fell within the scope of quality improvement.
These reports have rightly triggered demands that the medical professionals involved be held accountable for actions that so grossly violated medical ethics and were integral to human rights violations. In holding these professionals to account we must also keep in mind the complex system – the torture system – in which they were workers and moral agents, and keep pressing for more accounts of how this system came to be created and tolerated, and what factors bred the complicity that now shocks the conscience.
Nancy Berlinger is deputy director of The Hastings Center.