looked a lot more like a guess, and guessing, of course, leads to mistakes. Somewhere between 2 and 8 percent of the time in American hospitals, a patient having a genuine heart attack gets sent home—because the doctor doing the examination thinks for some reason that the patient is healthy. More commonly, though, doctors correct for their uncertainty by erring heavily on the side of caution. As long as there is a chance that someone might be having a heart attack, why take even the smallest risk by ignoring her problem?
“Say you’ve got a patient who presents to ER complaining of severe chest pain,” Reilly says. “He’s old and he smokes and he has high blood pressure. There are lots of things to make you think, Gee, it’s his heart. But then, after evaluating the patient, you find out his ECG is normal. What do you do? Well, you probably say to yourself, This is an old guy with a lot of risk factors who’s having chest pain. I’m not going to trust the ECG.” In recent years, the problem has gotten worse because the medical community has done such a good job of educating people about heart attacks that patients come running to the hospital at the first sign of chest pain. At the same time, the threat of malpractice has made doctors less and less willing to take a chance on a patient, with the result that these days only about 10 percent of those admitted to a hospital on suspicion of having a heart attack actually have a heart attack.
This, then, was Reilly’s problem. He wasn’t back at Dartmouth or over in one of the plush private hospitals on Chicago’s north side, where money wasn’t an issue. He was at Cook County. He was running the Department of Medicine on a shoestring. Yet every year, the hospital found itself spending more and more time and money on people who were not actually having a heart attack. A single bed in Cook County’s coronary care unit, for instance, cost roughly $2,000 a night—and a typical chest pain patient might stay for three days—yet the typical chest pain patient might have nothing, at that moment, wrong with him. Is this, the doctors at Cook County asked themselves, any way to run a hospital?
“The whole sequence began in 1996,” Reilly says. “We just didn’t have the number of beds we needed to deal with patients with chest pain. We were constantly fighting about which patient needs what.” Cook County at that time had eight beds in its coronary care unit, and another twelve beds in what’s called intermediate coronary care, which is a ward that’s a little less intensive and cheaper to run (about $1,000 a night instead of $2,000) and staffed by nurses instead of cardiologists. But that wasn’t enough beds. So they opened another section, called the observation unit, where they could put a patient for half a day or so under the most basic care. “We created a third, lower-level option and said, ‘Let’s watch this. Let’s see if it helps.’ But pretty soon what happened is that we started fighting about who gets into the observation unit,” Reilly went on. “I’d be getting phone calls all through the night. It was obvious that there was no standardized, rational way of making this decision.”
Reilly is a tall man with a runner’s slender build. He was raised in New York City, the product of a classical Jesuit education: Regis for high school, where he had four years of Latin and Greek, and Fordham University for college, where he read everything from the ancients to Wittgenstein and Heidegger and thought about an academic career in philosophy before settling on medicine. Once, as an assistant professor at Dartmouth, Reilly grew frustrated with the lack of any sort of systematic textbook on the everyday problems that doctors encounter in the outpatient setting—things like dizziness, headaches, and abdominal pain. So he sat down and, in his free evenings and weekends, wrote an eight-hundred-page textbook on the subject, painstakingly reviewing the available evidence for the most common problems a general practitioner might encounter. “He’s always exploring different topics, whether it’s philosophy or Scottish poetry or the history of medicine,” says his friend and colleague Arthur Evans, who worked with Reilly on the chest pain project. “He’s usually reading five books at once, and when he took a sabbatical leave when he was at Dartmouth, he spent the time writing a novel.”
No doubt Reilly could have stayed on the East Coast, writing one paper after another in air-conditioned comfort on this or that particular problem. But he was drawn to Cook County. The thing about a hospital that serves only the poorest and the neediest is that it attracts the kinds of nurses and doctors who want to serve the poorest and neediest—and Reilly was one of those. The other thing about Cook County was that because of its relative poverty, it was a place where it was possible to try something radical—and what better place to go for someone interested in change?
Reilly’s first act was to turn to the work of a cardiologist named Lee Goldman. In the 1970s, Goldman got involved with a group of mathematicians who were very interested in developing statistical rules for telling apart things like subatomic particles. Goldman wasn’t much interested in physics, but it struck him that some of the same mathematical principles the group was using might be helpful in deciding whether someone was suffering a heart attack. So he fed hundreds of cases into a computer, looking at what kinds of things actually predicted a heart attack, and came up with an algorithm—an equation—that he believed would take much of the guesswork out of treating chest pain. Doctors, he concluded, ought to combine the evidence of the ECG with three of what he called urgent risk factors: (1) Is the pain felt by the patient unstable angina? (2) Is there fluid in the patient’s lungs? and (3) Is the patient’s systolic blood pressure below 100?
For each combination of risk factors, Goldman drew up a decision tree that recommended a treatment option. For example, a patient with a normal ECG who was positive on all three urgent risk factors would go to the intermediate unit; a patient whose ECG showed acute ischemia (that is, the heart muscle wasn’t getting enough blood) but who had either one or no risk factors would be considered low-risk and go to the short-stay unit; someone with an ECG positive for ischemia and two or three risk factors would be sent directly to the cardiac care unit—and so on.
Goldman worked on his decision tree for years, steadily refining and perfecting it. But at the end of his scientific articles, there was always a plaintive sentence about how much more hands-on, real-world research needed to be done before the decision tree could be used in clinical practice. As the years passed, however, no one volunteered to do that research—not even at Harvard Medical School, where Goldman began his work, or at the equally prestigious University of California at San Francisco, where he completed it. For all the rigor of his calculations, it seemed that no one wanted to believe what he was saying, that an equation could perform better than a trained physician.
Ironically, a big chunk of the funding for Goldman’s initial research had come not from the medical community itself but from the navy. Here was a man trying to come up with a way to save lives and improve the quality of care in every hospital in the country and save billions of dollars in health care costs, and the only group that got excited was the Pentagon. Why? For the most arcane of reasons: If you are in a submarine at the bottom of the ocean, quietly snooping in enemy waters, and one of your sailors starts suffering from chest pain, you really want to know whether you need to surface (and give away your position) in order to rush him to a hospital or whether you can stay underwater and just send him to his bunk with a couple of Rolaids.
But Reilly shared none of the medical community’s qualms about Goldman’s findings. He was in a crisis. He took Goldman’s algorithm, presented it to the doctors in the Cook County ED and the doctors in the Department of Medicine, and announced that he was holding a bake-off. For the first few months, the staff would use their own judgment in evaluating chest pain, the way they always had. Then they would use Goldman’s algorithm, and the diagnosis and outcome of every patient treated under the two systems would be compared. For two years, data were collected, and in the end, the result wasn’t even close. Goldman’s rule won hands down in two directions: it was a whopping 70
Why is the Cook County experiment so important? Because we take it, as a given, that the more information decision makers have, the better off they are. If the specialist we are seeing says she needs to do more tests or examine us in more detail, few of us think that’s a bad idea. In Millennium Challenge, Blue Team took it for granted that because they had more information at their fingertips than Red Team did, they had a considerable advantage. This was the second pillar of Blue Team’s aura of invincibility. They were more logical and systematic than Van Riper, and they knew more. But what does the Goldman algorithm say? Quite the opposite: that all that extra information isn’t actually an advantage at all; that, in fact, you need to know very little to find the underlying
