Samburu or the Rendille were living at a bare subsistence level. To explain away Mann’s research on the Masai, Keys then evoked more recent research suggesting that the Masai, living in nomadic isolation for thousands of years, must have somehow evolved a unique “feedback mechanism to suppress endogenous cholesterol synthesis.” This mechanism, Keys suggested, would bestow immunity on the Masai to the cholesterol-raising effects of fat.
To believe Keys’s explanation, we would have to ignore Mann’s further research reporting that the Masai indeed had extensive atherosclerosis, despite their low cholesterol, without suffering heart attacks or any other symptoms of coronary heart disease. And we’d have to ignore still more research reporting that when the Masai moved into nearby Nairobi and began eating traditional Western diets, their cholesterol rose considerably. By 1975, Keys had relegated the Masai, and even the Samburu and the Rendille, to the sidelines of the controversy: “The peculiarities of those primitive nomads have no relevance to diet-cholesterol-CHD [coronary heart disease] relationships in other populations,” he wrote.
Once having adopted firm convictions about the dangers of dietary fat based on his own limited research among small populations around the world, Keys repeatedly preached against the temptation to adopt any firm contrary convictions based on the many other studies of small populations that seemed to repudiate his hypothesis. “The data scarcely warrant any firm conclusion,” he would write about such contradictory evidence. When a 1964 article in JAMA,
The Framingham Heart Study was an ideal example of this kind of selective thinking at work. The study was launched in 1950 under Thomas Dawber’s leadership to observe in a single community aspects of diet and lifestyle that might predispose its members to heart disease—risk factors of heart disease, as they would come to be called. The factory town of Framingham, Massachusetts, was chosen because it was what Dawber called a “reasonably typical” New England town. By 1952, fifty-one hundred Framingham residents had been recruited and subjected to comprehensive physicals, including, of course, cholesterol measurements. They were then re-examined every two years to see who got heart disease and who didn’t. High blood pressure, abnormal electrocardiograms, obesity, cigarette smoking, and genes (having close family with heart disease) were identified as factors that increased the risk of heart disease. In October 1961, Dawber announced that cholesterol was another one. The risk of heart disease for those Framingham men whose cholesterol had initially been over 260 mg/dl was five times greater than it was for men whose cholesterol had been under 200. This is considered one of the seminal discoveries in heart- disease research. It was touted as compelling evidence that Keys’s hypothesis was correct.
But there were caveats. As the men aged, those who succumbed to heart disease were ever more likely to have low cholesterol (as had Eisenhower) rather than high cholesterol. The cholesterol/heart-disease association was tenuous for women under fifty, and nonexistent for women older. Cholesterol has “no predictive value,” the Framingham investigators noted in 1971. This means women over fifty would have no reason to avoid fatty foods, because lowering their cholesterol by doing so would not lower their risk of heart disease. None of this was deemed relevant to the question of whether Keys’s hypothesis was true.
The dietary research from Framingham also failed to support Keys’s hypothesis. This never became common knowledge, because it was never published in a medical journal. George Mann, who left the Framingham Study in the early 1960s, recalled that the NIH administrators who funded the work refused to allow publication. Only in the late 1960s did the NIH biostatistician Tavia Gordon come across the data and decide they were worth writing up. His analysis was documented in the twenty-fourth volume of a twenty-eight-volume report on Framingham released in 1968. Between 1957 and 1960, the Framingham investigators had interviewed and assessed the diet of a thousand local subjects. They focused on men with exceedingly high cholesterol (over 300) and exceedingly low cholesterol (under 170), because these men “promised to be unusually potent in the evaluation of dietary hypotheses.” But when Gordon compared the diet records of the men who had very high cholesterol with those of the men who had very low cholesterol, they differed not at all in the amount or type of fat consumed. This injected a “cautionary note” into the proceedings, as the report noted. “There is a considerable range of serum cholesterol levels within the Framingham Study Group. Something explains this inter-individual variation, but it is not diet (as measured here).”
“As measured here” encapsulates much of the challenge of scientific investigation, as well as the loophole that allowed the dietary-fat controversy to evolve into Henry Blackburn’s two strikingly polar attitudes. Perhaps the Framingham investigators failed to establish that dietary fat caused the high cholesterol levels seen in the local population because (1) some other factor was responsible or (2) the researchers could not measure either the diet or the cholesterol of the population, or both, with sufficient accuracy to establish the relationship.
As it turned out, however, the Framingham Study wasn’t the only one that failed to reveal any correlation between the fat consumed and either cholesterol levels or heart disease. This was the case in virtually every study in which diet, cholesterol, and heart disease were compared
Ever since Sir Francis Bacon, in the early seventeenth century, scientists and philosophers of science have cautioned against the tendency to reject evidence that conflicts with our preconceptions, and to make assumptions about what assuredly would be true if only the appropriate measurements or experiments could be performed. The ultimate danger of this kind of selective interpretation, as I suggested earlier, is that a compelling body of evidence can be accumulated to support
To Keys, Stamler, Dawber, and other proponents of the dietary-fat hypothesis, the positive evidence was all that mattered. The skeptics considered the positive evidence intriguing but were concerned about the negative evidence. If Keys’s hypothesis was incorrect, it was only the negative evidence that could direct investigators to the correct explanation. By the 1970s, it was as if the two sides had lived through two entirely different decades of research. They could not agree on the dietary-fat hypothesis; they could barely discuss it, as Henry Blackburn had noted, because they were seeing two dramatically different bodies of evidence.
Another revealing example of selection bias was the reanalysis of a study begun in 1957 on fifty-four hundred male employees of the Western Electric Company. The original investigators, led by the Chicago cardiologist Oglesby Paul, had given them extensive physical exams and come to what they called a “reasonable approximation of the truth” of what and how much each of these men ate. After four years, eighty-eight of the men had developed symptoms of coronary heart disease. Paul and his colleagues then compared heart disease rates among the 15 percent of the men who seemingly ate the most fatty food with the 15 percent who seemingly ate the least. “Worthy of comment,” they reported, “is the fact that of the 88 coronary cases, 14 have appeared in the high-fat
