The Tell-Tale Brain

FIGURE 3.4 A cluster of 2s scattered among 5s. It is difficult for normal subjects to detect the shape formed by the 2s, but lower synesthetes as a group perform much better. The effect has been confirmed by Jamie Ward and his colleagues.

FIGURE 3.5 The same display as Figure 3.4 except that the numbers are shaded differently, allowing normal people to see the triangle instantly. Lower synesthetes (“projectors”) presumably see something like this.

Ed and I realized that, for the first time since Francis Galton, we had clear, unambiguous proof from our experiments (grouping and popout) that synesthesia was indeed a real sensory phenomenon—proof that had eluded researchers for over a century. Indeed, our displays could not only be used to distinguish fakes from genuine synesthetes, but also to ferret out closet synesthetes, people who might have the ability but not realize it or not be willing to admit it.

ED AND I sat back in the cafe discussing our findings. Between our experiments with Francesca and Mirabelle, we had established that synesthesia exists. The next question was, why does it exist? Could a glitch in brain wiring explain it? What did we know that could help us figure this out? First, we knew that the most common type of synesthesia is apparently number-color. Second, we knew that one of the main color centers in the brain is an area called V4 in the fusiform gyrus of the temporal lobes. (V4 was discovered by Semir Zeki, professor of neuroesthetics at University College of London, and a world authority on the organization of the primate visual system.) Third, we knew that there may be areas in roughly the same part of the brain that are specialized for numbers. (We know this because small lesions to this part of the brain cause patients to lose arithmetic skills.) I thought, wouldn’t it be wonderful if number-color synesthesia were simply caused by some accidental “cross-wiring” between the number and color centers in the brain? This seemed almost too obvious to be true—but why not? I suggested we look at some brain atlases to see exactly how close these two areas really are in relation to each other.

“Hey, maybe we can ask Tim,” Ed responded. He was referring to Tim Rickard, a colleague of ours at the center. Tim had used sophisticated brain-imaging techniques like fMRI to map out the brain area where visual number recognition occurs. Later that afternoon, Ed and I compared the exact location of V4 and the number area in an atlas of the human brain. To our amazement, we saw that the number area and V4 were right next to each other in the fusiform gyrus (Figure 3.6). This was strong support for the cross-wiring hypothesis. Can it really be a coincidence that the most common type of synesthesia is the number-color type, and the number and color areas are immediate neighbors in the brain?

FIGURE 3.6 The left side of the brain showing the approximate location of the fusiform area: black, a number area; white, a color area (shown schematically on the surface).

This was starting to look too much like nineteenth-century phrenology, but maybe it was true! Since the nineteenth century a debate has raged between phrenology—the notion that different functions are sharply localized in different brain areas—versus holism, which holds that functions are emergent properties of the entire brain whose parts are in constant interaction. It turns out this is an artificial polarization to some degree, because the answer depends on the particular function one is talking about. It would be ludicrous to say that gambling or cooking are localized (although there may be aspects of them that are) but it would be equally silly to say that the cough reflex or the pupils’ reflex to light is not localized. What’s surprising, though, is that even some nonstereotyped functions, such as seeing colors or numbers (as shapes or even as numerical ideas), are in fact mediated by specialized brain regions. Even high-level perceptions such as tools or vegetables or fruits—which border on being concepts rather than mere perceptions—can be lost selectively depending on the particular small region of the brain that is damaged by stroke or accident.

So what do we know about brain localization? How many specialized regions are there, and how are they arranged? Just as the CEO of a corporation delegates different tasks to different people occupying different offices, your brain parcels out different jobs to different regions. The process begins when neural signals from your retina travel to an area in the back of your brain where the image gets categorized into different simple attributes such as color, motion, form, and depth. After that, information about separate features gets divvied up and distributed to several far-flung regions in your temporal and parietal lobes. For example, information about the direction of moving targets goes to V5 in your parietal lobes. Color information gets sent mainly to V4 in your temporal lobes.

The reason for this division of labor is not hard to divine. The kinds of computation you need for extracting information about wavelength (color) is very different from the computations required for extracting information about motion. It may be simpler to accomplish this if you have separate areas for each task, keeping the neural machinery distinct for economy of wiring and ease of computation.

It also makes sense to organize specialized regions into hierarchies. In a hierarchical system, each “higher” level carries out more sophisticated tasks but, just like in a corporation, there is an enormous amount of feedback and crosstalk. For example, color information processed in V4 gets relayed to higher color areas that lie farther up in the temporal lobes, near the angular gyrus. These higher areas may be concerned with more complex aspects of color processing. The eucalyptus leaves I see all over campus appear to be the same shade of green at dusk as they do midday, even though the wavelength composition of light reflected is very different in the two cases. (Light at dusk is red, but you don’t suddenly see leaves as reddish green; they still look green because your higher color areas compensate.)

Numerical computation, too, seems to occur in stages: an early stage in the fusiform gyrus where the actual shapes of numbers are represented, and a later stage in the angular gyrus concerned with numerical concepts such as ordinality (sequence) and cardinality (quantity). When the angular gyrus is damaged by a stroke or a tumor, a patient may still be able to identify numbers but can no longer divide or subtract. (Multiplication often survives because it is learned by rote.) It was this aspect of brain anatomy—the close proximity of colors and numbers in the brain in both the fusiform gyrus and near the angular gyrus—that made me suspect that number-color synesthesia was caused by crosstalk between these specialized brain areas.

But if such neural cross-wiring is the correct explanation, why does it occur at all? Galton observed that synesthesia runs in families, a finding that has been repeatedly confirmed by other researchers. Thus it is fair to ask whether there is a genetic basis for synesthesia. Perhaps synesthetes harbor a mutation that causes some abnormal connections to exist between adjacent brain areas that are normally well segregated from each other. If this mutation is useless or deleterious, why hasn’t it been weeded out by natural selection?

Furthermore, if the mutation were to be expressed in a patchy manner, it might explain why some synesthetes “cross-wire” colors and numbers whereas others, like a synesthete I once saw named Esmerelda, see colors in response to musical notes. Consistent with Esmerelda’s case, hearing centers in the temporal lobes are close to the brain areas that receive color signals from V4 and higher color centers. I felt the pieces were starting to fall into place.

The fact that we see various types of synesthesia provides additional evidence for cross-wiring. Perhaps the