The Tell-Tale Brain

popout/segregation experiment had been done, I noticed something strange in one of the many synesthetes we had been recruiting. He saw numbers as being colored—nothing unusual so far—but what surprised me was his claim that some of the numbers (for example, 8) had different portions colored differently. To make sure he wasn’t making this up, we showed him the same numbers a few months later—without letting him know ahead of time that he would be retested. The new drawing he produced was virtually identical to the first, making it unlikely that he was fibbing.

     This observation provides further evidence that, at least in some synesthetes, the colors should be seen as emerging from (to use a computer metaphor) a glitch in neural hardware rather than from an exaggeration of memories or metaphors (a software glitch) Associative learning cannot explain this observation; for example, we don’t play with multicolored magnets. On the other hand, there may be “form primitives” such as line orientation, angles, and curves that get linked to color neurons that execute an earlier stage of form processing within the fusiform than the one at which full-fledged graphemes are assembled.

     (b) As previously noted, in some synesthetes the evoked color becomes less vivid when the number is viewed off-axis (in peripheral vision). This probably reflects the greater emphasis on color in central vision (Ramachandran & Hubbard, 2001a; Brang & Ramachandran, 2010). In some of these synesthetes the color is also more saturated in one visual field (left or right) relative to the other. Neither of these observations supports the high-level associative learning model for synesthesia.

     (c) An actual increase in anatomical connectivity within the fusiform area of lower synesthetes has been observed by Rouw and Scholte (2007) using diffusion tensor imaging.

     (d) The synesthetically evoked color can provide an input to apparent motion perception (Ramachandran & Hubbard, 2002; Kim, Blake, Palmeri, 2006; Ramachandran & Azoulai, 2006).

     (e) If you have one type of synesthesia, then you are more likely to have a second unrelated one as well. This supports my “increased cross activation model” of synesthesia; with the mutated gene being more prominently expressed in certain brain regions (in addition to making some synesthetes more creative).

     (f) The existence of color-blind (strictly speaking, color anomalous) synesthetes who can see colors in numbers that they can’t see in the real world. The subject couldn’t have learned such associations.

     (g) Ed Hubbard and I showed in 2004 that letters that are similar in shape (e.g., curvy rather than angular) tend to evoke similar colors in “lower” synesthetes. This shows that certain figural primitives that define the letters cross-activate colors even before they are fully processed. We suggested that the technique might be used to map an abstract color-space in a systematic manner onto form-space. More recently David Brang and I confirmed this using brain imaging (MEG or magnetoencephalography) in collaboration with Ming Xiong Huang, Roland Lee, and Tao Song.

     Taken collectively these observations strongly support the sensory cross- activation model. This is not to deny that learned associations and high-level rules of cross-domain mapping are not also involved (see Notes 8 and 9 for this chapter). Indeed, synesthesia may help us discover such rules.

4. The model of cross-activation—either through disinhibition (a loss or lessening of inhibition) of back projections, or through sprouting—can also explain many forms of “acquired” synesthesia that we have discovered. One blind patient with retinitis pigmentosa whom we studied (Armel and Ramachandran, 1999) vividly experienced visual phosphenes (including visual graphemes) when his fingers were touched with a pencil or when he was reading Braille. (We ruled out confabulation by measuring thresholds and demonstrating their stability across several weeks; there is no way he could have memorized the thresholds.) A second blind patient, whom I tested with my student Shai Azoulai, could quite literally see his hand when he waved it in front of his eyes, even in complete darkness. We suggest that this is caused either by hyperactive back projections or by disinhibition caused by visual loss, so that the moving hand is not merely felt but is also seen. Cells with multimodal receptive fields in the parietal lobes may also be involved in mediating this phenomenon (Ramachandran and Azoulai, 2004).

5. Although synesthesia often involves adjacent brain areas (an example is grapheme-color synesthesia in the fusiform), it doesn’t have to. Even far-flung brain regions, after all, may have preexisting connections that could be amplified (through disinhibition, say). Statistically speaking, however, adjacent brain areas tend to be more “cross-wired” to begin with, so synesthesia is likely to involve those more often.

6. The link between synesthesia and metaphor has already been alluded to. The nature of the link remains elusive given that synesthesia involves arbitrarily connecting two unrelated things (such as color and number), whereas in metaphor there is a nonarbitrary conceptual connection between two things (for example, Juliet and the sun).

     One potential solution to this problem emerged from a conversation I had with the eminent polymath Jaron Lanier: We realized that any given word has only a finite set of strong, first-order associations (sun = warm, nurturing, radiant, bright) surrounded by a penumbra of weaker, second-order associations (sun = yellow, flowers, beach) and third-and fourth-order associations that fade way like an echo. It is the overlapping region between two halos of associations that forms the basis of metaphor. (In our example of Juliet and the sun, this overlap derives from observations that both are radiant, warm, and nurturing). Such overlap in halos of associations exists in all of us, but the overlaps are larger and stronger in synesthetes because their the cross-activation gene produces larger penumbras of associations.

     In this formulation, synesthesia is not synonymous with metaphor, but the gene that produces synesthesia confers a propensity toward metaphor. A side effect of this may be that associations that are only vaguely felt in all of us (for example, masculine or feminine letters, or good and bad shapes produced by subliminal associations) become more explicitly manifest in synesthetes, a prediction that can be tested experimentally. For instance, most people consider certain female names (Julie, Cindy, Vanessa, Jennifer, Felicia, and so on) to be “sexier” than others (such as Martha and Ingrid). Even though we may not be consciously aware of it, this may be because saying the former involves pouting and other tongue and lip movements with unconscious sexual overtones. The same argument would explain why the French language is often thought of as being more sexy than German. (Compare Busten-halten with brassiere.) It might be interesting to see if these spontaneously emerging tendencies and classifications are more pronounced in synesthetes.

     Finally, my student David Brang and I showed that completely new associations between arbitrary new shapes and colors are also learned more readily by synesthetes.

     Taken collectively, these results show that the different forms of synesthesia span the whole spectrum from sensation to cognition, and indeed this is precisely why synesthesia is so interesting to study.

     Another familiar yet intriguing kind of visual metaphor, where meaning resonates with form, is the use (in advertising, for example) of type that mirrors the meaning of the word; for example, using tilted letters to print “tilt,” and wiggly lines to print “fear,” “cold,” or “shiver.” This form of metaphor hasn’t yet been studied experimentally.

7. Effects similar to this were originally studied by Heinz Werner, although he didn’t put it in the broader context of language evolution.

8. We have observed that chains of associations, which would normally evoke only memories in normal individuals, would sometimes seem to evoke qualia-laden sense impressions in some higher synesthetes. So the merely metaphorical can become quite literal. For example, R is red and red is hot so R is hot, and so forth. One wonders whether the hyperconnectivity (either the sprouting or disinhibition) has affected back projections between different areas in the neural hierarchy in these subjects. This would also explain an observation David Brang and I made—that eidetic imagery (photographic memory) is more common in synesthetes. (Back projections are thought to be involved in visual imagery.)

9. The introspections of some higher synesthetes are truly bewildering in their complexity; as they go completely “open loop.” Here is a quotation from one of them: “Most men are shades of blue. Women are more colorful. Because people and names both have color associations, the two don’t necessarily match.” Such remarks imply that