The Tell-Tale Brain

compassion, and ambition. A thorough understanding of how the visual system really works may therefore provide insights into the more general strategies the brain uses to handle information, including the ones that are unique to us.

A FEW YEARS ago I was at an after-dinner speech given by David Attenborough at the university aquarium in La Jolla, California, near where I work. Sitting next to me was a distinguished-looking man with a walrus moustache. After his fourth glass of wine he told me that he worked for the creation science institute in San Diego. I was very tempted to tell him that creation science is an oxymoron, but before I could do so he interrupted me to ask where I worked and what I was currently interested in.

“Autism and synesthesia these days. But I also study vision.”

“Vision? What’s there to study?”

“Well, what do you think goes on in your head when you look at something—that chair for example?”

“There is an optical image of the chair in my eye—on my retina. The image is transmitted along a nerve to the visual area of the brain and you see it. Of course, the image in the eye is upside down, so it has to be made upright again in the brain before you see it.”

His answer embodies a logical fallacy called the homunculus fallacy. If the image on the retina is transmitted to the brain and “projected” on some internal mental screen, then you would need some sort of “little man”—a homunculus—inside your head looking at the image and interpreting or understanding it for you. But how would the homunculus be able to understand the images flashing by on his screen? There would have to be another, even smaller chap looking at the image in his head—and so on. It is a situation of infinite regress of eyes, images, and little people, without really solving the problem of perception.

In order to understand perception, you need to first get rid of the notion that the image at the back of your eye simply gets “relayed” back to your brain to be displayed on a screen. Instead, you must understand that as soon as the rays of light are converted into neural impulses at the back of your eye, it no longer makes any sense to think of the visual information as being an image. We must think, instead, of symbolic descriptions that represent the scenes and objects that had been in the image. Say I wanted someone to know what the chair across the room from me looks like. I could take him there and point it out to him so he could see it for himself, but that isn’t a symbolic description. I could show him a photograph or a drawing of the chair, but that is still not symbolic because it bears a physical resemblance. But if I hand the person a written note describing the chair, we have crossed over into the realm of symbolic description: The squiggles of ink on the paper bear no physical resemblance to the chair; they merely symbolize it.

Analogously, the brain creates symbolic descriptions. It does not re-create the original image, but represents the various features and aspects of the image in totally new terms—not with squiggles of ink, of course, but in its own alphabet of nerve impulses. These symbolic encodings are created partly in your retina itself but mostly in your brain. Once there, they are parceled and transformed and combined in the extensive network of visual brain areas that eventually let you recognize objects. Of course, the vast majority of this processing goes on behind the scenes without entering your conscious awareness, which is why it feels effortless and obvious, as it did to my dinner companion.

I’ve been glibly dismissing the homunculus fallacy by pointing out the logical problem of infinite regress. But is there any direct evidence that it is in fact a fallacy?

First, what you see can’t just be the image on the retina because the retinal image can remain constant but your perception can change radically. If perception simply involves transmitting and displaying an image on an inner mental screen, how can this be true? Second, the converse is also true: The retinal image can change, yet your perception of the object remains stable. Third, despite appearances, perception takes time and happens in stages.

The first reason is the most easy to appreciate. It’s the basis of many visual illusions. A famous example is the Necker cube, discovered accidentally by the Swiss crystallographer Louis Albert Necker (Figure 2.1). He was gazing at a cuboid crystal through a microscope one day, and imagine his amazement when the crystal suddenly seemed to flip! Without visibly moving, it switched its orientation right in front of his very eyes. Was the crystal itself changing? To find out he drew a wire-frame cube on a scrap of paper and noticed that the drawing did the same thing. Conclusion: His perception was changing, not the crystal. You can try this on yourself. It is fun even if you have tried it dozens of times in the past. You will see that the drawing suddenly flips on you, and it’s partly—but only partly—under voluntary control. The fact that your perception of an unchanging image can change and flip radically is proof that perception must involve more than simply displaying an image in the brain. Even the simplest act of perception involves judgment and interpretation. Perception is an actively formed opinion of the world rather than a passive reaction to sensory input from it.

FIGURE 2.1 Skeleton outline drawing of a cube: You can see it in either of two different ways, as if it were above you or below you.

FIGURE 2.2 This picture has not been Photoshopped! It was taken with an ordinary camera from the special viewing point that makes the Ames room work. The fun part of this illusion comes when you have two people walk to opposite ends of the room: It looks for all the world as if they are standing just a few feet apart from each other and one of them has grown giant, with his head brushing the ceiling, while the other has shrunk to the size of a fairy.

Another striking example is the famous Ames room illusion (Figure 2.2). Imagine taking a regular room like the one you are in now and stretching out one corner so the ceiling is much taller in that corner than elsewhere. Now make a small hole in any of the walls and look inside the room. From nearly any viewing perspective you see a bizarrely deformed trapezoidal room. But there is one special vantage point from which, astonishingly, the room looks completely normal! The walls, floor, and ceiling all seem to be arranged at proper right angles to each other, and the windows and floor tiles seem to be of uniform size. The usual explanation for this illusion is that from this particular vantage point the image cast on your retina by the distorted room is identical to that which would be produced by a normal room—it’s just geometric optics. But surely this begs the question. How does your visual system know what a normal room should look like from exactly this particular vantage point?

To turn the problem on its head, let’s assume you are looking through a peephole into a normal room. There is in fact an infinity of distorted trapezoidal Ames rooms that could produce exactly the same image, yet you stably perceive a normal room. Your perception doesn’t oscillate wildly between a million possibilities; it homes in instantly on the correct interpretation. The only way it can do this is by bringing in certain built-in knowledge or hidden assumptions about the world—such as walls being parallel, floor tiles being squares, and so on—to eliminate the infinity of false rooms.

The study of perception, then, is the study of these assumptions and the manner in which they are enshrined in the neural hardware of your brain. A life-size Ames room is hard to construct, but over the years psychologists have