Catherine Kavassalis ckava@sympatico.ca

Cognitive Science HDP 1234 University of Toronto

05/01/01 Return to Index

Synesthesia:

The Implications of Cross Modal Communication

It was a chance invitation to dinner that led Neurologist Richard Cytowic to the man who tasted shapes. Michael Watson was a seemingly normal well-adjusted lighting designer for North Carolina’s School of the Arts. Yet this man had an unusual ability. In an interview with Robin Hughes of ABC Radio National, Cytowic recalled that first dinner.

“He'd made roast chickens and he delayed us sitting down to table with the admonition that there aren't enough points on the chicken. And he turned beet red and said, 'Oh my god, I shouldn't have said that.” (Hughes, 1996, e9)

To Michael the taste of a properly cooked chicken is synonymous with pointed prickly shapes. Michael is a synesthete, an individual for whom the senses merge. He became the focus of Cytowic’s book The Man Who Tasted Shapes (1993) and one of now thousands of documented cases.

Synesthesia or synaesthesia is described as the commingling of sensory perceptions or the union of sensory perceptions and conceptions. It is an inherited trait (although it may be artificially induced), automatic, unconscious and consistent over time. There are dozens of variants of which Watson’s smell to shape connection is just one case. Here are just a few alternative expressions of synesthesia:

Ø 'I avoid playing middle C and nearby notes because of their earthy odor and mud-brown colors, which I dislike intensely' (auditory ‘pitch’/olfactory + visual ‘color’)

Ø 'If I happen to see a red or scarlet object, I hear its high pitched sound' (color/ auditory ‘pitch’)

Ø 'If I have difficulty remembering certain telephone numbers, I realize the colors are incompatible' (number ‘conceptual memory’/ color)

Ø 'Vancouver tastes like rice pudding but with raisins' (‘conceptual’ /taste)(Williams, 1999, e4)

“That’s all very curious,” you might say, “but what can by learned from understanding this seemingly odd and relatively rare condition?” I believe the occurrence of synesthesia and other more common forms of cross modal association have profound implications for cognitive science. In particular I propose that the existence of these cross domain connections and communications early in the cognitive process refutes the Fodorian modularity hypothesis (Fodor, 1983) and a corollary, the multiple format hypothesis (Pinker, 1996). To make this case, I will introduce these hypotheses and present a primarily evidential argument against them. Because our assumptions about such things as modularity and language of thought guide our research and our interpretation of observations, I believe it is important to challenge these theses so that we do not operate under false assumptions.

Modularity

In Modularity of Mind (1983), Jerry Fodor outlined his conception of the organizational structure of the mind. According to Fodor, the mind is ‘trichotomous’, made up of transducers, input systems, and central processors. Transducers compile and translate sensory information (light, sound, scent, etc.) into a format that can be used by input systems, which then interpret this information and make it available to central processors. Elaborating on Gall’s idea of vertical faculties, Fodor suggests that there are six vertical faculties within the mind that correspond to the five perceptual input systems (sight, hearing, smell, touch, taste) and a linguistic input system. Perceptual input systems and the language input system are understood by Fodor to be functionally equivalent in that they, “ both serve to get information about the world into a format appropriate for access by such central processes as mediate the fixation of belief.” (46) Whereas Fodor asserts that the six domain specific input systems are modular, he concludes that central processors are nonmodular faculties requiring ‘neural equipotentiality.’ (127) Thus the mind is split into modular and nonmodular components characterized by their possessing, or not possessing, certain properties.

Fodor identifies nine properties that characterize modularity.

1. Modules should be domain specific. That is the computational inferences made by input systems should be consistent with the perceptual input provided by the associated transducer. (47)

2. Modules should operate automatically without the intervention of higher cognitive processes. Their operation is mandatory. (52)

3. Modules should have “’interlevels’ of input representation” which are inaccessible to central processes, or cognitively unpenetrable. (55)

4. Modules should be fast. (61)

5. Modules should be informationally encapsulated in that “input systems do not exchange subdoxastic [subpersonal unconscious] information with central processes or with one another.”(85)

6. Modules should have ‘shallow’ outputs. They can only provide for basic categorizations which can be “inferred, with reasonable accuracy, from …properties of the stimulus.” (97)

7. Modules should have fixed neural architecture characterized by “hardwired connections which indicate privileged paths of informational access.” (98)

8. Modules should exhibit characteristic and specific breakdown patterns typified by such pathologies as agnosias and aphasias. (99)

9. Modules should exhibit a characteristic ontogenetic development. However Fodor concedes that “the available information is so fragmentary, that I offer this point more as a hypothesis that a datum.” (100)

Of these properties, Fodor deems informational encapsulation and domain specificity the most critical. To show that central processes are nonmodular, Fodor cites evidence that the processes “cut across cognitive domains” and that the “processes are subserved by nonmodular (e.g., information unencapsulated) mechanisms?” (101) Therefore to show that the input systems are nonmodular, I need to provide similar evidence. That evidence comes from research on synesthesia. I have included a summary of synesthesia information and research as an appendix and will only cite relevant material.

A Refutation of Modularity

As suggested by the opening quotes from synesthetes, there are at least some individuals for whom input systems are not modular. Although input systems of synesthetes exhibit automatic fast operation, with shallow outputs and unconscious ‘interlevels,’ the research shows that they do not display the essence of modularity: informational encapsulation and domain specificity. Instead, for individuals with idiopathic (inherited) synesthesia, the research indicates that low level and intermediate level communication between perceptual domains and or the language domains occurs normally. (Cytowic, 1995; Baron-Cohen, 1996; Grossenbacher, 2001) Such low level interdomain communication clearly breaches modularity of input systems. Does it represent a ‘breakdown’ as Fodor suggests agnosia does? If so, synesthetes should exhibit a characteristic pattern of dysfunction. Simon Baron-Cohen looked for such evidence. He noted, “Curiously, most individuals with colored hearing synaesthesia do not complain of their condition. For them, it is their normal perception of the world, and they are not aware of it causing any disadvantages or interference.” (Baron-Cohen 1996, e4) Although there are two cited cases of maladaptive synesthesia (see appendix), the majority of synesthetes are well-adapted individuals of high intelligence and with excellent memories. Thus we can conclude that synesthetes have nonmodular input systems. That conclusion however is not sufficient to refute modularity for the rest of us.

This is where the story becomes interesting. I will provide evidence from four distinct areas of research to show that the minds of nonsynesthetes are also nonmodular. The first evidence comes from developmental studies; the second, from nonidiopathic synesthetic experience; the third, from a growing body of evidence on multimodal and multifunctional neurons within animal brains; and finally, I will consider metaphoric thinking as an extension of synesthesia.

With the improvement in neuroimaging techniques, researchers are able to study patterns of activity in the brain associated with various mental processes. They have established a correlation between the pattern of sensory processing seen in synesthetes and that seen in infants, suggesting “that synaesthesia may be a prototype sensory perception that is done at subcortical level.” (Carter, 1998, 179) This is supported by a variety of observations. For instance Stein and Meredith observed, " That sound as well as visual objects evoke spatially coordinated eye movements in neonates. This has been known for some time in older infants, but now is being examined far earlier. There is at least one case in which auditory-evoked eye movements have been claimed within moments of birth, and in another a congenitally blind infant showed convergence of the eyes to approaching sounds and divergence of the eyes to receding sounds. The presence of a visual target is also known to enhance the spatial coordination of the eye movements evoked by an auditory stimulus in neonates. These observations are not explainable by mere arousal effects or traditional learning theories, and suggest a prenatal link between vision and audition, at least in terms of spatial relations." (1993, 14) The Neonatal Synaesthesia hypothesis (Mauer, 1993) and the Cross-Modal Transfer hypothesis (Baron-Cohen, 1996) were developed based on an abundance of such developmental evidence which all suggest that “human babies experience sensory input in an undifferentiated way. Sounds trigger auditory, visual and tactile experiences all at once.”(Grossenbacher, 2000, Conf. Abstract) Based on the CTM hypothesis and the modularity hypothesis, Baron-Cohen suggested that adult synesthetes do not develop normally in that natural pruning of intersensory connections leading to the modularization of the mind does not occur. However, this proposal is problematic because this breakdown of modularity is not typically dysfunctional and it does not explain nonidiopathic synesthesia.

In 1995, Cytowic commented, “hypothetical modules presently drive experiment instead of theory being driven by phenomenology.” (e15). Cytowic (1989, 1995) and later Grossenbacher (1997, 2001) chose to consider explanations that ignored the modularity hypothesis in order to understand synesthetic phenomena. Noting that synesthetic experiences can be induced by a variety of means: epileptic seizure, drugs (particularly hallucinogenic compounds like LSD), neuron degeneration, brain damage spinal cord damage, and concussion, these researchers concluded that cross modal connections must be part of the brain’s normal architecture, but for the majority of people multisensory signals do not reach consciousness. (Cytowic and Grossenbacher have developed distinct theories to explain this: limbic mediation and feedback disinhibition- see appendix.) “Although synesthetes exhibit abnormal patterns of brain activity, there is no evidence suggesting that the neuroanatomical connections between sensory modalities differ between synesthetes and non-synesthetes. To the extent that neuroanatomical connectivity is indeed normal in synesthetes, the intersensory links evident in synesthesia may inform our understanding of normal human intersensory connectivity.” (Grossenbacher, 2000, e1) Thus if the neural structure in synesthetes is nonmodular so too is the neural structure in non-synesthetes.

Further support for cross-domain connectivity comes from the growing body of multisensory research, which has resulted in the recent establishment of the Multisensory Research Forum (http://www.wfubmc.edu/nba/IMRF/). A quick scan of the literature soon generates an overwhelming number of studies reporting evidence for cross modal brain connections and multimodal multifunctional neurons in every imaginable animal including: gastropods (Safonova 1996), ferrets (Pallas, 2000), cats (Rauschecker, 1995), monkeys and humans (Bremmer, 2000, Foxe 2000, Lewkowicz, 2000), etc. Most of the research on multisensory integration has focused on cats. Neurons in the cat’s superior colliculus (SC) and the anterior ectosylvian sulcus (AES) “respond to combinations of auditory, visual, and somatosensory input and alter their firing according to the presence, intensity, and location of multiple sensory evets—that is, they are able to integrate information from multiple senses. “ (Lovelace, 2001, e1-2) Foxe investigating multisensory interactions produced by simultaneous auditory and somatosensory stimulation in humans found that, “ Both the topography and timing of these interactions are consistent with multisensory integration early in the cortical processing hierarchy, in brain regions traditionally held to be unisensory.” (Foxe, 2000, e1)

In 1967, Teuber, who specialized in brain mechanisms involved in visual perception wondered, "if it were generally true that the different senses are hermetically sealed off against each other, it would be extremely difficult to understand the function of those large clusters of cortical and sub-cortical neurones which seem to respond to more than one kind of sensory input ... It would seem essential that there be some central mechanism for transcending the division between the different senses, for identifying an object felt with an object seen, and both with the object we can name; there should be some form of cross-modal processing resulting in supra-modal, rather than sensory categories, extracted from or imposed upon experience.” (in Allott, 1972, 1. ii) Clearly, Teuber was correct in assuming that there must be mechanisms that are informationally unencapsulated. These mechanisms allow for that multisensory integration in the midbrain and cortex. The mechanisms of information-processing integration are not well understood but their existence contravenes information encapsulation.

As the modularity hypothesis falls away, and the distinction between vertical input systems and horizontal central processes disappears, implications of the underlying synesthete in all of us emerge. Metaphoric thinking is perhaps evidence of this emergent cross modal process. How is it that we come to say, “I’m feeling blue,” “She’s green with envy? Or ” What a bitter memory.” In a recent article by Seitz, it is argued that metaphoric abilities may be “conceptualized as (a) largely intuitive and nonconscious and thus, (b) arise early in development, are innately constrained, and follow separate developmental pathways, (c) are independent of logical reasoning abilities, (d) are not an epiphenomenon of language and thus reflect disparate underlying cognitive processes, and, therefore, (e) involve different types of cognitive processes that develop at different ages.” Seitz goes on to suggest that “synesthesia provides the biological infrastructure for metaphorical mapping and these … represent primitive perceptual forms that are created by means of polysensory cortical neurons and integrated at cortical association areas. “ (Seitz, 2001, e1,4) Although the connection between metaphoric thinking and synesthesia is largely speculative, it can only be fully investigated as a possibility if the modularity hypothesis is abandoned. I believe the preponderance of evidence cited here provides a clear basis to do so.

In summary, it has been shown that input systems are not domain specific or informationally encapsulated in the case of synesthesia. It has further been shown that infants share this characteristic with synesthetes. Next it was argued on the basis on nonidiopathic synesthesia that the neural structure in synesthetes is normal and therefore all humans are architecturally synesthetic. This contention was substantiated by the presence of multisensory multifunctional neurons in a variety of animals including humans. Finally it was suggested that metaphoric thinking is a manifestation of underlying cross modal association mechanisms in all of us. Thus the case has been made that input systems are nonmodular. Therefore the mind is nonmodular in the Fodorian sense.

Multiple format hypothesis:

In How The Mind Works, (1996) Steven Pinker moves away from the idea of hermetic modules. He suggests instead that modules should be “defined by the special things they do with the information available to them, not necessarily by the kinds of information they have available.” (31) However, his version of modularity is problematic. The problem is particularly evident when he introduces his conception of formats of representation with which different ‘organs’ carry out the ‘special things they do.’ Pinker states that “the human brain uses at least four major formats of representation. One format is the visual image, which is like a template in a two-dimensional, picturelike mosaic. …Another is a phonological representation, a stretch of syllables that we play in our minds like a tape loop, planning out the mouth movement and imagining what the syllables sound like. …A third format is the grammatical representation: nouns and verbs, phrases and clauses, stems and roots, phonemes and syllables, all arranged into hierarchical trees. …The fourth format is mentalese, the language of thought in which our conceptual knowledge is couched.”(89-90) The idea that the mind uses formats, or meta-languages, suggests that communication between Pinker’s modules must be restricted or at least requires a translator, a transducer-compiler in computer lingo, to intervene. Thus Pinker’s modules seem to exhibit a kind of information encapsulation that I will argue is unwarranted.

Again using evidence regarding synesthesia and multisensory processing, I will make the case that the mind must process information using a single coherent code, the fundamental language of thought. I will briefly argue that although formats make sense to computer programmers, they make no sense to machines or minds. Finally I will suggest that the mind uses natural language, and perceptual languages as they are manifested by this fundamental code.

A Refutation of Multiple Formats:

Pinker provides the following anecdotal evidence in support of his position that the mind uses mentalese and not natural language to think. “When you put down a book, you forget almost everything about the wording and typeface of the sentences and where they sat on the page. What you take away is their content of gist.” (90) I would agree that this is the case for me, but the synesthete would remember something more, not just the content but also the color, or the sound, or the smell, or the taste. (Remember such phrases as: 'Vancouver tastes like rice pudding but with raisins') They are able to merge conceptual understanding with perceptual images. In addition, the fact that many synesthetes regularly combine phonetic information with visual imagery suggests that such cross modal information can be summed without great difficulty. Although it is possible that the mind is full of transducers allowing a kind of gated information flow, it seems improbable.

It is particularly improbable given the research on plasticity and development which shows that neurons are not specific to data type. Making a case for “general brain architecture rather that a specific one based on genetics,” Kreger cites numerous plasticity studies, (1999, e6.) In such studies examining the effects of sensory loss or deprivation it has been shown that areas of the brain that would normally process one type of sensory information are able to process information from a different modality. Reporting on interspecies brain transplantation experiment, described by Deacon, (1997: 199-200), Kreger states that “the chimeric brains that resulted should have been dysfunctional because of the differing connections the pig tissue would make, and the differing signals it would send. This was … not the case. The resultant brains worked well, with the pig tissue making the same connections and transferring information in the same way the rat tissue would. This indicates that neural tissue is functioning along general mammalian guidelines versus total species-specific deterministic genetics.” (1999: e6) This general processing ability of neural tissue was also demonstrated by Sarah Pallas, (2000). Pallas made a series of lesions in the brains of ferrets and essentially rerouted visual input through auditory systems. The animals were still able to distinguish between visual stimuli suggesting that the format used to communicate visual information is not distinct from the format used to communicate auditory information.

Defining mentalese as the language of thought, Pinker states: “Mentalese is also the mind’s ligua franca, the traffic of information among mental modules that allow us to describe what we see, imagine what is described to us, carry out instructions, and so on. This traffic can actually be seen in the anatomy of the brain. The hippocampus and connected structures, which put our memories into long-term storage, and the frontal lobes, which house the circuitry for decision making, are not directly connected to the brain areas that process raw sensory input (mosaic of edges and colors and the ribbon of changing pitches).” (90) Although I would agree that traffic does flow through the hippocampus, research from synesthesia and evidence for the existence of multimodal multisensory neural centers in that region suggests that ‘raw’ sensory input does indeed merge with conceptual input, and according to Cytowic at least that merging occurs in the hippocampus. “The hippocampus is an important and probably obligate node in whatever neural structures generate the synesthetic experience.” (Cytowic, 1995, e6.3)

Pinker himself raises the question, “Why so many kinds of representations?” His response comes out of computer programming and he suggests an ‘Esperanto of the mind would be hellishly complicated,” and programmers know to “choose a data representation that makes the program simple.”(1996, 91) Of course, it would be hellish if we had to program the mind, but we don’t. Metalanguages are for the programmers ease not the computers. Formats used to program specific operations must be compiled and translated into machine code. Why ignore Occam’s Razor? The mind does not require multiple metalanguages to perform operations. [It is probable that the underlying notion of representation must be rethought, but I will leave that to more philisophical minds. (Olsen, in progress).] Machine code is the fundamental format used by the machine just as an electrochemical code is the fundamental mental format.

Then what is the language of thought? David Cole looks very carefully at the arguments for mentalese and states: “The question then is, would an image, say of acoustic spoken words and sentences in natural language, _have_ to be translated into a non-acoustic language, mentalese, in order for it to be the subject of logical operations, association, and other computational processes characteristic of thought? We haven’t encountered an argument yet that shows that the answer must be "yes".” (Cole, 1998, e14) Cole argues that the language of thought must therefore be natural language and I would partially agree. The process of thinking requires operations done in the fundamental code that inscribes natural language, and perceptual input. Therefore thinking in fundamental code can manifest itself as thinking in natural language or a combination of say natural language and mental imagery. This is consistent with the synesthetic experience, which according the Cytowic and Grossenbach is consistent with our experience.

To summarize, there is no need to hypothesize multiple formats or meta-languages to explain mental processes. Evidence that synesthetes easily merge perceptions and conceptions refutes the idea that they are stored as distinct formats as does evidence from multisensory research. Although computer programmers use specialized language formats for ease in programming, it does not follow that the mind requires metalanguages to function. Thus we can to think of the mind as fluidly processing information in a common format. This allows us to fully abandon the modularity hypothesis.

Conclusions:

Although many questions remain unsolved about the mind, research continues to shed some light on its inner functioning. However, false assumptions cloud our understanding, [metaphors intended]. By adhering to Fodorian modularity, Baron-Cohen was unable to explain the phenomenological evidence. Only by letting go of this assumption could Cytowic and Grossenbacher develop more comprehensive hypotheses. Although their respective theories remain to be proven, they were correct to relinquish the supposition of modularity.

Combined evidence from synesthetic research, developmental studies, plasticity and multimodal multisensory investigations refutes both the modularity hypothesis and the multiple format hypothesis. Although the adult mind appears to be organized into task specific regions, it is clear that these regions are not modular and can communicate freely when connected. But the situation is far from bleak as Fodor suggests it would be in the absence of modularity. Alternatives to innate modules are abundant and include neural Darwinism (Edelman, 1987), chaotic dynamics (Freeman, 1990), brain acclimation (Kreger, 1999) and my personal favorite neural constructivism (Quartz and Sejnowski,1997). I am confident that progress in understanding will continue and the mysteries of mind will be unraveled.

Appendix

“Synesthesia is an involuntary joining in which the real information of one sense is accompanied by a perception in another sense. In addition to being involuntary, this additional perception is regarded by the synesthete as real, often outside the body, instead of imagined in the mind's eye. It also has some other interesting features that clearly separate it from artistic fancy or purple prose. Its reality and vividness are what make synesthesia so interesting in its violation of conventional perception. Synesthesia is also fascinating because logically it should not be a product of the human brain, where the evolutionary trend has been for increasing separation of function anatomically.” (Cytowic, 1989, 1)

When Cytowic first wrote about synesthesia, there was only a small data base of synesthetes on which to base his findings and thus initially Cytowic felt that synesthesia was an extremely rare phenomenon, one in 300, 000. As interest and information about the condition has spread, Cytowic and other researchers have became aware of a much larger population of synesthetes. Cytowic currently estimates that one in 25,000 (Cytowic, 1995) individuals display some form of synesthesia while Simon Baron-Cohen, an experimental psychologist from the University of Cambridge, believes the occurrence rate to be even higher, one in 2000, (Lemley, 1999). Peter Grossenbacher, psychologist and organizer of the American Synesthesia Association, suggests it may be as high as one in 300, (Lichtenstein, 2001, e1). With a larger population of individuals to examine, researchers have been able identify characteristics of synesthetes and develop explanations for its occurrence.

Characteristics:

As mentioned in the Cytowic quote (above), synesthesia is involuntary but can by elicited by particular stimuli. That stimuli can be referred to as an inducer and the associated perception or conception is referred to as the concurrent, (Grossenbacher, 2001). The concurrent sensation is generally perceived externally, within arms length, but can be perceived within the mind’s eye like mental imagery. The phenomenon is durable and repeatable over time. For instance, a child who associated word sounds to his own body position (only one such case cited in literature) was able to exactly repeat the patterns ten years after initial testing, (Cytowic, 1995, 3.6). In addition the induced or concurrent sensation is generic or simple. Concurrents may be colors, or tones, shapes (such as those used at the head of the appendix), tastes, odors, somatosensory, etc. Although concurrent sensations are idiosyncratic, there is some evidence that they follow particular trends, (e.g. according to data compiled by Lawrence Marks on colored-hearing synesthetes, there is a near linear relationship between hue and sound structure, brightness and pitch,1996). The synesthetic experience is holistic and noetic - individuals feel quite certain about their perceptions. However most synesthetes hide their ability for it is not socially acceptable. This is illustrated by the anecdotal tale of a non-drug-using synesthetic teenager who was placed in a drug treatment program when she told a school official that her boyfriends’ kiss made her see orange-sherbet foam, (Baron-Cohen, 1987).

The synesthetic experience is usually but not always unidirectional. Usually it involves only two modalities, but it can involve multiple modalities. A Soviet neurophysiologist, A. Luria, documented the most famous case of multiple modal associations in his book, The Mind of a Mnemonist. His subject Shereshevsky had a phenomenal memory. Although most synesthetes score in the superior range on Wechsler Memory Scale (Cytowic, 1995, 2.8), Shereshevsky was extraordinary. Combining sight, sound, taste and touch to form complex sensorial databases, he could recall vast quantities of information. Whereas unidirectional bimodal synesthesia does not adversely affect individuals, bi-directional or multimodal synesthesia may be problematic. Cytowic comments, “I have encountered no one whose synesthesia was so markedly disruptive to rational thought as it was in Luria's famous male subject, S.” (Cytowic, 1995, 2.9) Baron-Cohen, commenting on a bi-direction color-sound synesthete, suggests “this form of synesthesia leads to massive interference, stress, dizziness, a feeling of information overload, and a need to avoid those situations that are either too noisy or too colourful. …[and therefore causes] social withdrawal, and interference with ordinary life. (Baron-Cohen, 1996, e4) This is not the typical experience of the vast majority of synesthetes who are “normal in the conventional sense. They are bright and hail from all walks of life. …their memories are excellent, …[they display] overall high intelligence.” (Cytowic, 1995, 2.7-2.9)

Idiopathic forms of synesthesia show a familial pattern of inheritance. Because the high ratio of female to male synesthetes, the condition is thought to be sex-linked and dominant. [I wonder though, if it has more to do with general differences in the connectivity of the female brain and thus manifests itself more readily.] "Perhaps the most famous family case is that of the Russian novelist Vladimir Nabokov. When, as a toddler, he complained to his mother that the letter colors on his wooden alphabet blocks were ‘all wrong,’ she understood the conflict he experienced between the color of the painted letters and his lexically-induced synesthetic colors." (Cytowic, 1995, 2.4) Concurrent experiences even in families, tend to be idiosyncratic, that is associations for instance between middle C and the color brown are not consistent between individuals [This might have some interesting implications for cultural learning theories and discussions of qualia].

Figure 1: Kluver forms

An interesting area of investigation is in the examination of the synesthetic shapes that are concurrent to various inducing stimuli. Synesthetes do not see elaborate scenes as concurrents, but rather they see geometic forms, blobs, lines, spirals, lattices, etc. It is curious that these forms resemble those developed by Heinrich Kluver in the 1920's known as Kluver’s "form constants" (shown in Fig 1). “These generic shapes are common to synesthesia, hallucinations and are frequently seen in primitive art. " (Lyons,1998, e3) Some researchers suggest that these forms may represent conceptual primitives like Biederman’s geons (1987) that are used to construct more complex entities. Alternatively, Goldstone and Shyn believe that these primitive shapes “are neither fixed nor finite, but rather adapt to the requirements of the tasks for which they are employed,” (1997, e2) There are many opportunities for discovery in this area.

Neural Basis of Synesthesia

The wide variety of synesthetic experiences, and variable experimental evidence suggests that synesthesia may be a group of phenomena rather than a singular phenomenon, (de F. Costa, 1996). For instance, Cytowic notes decreased blood flow in the neocortex when examining smell-shape synesthete using xenon inhalation, whereas Baron-Cohen (1987) notes increased blood flow in the neocortex when examining phoneme -color synesthetes using positron emission technology. The lack of neocortical blood flow, recorded by Cytowic may be specific to the type of synesthete he was examining, (taste, smell – shape), or be anomalous for the individual. Research on specific types of synesthetes is needed to resolve this inconsistency.

That said, based on several addition points of evidence, Cytowic proposed the Limbic Mediation theory, LMT (Cytowic, 1989,1993,1995). According to this theory, everyone has cross-modal mixing which occurs at “a low to intermediate level of the neuraxis, rather than a higher level involving more mental mediation” (1995, 6.1). He feels that synesthesia depends primarily on processes occurring in the left hemisphere and is mediated by the limbic system, particularly the hippocampus in which multiple sensory signals converge. According to this hypothesis then, synesthetes simply have more connective fibers leading into the neocortex and are aware or conscious of these cross-modal associations that occur in everyone. Baron-Cohen’s PET scans do not show the same hemispheric dependency but their subjects and studies cannot easily be compared. Citing different empirical evidence they have proposed an alternative theory.

Focusing on colored-hearing synesthetes, Baron-Cohen’s group have, found significant differences in the areas of activity within their brains when compared with nonsynesthetes, as illustrated in the PET scans on the following page. (Baron-Cohen et al. 1993; Paulesu et al. 1995). With subjects blindfolded and listening attentively to a series of tones (control condition) or spoken words (activation condition or inducer), researches found increased activation in several regions of the brain including those associated with these visual areas have been closely linked to human visual area V4, located in the right occipital region, which contains many cells sensitive to both color and shape (and some decreased activation the left insula left lingual gyrus not readily seen in above pictures).

The top row of the block of images shows the averaged differences in synaesthesic subjects when the bloodflow maps for listening to tones was subtracted from those for words. The pictures show different slices taken through the brain, with the images to the far left taken fairly low in the brain and the far right image showing the topmost 'slice'. The image on the far left is labelled -16, indicating that it is 16mm below the central plane. The central plane is known as the 'AC-PC' plane. This is because in order to take a point of reference PET researchers locate two small ascending structures known as the Anterior Commissure (AC) and the Posterior Commissure (PC) and draw a plane to connect them. A '+' figure implies that the slice lies above the AC-PC line.

The second row of images shows the activation differences when the blood flow maps for controls were subtracted from those of synesthetes. The row again shows low 'slices' on the left and high 'slices' on the right. (Paulesu, 1995, e2)

Baron-Cohen noted the interesting similarity in the PET scans of synesthetes to neonatal scans. In addition to neuralimaging studies, numerous behavioral studies show that infants use multisensory processing.” (Baron-Cohen, 1996, e2) Baron-Cohen cites some of the cross-modal evidence that led to Mauer’s neonatal synesthesia (NS) hypothesis (1993). This hypothesis basically purports that everyone is born synesthetic. He then argues that if neonates are synesthetic and adults are not it must be the result in the breakdown of cortical modularity from either lack of pruning of connections present at birth or abnormal connections that develop after birth. This conclusion is based on the assumption that minds are genetically predisposed to Fodorian modularity, but is problematic in that it does not explain non-idiopathic synesthesia or the fact that the vast majority of synesthetes do not seem adversely affected by this breakdown of modularity, in fact for some it is both pleasant and beneficial, particularly for memory.

Several researches argue that synesthesia is not the result of aberrant connectivity as Cytowic and Baron-Cohen’s theories tend to suggest. Grossenbacher proposed Cortical Feedback Disinhibition, (CFD) (1997, 2001). According to CFD, synesthesia is mediated by neural connections that exist in normal adult human brains and results from the lack of inhibition of normally occurring activity in multi-sensory areas of the cortex. This explains, according to Grossenbacher, the occurrence of drug induced synesthetic experiences in non-synesthetes.

Luciano da F. Costa of the Cybernetic Vision Research Group, contends that synesthesia is "a much more generalized effect and may take place throughout a range of scales in the brain." (da Costa, 1996, e1) Citing evidence of cross-modal transfer in mammalian brains, (cats and monkeys whose visual neurons are able to receive and process auditory and somatosensory input), da F. Costa suggests that cross-modal transfer is not a dysfunction, but a normally occurring function of the brain, synesthetes simply have more cross-modal connections. [ It is unclear why da F. Costa sees Cytowic’s increased connectivity as aberrant and his connectivity as normal.]

The difference in the three models, LMT, NS, CFD is illustrated in the diagram below:

(Grossenbacher, 2000, conf.poster)

There is still insufficient data to distinguish between these mechanisms, although Cytowic’s and Grossenbacher’s explain a wider range of phenomenon. There may in fact be multiple mechanisms necessary to explain different types of synesthesia. What is commonly understood is that forms of synesthesia must result from either low level cross modal association or intermediate cross modal association. Whether synesthesia is the result of differential connectivity or differential cortical feedback is yet to be determined.

Although the research on synesthesia is just beginning, it can potentially provide much insight into how the mind works. As a teacher I will definitely be on the look out for such students, who in the past I may have thwarted by not recognizing their gift.

Bibliography:

Baron-Cohen, S., Wyke, M., & Binnie, C. (1987). Hearing words and seeing colours: an experimental investigation of a case of synaesthesia. Perception, 16, 761-67.Available online: http://www.psychiatry.cam.ac.uk/isa/expinv.html#1

Baron-Cohen, S., Harrison, J., Goldstein, L., & Wyke M. (1993). Coloured Speech PerceptionIs Synaesthesia What Happens When Modularity Breaks Down? Perception, Volume 22, 419-426. Available online: http://www.psychiatry.cam.ac.uk/isa/expinv.html#2

Baron-Cohen, Simon. (1996). Is There a Normal Phase of Synaestheia in Development. PSYCHE, 2(27). Available online: http://psyche.cs.monash.edu.au/v2/psyche-2-27-baron_cohen.html

Bremmer, F. et al. Polymodal Motion Processing in Posterior Parietal and Premotor Cortex: A Human fMRI Study Strongly Implies Equivalencies between Humans and Monkeys. Neuron. Vol. 29, 287296, Available online: http://www.google.com/search?q=cache:www.kfajuelich.de/ime/Bremmer_01_polymodal_motion.pdf+bremmer+2000+human+brain&hl=en

Cole, David 1998). I Don’t Think So:Pinker on the Thinker; mentalese monopoly in thought not vindicated. Available online:

http://www.d.umn.edu/~dcole/pinker.htm

Cytowic, R. (1989). Synesthesia: A Union of the Senses.Springer-Verlag, NY.

Cytowic, Richard, (1993). The Man Who Tasted Shapes, New York: Warner Books.

Cytowic, R. (1995). Synesthesia: Phenomenology And Neuropsychology. Psyche, 2(10) Available online: http://psyche.cs.monash.edu.au/v2/psyche-2-10-cytowic.htm

da F. Costa, Luciano. (1996). Synesthesia - A Real Phenomenon? Or Real Phenomena? PSYCHE, 2(10), 2(26). Available online: http://psyche.cs.monash.edu.au/v2/psyche-2-26-dacosta.html

Deacon, Terrence W. (1997). The Symbolic Species: The Co-evolution of Language and the Brain. New York: W.W. Norton & Company, Inc.

Edelman, G. (1987) Neural Darwinism: The Theory of neuronal group selection. Basic Books.

Fodor, Jerry. (1983) Modularity of Mind. Cambirdge, MIT Press.

Freeman WJ, Skarda CA (1990). Chaotic dynamics versus representationalism. Behavioral & Brain Sciences 13: 167-168.

Goldstone, R. L., Schyns, P. G., & Medin, D. L. (1997). Learning to bridge between perception and cognition.  In L. Goldstone, P. G. Schyns, & D. L. Medin (Eds.)  Psychology of Learning and Motivation: Perceptual Learning, Vol. 36.  (pp. 1-14).  San Diego, Academic Press. Available online: http://cognitrn.psych.indiana.edu/rgoldsto/plm.html

Grossenbacher, Peter. (1997). Perception and sensory information in synesthaetic experience. In S. Baron-Cohen and J. Harrison (Eds.); Synaesthesia: Classic and Contemporary Readings; Oxford, England: Blackwell. Pp. 148-172.

Grossenbacher, P. G. & Lovelace, C. T. (1999). Synesthesia then and now: Contrasting scientific views from a century ago and today. Society for Neuroscience Abstracts, 25(Part 1), 258.  Abstract available online: http://www.wfubmc.edu/nba/postdocs/lovelace/abstracts/sfnabs1999.html

Grossenbacher Peter. (2000). Bias among human intersensory links revealed by individual differences in synesthetic perception. 2^nd Annual Multisensory Research Conference. Abstract available Online: http://www.wfubmc.edu/bgsm/nba/IMRF/meeting00/abstracts.html

Grossenbacher, P. G., & Lovelace, C. T. (2001). Mechanisms of synesthesia: Cognitive and physiological constraints. Trends in Cognitive Sciences, 5(1), 36-41. Available online: http://www.wfubmc.edu/nba/postdocs/lovelace/abstracts/TICSpreprint.doc

Hughes, Robyn. (1996). SYNESTHESIA. The Health Report. (July, 8). ABC Radio National: Transcript available online: http://www.abc.net.au/rn/talks/8.30/helthrpt/hstories/hr080796.htm

Hughes, Robyn. (1999). Synaesthesia: An Unusual Union of the Senses. Ockham’s Razor. (Nov. 28). ABC Radio National: Transcript available online: http://www.abc.net.au/rn/science/ockham/stories/s69264.htm

Istvan, J. et al. (2000). Multisensory auditory--somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Cognitive Brain Research Vol. 10 (1-2) pp. 77-83 Available online: http://www.elsevier.nl/cgibin/cas/tree/store/bresc/cas_sub/browse/browse.cgi?year=2000&volume=10&issue=1-2&aid=30242
 
Kreger, David. (1999). Did Neanderthals Have Language? Web Published: http://www.humanevolution.f2s.com/anth309.html
 
 
Lemley, Brad. (1999). Do You See What They See? Discover 20(12). Available online: http://www.findarticles.com/cf_0/m1511/12_20/58050561/print.jhtml
 
Lewkowicz, David J. (2000).The Development of Intersensory Temporal Perception: An Epigenetic Systems/Limitations View, Psychological Bulletin, 126 (2), 281-308, Abstract available online: http://www.library.csi.cuny.edu/users/Lewkowicz/Abstracts.htm
Lichtenstein, Fred. (2001. Synesthesia. The Infinite Mind. Lichtenstein Creative Media, (Jan 10). Transcript available online: http://www.theinfinitemind.com/mind149.htm
 

Lovelace, C., Grossenbacher, P. Crane, C. (1999). Functional Connectivity Underlying Synesthetic Perception: Theories and Data. CNS Poster Available online: http://www.wfubmc.edu/nba/postdocs/lovelace/cnsposter99/cns.html

Lovelace, C. (2001). Integrating sensory integration. Trends in Cognitive Sciences. (Feb.). Summary available online: http://www.wfubmc.edu/bgsm/nba/IMRF/meeting00/rpt-preprint.html

Lyons, Andrew. (1998). Synaesthesia - A Cognitive Model of Cross Modal Association, ACMA, Available online: http://www.users.bigpond.com/tstex/synaesthesia.htm

Marks, Lawrence. (1996). On Coloured-Hearing Synesthesia. In Synesthesia: Classic and Contemporary Readings, ed. by S. Baron-Cohen, and J. Harrison. Oxford: Blackwells

Maurer, D. (1993). Neonatal synesthesia: implications for the processing of speech and faces. In de Boysson-Bardies, B., Jusczyk, P., MacNeilage, P., Morton, J., & deSchonen, S. (Eds.). Developmental Neurocognition: Speech and face processing in the first year of life. Dordrecht, Kluwer Academic Publishers. pp. 109-124.

Motluk, A. (1995). Current Work And Theory Of Synaesthesia: How many people hear in colour? New Scientist, 5/27,18. Available online: http://www.psychiatry.cam.ac.uk/isa/expinv.html#4

Rauschecker, J.P. (1995). Compensatory plasticity and sensory substitution in the cerebral cortex. Trends in Neurocience, 18, 36-43.

Pallas, Sarah. (2000). Cross-modal Plasticity "Visualizes" Auditory Cortex,” 2^nd Annual Multicensory Research Conference. Available online: http://www.wfubmc.edu/bgsm/nba/IMRF/meeting00/abstracts.html

Paulesu, E., Harrison, J., Baron-Cohen, S., Watson, J.D.G., Goldstein, L. Heather, J., Frackowiak, R.S.J., Frith, C.D. (1995). The physiology of coloured hearing. A PET activation study of colour-word synaesthesia. Brain 118: 661-676. Available online: http://www.psychiatry.cam.ac.uk/isa/frames.html

Quartz, S. & Sejnowski, T.J. (1997). The neural basis of cognitive development: A constructivist manifesto. Behavioral and Brain Sciences 20 (4): 537-596. Available online: http://www.cogsci.soton.ac.uk/bbs/Archive/bbs.quartz.html

Safonova, T., Zhuravlev, L. (1996). Multimodal and Multifunctional Neurons of Simple Nervous Networks. Towards a Science of Consciousness. Conference Poster. Available online: http://www.maurer.demon.co.uk/SPECIAL/02_15.html#177

Seitz, J. A. (2001). A developmental theory of metaphor and creativity. Journal of Cognition and Development. Available online: http://www.york.cuny.edu/~seitz/metaphoranal.htm

Stein, B.E., and Meredith, M.A., (1993). The Merging of the Senses. Cambirdge, MIT Press.

Teuber, H. (1967). Lacunae and research approaches to them. In Brain mechanisms underlying speech and language. ed. by C.H.Millikan & F.M.Darley, New York: Grune & Stratton 204-216. cited by Robin Allot, 1973,in The Physical Foundation of language, Seaford:ELB. Chpt. 1 (ii) Integral Operation of the Brain, http://www.percepp.demon.co.uk/pfol1st.htm#inte