Neuropyrhulogin, Vol . 32. No . 12 . pp. 1465-1473, 1994
Elsevier Seiena Ltd
Printed in Great Britain
W25-3932/94 $7 .00+0.00

Pergamon

0028-3932(94)00078-6

MULTIPLE MEANING SYSTEMS IN THE BRAIN : A CASE FOR
VISUAL SEMANTICS
ELIZABETH K . WARRINGTON and ROSALEEN A . McCARTHY*
National Hospital for Neurology and Neurosurgery, Queen Square, London WCIN 3BG, U .K .
(Received 13 November 1993 ; accepted 25 May 1994)

Abstract-In this study we report our investigations of a category specific visual associative agnosia .
The patient D .R .S .'s spoken language skills were excellent . Although he could name objects from
their description, he was unable to name them by sight nor was he able to mime their use . On
visual-visual matching tasks his performance was impaired and affected by semantic proximity . In
two tasks his knowledge of visual objects was demonstrated to be significantly more impaired than his
knowledge of visual living things . It is argued that these findings support a multiple knowledge base
hypothesis.
Key Words : visual agnosia ; semantic memory ; category specificity ; clinical study ; cerebral vascular
disease .

INTRODUCTION
The issue of whether knowledge bases for visual and verbal domains are fully integrated, or in
some sense separable, has profound implications for cognitive and neuroscientific theory .
Patients with selective preservation or selective loss of knowledge in one domain, sparing
knowledge in the other have provided evidence consistent with the multiple knowledge base
hypotheses [11, 17, 19, 21] . By and of themselves, the frequently reported category specific
dissociations between living things and objects do not necessarily contribute to the debate as
to whether there is a unitary all-purpose knowledge system [1, 2, 5, 18, 28] . Indeed parallel
category effects in two modalities have been held to favour the unitary hypothesis rather than
the less parsimonious formulation that these reflect parallel deficits in two separate
knowledge bases . We will argue it is the co-occurrence of disorders that are category specific
and modality specific that are of critical significance .
In the classical syndrome of visual associative agnosia, patients' semantic knowledge of
many visual objects is gravely compromised [9, 13] : objects, to quote Tcuher [20], arc
,. percepts stripped of their meaning" . The analogous syndrome in the verbal domain is
observed in patients with one subtype of transcortical sensory aphasia . For these patients, it
is the spoken word that is stripped of its meaning [e .g . 6, 26] . A lack of concordance between
verbal and visual knowledge does not necessarily provide conclusive evidence for separable
meaning systems . There are two main problems : First, the patient's deficit may be due to a
peripheral disorder of visual or auditory perception, second there may be a failure of
*Present address : Derailment of Experimental Psychology, Univer
Cambridge C.B2 3EB, U .K .
1465

V

of Cambridge, Downing Strecl,

1466

IS .K WAR RIVGTON :mdR .A .MLCARI'HY

communication between early sensory processing systems and a unitary central knowledge
base .
However, in cases where there is evidence of a domain specific deficit confined to one
semantic category then it becomes less plausible to ascribe the deficit to impaired sensory or
perceptual processing [e .g . 7, 8] . Problems in communication between input systems and
semantic knowledge have been less straightforward to differentiate from a primary deficit
within the knowledge base itself . Once again . we can appeal to evidence of category
specificity to rule out most variants of a disconnection hypothesis . For a deficit to be specific
to a specific semantic category, some semantic analysis must have been carried out by the
system . We consider it a contradiction to Suppose that a doubly dissociated category specific
deficit can arise at a pre-semantic level of processing .
We have previously described a patient in whom category and domain specific deficits cooccurred [14, 15] . Although TOB's visual knowledge of both animals and objects was
preserved, his verbal knowledge of animals but not objects was impaired . We argued that
T .O .B . provided evidence for categorically organised domain specific meaning systems in the
brain . The fact that the disorder was specific to the category of living things implicated
semantic processing ; the fact that the disorder was confined to the spoken word indicated
that there was a separation between visual and verbal semantics . In this report we describe a
further critical case that contributes to this debate . The patient had a visual associative
agnosia that appeared to he disproportionately severe for common objects . His verbal
knowledge appeared to be entirely intact .
CASE REPORT
.
was
a
59-year-old
physical
scientist who was referred to the National Hospital for
D .R .S
further investigations of his cerebra-vascular disease . On examination there were no
neurological signs other than a right-sided hemianopic field defect . His corrected visual
acuity was 6/9 and his colour discrimination was good (James, unpublished) . A CT scan
revealed bilateral but asymmetric infarcts in the right parietal lobe and inferior left occipitotemporal region .
D .R .S . was, investigated in the Neuropsychology Department over a 10-day period in
December 1987 . On a shortened form of the WAIS he obtained a Verbal IQ of 121 and a
Performance IQ of 72 [25] . His recognition memory for both words (43/50) and faces (42/50)
appeared to be intact [22] .
His performance on a range of tests of visual competence and visual perception was
adequate . On the Efron Squares test [3] of form perception his score was normal (19/20
correct) on the hardest discrimination (see Fig . 1) . On a figure detection task [24] his
performance, 16/20 correct, was creditable (see Fig . I) . On tests of perceptual analysis his
performance was good . There was no effect of an angle of view manipulation insofar as
having obtained the poor score of 12/20 correct [23] on the Unusual Views there was no gain
on the Usual Views (12/20 correct) . On a demanding object-decision task (see Fig . 1) he
obtained a normal score of 17/20 correct .
Although he was unable to read, his spelling was at a high level . His propositional speech
was fluent and sophisticated both in content and vocabulary . On a test of naming objects
from description his performance was error-free (15/15) . However . on a graded difficulty
object-picture naming test he scored only 1 1/30 [ 16] : here it appeared that his difficulty was
not merely one of retrieving the target name : but rather, that he had difficulty in recognising

B

C

Fig .] . (A) Example oft he most difficult discrimination oft he Efron squares test . I n a series of randomly presented items the subject is required to identify the
stimulus as a square or an oblong . (B) Example of shape detection test . The subject is required to determine whether the central "X" is present or not . (C) One
example of the object-decision test . The subject is required to point to the one real object, the other three shapes being nonsense shapes .

A

%& L

A
c

V1st :AL SEMANTICS

1469

the object (e .g . TWEEZERS "device for threading needles") . On a test requiring him to name
objects and animals (distinct coloured photographs) ranging in frequency from very
common to very infrequent, he named 7/20 objects and 11/20 animals (Thorndikc-Lorge
Mean Frequency 38 .1, 19 .0). Again, his errors indicated grave difficulty in identification of
some very common target objects (e .g . fork-rlighthulb, clock-sewing machine) . His
naming was tested on three further categories : occupations, sports and actions, Whilst he
scored a creditable 27/30 in identifying pictures of occupations, he was somewhat weaker at
naming pictures representing sports (22,/32) or actions (14/22) [12] . His ability to name
objects presented for identification via touch was considerably better than when the same
objects were presented visually : he scored 9/11 and 3/11, respectively .
Our investigation has focused on the analysis of his visual recognition deficit .
EXPERIMENTAL INVESTIGATIONS
Test L .Naming-rlsual is rerhal prewniarion
His ability to name objects from verbal description vs his ability to name from pictures was directly compared .
Twenty low frequency three-syllable words (e g . telescope and unicorn) and 20 low frequency one-syllable words
le .g . hoof-moat) were tested using both a verbal definition and a simple prototypical black and white line drawing .
.R picture naming was stow, laborious and inaccurate (25/40) . His naming from description was fast and
D
.
.S
efficient (37 ;401 . This dissociation establishes a domain specific naming difficulty .
'fe.sr 2r/nnintnitne ac ti . . .i

- visual us aerhal elicitation

D .R .S : s ability to produce pantomimes in response to a spoken object name or a visually presented object was
assessed . The stimuli consisted of 24 common household objects and tools with an associated characteristic and
distinctive action . l'he teststimuli were presented in sets of six items that were functionally and contextually similar
le .g . pliers, hammer, screwdriver, hacksaw, trowel, paintscraper ; paintbrush, hairbrush, shoebrush, toothbrush,
nailbrush . slmvingbrushl . Each set of objects was tested (I) for naming (2) for pantomime given the spoken name
and (3I for pantomime when shown the object out of reach . The conditions were tested in a counterbalanced order,
each item being sampled twice in each condition . D .R .S .'s mimes to the spoken name were flawless and
unmistakable (24/24 correct). However . his performance on the miming to object task was very impaired (13/24
correct) and at the same impaired level as object naming (13/24 correct) . On the miming task his performance was
characterised by semantic errors he g . pliers demonstrated as a drill ; screwdriver demonstrated as a hammer) . These
results establish that his deficit is not merely one of word retrieval, but affects his ability to identify common objects .
Test 3 : Word pirIure inarrldng-..semanrie prazirnity

In order to further determine the locus of D .R .S : s visual recognition problems he was tested on a word-picture
matching test which manipulated the semantic relatedness of targets [27] . The stimuli consisted of] 2 arrays of six
pictures (selected front the Sued grass and Vanderwartpool), six oft he arrays consisted of visually distinct exemplars
from a narrow semantic category (i .e . six items of clothing, of desktop objects, of containers, etc .) . The other six
arrays comprised one item drawn from each of six of these narrow categories (e .g . table, jumper, watch, pen, car,
howl I . Each item was probed twice in both the close and distant array conditions . He scored 56/72 in the close arrays
and 71/7' on the distant arrays . In view of his excellent verbal abilities we would infer that these findings reflect an
impairment in the visual component of the task . His sensitivity to the close/distant manipulation provides further
evidence that his disorder is at the level of visual object identification in visual semantics .
Test 4: t'isual eisoal maldtiny-rein objecis and pictured objects

In order to confirm that D .R .S . had a deficit at the level of visual identification, two visual-visual matching tasks
were devised- In these tasks the subject is required to match an object to its physically dissimilar "synonytti" pair (e .g .
two glasses) The stimuli in the first task consisted of three sets of four pairs of real objects . Each object in each array
was probed three times (a total of 36 responses) . Ile scored 26 .: 36 correct . The stimuli in the second task consisted of
two sets of four pairs of coloured object photographs . Each object in each pair was probed once, He scored 6,% 16
correct. Although he was slow and careful on these tasks his performance was nevertheless gravely impaired . His
errors included confusions between common items such as a wine glass and a light bulb .
Test 5 : Visual visual nmtchbrg-catepmrpspecificity

In the previous visual visual matching task, all the items probed were common objects . D .R .S . protested
vociferously when presented with these stimuli commenting' not another tat*t common object, I don't recognise

1470

E . K . WARRINGTON and R . A . McCARTHY

those!" We therefore attempted to broaden our stimulus range and tested him on a further matching test in which we
compared his performance on visual-visual and auditory-visual matching with common objects, animals and
vehicles . The stimuli consisted of four arrays of five simple clear lined rawings .In the visual-visual condition U.R .S .
was given a physically dissimilar line drawing of one item in the array and asked to match it to its pair (see Fig . 2) . In
the auditory condition he was asked to match a spoken word to a target item in the array, each item in each array was
probed eight times . Auditory and visual conditions were ordered in an ABBA design . The percent correct and
response latency for each array is given in Table I . By broadening the number of categories probed it appears that
there is a category effect insofar as his score on visual-visual matching of manipulable common objects is
significantly worse than his matching of animals ( x 2=13 .3, P<0 .001) . A similar category profile is seen in the
verbal visual matching task (y''=9 .60, P<0_01), albeit that his overall level is somewhat less impaired .

Fig . 2 . Examples of visual-visual matching test. The subject is required to point to the stimulus item in
the array that has the same "identity' as the singly presented lest item (i .e . the key on the door . the
lying down cow) .

1 47 1

VISUAL SEMANTICS

Test 6 : Word-picture matching-category specificity
In order to explore the category effect further we turned to a verbal-visual matching test that has previously been
found to show category effects [27] . The stimuli consisted of three arrays of distinctive coloured drawings of five item
each : five flowers, five common objects and five animals . He was tested in blocks of 20 trials in which each item was
probed four times. Each of the three arrays was tested three times using a Latin Square design . His scores were as
follows : Objects 38/60; Animals 56/60 ; Flowers 56/60. His score on the objects condition was significantly worse
than on the other two (Z2 -6 .0, P<0 .02) . These findings both replicate and extend the findings of category
specificity documented in Test 5 .

Table l . Percent correct and mean latency of response for each
condition .

Common objects 1
N=40
Common objects 2
N=40
Vehicles
N=40
Animals
N=40

Visual-visual

Auditory-visual

60%
17.4"
30%
29.2"
75%
13 .0"
80%
8 .1"

80
5 .7"
68
5 .6"
83%
5 .8"
95%
4 .5"

DISCUSSION
In this investigation we have documented a patient with a category-specific visual agnosia .
Although D .R .S . had excellent and very high-level verbal skills, he was impaired in
identifying visually presented common objects . His deficit was entirely within the visual
domain . The major feature of interest in this visual agnosic syndrome was its category
specificity . D .R .S . had grave difficulty in identifying common objects such as cups and
saucers or knives and forks but he could, nevertheless, identify a wide range of stimuli drawn
from other semantic categories . His performance was particularly competent with living
things . In this discussion we shall focus on the importance of this hitherto undocumented
dissociation .
First, it is important to establish the component of the visual information processing
system that is implicated by these impairments, We wish to argue that D .R .S .'s deficits are
accounted for by a disorder at the level of assigning meaning to a structured visual input . In
order to support this analysis, it is important to eliminate any sufficient causes of the disorder
arising at earlier stages in the visual information processing system : specifically at the stages
of sensory processing or structured perception . D .R .S . was competent on tasks requiring him
to detect, discriminate and group visual sensory input . In particular, his performance on tests
of figure-ground discrimination and form discrimination were adequate . These results
indicate that a deficit in the early stages of object processing cannot explain D .R .S .'s
disorder . Moreover, his ability to derive structural information from drawings and pictures
was good . D .R .S . did not appear to be differentially affected by presenting stimuli in a
visually complex or degraded format or in the context of perceptually similar discriminanda .
He was not additionally disadvantaged by presenting stimuli in a foreshortened view and he

1 47 2

F . K . WARRING ION and R. A . McCARTHY

performed within the normal range on a difficult object decision task . These results effectively
eliminate the possibility that his agnosia can be attributed to a deficit in deriving a volumetric
structural description ; the stage of information processing that is compromised by some
right parietal lesions (e .g . [23]) .
The locus of D .R .S .'s deficit therefore appears to lie either within those components of the
processing system that are responsible for assigning meaning to a structured percept, or in
access to them . Since D .R .S .'s deficit is also category specific we can go somewhat further
with our analysis . We wish to argue that his impairment is not merely a disconnection
between perceptual input systems and semantic processing but rather, that it implicates a
functionally distinct visual semantic system [e .g . 13] . Category specificity appears to
implicate a disorder within the procedures required for computing meaning rather than in
those processes that relay information from one level of coding to another .
There have been two other attempts to account for category by modality interactions that
merit discussion here . First, Caramazza and his colleagues [2] have proposed that there is a
central categorically organised semantic system . On their framework, the sparing of visual
living things (documented in our previous case, T .O .B .) was attributed to privileged
activation of this system from vision . However, asymmetries that favour the verbal domain
over the visual domain, as documented in the present case, D .R .S ., pose a serious problem for
this position . They would seem to refute it .
Secondly, Farah and McClelland [4] have proposed that knowledge is organised in two
parallel sub-systems, one organised for sensory information and one for functional
information . They suggest that the locus of the category by modality interaction arises prior
to addressing the full semantic representation . Thus, in the case of our previous case T .O .B .,
they argued that input from peripheral verbal systems to a visual/sensory semantic processor
was disconnected . The same formulation could, of course, be applied to D .R .S . : input from a
peripheral visual system might be disconnected from a functional/associative semantic
subprocessor .
While acknowledging that virtually any argument for a representational deficit might be
countered by a complex combination of disconnections interacting with differential trace or
connection strengths, we prefer a simpler interpretation . We propose that semantic
information can be, and is, computed in dissociable modality-specific meaning systems . The
fact that these modality-specific semantic systems appear to be sensitive to similar
independent variables (e .g . the distinction between sensory and functional semantic
categories) suggests that they may have comparable organisational characteristics . In
essence, our hypothesis is that processing domain (e .g . visual semantics vs verbal semantics)
is orthogonal to category specificity .
Similarities in organisation across visual and verbal domains are unlikely to reflect a
simple duplication of knowledge . These domains may differ because they arc required to be
different for normal function . They do appear to have a different timecourse in development :
synonymy and equivalence relations are different for vision and language and visual and
verbal knowledge may partition the world along different lines (and in different ways
depending on the characteristics of specific languages) . It therefore makes computational
sense for the visual and verbal systems to have the potential for a considerable degree of
autonomy since they have different inputs, often have different outputs, and typically differ in
their requirements for stimulus generalisation and equivalence . We want to understand what
we see without interference with what we say-but to have the option of making the two
domains co-referential .

VISUAL SEMANTICS

1473

This account is testable : it predicts that two further dissociations ought to be observed,
namely failure to recognise visual animals and failure to comprehend named objects . We do
not think the story is likely to stop there . Indeed, given the complexity and range of the
current catalogue of category specific effects this framework appears to be the most easily
extended to accommodate finer grain and broader ranges of deficit .
dcknoohd9emenr-R .A .M .'s work was partially supported by a grant from the Lenerhulme Trust to King's
College . Cambridge.

REFERENCES
L Basso, A . Capitani, F . and Laiacona, M . Progressive language impairment without dementia : A case with
isolated category specific semantic defect . J. Neurol . Neurosurg . Psychiat . 51, 1201-1207, 1988 .
2 . Caramazza, A ., Hillis . A . E ., Rapp, B . C . and Romani, C . The multiple semantics hypothesis : Multiple
confusions? Cognit . NeuropsVchol .7, 161-189, 1990 .
3 . Efron .R .In Bo .smn Sludie.vinthePhilosophpu/Sc'ience,R .F-CohenandM .Wartofsky(Editors),pp .137-17 3 .
Humanities Press, New York . 1968 .
4 . Farah . M . J . and McClelland, J . L . A computational model of semantic memory impairment : Modalityspecificity and emergent category-specificity . J. exp . Psycho! . 120, 339 357, 1991 .
5 . Farah, M . J ., McMullen, P . A . and Meyer, M . M . Can recognition of living things he selectively impaired?
Neuropsychologia 29, 185 193 . 1991 .
6 . Gesehwind, N ., Quadfascl, F . A . and Segarra, J . M . Isolation of the speech area. Neuropsychologia 6,327-340,
1968 .
7 . Hart,J ., B erndt .R .S .andCaramazza .A .Categoryspecificnamingdeficitfollowingcerehralinfarction .Nature
316,439 440, 1985 .
8 . Hart . J . and Gordon . B . Neural subsystems for object knowledge . Nature 359, 60-64 . 1992.
9 . Hecaen, EL . Goldblum, M . C ., Masure . M . C . and Rainier, A . M . tine nouvelle observation d'agnosied'objet .
Deficit de l'association Oil de la categorisation, specifique de In modalite visuelle. Neuropsychologia 12,447 464 .
1974 .
10 . Hillis, A . F . and Caramazza, A . Category-specific naming and comprehension impairment : A double
dissociation . Brain 114, 2081 2094, 1991 .
11 . Marshall, J . C . Sensation and semantics . Nature 334, 378, 1988 .
12 . McCarthy, R .A .andWarrington .E .K .Categoryspecificityinanagrammaticpatient .Therelativeinrpainnen t
of verb retrieval and comprehension . Neuropsychologia 23, 709-727, 1985 .
13 . McCarthy, R . A . and Warrington, E . K . Visual associative agnosia : A clinico-anatomical study of a single case .
J . Neuroi . Neurosurg . Psychiat . 49, 1233 1240, 1986 .
14 . McCarthy, R . A . and Warrington, F . K . Evidence for modality-specific meaning systems in the brain . Nature
334,428 430 . 1988 .
15 . McCarthy, R . A . and Warrington, E . K . The dissolution of semantics . Nature 343, 599 . 1990 .
16 . McKenna . P . and Warrington, F . K . Testing for nominal dysphasia . J. Neural. Neurosurg . Psychiat. 43,
781 788 .1980 .
17 . Riddoch, M . J . . Humphreys . G . W ., Coltheart, M . and Funnel], F . Semantic systems or system?
Ncuropsychologicalevidence re-examined . Cognir . Neuropsychol . 5, 3-25, 1988 .
18 . Sartori, C . and Job, R . The oyster with four legs : A neuropsychologieal study on the interaction of visual and
semantic information . Count . Neuropsychol . 5, 105-132 . 1988 .
19 . Shallice, T . Multiple semantics : Whose confusions? Cognit . Neuropsychol. 10, 251-261, 1993 .
20 . Teuber, H .-L . Some needed revisions of the classical views of agnosia . Neuropsychologia 3, 371-378, 1965 .
21 . Warrington, E . K . The selective impairment of semantic memory . Q . .f . exp . Psycho! . 27, 635-657, 1975 .
22 . Warrington, F . K . Recognition Memory Test. NFFR-Nelson, Windsor, Bcrks, 1984 .
23 . Warrington, E . K . and James . M . Visual apperceptive agnosia : A clinico-anatomical study of three cases.
Cortex 24, 13 32 . 1988 .
24 . Warrington, E . K . and James, M . The Visual Object and Spare Perception Battery . Thames Valley Test
Company, Bury St Edmunds, Suffolk, 1991 .
25 . Warrington, F . K ., James, M . and Maeiejewski, C . The WAIS as a lateralising and localising diagnostic
instrument : A study of 656 patients with unilateral cerebral lesions . Neurupsycholoyio 24, 223 239, 1986 .
26 . Warrington, F . K . and McCarthy, R . A . Category specific access dysphasia. Brain 106, 859-878, 1983 .
27 . Warrington, E . K . and McCarthy, R . A . Categories of knowledge : Further fractionations and an attempted
integration . Brain 110,1273-1296,1987 .
28 . Warrington, IL K . and Shallice, T . Category specific semantic impairments . Brain 107, 829 853, 1984 .