Neuropsychologia 38 (2000) 1235±1241 www.elsevier.com/locate/neuropsychologia Neuronal representation of object orientation H.-O. Karnath a,*, S. Ferber a, H.H. BuÈltho€ b a Department of Cognitive Neurology, University of TuÈbingen, Hoppe-Seyler-Str 3, 72076 TuÈbingen, Germany b Max-Planck-Institute for Biological Cybernetics, TuÈbingen, Germany Received 3 September 1999; received in revised form 10 February 2000; accepted 14 February 2000 Abstract The dissociation between object identity and object orientation observed in six patients with brain damage, has been taken as evidence for a view-invariant model of object recognition. However, there was also some indication that these patients were not generally agnosic for object orientation but were able to gain access to at least some information about objects' canonical upright. We studied a new case (KB) with spared knowledge of object identity and impaired perception of object orientation using a forced choice paradigm to contrast directly the patient's ability to perceive objects' canonical upright vs non-upright orientations. We presented 2D-pictures of objects with unambiguous canonical upright orientations in four di€erent orientations (08, ÿ908, +908, 1808). KB showed no impairment in identifying letters, objects, animals, or faces irrespective of their given orientation. Also, her knowledge of upright orientation of stimuli was perfectly preserved. In sharp contrast, KB was not able to judge the orientation when the stimuli were presented in a non-upright orientation. The ®ndings give further support for a distributed view-based representation of objects in which neurons become tuned to the features present in certain views of an object. Since we see more upright than inverted animals and familiar objects, the statistics of these images leads to a larger number of neurons tuned for objects in an upright orientation. We suppose that probably for this reason KB's knowledge of upright orientation was found to be more robust against neuronal damage than knowledge of other orientations. 7 2000 Elsevier Science Ltd. All rights reserved. Keywords: Object recognition; Object orientation; Agnosia; Brain damage; Human 1. Introduction Our ability to recognize objects from many viewpoints is remarkable. To identify objects regardless of position, scale or viewpoint, we must match them with mental representations of previously seen objects. Viewpoint-dependent theories hold that these representations are stored in a viewer-centered frame of reference determined by the location of the viewer in relation to the object. Accordingly, objects might be recognized by interpolating between previously seen and stored views [2] or by transforming either the input view, the stored view, or both (e.g., by mental rotation [11,14], or by alignment [18,19]). On the con* Corresponding author. E-mail address: karnath@uni-tuebingen.de (H.O. Karnath). trary, viewpoint-invariant theories hold that the representations are stored in object-centered frames of references based on the objects' geometry, e.g., their principal axes [6] or by orientation-free unique features of the object [3]. Object recognition thus would not require mental transformations to align the respective image of an object with a represented view of it. To our knowledge, Best [1] ®rst described a braindamaged patient who showed normal object recognition skills but an impaired sense of object orientation (for a translation of Best's case see Ref. [4]). Five further patients showing the same dissociation were recently investigated. The cases seemed to indicate that object recognition indeed can be achieved without viewer-centered information, that is without knowledge of the orientation of the object with respect to the subject. 0028-3932/00/$ - see front matter 7 2000 Elsevier Science Ltd. All rights reserved. PII: S 0 0 2 8 - 3 9 3 2 ( 0 0 ) 0 0 0 4 3 - 9 1236 H.-O. Karnath et al. / Neuropsychologia 38 (2000) 1235±1241 Best [1] studied a patient (case 38, Z) with a gunshot lesion a€ecting closely the same region in the inferior parietal lobes of both hemispheres as revealed by autopsy. The author presented a hand in the four cardinal orientations as well as objects and pictures of persons either in their canonical upright orientation or inverted through 1808. In each condition, the patient could identify the stimuli but was not able to judge their orientation. Solms et al. [12] asked a patient (WB) with bifrontal abscesses and episodes of inverted vision to sort letters according to whether they were presented in their upright orientation or inverted through 1808. Although WB could read and identify the letters in both orientations, he incorrectly classi®ed their orientation in 35% of the upright presented and in 82% of the inverted stimuli. A comparable observation was reported from a patient with BaÂlint± Holmes syndrome and bilateral parieto-occipital lesions [10]. The authors presented two letters either upright or inverted. While patient RM correctly named the letters on every trial except one, he incorrectly classi®ed their orientation in 14% of the upright presented and in 64% of the inverted stimuli. Turnbull et al. [16,17] reported three further cases. One patient (LG) su€ered from multiple strokes emanating from an arterio-venous malformation in the right temporoparietal region, two patients had strokes in the right parietal (NL) and the right temporo-parietal area (SC). Like the case reported by Solms et al. [12], the patients showed a tendency to rotate ®gures in copying while maintaining the correct internal structure of the object. When they were asked to indicate the canonical upright orientation of those objects that they perfectly identi®ed in any of four possible orientations (08, ÿ908, +908, 1808), the three patients performed 57, 53 and 13% incorrect orientation judgements. Since Best's [1] ®rst interpretation, the discrepant ability between determining object orientation and identifying these objects observed in these patients was taken as evidence that object recognition can be achieved without knowledge of object orientation. However, there was also some indication that the patients were not generally agnosic for object orientation but were able to gain access to at least some information about objects' canonical upright position. For example, LG was able to match the canonical upright of di€erent objects when an upright oriented model was shown next to the test stimulus. When copying di€erent objects presented in four possible orientations (08, ÿ908, +908, 1808), NL copied the components of each object quite accurately but made some rotational errors. Interestingly, all these incorrect copies showed objects rotated to their canonical upright. The latter observations could indicate that the perception of an object's canonical upright orientation is more robust against neuronal damage and thus dissociates from the ability to judge non-upright orientations. To closer investigate this assumption, we studied a new case with spared knowledge of object identity and impaired perception of object orientation using a forced choice paradigm to contrast directly the patient's ability to perceive objects' canonical upright vs non-upright orientations. 2. Methods 2.1. Patient KB KB, a 69-year-old, right-handed woman, was admitted to our department after sudden onset of apparent `blindness'. Three days before admission, the patient had already experienced discrete double vision, gait ataxia and reduced vigilance. Magnetic resonance imaging (MRI), including di€usion-weighted and T2weighted MRI, one day and four weeks post-admis- Fig. 1. T1-weighted magnetic resonance image (MRI) of patient KB performed one day after admission. The scan was coregistered in Talairach stereotaxic space using the linear normalization functions of SPM99 (www.®l.ion.bpmf.ac.uk/spm). Lesion location was determined using di€usion-weighted and T2-weighted MRIs performed one day and four weeks post-admission. The lesions were mapped using MRIcro software (www.mrc-cbu.cam.ac.uk/ 0 chris.rorden/ mricro.htm). Imaging revealed one old lesion (outline drawings ) in the left middle frontal gyrus and new infarcts ( ®lled drawings ) involving the parietal and the occipital lobes bilaterally. H.-O. Karnath et al. / Neuropsychologia 38 (2000) 1235±1241 sion revealed one old lesion in the left middle frontal gyrus and new infarcts involving the parietal and the occipital lobes bilaterally (Fig. 1). The new left hemispheric lesions were located in the lateral part of the parieto-occipital cortex, while the right hemispheric lesion a€ected its medial part. Left parietal damage included the superior parietal lobule, slightly extending into the superior part of the inferior parietal lobule (superior of the left supramarginal gyrus). A second left-sided lesion a€ected the occipital gyri. The lesion in the right hemisphere extended from the precuneus in the medial parietal lobe to the cuneus in the medial occipital lobe and to the posterior cingulum. During the ®rst days, KB behaved like a blind person. Seven days after admission, she had developed the cardinal symptoms of BaÂlint±Holmes syndrome: simultanagnosia, spatial disorientation, optic ataxia and impaired oculomotor behavior. She could recognize a square or a circle drawn on a sheet of paper, but when both objects overlapped each other, she had considerable problems with identi®cation of at least one ®gure. From complex scenes, KB was able to identify only single objects without recognizing the picture's general context or theme. On the ward, she could not ®nd her room, her bed, or other locations. Reaching for objects 1237 in peripersonal space was grossly ataxic with either hand in both visual half-®elds. KB did not show any signs of spatial neglect or visual ®eld defects but could not perform pursuit eye movements and showed no optokinetic nystagmus. Beyond BaÂlint±Holmes syndrome, clinical examination revealed a further symptom. The patient showed a striking discrepancy between the ability to recognize objects and to determine their orientation. When we presented pictures of tilted objects, e.g. a christmas tree, KB immediately recognized the object and associated the correct season but was not able to tell us that we had presented the picture upside-down. Due to her severe BaÂlint±Holmes syndrome, KB was not able to copy or write. However, when writing was tested with wooden letters, she frequently chose correct letters but rotated them by 2908 or placed them mirror-reversed. 2.2. Procedure The investigation started 12 days after admission and covered a period of six days in several sessions. Three item groups were selected for three di€erent sets of stimuli (Fig. 2): (a) 14 letters; (b) 24 black-andwhite pictures of simple animals and objects; and (c) Fig. 2. Twelve example stimuli from the three sets of stimuli (letters/animals and objects/faces). 1238 H.-O. Karnath et al. / Neuropsychologia 38 (2000) 1235±1241 60 photographs of faces (20 famous and 40 unfamiliar faces). All of the stimuli had unambiguous canonical upright orientations and were placed individually on square cards in front of the subject. Each letter was presented three times in its canonical upright position, tilted 908 towards the left, 908 towards the right and inverted through 1808. The ®rst stimulus set thus consisted of a total of 168 stimuli, 42 in each orientation. In the second stimulus set, each animal/object was presented two times in its canonical upright position, tilted 908 towards the left, 908 towards the right and rotated by 1808 (upside-down), yielding a total of 192 stimuli, 48 in each orientation. The faces were only presented in their canonical upright position and upside-down. This third stimulus set thus consisted of 120 stimuli, 60 in each orientation. Within each of the three sets, the stimuli and orientations were arranged in a pseudo-randomized order. 2.3. Experiment I KB was sitting at a table and the stimulus cards were presented in front of her while she was not allowed to touch them. The patient was asked to name each stimulus (identi®cation task) and subsequently to determine in a forced-choice manner whether the stimulus was presented in its upright orientation or not (orientation task). The identi®cation task was slightly di€erent with the photographs. KB was not required to give the full name of the subject but simply determined whether the person was famous or unknown. 3. Results 3.1. Identi®cation of stimuli KB did not show any impairment in identifying letters, animals/objects, or faces irrespective of their orientation. From the three sets of stimuli presented in Experiment I, she identi®ed correctly 100% of the letters (ntotal=168), 91.4% of the animals/objects (ntotal=192), and 97.5% of the faces (ntotal=120). 3.2. Determining stimulus orientation In both Experiment I and Experiment II, KB was able to judge the canonical upright orientation of the three di€erent sets of items when the stimuli were presented in their upright position (Fig. 3a±c). In contrast, her performance was at chance or close to chance level when the stimuli had a non-upright orientation, i.e. were oriented ÿ908, +908 or 1808 (Fig. 3a± c). We found no signi®cant di€erences between Experiment I and Experiment II indicating that explicit identi®cation by naming of a stimulus had no in¯uence 2.4. Experiment II To investigate whether the explicit identi®cation by naming of a given stimulus in¯uences the perception of its orientation, we presented exactly the same stimuli as in Experiment I but with a di€erent task. KB now only had to determine whether the stimulus was upright or not (forced-choice paradigm). Naming of the stimulus was neither required nor allowed. 2.5. Experiment III Experiments II and III were conducted the same days. In Experiment III, only the set of black-andwhite pictures of animals/objects was used. Like in Experiment II, naming of the stimulus was neither required nor allowed. However, KB was now asked to touch the cards and to rotate them to their canonical upright. No time limit was used. The subject indicated when the `upright orientation' was reached. Fig. 3. Percentage of KB's correct orientation judgements in the three sets of stimuli: letters (A), animals and objects (B), faces (C). The dashed line indicates the 50% chance level, the grey area marks the 95% con®dence interval. Black bars, Experiment I. White bars, Experiment II. H.-O. Karnath et al. / Neuropsychologia 38 (2000) 1235±1241 on the perception of its orientation (letters: w 2=1.38, ns; animals/objects: w 2=1.28, ns; faces: w 2=0.5, ns). In the set of animals/objects stimuli (Fig. 3b), KB showed a systematic error of determining object orientation when the stimuli had either a ÿ908 or +908 orientation which she judged in 89% of the presentations as upright (Experiments I and II). Such a systematic error was not found for the letter stimulus set (Fig. 3a). When KB in Experiment III was required to rotate the animal/object cards to their canonical upright, she showed no severe diculties to determine the correct upright orientation of the stimuli (Fig. 4). 3.3. Time course of de®cit Nineteen days after admission we repeated Experiment I with a reduced sample of stimuli. Eight di€erent letters and 15 di€erent animals/objects were presented in any of the four possible orientations (08, ÿ908, +908, 1808). The third stimulus set was reduced to 35 di€erent faces, each presented twice in its canonical upright and upside-down orientation. In all three stimulus sets, KB now performed at ceiling giving 100% correct responses in both the identi®cation as well as the orientation task. 4. Discussion Like the six previously reported cases [1,10,12,16,17], KB showed normal object recognition but an impaired sense of object orientation. When we directly contrasted the patient's ability to perceive objects' canonical upright vs non-upright orientations, we found KB's knowledge of upright orientation perfectly preserved while she was not able to judge the Fig. 4. Percentage of KB's correct rotations of `animals and objects' stimuli to their upright position. n = 48 stimuli were presented in each initial orientation. 1239 orientation when the stimuli were presented in a nonupright orientation. Explicit identi®cation by naming a stimulus had no in¯uence on the perception of its orientation. In contrast, KB showed no diculties to ®nd the correct upright orientation when she was allowed to rotate the stimuli. The results clearly show that the patient was not generally agnosic for object orientation. When a stimulus was presented in its upright position or when KB was free to rotate the stimulus, the patient showed undisturbed knowledge of upright orientation. Given that the previously studied patients in other experimental tasks also showed some knowledge of the canonical upright orientation of objects, the results obtained in the present and the previous cases rather argue for a speci®c weakness or inability to determine an object's orientation when that object is presented in a nonupright orientation. Therefore, it is dicult to agree with Turnbull et al. [16,17] who interpreted these patients' de®cit as an `agnosia for object orientation'. Quite similar to Best's [1] original explanation, these authors interpreted the discrepancy between object identi®cation and the perception of its orientation as evidence for the existence of an orientation-dependent and an orientation-independent route to object recognition in humans. They further assumed that their view can be accommodated within the two visual systems account of Milner and Goodale [7]. The latter assumed a ventral and a dorsal stream of processing information about the properties of objects and their spatial locations. While the dorsal stream provides the instantaneous and egocentric features of objects, the ventral stream permits the formation of perceptual and cognitive representations which embody the enduring characteristics of objects. Turnbull et al. [16] interpreted the observation of normal object recognition but an impaired sense of object orientation as an operation mediated via the viewindependent ventral stream in the absence of the dorsal stream that carries orientation information. However, in our opinion, the present as well as the previously reported cases showing this dissociation of processing object information argue for a di€erent view. Beyond the obvious discrepancy between the patients' ability to identify objects and to determine their orientation, they also show a discrepancy between their knowledge of upright orientation vs non-upright orientation of objects. It seems that their ability to determine the objects' upright orientation is more robust against neuronal damage. The ®nding of preserved knowledge of upright orientation of objects contradicts Best's [1] and Turnbull et al.'s [16] corresponding interpretation of the disorder as induced by an absence of a neural system that processes object orientation information. However, the same observation is in rather good agreement 1240 H.-O. Karnath et al. / Neuropsychologia 38 (2000) 1235±1241 with Perrett's thesis that one does not need mental rotation to recognize familiar objects in non-canonical orientations [8]. The authors proposed that objects are represented by neurons tuned to view, orientation and size and that the number of tuned neurons for a particular orientation depends on the amount of experience with it. Logothetis et al. [5] showed that exposure to a particular view of an unfamiliar object increases the number of cells tuned to that particular view. Since most natural objects are oriented in a gravity-based upright orientation we can assume that many more neurons are recruited to code an object in that orientation. This numerical bias is sucient to explain why patient KB and also the previous cases showing normal object recognition but an impaired sense of object orientation were not generally agnosic for object orientation. They showed a dissociation between their knowledge of objects' canonical upright orientation and their ability to perceive objects' orientation in non-upright positions. A larger number of neurons tuned for upright orientation would predict that the perception of objects' canonical upright is more robust against neuronal damage or may recover earlier than knowledge of other orientations. Nevertheless, this latter view cannot answer all of the questions raised so far by the patients exhibiting this peculiar dissociation of object recognition and impaired sense of object orientation. For example, it remains unclear why KB's knowledge of upright orientation did not help her to identify when an object had a di€erent, i.e. a non-upright orientation. Why couldn't she deduce that objects which were not in that (upright) orientation must be misoriented? Furthermore, it remains unanswered why such patients rotate objects by 908 or 1808 in copying [12,16,17] or in writing with wooden letters (patient KB, see Methods above). A preserved knowledge of upright orientation of objects should prevent such errors. (Rotational errors in copying tasks were observed in even more patients su€ering from cortical lesions but, unfortunately, their knowledge of object orientation and object recognition was not formally compared [9,13,15].) A possible answer to these questions might be the dramatic change of the pattern of neuronal activity after the stroke. In the normal brain, recognition of objects in non-upright orientations involve neurons tuned to the appearance of objects in such orientations. While perception of objects' canonical upright orientation post-stroke still is possible due to the larger number of neurons tuned for upright views, the discharge of those neurons that have been recruited to code non-upright orientations is lacking or much weaker. The latter leads to the patient's inability to determine the orientation of a rotated object in the acute stage of the disease. It is not plausible to assume that shortly after the stroke the patient can compensate for this loss of neuronal activity by interpreting the absence of signals from those neurons tuned for the canonical upright and from those tuned for nonupright orientations as the new signal for the appearance of a rotated object. Thus, it seems implausible that the preserved knowledge of upright orientation should help the subject to judge a non-upright orientation or that this knowledge should prevent rotational errors in copying or writing. However, it is possible that during recovery, the subject may learn to interpret the absence of discharge from these neuronal populations as the equivalent of an object appearing in a non-upright orientation. We would then expect that the patients' inability to determine non-upright orientations improves in course of time. Indeed, the present investigation revealed that 19 days after admission when Experiment I was repeated, KB now performed at ceiling giving 100% correct responses in both the identi®cation as well as the orientation task. However, since the present observation is the ®rst report of the de®cit's time course, it is too early to speculate whether or not the inability to determine non-upright object orientations is in general a transient phenomenon or may persist over longer time periods in some subjects. These questions remain issues for future studies of this interesting and recently re-discovered disturbance. Nevertheless, one may tentatively conclude that it is not `agnosia for object orientation' [16,17] in these patients but much rather a disability to determine the orientation of predominantly those objects that have a non-upright orientation. Thus, one should consider whether a term such as `agnosia for rotated object orientation' is more suitable to characterize the disorder. Acknowledgements We are grateful to Oliver Turnbull and David Perrett for their helpful comments and thoughtful advice on a previous version of the manuscript. This work was supported by grants from the Deutsche Forschungsgemeinschaft awarded to the ®rst author. References [1] Best F. Hemianopsie und Seelenblindheit bei Hirnverletzungen. vGraefes Archiv fuÈr Ophthalmologie 1917;93:49±150. [2] BuÈltho€ HH, Edelman S. Psychophysical support for a 2-D view interpolation theory of object recognition. Proceedings of the National Academy of Science 1992;89:60±4. [3] Corballis MC. Recognition of disoriented shapes. Psychological Review 1988;95:115±23. [4] Ferber, S, Karnath, H-O. Friedrich Best's case Z. with misiden- H.-O. Karnath et al. / Neuropsychologia 38 (2000) 1235±1241 ti®cation of object orientation. In: Code C, Wallesch C-W, Joanette Y, Lecours AR, editors. Classic cases in neuropsychology, vol. II. East Sussex: Taylor & Francis, in press. [5] Logothetis NK, Pauls J, Poggio T. Shape representation in the inferior temporal cortex of monkeys. Current Biology 1995;5:552±63. [6] Marr D, Nishihara HK. Representation and recognition of the spatial organization of three-dimensional shapes. Proceedings of the Royal Society of London 1978;B200:269±94. [7] Milner AD, Goodale MA. The visual brain in action. Oxford: Oxford University Press, 1995. [8] Perrett DI, Oram MW, Ashbridge E. Evidence accumulation in cell populations responsive to faces: an account of generalisation of recognition without mental transformations. Cognition 1998;67:111±45. [9] Pillon B. Troubles visuo-constructifs et methodes de compensation: resultats de 85 patients atteints de lesions cerebrales. Neuropsychologia 1981;19:375±83. [10] Robertson L, Treisman A, Friedman-Hill S, Grabowecky M. The interaction of spatial and object pathways: evidence from Balint's syndrome. Journal of Cognitive Neuroscience 1997;9:295±317. 1241 [11] Shepard RN, Cooper LA. Mental images and their transformations. Cambridge, MA: MIT Press, 1982. [12] Solms M, Kaplan-Solms K, Saling M, Miller P. Inverted vision after frontal lobe disease. Cortex 1988;24:499±509. [13] Solms M, Turnbull OH, Kaplan-Solms K, Miller P. Rotated drawing: the range of performance and anatomical correlates in a series of 16 patients. Brain & Cognition 1998;38:358±68. [14] Tarr MJ. Rotating objects to recognize them: a case study on the role of viewpoint dependency in the recognition of threedimensional objects. Psychonomic Bulletin & Review 1995;2:55± 82. [15] Turnbull OH. Rotated drawing and object recognition. Brain & Cognition 1996;32:120±4. [16] Turnbull OH, Laws KR, McCarthy RA. Object recognition without knowledge of object orientation. Cortex 1995;31:387± 95. [17] Turnbull OH, Beschin N, Della Sala S. Agnosia for object orientation: implications for theories of object recognition. Neuropsychologia 1997;35:153±63. [18] Ullman S. Aligning pictorial descriptions: an approach to object recognition. Cognition 1989;32:193±254. [19] Ullman S. High level vision. Cambridge, MA: MIT Press, 1996.