This article was downloaded by: [University of Sydney] On: 05 January 2015, At: 02:21 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: 1072954 Registered office: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK Neurocase: The Neural Basis of Cognition Publication details, including instructions for authors and subscription information: http://www.tandfonline.com/loi/nncs20 Exploring the Role of Motion in Prosopagnosia: Recognizing, Learning and Matching Faces a b Karen Lander , Glyn Humphreys & Vickl Bruce c a Department of Psychology, University of Manchester , Manchester, UK b School of Psychology, University of Birmingham , Birmingham, UK c College of Humanities and Social Science, University of Edinburgh , Edinburgh, UK Published online: 17 Aug 2010. To cite this article: Karen Lander , Glyn Humphreys & Vickl Bruce (2004) Exploring the Role of Motion in Prosopagnosia: Recognizing, Learning and Matching Faces, Neurocase: The Neural Basis of Cognition, 10:6, 462-470, DOI: 10.1080/13554790490900761 To link to this article: http://dx.doi.org/10.1080/13554790490900761 PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the “Content”) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at http:// www.tandfonline.com/page/terms-and-conditions Neurocase, 10(6): 462–470, 2004 Copyright © Taylor & Francis Inc. 1355-4795/04/1006–462$16.00 Neurocase Exploring the Role of Motion in Prosopagnosia: Recognizing, Learning and Matching Faces Karen Lander1, Glyn Humphreys2 and Vickl Bruce3 1 Department of Psychology, University of Manchester, Manchester, UK, 2School of Psychology, University of Birmingham, Birmingham, UK and 3College of Humanities and Social Science, University of Edinburgh, Edinburgh, UK Downloaded by [University of Sydney] at 02:21 05 January 2015 Abstract HJA has been completely unable to recognize faces since suffering a stroke some 22 years ago. Previous research has shown that he is poor at judging expressions from static photographs of faces, but performs relatively normally at these judgements when presented with moving point-light patterns (Humphreys et al., 1993). Recent research with non-prosopagnosic participants has suggested a beneficial role for facial motion when recognizing familiar faces and learning new faces. Three experiments are reported that investigate the role of face motion for HJA when recognizing (Experiment 1), learning (Experiment 2) and matching faces (Experiment 3). The results indicate that HJA is unable to use face motion to explicitly recognize faces and is no better at learning names for moving faces than static ones. However, HJA is significantly better at matching moving faces for identity, an opposite pattern to that found with age-matched and undergraduate control participants. We suggest that HJA is not impaired at processing motion information but remains unable to use motion as a cue to identity. Introduction Prosopagnosic patients are unable to recognize once familiar faces. This problem can arise even though, in some cases, the patients are able to accurately judge expression and gender from faces (see Tranel et al., 1988). Prosopagnosics can also maintain accurate memories of many facial properties of familiar people (Young and De Haan, 1992; Michelon and Bicderman, 2003), but they seem impaired either at deriving the appropriate perceptual representation to access these stored representations (de Gelder and Rouw, 2000; Boutsen and Humphreys, 2002) or at activating these representations sufficiently for explicit recognition to take place (McNeil and Warrington, 1991). Consistent with this last proposal is evidence indicating that, at least in some patients, there is ‘covert’ recognition, as indicated in tasks such as learning correct versus incorrect name pairings to familiar faces (e.g., see De Haan et al., 1987; De Haan et al., 1992; Ellis et al., 1993; Farah et al., 1997). Despite the interest in prosopagnosia, few studies have directly investigated whether the manner in which the face is presented is important. An exception to this is the study by Humphreys et al. (1993). They report the cases of two patients with face processing impairments that showed different patterns of performance on identity and expression tasks, to both static and moving images. One participant, HJA, a prosopagnosic patient, was poor at judging expressions and gender from static photographs of faces, but performed relatively nor- mally on expression judgements when presented with moving point-light face patterns. The second patient, GK, performed better on identity judgements but was poor at judging expressions from both moving and static faces. Humphreys et al. (1993) propose that these cases suggest the existence not only of separate processes for encoding face identity and facial expression, but also separate processes for coding expressions from moving and static faces (also see Archer et al., 1994 for work with schizophrenic patients). Other recent work has investigated the role of motion for non-prosopagnosic participants in the recognition of familiar faces (see O’Toole et al., 2002; Roark et al., 2003 for current reviews). Results suggest that viewing a moving face aids the recognition of identity (see Knight and Johnston, 1997; Lander et al., 1999; Lander and Bruce, 2000). Faces can move in both rigid and non-rigid ways. During rigid motion, the face maintains its three-dimensional form, while the whole head changes its relative position and/or orientation. In non-rigid motion, individual parts of the face move in relation to one another, as during the formation of expressions or the articulation of speech. Rigid and non-rigid motion may contribute in different ways to familiar face recognition and unfamiliar face learning. O’Toole et al. (2002) speculate that seeing the face move may enhance the perception of its static structure, which in turn is helpful for recognition (the ‘representational enhancement hypothesis’). Such an effect would not depend Correspondence to: Dr Karen Lander, Department of Psychology, University of Manchester, Oxford Road, Manchester, M13 9PL, UK. Tel: +44 (0) 161 275 2598; Fax: +44 (0) 161 275 2588; e-mail: karen.lander@man.ac.uk DOI: 10.1080/13554790490900761 Downloaded by [University of Sydney] at 02:21 05 January 2015 Face motion and prosopagnosia on having prior experience with the particular face. Instead, seeing a face move would provide a richer or more complete structural description relative to a static image (Knight and Johnston, 1997). Alternatively, individual faces may have characteristic motion signatures, which capture the idiosyncratic aspects of their facial movement patterns (the ‘supplemental information hypothesis’, see O’Toole et al., 2002). Here, experience with the face is needed in order to learn an individual’s characteristic motion style. Whatever the theoretical basis for this recognition advantage, it is certainly true that seeing a face move adds information useful in the recognition of identity, particularly when the face is shown in non-optimum viewing conditions (Knight and Johnston, 1997; Lander et al., 1999; Lander and Bruce, 2000; Lander et al., 2001). Research has also suggested that there may be a beneficial role for motion in face learning. For example, Pike et al. (1997) filmed actors rotating on a motorised chair, which was illuminated from a single light source. In the learning phase, participants were asked to try and learn the identity of previously unfamiliar faces from dynamic sequences (360-degree rotation), multiple static images or a single static image. At test, participants viewed a single full-face static image, different from any shown in the learning phase and were asked to decide if the face shown had been present in the earlier learning phase. The data indicated that there was a significant advantage for faces learned via a coherent rigid motion sequence. Recent work has also suggested a beneficial role for non-rigid motion when learning new faces (Lander and Bruce, 2003; but see Christie and Bruce, 1998). Thus, prior work suggests that there is a beneficial role for facial motion when recognising already familiar faces (Knight and Johnston, 1997; Lander et al., 1999; Lander and Bruce, 2000) and when learning new faces (Pike et al., 1997; Lander and Bruce, 2003). These findings, coupled with earlier research showing a distinction between perceiving emotion in moving and static stimuli for some prosopagnosics, suggests that further work is needed to investigate the role of motion in face coding and recognition in prosopagnosic patients. This is the aim of the present paper. We ask whether motion can act as a cue to recognizing facial identities, tested under forced choice conditions (Experiment 1). We also explore the contribution of motion to face learning (Experiment 2) and face coding (assessed through face matching, in Experiment 3). The experiments were carried out with HJA, a patient with poor perceptual encoding of static faces (e.g., Boutsen and Humphreys, 2002), but who has demonstrated an ability to use motion for expression judgements (Humphreys et al., 1993). We assess whether motion contributes selectively to expression decisions or whether it can provide a more general contribution to face coding and recognition. Case report HJA suffered a stroke preoperatively in 1981, when aged 61. This produced bilateral damage in the territory of the posterior cerebral artery, affecting the ventral occipital-temporal 463 regions including lingual and fusiform gyri (see Riddoch et al., 1999, for an MRI). He has a superior altitudinal field defect along with good detection in his lower visual fields, and his eye movement patterns are normal (Riddoch and Humphreys, 1987). HJA suffers a range of neuropsychological problems that have been documented in several papers (Humphreys and Riddoch, 1987; Riddoch and Humphreys, 1987; Humphreys et al., 1992a, b; Riddoch et al., 1999; Giersch et al., 2000). In particular, he is severely agnosic, being especially poor at identifying objects drawn from categories with many visually similar exemplars (animals, insects, fruits, vegetables; see Riddoch and Humphreys, 1987; Humphreys and Riddoch, 2003). He is acromatopsic (Humphreys et al., 1992b), reads letter-by-letter and also has topographical agnosia (Humphreys and Riddoch, 1987). Tactile object recognition is intact, as are many other general intellectual abilities, including mental rotation (Riddoch et al., 2003). He is able to carry out some low level perceptual tasks including copying (albeit slowly and slavishly; Riddoch and Humphreys, 1987), Efron shape matching (Humphreys et al., 1992a) and matching across unusual views (Humphreys and Riddoch, 1984). His object recognition is also influenced by properties of visual stimuli and he is aided by addition of depth information gained through binocular disparity and head movements (Chainay and Humphreys, 2001). In addition to the above, HJA is profoundly prosopagnosic, failing even to recognize his close family. Person recognition instead relies on cues such as voice and walking gait (Humphreys and Riddoch, 1987). He is also able to make accurate categorizations of lip movements (Campbell, 1992). Although HJA maintains long-term knowledge of some facial properties of people (Young and De Haan, 1992), he has a severe impairment of coding configural information from static facial images (Boutsen and Humphreys, 2002). Prior evidence indicates a contribution of stimulus properties to HJA’s object recognition (Chainay and Humphreys, 2001), and that he can use motion when categorizing lip movements, expressions (Humphreys et al., 1992a, b) and gait (Humphreys and Riddoch, 1987). Hence he is a suitable patient for a study of the contribution of motion to face processing, since basic motion processing seems preserved. Experiment 1: Identity judgements The aim of Experiment 1 was to provide a reliable assessment of HJA’s overt face recognition abilities (see Humphreys et al., 1993) and to investigate the effect of face motion on this task. In this experiment, HJA was presented with a face clip and was asked to identify (by name or other semantic information) the person shown. If unable to provide any information about the person shown, he was asked to choose whether the face belonged to a famous or an unknown person (forced choice decision). This familiarity discrimination task was used to assess whether HJA had any access to stored information about faces, even if he was unable to access semantic or name information. Variations of this task have 464 K. Lander et al. been used in studies of face recognition with both prosopagnosic patients (Tranel and Damasio, 1987; De Haan et al., 1992) and non-prosopagnosic participants (Knight and Johnston, 1997; Lander et al., 1999). By examining performance with both static and moving faces, we test whether facial movements can provide an independent input to face recognition procedures, even when the derivation of perception representations of static faces is impaired (Boutsen et al., 2002). Downloaded by [University of Sydney] at 02:21 05 January 2015 Method Stimuli and procedure The stimuli consisted of 18 faces of famous people, and 18 faces of unknown people selected to match the famous faces on general visual appearance and clip type. Each face was shown both as a moving clip and as a static image, which resulted in 72 different clips (36 faces × 2 conditions). The length of each clip was set to 2.5 s. All moving clips displayed at least the head and shoulders of the person, from a frontal viewpoint. As the images were taken from TV and video productions, some people were shown from the waist upwards. The movement shown was mainly of a non-rigid nature (expressions and speech) with some additional rigid motion (head nodding and turning as well as rotation about the waist). The static clips comprised of a single freeze-frame (shown for 2.5 s) selected from the moving sequence. It was judged to be typical of the moving sequence in that it avoided any unusual momentary expressions, as well as unusual head movement or angle. The test sequences were displayed using a 30 cm by 21 cm Sony Trinitron TV monitor and the videotapes were played on a Sony VHS video recorder. All images were greyscale and filled the television screen, although the size of the face varied for different identities. The famous faces were rated for familiarity by a group of 10 independent age-matched volunteers (mean age 80.2 years). Each face was rated, by each volunteer, from 1 (unknown) to 7 (very familiar). Mean ratings were used to construct the two familiarity matched groups of faces (Group 1 average familiarity rating = 5.8, Group 2 average familiarity rating = 6.0). Overall, the 36 faces used were split into two groups of 18 faces (Groups 1 & 2), each comprising 9 famous and 9 unknown faces. Faces in one group were shown as moving clips first, whereas the other group were shown as static clips first. One or more different identities were always shown between each presentation of the same face. When each clip was presented, the videotape was paused and HJA was asked if he could identify the person shown. If unable to do so, he was asked to decide if the face was a famous one or an unknown one. HJA was told that half of the clips shown would belong to famous people. Testing was carried out in October 2000. Results The results are summarized in Table 1. HJA was unable to identify overtly any of the famous people from their faces, Table 1. The number (and percentage) of correct responses given by HJA using moving and static sequences of faces Type of face clip Identity decisions Familiarity decisions False alarm Moving Static 1/18 (6) 10/18 (56) 11/18 (61) 0/18 (0) 5/18 (28) 4/18 (22) with the exception of “Margaret Thatcher”, whom he correctly named when shown as a moving image. Interestingly, he was unable to identify Margaret Thatcher when later shown a static image. When questioned he said that none of the other faces felt overtly familiar and that his familiarity decisions were based either on hairstyle and clothing, or (for the moving clips) the way the people moved in front of the camera. For example, when shown Princess Anne moving he answered “Famous–Woman in public eye, knows how to act in front of camera”. Familiarity discrimination was at chance level, though HJA used both response categories, making 15 “familiar” and 21 “unfamiliar” responses to the famous faces. A McNemar Change Test was used to assess whether there were significantly more cases in which HJA correctly classified the moving face as famous and the static face as unfamiliar, than the opposite pattern. Results indicated no significant difference, χ2(1) = 1.92, p > 0.05. A similar analysis on false positive rates for the unfamiliar faces also indicated no significant difference across the moving and static conditions, χ2(1) = 3.76, p > 0.05. Furthermore, when the data from the famous and unfamiliar faces were pooled, HJA did not respond correctly significantly more often to moving faces than to static ones, χ2(1) = 0.36, p > 0.05. Thus, HJA was unable to use motion as a cue to access the identity of the viewed faces. This poor performance was not because HJA did not know the famous faces used. Indeed he was able to provide a written description about all the famous people, when given their name. We asked 20 undergraduate students at the University of Manchester to judge whether HJA had provided an accurate description of the famous person, and to rate each written description for visual and semantic content. The results confirmed that HJA did know the famous faces. Indeed, of the 20 raters, all thought HJA was familiar with at least 18 of the 20 famous faces used. The results also suggested that written descriptions of the famous people provided by HJA contained more semantic than visual information, by rater, t(19) = 2.12, p < 0.05 and by face, t(17) = 4.48, p < 0.05. Discussion Experiment 1 confirms that HJA is prosopagnosic. Despite knowing all the famous people used he failed to identify people from their faces. Furthermore, he found it difficult to make familiarity judgements to faces and performed at chance Downloaded by [University of Sydney] at 02:21 05 January 2015 Face motion and prosopagnosia level. Overall HJA was not significantly more correct when the faces were viewed as moving images and thus was unable to use facial motion to access stored information about known faces. It may be that HJA’s overt recognition abilities are so impaired that he is unable to recognize faces in any circumstance. The study fails to provide any evidence for an independent direct input to face recognition from motion information, distinct from the information derived from static faces. Nevertheless, despite not being able to use motion as a a cue to directly access existing face representations, HJA may be able to use motion as cue to learn new faces. Recent work with undergraduate students has suggested that viewing a moving image of a previously unfamiliar face may aid face learning (Pike et al., 1997; Lander and Bruce, 2003). We test whether HJA can also use motion to help learn faces, in Experiment 2, using a methodology designed to test covert recognition abilities. Experiment 2: Face-name learning This task was designed following the paradigm of De Haan et al. (1987) and Sergent and Poncet (1990), in which patients were asked to learn true and untrue pairing of names and faces. Covert recognition is suggested by faster learning of the true than untrue pairings. Four versions of this task were conducted, using 2 different sets of 6 famous faces, which were matched for familiarity (see description of familiarity matching in Experiment 1). Previous studies have failed to demonstrate any evidence of covert recognition in HJA, using tests of both object and face recognition (e.g., Humphreys et al., 1992b). We ask whether evidence for covert processing might emerge given an appropriate stimulus input (i.e., is there better learning of true relative to untrue face-name pairings, for moving faces?). In addition, we assess whether there is simply better learning of moving facial images, for both true and untrue pairings alike. Method Stimuli and procedure In each version of the task, 3 faces were paired with their true name (e.g. Barbara Winsor’s face with Barbara Winsor’s name) and 3 faces were paired with untrue names (e.g. Joan Collins’s face with Liza Minelli’s name). One trial consisted of learning and testing phases. In the learning phase each famous face was presented one at a time on a television screen for a total of 12 s, with the appropriate name (true or untrue) called out by the investigator. The faces were displayed as moving sequences or static images. The moving images consisted of a single twelve-second clip of the person, from the head and shoulders up. The motion displayed was predominately non-rigid (talking and expressions) with some limited head nodding and shaking. The static image was a freeze-frame selected from the moving sequence. The static image used did not show any unusual 465 facial expression or pose. HJA was asked to try and associate the viewed face with the heard name. Following the presentation of all 6 face/name pairs, HJA proceeded to the testing phase. Here the 6 faces were displayed as static images, arranged in a 3 × 2 configuration on a table. Each of the names was called out; HJA’s task was to point to the face associated with that name. In each version of the task there were 10 trials. On each trial, faces were learnt in a different order (always paired with same name within a task) and the positions of the static test images were shuffled between trials. There were four versions of the task, as follows: In version 1 the faces were learnt as moving images. In version 2, the faces were learnt as static images. A different static image was shown in the learning and testing phases, to reduce picture-matching effects. Version 3 was the same as version 2, except the true pairings became untrue and vice versa. Finally version 4 was the same as version 1 (learnt as moving images) but again the true and untrue pairings were swapped. Testing was carried out in October 2000 (versions 1 & 2) and November 2001 (versions 3 & 4). Results The differences between the true and untrue pairing for both static and moving faces were tested using the sign test. Each of the 10 trials involved 6 responses, and the number of times HJA pointed to the correct picture was counted separately for each trial (maximum = 3) for the true and untrue face-name pairings. Raw scores are presented in Table 2. Each trial was scored with either a “+” (better on true than untrue), “0” equal performance on true and untrue pairings), or a “−” (better on the untrue than true pairings). Table 2. Number of correct responses (max = 3) per trial on the face-name learning task with true and untrue pairings Trial Version 1: Moving faces True pair Untrue pair Version 2: Static faces True pair Untrue pair Version 3: Static faces True pair Untrue pair Version 4: Moving faces True pair Untrue pair 1 2 3 4 5 6 7 8 9 10 1 2 − 1 2 − 2 1 + 3 2 + 3 2 + 2 2 0 2 1 + 2 2 0 2 2 0 3 3 0 0 1 − 1 0 + 3 2 + 1 0 + 2 0 + 1 0 + 3 0 + 2 2 0 1 1 0 3 1 + 3 3 0 2 2 0 2 3 − 2 2 0 3 3 0 1 2 − 3 2 + 2 1 + 1 2 − 3 3 0 1 1 0 0 1 − 0 1 − 0 2 − 1 1 0 1 1 0 1 0 + 1 2 − 1 3 − 1 1 0 466 K. Lander et al. We compared the number of correct face-name pairings (irrespective of whether true or false) for the moving stimuli compared with the static stimuli, within a testing session (versions 1 & 2; 3 & 4). There was no significant difference between learning from moving and static faces (sign test; n = 17, x = 8, p > 0.05). We also compared the number of correct face-name pairings (irrespective of whether moving or static) for the true pairings compared with the false pairings. Similarly there was no significant difference between performance between learning in the true and untrue conditions (sign test: n = 25 x = 11, p > 0.05). Downloaded by [University of Sydney] at 02:21 05 January 2015 Discussion HJA was no better at learning the face-name pairings when shown moving faces compared to static ones. Thus, HJA was not able to use motion as a cue to learn new faces. This result is perhaps not unexpected since, anecdotally, HJA does not appear to learn faces even when the communicates with them personally on a daily basis. In line with previous findings, HJA also showed no evidence of covert recognition, and this held for both moving and static faces (see also Humphreys et al., 1992b). To summarise, although previous experiments (Humphreys et al., 1993) have suggested that HJA can utilise motion to recognize expressions and facial speech, he is no better at overtly recognising moving famous faces (Experiment 1) and cannot use motion as a method of learning new faces (Experiment 2). Despite this, motion may still be useful to HJA in face tasks where identity decisions are not directly required. We explore this possibility in Experiment 3, using a face matching task. Experiment 3: Face matching In this final experiment we compared how well HJA could match moving (rigid and non-rigid motion) and static faces for identity. This experiment was also run with a group of age-matched participants and a group of undergraduate students. Two recent studies have investigated the effect of motion on face identity matching. First, Bruce et al. (1999) found no advantage for moving faces. In this experiment, participants were asked to match a moving face (head nodding, shaking, speaking & smiling) or static face to one presented in a tenface static array (Experiment 3; Bruce et al., 1999). In contrast, Thornton and Kourtzi (2002) found that moving prime images, in the form of short video sequences (nonrigid change in expression), facilitated matching responses relative to a single static prime image. Here the nature of the initial or prime image was manipulated (moving/static) while the second ‘target’ image was always a single still image. This matching advantage was observed whenever the prime and target showed the same face but an identity match was required across expression (Experiment 1) or view (Experiment 2). No advantage was found for identical dynamic prime sequences when the matching dimension was shifted from identity to expression (Experiment 3). Thus, it is unclear whether or not, with normal participants, viewing a face moving facilitates face identity matching. One point to note is that both of the studies with normal participants manipulated whether the face was first viewed as moving or static, and matching was required to static images (an array in Bruce et al., 1999; single static images in Thornton and Kourtzi, 2002). Furthermore, existing experiments utilised either a mixture of rigid and non-rigid motion (Bruce et al., 1999) or non-rigid motion alone (Thornton and Kourtzi, 2002). In our experiment (Experiment 3) we used a matching task, in which participants were asked to decide if two consecutively shown faces belonged to the same person or to different people (respond ‘same’ or ‘different’). We manipulated the effect of motion, such that both faces could be moving (rigid or non-rigid motion), one could be moving (other static) or both could be static. We ask whether HJA performs this matching task better with moving than with static images, and whether a similar profile is found with normal observers. In addition we investigate whether contrasting types of motion have differential effects on face matching. Method Stimuli and procedure HJA was shown two sequentially presented clips of faces (500 ms black screen shown between clips) and asked to decide whether they were of the ‘Same’ person or two ‘Different’ people. All trials involved images of unfamiliar faces, and there were 112 trials in total. On half the trials the sequences were of the same person and, for the other half, they were different people. In total eight different faces were used in this experiment. All were selected from a database of 64 police trainees, collected by Vicki Bruce and Mike Burton. Moving VHS video sequences of the police trainees were available, and clips were trimmed to be 2.5 s in length. For each face, we selected two clips displaying rigid (rotational) motion (one where the person was looking up and down, one looking left and right) and two displaying non-rigid (expression and talking) motion. From these moving clips we also created two static clips of each face by selecting a single frame from the moving sequence. Each static image was representative of the moving sequence, whereby no unusual expression or head angles were displayed. The two static images for each face were slightly different from one another (e.g. one looking neutral, one with mouth slightly open). See Fig. 1 for examples of static images used. The eight selected faces were divided into two groups of four, based on previous distinctiveness data (faces rated for distinctiveness on 1–7 scale; Group 1 mean distinctiveness = 3.2, Group 2 mean = 3.1). Within the group of 4 faces, each face appeared with itself and another face within the group (8 trials). Seven different pairings of clips were created, as follows: non-rigid motion & non-rigid motion; non-rigid & Downloaded by [University of Sydney] at 02:21 05 January 2015 Face motion and prosopagnosia 467 Fig. 1. Examples of the static images used for matching in Experiment 3. static; static & non-rigid, rigid & rigid; rigid & static; static & rigid; static & static. Thus, there were seven conditions with 8 trials per condition, for both the same and different trials (7 × 8 × 2 = 112 trials). When a face was paired with itself (same condition) a different static image or moving sequence of the same face was utilized, thus preventing matching on the basis of pictorial or episodic factors alone. Similarly, different trials showed two different faces but again, for example, one was smiling and the other was talking. When a static image was paired with a moving sequence, the static image was one of the frames that made up the moving sequence. In addition to HJA, ten age-matched controls (8 males; 2 females; age from 78 to age 85 years, mean age = 80 years 11 months) and ten undergraduate students (5 males; 5 females; age from 17 to age 21 years, mean age = 19 years 3 months) completed the matching task. The undergraduate students were asked to make their matching decisions as quickly as possible, in an attempt to induce some errors. The speed stress was not emphasized for either HJA or the age-matched (elderly) controls. Results HJA Overall performance was not significantly different for same trials compared to the different trials (t(9) = 0.00, p > 0.05). Thus, the same and different trials were analysed together. Performance was analysed across conditions (same and different) using a series of McNemar Change Tests. There were significantly more correct responses when HJA viewed the non-rigid & non-rigid condition, compared to the static & static condition, χ2(1) = 3.20, p < 0.05. Similarly, there were also significantly more correct responses when HJA viewed the rigid & rigid condition, compared to the static & static condition, χ2(1) = 2.64, p < 0.05. No other comparisons reached significance. Thus, HJA performed best with moving faces i.e. in the rigid & rigid and non-rigid & non-rigid conditions. Age matched controls Overall performance was significantly better on same trials compared to the different trials (t(9) = 2.632, p < 0.05). Thus, the same and different trials were analysed separately. A oneway within subjects ANOVA of the same trial results revealed no significant main effect of condition (F(6,54) = 1.809, p > 0.05). Here, performance approached ceiling, and this may have contributed to the lack of any significant effects. A one-way within subjects ANOVA of the different trial results revealed a significant main effect of condition (F(6,54) = 3.683, p < 0.05). Further analysis using NewmanKeuls post-hoc pairwise comparisons revealed performance in the static-static condition was significantly better than in the non-rigid & non-rigid condition (p = 0.029), the static & non-rigid condition (p = 0.011), the rigid & rigid condition (p = 0.002) and the rigid & static condition (p = 0.007). The only other difference that reached significance was that performance in the static & rigid condition exceeded that in the rigid & rigid condition. Thus, performance was best in the static & static condition and worst in the rigid & rigid condition. Undergraduate students Overall performance was not significantly different in the ‘same’ trials compared to the ‘different’ trials (t(9) = 1.500, p > 0.05). Thus, the same and different trials were analysed together. A one-way within subjects ANOVA of matching performance results revealed a significant main effect of condition (F(6,54) = 5.137, p < 0.05). Newman-Keuls post-hoc pairwise comparisons revealed that performance in the rigid & rigid condition was significantly worse than in any other condition; non-rigid & non-rigid condition (p = 0.001); nonrigid & static condition (p = 0.002); static & non-rigid condition (p < 0.001); rigid & static condition (p = 0.015); static & 468 K. Lander et al. rigid condition (p = 0.006); static & static condition (p = 0.003). No other differences reached significance. Thus, performance was worse in the rigid & rigid condition. A one-way within subjects ANOVA of the reaction time data revealed no significant effect of condition (F(6,54) = 1.32, p > 0.05). Downloaded by [University of Sydney] at 02:21 05 January 2015 Discussion The pattern of the results was different for HJA compared to either the age-matched controls or undergraduate students. HJA was significantly worse in the static-static matching condition, compared to both the non-rigid & non-rigid and the rigid & rigid conditions. Conversely, the age-matched controls and the undergraduate students found the static & static condition relatively easy and performed worse in the moving matching conditions. The data for the control participants here fail to match prior reports, where there has either been no effect of facial motion on identity matching (Bruce et al., 1999) or a positive effect (Thornton and Kourtzi, 2002). It is not clear why this different result emerged here, though it may reflect the quality of the moving facial studies in the various studies. Certainly the result indicates that moving images were not simply better quality than the static images of faces. Despite this, HJA showed better performance with moving faces. This result emphasizes that HJA is poor at encoding static facial information and at using any resulting structural description of a face for identity matching. Interestingly, he was better able to use the information conveyed by moving faces, with performance being particularly good when the movement matched across both presentations (rigid & rigid, non-rigid & non-rigid). Indeed, in these last conditions his performance did not differ from the controls (see Table 3). Note, though that the precise motion patterns presented differed on both the same and different trials. Thus HJA’s performance could not be based on his matching exact patterns of motion. Instead, it seems clear that HJA can pick out invariances in spatio-temporal patterns, and use this information to make matching decisions. Such information may be image-based (e.g. reflect similarity of motion patterns available across sequences) or be related to person identity Table 3. Number of correct responses (max = 8) in each condition of the identity matching task, for patient HJA, age matched controls and undergraduate students. Reaction times (where appropriate) are shown in brackets Condition NR–NR NR–S S–NR R–R R–S S–R S–S HJA Age matched controls Undergraduate students Same Different Same Different Same Different 8 6 6 7 5 3 2 7 6 4 6 6 6 3 7.8 7.8 7.8 7.5 7.4 7.9 7.9 6.9 6.9 6.6 6.1 6.4 7.0 7.8 7.8 (918) 7.7 (980) 7.8 (987) 7.1 (999) 7.6 (948) 7.8 (920) 7.6 (1021) 7.6 (932) 7.5 (928) 7.8 (964) 7.2 (1033) 7.3 (983) 7.5 (940) 7.7 (974) information (e.g. movement may be used to help encode a moving structural description of a face). Interestingly, it is possible that the same individual can be characterised by person-specific motion parameters that are derived across different sequences. For example, Hill and Johnston (2001) showed that participants could discriminate between individuals using motion information alone. HJA may be able to derive such person-specific motion parameters for face matching, even if he is unable to use such parameters either to recognize previously familiar faces or to learn new face-name pairings. Alternatively, HJA’s ability to match moving faces may reflect a more general preserved motion processing ability. Whatever the theoretical interpretation, Experiments 1 and 2 indicate that HJA cannot use facial motion to aid face recognition or learning. General discussion The three experiments reported here allow us to make the following points: First, HJA cannot use motion as a cue to explicitly recognize once familiar faces (Experiment 1). Indeed, HJA was not significantly better at forced choice familiarity decisions when a face was moving compared with when it was shown as a static image. These data not only confirm that HJA is densely prosopagnosic, but also that facial movements do not aid his access to stored face memories. Second, HJA cannot use motion as a cue to learn new faces. This was demonstrated by the fact that he was no better at learning names for moving faces than static ones (Experiment 2). HJA also did not display covert recognition; he was no better at learning true names for famous faces than untrue ones, for static and moving faces alike. HJA was better at matching faces for identity when they were shown moving, compared to static images. This pattern of results was also opposite to that present with age-matched controls and undergraduate students, who found it easiest to match for identity when two static images were shown. The advantage HJA showed when matching moving faces could arise for a number of reasons. One is that, with static faces, he may tend to focus on particular details in the image rather than seeing the face as a whole. This would be consistent with other evidence indicating that HJA is impaired at integrating ‘parts’ into ‘wholes’ (Riddoch and Humphreys, 1987; Giersch et al., 2000; see Boutsen and Humphreys, 2002 for evidence with faces). In contrast, facial movement may encourage a more holistic encoding strategy, facilitating the encoding of a structural description supporting same-different matching. If this was the case, however, the data from Experiments 1 and 2 also indicate that such a structural description is not useful for his face recognition or learning. However the description may still be impoverished but sufficient to support face matching. An alternative possibility is that HJA used motion information directly for matching, whilst being unable to use it for matching to stored memories of for new face–name learning. This last view would be Downloaded by [University of Sydney] at 02:21 05 January 2015 Face motion and prosopagnosia consistent with HJA having problems with the processing of ‘form’ information, whilst being relatively unimpaired with processing ‘motion’ information (Desimone and Ungerleider, 1989; Oram and Perrett, 1996). The interesting point, then, is that HJA was able to use motion information in a way that normal subjects were not. Normal subjects found it relatively difficult to match moving faces (perhaps due to the particular images used here). Furthermore, given that HJA can use motion to compute facial expression (Humphreys et al., 1993), it would follow that the processes that use motion information to compute expression dissociate from those that use motion information for deriving structural descriptions and facial identities. Such a suggestion adds to recent discussion by Haxby et al. (2000) who propose that two neural systems are involved in encoding face information – one for changeable aspects of the face (e.g. perception of eye gaze, expression and lip movement) and the other for invariant aspects (e.g. perception of unique identity). Similarly, O’Toole et al. (2002) suggest that the ventrally-based stream processes the static structure of the face, and the dorsally-based stream processes facial motion (social communication signals and person-specific dynamic facial signatures). The degree of separation between the roles played by different systems or streams is unclear. For example, Haxby et al. (2000) suggest that the face identity region (lateral fusiform gyrus) may play a supportive role in the perception of expression, as different individuals may have characteristic expressions that we associate uniquely with them. Nevertheless, expression as indicated by HJA’s case might be computed from facial motion independently of the use of motion signals for identification (with only the latter being impaired in his case). Given the site of HJA’s lesion in the lingual and fusiform gyri, we suggest that motion signals for identification are processed in the ventral visual stream and are damaged in his case. In contrast, he remains able to process facial motion for emotion (Humphreys et al., 1993) and for perceptual matching (Experiment 3 here). This is consistent with facial motion also being processed independently in a more dorsal visual stream, spared in his case. Acknowledgements This work was supported by an ESRC grant to the third and first author and an MRC (UK) grant to the second author. We thank HJA for his great patience during our testing. References Archer J, Hay DC, Young AW. Movement, face processing and schizophrenia: Evidence of a differential deficit in expression analysis. British Journal of Clinical Psychology 1994; 33: 517–28. Boulsen L, Humphreys GW. Face context interferes with local part processing in a prosopagnosic patient. Neuropsychologica 2002; 40: 2305–13. Bruce V, Henderson Z, Greenwood K, Hancock PJB, Miller P, Burton AM. Verification of face identities from images captured on video. Journal of Experimental Psychology : Applied 1999; 5: 339–60. Campbell R. The neuropsychology of lipreading. Philosophical Transactions of the Royal Society 1992; 335B: 39–44. 469 Chainay H, Humphreys GW. Evidence for a role of surface and depth information in object recognition. Cognitive Neuropsychology 2001; 18: 175–91. Christie F, Bruce V. The role of dynamic information in the recognition of unfamiliar faces. Memory and Cognition 1998; 26: 780–90. de Gelder B, Rouw R. Structural encoding precludes recognition of face parts in prosopagnosia. Cognitive Neuropsychology 2000; 17: 89–102. De Haan EHF, Bauer RM, Greve KW. Behavioural and physiological evidence for covert face recognition in a prosopagnosic patient. Cortex 1992; 28: 77–95. De Haan EHF, Young AW, Newcombe F. Face recognition without awareness. Cognitive Neuropsychology 1987; 4: 385–415. Desimone R, Ungerleider LG Neural mechanisms of visual processing in monkeys. In: Handbook of Neuropsychology, vol 2. F. Boller and J. Gratman, editors Amsterdam: Elsevier Science 1989. Ellis HD, Young AW, Koenken G. Covert face recognition with prosopagnosia. Behavioural Neurology 1993; 6: 27–32. Farah MJ, O’Reilly RC, Vecera SP. The neural correlates of perceptual awareness: Evidence from covert recognition in prosopagnosia. In: Cohen JD, Schooler JW, editors. Scientific approaches to consciousness. Hillsdale, NJ, England: Lawrence Erlbaum Associates, Inc., 1997. Glersch A, Humphreys GW, Boucart M, Kovacs L. The computation of occluded contours in visual agnosia: Evidence for early computation prior to shape binding and figure-ground coding. Cognitive Neuropsychology 2000; 17: 731–59. Haxby JV, Hoffman EA, Gobbini MI. The distributed human neural system for face perception. Trends in Cognitive Sciences 2000; 4: 223–33. Hill H, Johnston A. Categorizing sex and identity from the biological motion of faces. Current Biology 2001; 11: 880–5. Humphreys GW, Donnelly N, Riddoch MJ. Expression computed separately from facial identity, and it is computed separately for moving and static faces: Neuropsychological evidence. Neuropsychologica 1993; 31: 173–81. Humphreys GW, Riddoch JM. Routes to object constancy: Implications from neurological impairments of object constancy. Quarterly Journal of Experiments Psychology: Human Experimental Psychology 1984; 36A: 385–415. Humphreys GW, Riddoch MJ. To see but not to see: A case study of visual agnosia. London; Lawrence Erlbaum, 1987. Humphreys GW, Riddoch MJ. A case series analysis of category-specific deficits of living things: The HIT account. Cognitive Neuropsychology 2003; 20: 263–306. Humphreys GW, Riddoch JM, Quinlan PT, Price CJ, Donnelly N. Parallel pattern processing and visual agnosia. Canadian Journal of Psychology 1992a; 46: 377–416. Humphreys GW, Troscianko T, Riddoch MJ, Boucart M, Donnelly N, Harding GFA. Covert processing in different visual recognition systems. In: Milner DA, Rugg MD, editors. The neuropsychology of consciousness. San Diego, CA, USA: Academic Press, Inc., 1992b. Knight B, Johnston A. The role of movement in face recognition. Visual Cognition 1997; 4: 265–73. Lander K, Bruce V. Recognizing famous faces; Exploring the benefits of facial motion. Ecological Psychology 2000; 12: 259–72. Lander K, Bruce V. The role of motion in learning new faces. Visual Cognition 2003; 10: 897–912. Lander K, Bruce V, Hill H. Evaluating the effectiveness of pixelation and blurring on masking the identity of familiar faces. Applied Cognitive Psychology 2001; 15: 101–16. Lander K, Christie F, Bruce V. The role of movement in the recognition of famous faces. Memory and Cognition 1999; 27: 974–85. McNeil JE, Warrington EK. Prosopagnosia: A reclassification. Quarterly Journal of Experimental Psychology: Human Experimental Psychology 1991; 43A: 67–287. Michelon P, Bicderman I. Less impairment in face imagery than face perception ln early prosopagnosia. Neuropsychologica 2003; 41: 421–41. Oram MW, Perrett DI Integration of form and motion in the anterior temporal potysensory area (STPa) of the macaque monkey. Journal of Neurophysiology 1996; 76: 109–29. O’Toole AJ, Roark DA, Abdi H. Recognizing moving faces: A psychological and neural synthesis. Trends in Cognitive Sciences 2002; 6: 261–6. Pike GE, Kemp RI, Towell NA, Phillips KC. Recognising moving faces: The relative contribution of motion and perspective view information. Visual Cognition 1997; 4: 409–37. Riddoch MJ, Humphreys GW. A case of integrative visual agnosia. Brain 1987; 110: 1431–62. Riddoch MJ, Humphreys GW, Gannon T, Blott W, Jones V. Memories are made of this: The effects of time on stored visual knowledge in a case of visual agnosia. Brain 1999; 122: 537–59. 470 K. Lander et al. Downloaded by [University of Sydney] at 02:21 05 January 2015 Riddoch MJ, Humphreys GW, Hardy E, Blott W, Smith A. Visual and spatial shortterm memory in visual agnosia. Cognitive Neuropsychology 2003; 20: 641–71. Roark DA, Barrett SE, Spence M, Abdi H, O’Toole AJ. Memory for moving faces: Psychological and neural perspectives on the role of motion in face recognition. Behavioral and Cognitive Neuroscience Reviews 2003; 2: 15–46. Sergent J, Poncet M. Fromi covert to overt recognition of faces in a prosopagnosic patient. Brain 1990; 113: 989–1004. Themton IM, Kourtzi Z. A matching advantage for dynamic human faces. Perception 2002; 31: 113–32. Tranel D, and Damasio AR Evidence for covert recognition of faces in a global amnesic patient. Journal of Clinical and Experimental Necropsychology 1987; 9: 15. Tranel D, Damasio AR, Damasio H. Intact recognition of facial expression, gender, and age in patients with impaired recognition of face identity. Neurology 1988; 38: 690–6. Young AW, De Haan EHF. Face recognition and awareness after brain injury. In: Milner DA, Rugg MD, editors. The Neuropsychology of consciousness. San Diego, CA, USA: Academic Press, Inc., 1992.