OBJECT-BASED NEGLECT: AN INVESTIGATION OF THE CONTRIBUTIONS OF EYE MOVEMENTS AND PERCEPTUAL COMPLETION R. Walkerl and A.W. Young2 eDepartment of Psychology, University of Durham, England; 2MRC Applied Psychology Unit, 15 Chaucer Road, Cambridge, England) ABSTRACT Eye movements made by a patient with object-based neglect and no hemianopia, RR, were recorded while he viewed whole, half, and chimaeric objects. When whole or chimaeric objects were presented centred on his initial fixation point, and when right half objects were presented, R.R. restricted his saccades to the right sides of the images. RR' s verbal identifications were consistent with these scan paths, since he correctly reported the identities of whole objects, right half objects, and the right sides of chimaeric objects, but failed to report objects on the left sides of chimaerics. When a left half object was presented to the left of initial fixation, however, R.R made left saccades to scan the image and correctly named these objects; hence his failure to scan the left sides of chimaeric object cannot simply be attributed to an inability to make contralesional saccades. We examined the possibility of perceptual completion by testing R.R with sets of whole and half shapes presented tachistoscopically and asking him to report what he had seen. When R.R viewed whole shapes he correctly reported seeing a whole shape on every trial. However, R.R. often reported seeing right half objects as whole, whereas he was much less prone to report left half objects as perceptually whole. We discuss the nature of this perceptual completion, and conclude that both scan paths and completion are consequences of objectbased neglect, not its causes. INTRODUCTION Unilateral neglect is a condition in which brain-injured patients ignore stimuli located contralaterally to the site of the lesion. This problem is usually more severe following right-sided brain injury, where the left side of the world is neglected. Neglect is known to operate in more than one frame of reference, and has been shown to be influenced by spatial and object-based co-ordinates. One of the first demonstrations of object-based neglect was provided by Gainotti, Messerli and Tissot (1972), who asked people to copy a model containing objects located along a horizontal axis (house, fence and trees). They found that some patients with right brain damage would omit the left sides of particular objects from their drawings even though they still drew other objects positioned further into the left side of the page. Object-based neglect has since been illustrated in other studies of patients' drawing performance (Halligan and Marshall, 1994; Hormak, 1995) and in perceptual experiments using nonsense shapes (Bermann and Tipper, 1994; Driver, Baylis, Goodrich et aI., 1994; Driver and Halligan, 1991). In some cases, dissociable forms of neglect have been found, and the Cortex, (1996) 32, 279-295 280 R. Walker and A. W. Young same patient may even neglect opposite sides of spatial and object-based representations (Humphreys and Riddoch, 1994, 1995). Object-based neglect has also been demonstrated with chimaeric stimuli made by joining the left half of one object to the right half of a different object at the vertical midline. Young, Hellawell and Welch (1992) showed that neglect patient B.Q. failed to report the left sides of chimaeric objects and chimaeric faces, even though she could identify these left sides when they were presented without any accompanying right half. Even more compellingly, if the two halves of a chimaeric face were separated by a gap, so that they no longer formed a single perceptual object, B.Q. was then able to identify the left sides. This observation was replicated with 5 further patients by Buxbaum and Coslett (1994). A contributory factor in demonstrations of object-based neglect may be the eye movements made while viewing the stimuli. An interesting study by Hornak (1995) showed that patients who drew only the right side of an object failed to scan the missing left sides of their own drawings. Hornak (1995) also found that when asked to recognise bilaterally symmetric objects, neglect patients only scanned the right sides, and that if right half objects were presented so that the missing left side would fall initially in the left visual field, neglect patients never made eye movements to scan the region of the missing left half. This held even for neglect patients who did not have left hernianopias. Object-based neglect can thus be associated with a failure to scan the left sides of seen objects. However, the nature of this link is uncertain because Hornak did not record eye movements made to left half objects presented in the patients' left visual fields. Without this condition, we do not know how easily neglect patients can make leftward saccades if there is no competing rightsided stimulation. This leaves it unclear whether their failure to fixate the left sides of objects is due to a generalised problem in making contralesional (leftward) saccades, or if the patients would only fail to make contralesional saccades when a stimulus is present in their ipsilateral (in this case, right) visual field. In the present study we therefore explored further the relation of eye movements to object-based neglect by recording eye movements made by a neglect patient when he viewed left half, right half, whole and chimaeric objects. Hornak (1995) also noted that her neglect patients verbally reported right half objects as being whole objects, and failed to notice the missing left sides. She thus considered the possibility that a second factor involved in object-based neglect is perceptual completion (Poppelreuter, 1917; Warrington, 1962). To investigate completion, Hornak (1995) presented left half shapes and right half shapes tachistoscopically, and asked neglect patients to report what they had seen. When these half shapes were aligned with the vertical midline running through the fixation point, so that a left half shape fell in the patient's left visual field and a right half shape fell in the patient's right visual field, all of Hornak's (1995) neglect patients incorrectly claimed that right half shapes (i.e., shapes with the left side missing) were 'whole' on at least some trials. In contrast, none of these neglect patients ever reported left half shapes as being whole. A control group of hemianopic patients without neglect did not make any similar completion responses with the half shapes. Object-based neglect 281 One way of interpreting thes~ completion responses for right half shapes is that neglect patients may in part fail to make appropriate eye movements to the left sides of symmetric objects because they already perceive a whole object by somehow mirror-imaging the scanned right sides. This would leave them unaware of the difference between whole objects and right half objects. On this account, object-based neglect could reflect a combination of unwillingness to scan the left sides of visual stimuli and symmetric completion of right-sided information onto the unscanned left side. To explore this hypothesis, we investigated eye movements and perceptual completion in a patient with object-based neglect without hemianopia, RR. In the first experiment, RR.'s eye movements were recorded when he viewed whole objects, chimaeric objects, and left and right half objects. Our aims were to show whether RR. could make contralesional saccades when a left half object was presented to the left of fixation, and whether or not he would do this with chimaeric objects. In the second experiment, we examined perceptual completion for RR In the discussion section we then outline the implications of our.results for the roles of eye movements and completion in object-based neglect. The findings show that both eye movement patterns and apparent perceptual completion are best considered to be consequences of object-based neglect, not its cause, and that the key factor in responses indicating perceptual completion lies in ignoring the contralesional sides of objects, not an active filling in. CASE HISTORY RR, a right-handed man born in 1934, suffered a stroke in 1990. The initial neurological examination revealed a left hemiparesis and left hemianaesthesia. A CT scaQ showed an extensive attenuation change in the fronto-temporo-parietal region, consistent with an infarct to the right middle cerebral artery. There was also a slight increase of attenuation within the right parietal region, which it was thought could have been due to an earlier haemorrhage. At this time, RR's scores on the Rivermead Behavioural Inattention Test (Wilson, Cockburn and Halligan, 19S7) indicated severe left sided neglect (Conventional score = 94, Behavioural score = IS). After discharge R.R made a mostly good recovery and could walk with the aid of a stick. Some aspects of left-sided neglect persisted, however, and he was referred to us in 1993. The investigations reported here were carried out in 1993 and 1994. A general assessment of RR.' s vision revealed good near visual acuity (NS). His contrast sensitivity was within the normal range for low spatial frequencies and only slightly below normal for middle and high spatial frequencies. RR' s visual fields were examined by presenting stimuli briefly to the left and right of a central fixation point on a VDU screen. Two fixation conditions were used; with a continuously displayed fixation point or a fixation point that was extinguished prior to stimulus presentation. This procedure has been used successfully to distinguish between left hemianopia and unilateral neglect (Walker, Findlay, Young et aI., 199 I). Unlike our previous case, who could only respond to left visual field stimuli when the fixation point was extinguished (Walker et aI., 1991), RR was able to report left stimuli under both fixation conditions. This demonstrates that he did not have a left hemianopia, and that he was able to attend to the left side of visual space even when there was a competing central stimulus. RR.'s visual fields were also plotted using an automatic perimeter (Henson CFS 3000). This automatic field plotter obtains a measure of the patient's ability to maintain fixation by presenting stimuli in the region of the blind spot. R.R reported only two out of a possible fourteen of these stimuli, showing that he was good at maintaining fixation (an essential requirement for accurate eye movement recording). 282 R. Walker and A. W. Young The resulting field plot confirmed the observation that R.R. did not have a left hemianopia, although he did show evidence of increased thresholds in the outer regions of his lower left quadrant beyond 10 degrees. His upper left visual field and upper and lower right visual fields were intact. R.R. did not show any signs of speech or verbal comprehension problems. His predicted pre-morbid IQ from the revised NART (Nelson, 1991) was 106, which was consistent with his educational background. He scored 18/20 on the Mini Mental State test (Folstein, Folstein and McHugh, 1975) and he was alert and well oriented with regard to time and place. He stated that his problems of neglect had been more severe immediately after the stroke, but had since improved considerably. Formal assessment in 1993 showed only a very mild tendency to neglect the left side of spatial displays. For example, when reading a short passage of text 10 line long (102 words), RR. omitted one word from the left side of one line. Similarly, in a crossing out task with four vertical columns of targets (10 targets positioned randomly within each column) R.R omitted only 4/40 targets which were all located in the far left column. Note that in both types of task many leftward eye movements would have to be made to sustain these levels of performance. In contrast to this evidence of only mild spatial neglect, RR showed marked neglect of the left sides of individual objects. For example, when asked to copy a scene containing eight objects, RR drew the right side of each individual object but omitted several details from the left sides of the objects. His drawings of a clock and a daisy from memory also showed left-sided distortions and omissions. When shown a set of chimaeric faces made from the left and right sides of two different people's faces, RR correctly named the face on the right of the chimaeric on 27/31 trials, but he was able to name the face on the left side of the chimaeric on only 12/31 trials. This type of performance involves what Humphreys and Riddoch (1994, 1995) call withinobject neglect, and it is clear that RR.'s object-based problems were more severe than the mild residual neglect of spatial displays shown in the reading and target cancellation tasks described above. RR. appeared to lack insight and awareness into this problem. When questioned about his own drawings he was unaware that the left sides of the individual objects were missing and satisfied that his copies were normal. When he failed to report the left sides of chimaeric stimuli, R.R. showed no awareness that the face on the left side was different from that on the right. In summary, then, at the time of our investigation RR had recovered from an initial severe neglect; he still showed object-based neglect, but only mild spatial neglect. The combination of differentially severe object-based neglect and no left hemianopia made RR an ideal person to explore eye movements and perceptual completion in object-based neglect. EXPERIMENT 1 Eye Movements and Object-based Neglect In Experiment 1, R.R' s eye movements were recorded while he viewed left half objects, right half objects, whole objects, and chimaeric objects. The objects used were familiar buildings and landmarks; these were chosen so that they formed a reasonably homogeneous set of grey-scale images from which chimaerics could be created. Chimaeric objects were made by joining the left half of one object to the right side of a different object. Our aim was to establish whether RR's object-based neglect could be attributed to an eye movement deficit. Object-based neglect 283 Mat~rials and Method A set of five photographs of famous buildings and landmarks was used to generate the stimulus set. The buildings and landmarks used were: Ayers Rock, the Colosseum, Stonehenge, the Arc de Triomphe and St. Peter's. The photographs were digitised and presented on an Apple Macintosh VDU at a distance of 57 cm; the whole object images were on average approximately 10 degrees wide. A set of left and right half objects was also made from the original whole images, by dividing each down the vertical midline. Five chimaeric objects were made by joining the left side of the image with the right side of another image. The left and right halves joined to form each chimaeric were selected so that they formed an approximate match for outline shape. At the start of each trial, a fixation square (sides of 1 degree visual angle) appeared in the centre of the VDU for I second. The offset of the fixation square was simultaneous with the onset of the target stimulus. The whole objects and chimaeric objects were presented centred on the fixation point, so that the left half of each appeared initially in RR.' s left visual field and the right half in his right visual field. Left half objects when presented in isolation were presented in the positions they occupied in corresponding whole objects or chimaerics, i.e. to the left of R.R's initial fixation position, and the right half objects were similarly presented to the right of fixation. Each stimulus was presented for 5 seconds; the offset of the stimulus was then followed by a delay of 5 seconds, during which time RR. was asked to verbally describe what he had seen. The order of stimulus presentation was entirely random. R.R's eye movements were recorded using a Skalar IRIS infrared limbus reflection eye movement recorder (Young and Sheena, 1975). One detector (left eye) monitored vertical eye movements and the other (right eye) monitored horizontal eye movements. The two channel (horizontal and vertical) analogue signal was sampled at 5 ms intervals and digitised using a labdriver interface. A chin rest was used to limit head movements. Sampling of the eye movement record started simultaneously with the stimulus image appearing on the monitor and stopped when the image disappeared. A number of calibration routines were performed to an array of nine stimulus circles (approx. 1 degree in diameter) surrounding a small central fixation point. There were three rows of circles (top, middle and bottom) spaced 6 degrees apart, with three circles in each row (left, centre and right) spaced 6 degrees apart. RR. was asked to move his eyes in a set sequence from top left to bottom right and to fixate on the centre of each of the calibration points in tum. To ensure that he could follow the desired sequence the experimenter moved a pen slowly from one point to another to enable him to locate each point with his eyes. The output from the eye movement recorder was sampled and stored to enable it to be translated into degrees of visual angle. The horizontal channel showed that R.R moved his eyes in the desired sequence and located all of the calibration points, including those located to the left of centre. The calibration record was later used to calculate the number of units (output from recorder) that corresponded to one degree of visual angle (linearity of signal was assumed). The eye movement records from each trial were analysed off-line with a program that enabled the calculation of the amplitude, direction and latency of each saccade. The starting point of each saccade was detected using a velocity criterion of 30 degrees/sec. Each record was examined individually and if contaminated by blinks or failure to maintain pre-trial central fixation it was not analysed; 1 record (St. Peter's in the whole object condition) was lost for this reason. The saccades identified in each record were drawn to scale and plotted as a scan path superimposed onto the original image. Results R.R. correctly named all of the whole images, left half images, and right half images. He correctly named all of the right sides of the chimaeric stimuli, but failed to report any of the left sides of the chimaerics. This combination of ability to identify left half objects when presented in isolation, but not when 284 R. Walker and A. W. Young - , ,)'~: ~ I: \\ \ } ",;; \ "/ ~ ,~ ,~"(' ....,.. . . ':. ' .~ 'r~ h~" "•.. ~ t' .siS~" ~$f -., ;! \. ". f~'" ~. ~:I '- ~i r~:l'~ ." • a~ - ~" , ,,~ Fig. I - R.R. 's eye movements made when viewing whole, left half, right half, and chimaeric stimuli, superimposed onto the originals. The square indicates the position of the initial fixation square. Solid black lines indicate the direction and amplitude of each saccade. and the arrows indicate the location of each fixation. Object-based neglect ~' -~ ~ ~ ~, ~\};)'\~:;.>\~~ "~!i'L ',\."~"\€~ , :I -t~~"', :"·~'7!\~:~y<~~~"'r.:})'~>l~ • 285 R. Walker and A. W. Young 286 TABLE I Proportion {~f total fixation time R.R. spent in each horizontal quadrant for whole objects, chimaeric objects, left half objects, and right half objects Whole Chimaeric Left half Right half Outer left Inner left Inner right Outer right 0% 0% 0% 0% 0% 0% 78% 0% 66% 69% 22% 63% 34% 31% 0% 37% they formed part of a chimaeric object, is consistent with observations in other reports where eye movements were not recorded (Buxbaum and Coslett, 1994; Young et aI., 1992). RR's eye movement records were superimposed onto the original images and are shown in Figure 1. As noted previously, the eye movement record from one whole building (St. Peter's) was not used as RR had not maintained central fixation prior to the onset of the image. When viewing right half images, RR always restricted his saccades to the right of the initial fixation point. When viewing whole objects RR also restricted his saccades and fixations to the right of the initial fixation point on every trial. Similarly, when viewing chimaeric images RR' s initial saccades were always made to the right of initial fixation. Furthermore, RR never made any saccades that crossed into the left side of the chimaeric images, and all fixations were restricted to the part on the right side. Despite this right-sided bias for right half objects, whole objects and chimaerics, RR did make leftward saccades to left half objects. On four out of five occasions when a left half image was presented, RR made some saccades to the left of initial fixation; on the other occasion the record showed that RR' s initial fixation position had been slightly to the left of the fixation point anyway. Hence, on every trial when a left half image was presented to the left of his initial fixation position RR was able to make left saccades and fixated on the half image. Although he was able to make saccades into left half objects, RR did not move as far into them as he went rightward with right half objects, wholes or chimaerics. To show this, we divided each whole object into four equal sized columns along the horizontal axis, to create outer left, inner left, inner right and outer right strips. We then calculated the proportions of the total fixation time RR spent in each of these horizontal quadrants for whole objects, chimaeric objects, left half objects and right half objects. These are shown in Table 1. The proportions for whole objects, chimaerics and right half objects are pretty much the same as each other, but there are two immediately striking features of the data for the left half objects. First, although RR fixated left half objects, he confined his fixations to the inner left area. The absence of outer left saccades to left half objects contrasts markedly with the fact that RR did make saccades into the opposite (outer right) region with all other types of figure. Second, some of RR' s saccades to left half objects fell in the inner right area, and he spent 22% of his time fixating this region of empty space. RR never fixated empty space when looking at right half objects. Object-based neglect 287 Discussion On trials when a left half object was presented to the left of fixation, RR did scan the image, and these left half objects were correctly reported on every trial. In contrast, when a chimaeric object was presented centred on the fixation point, RR restricted his saccades to the right of the image midline. On each trial with chimaerics, RR reported the object on the right side but failed to report the object on the left side. The eye movement records show that RR' s failure to scan the left sides of the chimaeric objects cannot simply be attributed to an inability to make contralesional saccades, since he made these to left half objects presented in isolation. Thus a failure to make saccades into the left sides of chimaeric objects should be regarded as being a consequence of RR ' s neglect, and not the primary cause of his deficit. That said, though, it is clear that the two problems are closely related, since RR showed only restricted leftward scanning of the left half objects. It is interesting to speculate that what he seemed to do was almost to treat the left half objects as if they were narrower versions of whole objects, and concentrate his eye movements onto what would then be their right sides (Le., the inner left region of Table I). Certainly, he seems to have moved his eyes only as far left as was necessary to identify each left half object. This point was also emphasised by Hornak (1995) for her cases, but it still does not account for RR' s failure to explore the left sides of chimaerics. RR knew that chimaeric stimuli were presented, and that his task was to report both sides. To do this, he would at most only have needed to make eye movements equivalent to those he made to left half objects; yet he was apparently unable to do so with the chimaeric. EXPERIMENT 2 Completion and Object-based Neglect Experiment 2 was performed to examine the possibility that RR showed a form of visual completion whereby he misperceived right half objects as being whole objects. This form of completion could be a factor that contributes to RR's unawareness of the left sides of chimaeric objects. To explore this, RR was shown whole and half shapes and asked to report what he had seen. Materials and Method A central fixation square (sides 0.5°) was positioned in the centre of a computer screen; this was made from black Jines (l pt.) on a white background, using a computer drawing package. Horizontal and vertical lines I cm long crossed in the centre of this square, forming an accurate centre point for fixation . Ten simple shapes were drawn using a computer package, and used as stimuli; they are shown in Figure 2. Shapes were drawn in black 4 pt. lines on a plain white background, and positioned so that they would fall within a template circle which measured 8° of visual angle in diameter at the viewing distance of 57 cm. Each of these whole shapes was presented centrally to R.R., so that it projected 4° into his left and 4° iQto his right visual fields. Sets of left and right half shapes were also made by dividing 288 R. Walker and A. W. Young D ] [ 6 ~ L D ] [ o ) C <> > < CJ 7 '= c=J ~ L * Jt C~ o ) ( C) Fig. 2 - Whole shapes, right half shapes, and left half shapes used in Experiment 2. Object-based neglect 289 each of the whole shapes at the vertical midline. This produced a total of 10 whole shapes, 10 left half shapes and 10 right half shapes. A Macintosh portable computer was used to present the stimuli (Powerbook 165). A tachistoscopic software package (SuperLab® 1.5) was used to control stimulus presentation. For each whole shape, its centre coincided with the centre of the fixation cross. The left and right half shapes were presented to the left and right of fixation respectively, occupying the same position that they would have had they been presented as a whole shape. The stimulus presentation sequence used for each trial was as follows; a fixation point was presented at the centre of the screen for 2 seconds, followed by a 50 ms presentation of the stimulus shape. The 50 ms stimulus duration was chosen from pilot work to be above RR's recognition threshold for the shapes used, but sufficiently brief to prevent any saccadic eye movements (Young, 1982). The offset of the stimulus shapes was followed by a blank screen, and RR was asked to report what he had seen; i.e. which shape, and whether it was a whole or half shape. The experimenter noted RR' s response- by making a key press which also initiated the start of the next trial. RR was tested on five separate blocks of trials during the same testing session. Each block contained one presentation of each of the whole, left half, and right half shapes (total of 30 trials). The presentation order of the whole shapes and half shapes was randomised throughout each block of trials. Thus RR received a total of 150 trials, during which he saw each of the whole and half shapes five times each. The instructions given were as follows: "Please look at the fixation square on the screen. You will then see a shape which will be presented very briefly. You will be asked to name that shape. On some occasions a whole shape will appear, but on other trials only a half shape will appear. Please state which shape you saw and if it was a sholw or a half shape". Results RR had little difficulty in correctly naming the briefly presented shapes. Occasionally he reported a "square" when a rectangle had been presented, and also reported a "circle" when the shape was actually elliptical. When such a mistake was made the experimenter corrected him for future reference and on each occasion RR agreed that the shape was actually a rectangle or circle. The percentages of shapes correctly identified and the percentages reported as being whole shapes or half shapes are shown in Table II. When a whole shape was presented, RR reported that he had seen a whole shape on every trial. With left half-shapes he correctly reported that a half shape had been presented on 41 trials, on 9 other trials he mistakenly said that a whole shape had been presented. RR' s ability to correctly identify left halfshapes as halves shows that he was able to correctly perceive a half shape and that he was not merely reporting "whole" on every trial. His ability to correctly identify a shape presented for 50 ms in his left visual field also confirms that he was not hemianopic. TABLE" Percentages of stimuli correctly identified by R.R., and the percentages of shapes that were reported as being either a whole or half shape Whole shapes Left half shapes Right half shapes Percentage of shapes correctly identified Percentage reported as whole 96% 92% 94% 100% 18% 52% Percentage reported as half 0% 82% 48% 290 R. Walker and A. W. Young Despite this, when presented with a right half shape, RR correctly identified it as being a half shape on 24 trials, but on the remaining 26 trials (52%) he mistakenly reported right half shapes as being whole. This tendency to report right half shapes as being whole is significantly greater than was shown for the left half shapes (X 2 =12.6, d.f.=l , p