Author’s Accepted Manuscript
Patterns of neural activity predict picture-naming
performance of a patient with chronic aphasia
Yune Sang Lee, Jihad T. Zreik, Roy H. Hamilton

www.elsevier.com/locate/neuropsychologia

PII:
DOI:
Reference:

S0028-3932(16)30411-0
http://dx.doi.org/10.1016/j.neuropsychologia.2016.11.010
NSY6176

To appear in: Neuropsychologia
Received date: 1 February 2016
Revised date: 6 October 2016
Accepted date: 13 November 2016
Cite this article as: Yune Sang Lee, Jihad T. Zreik and Roy H. Hamilton,
Patterns of neural activity predict picture-naming performance of a patient with
chronic
aphasia, Neuropsychologia,
http://dx.doi.org/10.1016/j.neuropsychologia.2016.11.010
This is a PDF file of an unedited manuscript that has been accepted for
publication. As a service to our customers we are providing this early version of
the manuscript. The manuscript will undergo copyediting, typesetting, and
review of the resulting galley proof before it is published in its final citable form.
Please note that during the production process errors may be discovered which
could affect the content, and all legal disclaimers that apply to the journal pertain.

Patterns of neural activity predict picture-naming performance of a patient with
chronic aphasia
Yune Sang Lee1, Jihad T. Zreik2, Roy H. Hamilton2,

1 Department of Speech and Hearing Science, The Ohio State University,

Columbus, OH USA
2 Department of Neurology, University of Pennsylvania, Philadelphia, PA USA

Number of figures: 5
Number of tables: 2
Abstract word count: 231
Introduction word count: 527
Discussion word count: 2142
Conflicts of interest: The authors declare no competing financial interests.

Key Words
Aphasia, picture-naming, fMRI, MVPA, semantic, speech production, language,
production

Please address correspondence to:
Roy H. Hamilton, MD, MS
Goddard Laboratories, Room 518
University of Pennsylvania
3710 Hamilton Walk
Philadelphia, PA 19104
email: roy.hamilton@uphs.upenn.edu
phone: 215-573-7090; fax: 215-898-1982

Yune Sang Lee, Ph.D.
104D Pressey Hall,
The Ohio State University
1070 Carmack Rd.
Columbus, OH 43210
email: lee.7966@osu.edu
phone: 614-292-1743

1

Acknowledgments:
The authors would like to thank Branch Coslett, Olu Faseyitan, Felix Gervitz
Myrna Scwartz, Grant Walker, Nicole White, Kate Corbeil, and John Detre. Our
special thanks go to the subject KL for his indefatigable energy and enthusiastic
participations.

Abstract
Naming objects represents a substantial challenge for patients with
chronic aphasia. This could be in part because the reorganized
compensatory language networks of persons with aphasia may be less
stable than the intact language systems of healthy individuals. Here, we
hypothesized that the degree of stability would be instantiated by
spatially differential neural patterns rather than either increased or
diminished amplitudes of neural activity within a putative compensatory
language system. We recruited a chronic aphasic patient (KL; 66 year-old
male) who exhibited a semantic deficit (e.g., often said “milk” for “cow”
and “pillow” for “blanket”). Over the course of four behavioral sessions
involving a naming task performed in a mock scanner, we identified
visual objects that yielded an approximately 50% success rate. We then
conducted two fMRI sessions in which the patient performed a naming
task for multiple exemplars of those objects. Multivoxel pattern analysis
(MVPA) searchlight revealed differential activity patterns associated with
correct and incorrect trials throughout intact brain regions. The most
robust and largest clusters were found in the right occipito-temporal
cortex encompassing fusiform cortex, lateral occipital cortex (LOC), and
middle occipital cortex, which may account for the patient’s propensity
for semantic naming errors. None of these areas were found by a
conventional univariate analysis. By using an alternative approach, we
extend current evidence for compensatory naming processes that operate
through spatially differential patterns within the reorganized language
system.

1. Introduction
Although some degree of language recovery occurs over time in many
patients with chronic aphasia, object naming remains a challenging task for
these individuals. One interesting observation of the naming deficits in
patients with aphasia is that performance on particular items often

2

fluctuates in an unpredictable and sometimes seemingly random manner;
this manifests itself during picture-naming tasks as inconsistent name
retrieval when the same pictures are repeatedly presented (e.g., ‘cat’ is
named either ‘cat’ or other similar animals such as ‘dog’).
Various neuroimaging techniques have offered helpful insights for
understanding the neuroanatomical basis of naming deficits in aphasia
(Saur & Hartwigsen, 2012, Thompson & Ouden, 2008). For instance, voxel
lesion symptom mapping (VLSM) is useful for identifying the distribution
of lesions associated with different subtypes of aphasia (Bates et al., 2003,
Schwartz et al., 2011). By contrast, functional magnetic resonance imaging
(fMRI) allows for online measurement of neural activity in spared brain
regions while patients with aphasia perform a particular language task such
as overt picture naming (Fridriksson, Baker, & Moser, 2009, Léger et al.,
2002, Meinzer et al., 2013, van Oers et al., 2010, Postman-Caucheteux et al.,
2009, Szaflarski et al., 2011). This approach has proven useful in
highlighting changes in compensatory networks over the course of
spontaneous language recovery (Saur et al., 2006).
However, standard fMRI analysis methods, which explicitly assume
‘greater’ or ‘less’ brain activation across different tasks or populations, are
limited and often produce puzzling results in aphasia research. For example,
Fridriksson and colleagues (2009) measured neural activity during a naming

3

task in both chronic aphasics and normal subjects. Despite normal subjects
clearly outperforming patients in the language task, no significant areas
were found to differentiate healthy controls from aphasics with respect to
neural activity in the language network. One potential explanation for this
discrepancy is that successful retrieval of names may not depend on the
intensity of activation but rather on the pattern of activation within the
newly engaged language network. The central aim of the present fMRI
study is to explore the compensatory neural processes that sustain language
performance in aphasic patients by testing this hypothesis. To do this, we
employed multivariate pattern-based analysis (MVPA), an alternative
approach for relating neural activity to behavioral success or failure using
machine-learning techniques (Mahmoudi, Takerkart, Regragui, Boussaoud,
& Brovelli, 2012). Figure1A depicts differences in the hypothesis between
MVPA and the standard univariate analysis.
In this proof-of-concept study, we carefully selected a set of pictures
that our subject could correctly name with 50% accuracy. These pictures
would allow us to directly compare patterns of neural activity for correct
versus incorrect trials, while holding constant a number of potential
confounds, including visual object features and the complexity of object
names (e.g., number of syllables). By adopting this strategy, we ensure that
any differences observed in the MVPA are attributable to performance

4

accuracy across trials. The participant underwent multiple behavioral
sessions involving object naming in order to identify the candidate items for
the main fMRI sessions. Consistent with our hypothesis, we found that
MVPA could be used to link patterns of neural activity to behavioral
outcomes but that standard fMRI analysis was insensitive to differences in
performance.

2. Methods
2.1. Participants and stimuli
We identified candidate participants and stimuli for this study from an
existing data set (Walker & Schwartz, 2012), in which 25 chronic aphasic patients
performed the Philadelphia Naming Test (PNT) twice on different days. Seven
candidate participants demonstrated naming scores that fell within the midrange [39-70%, mean=53%] among the 25 potential subjects and 29 picture items
that were neither too difficult nor easy (Figure 1B); these items yielded errors
once in either of the two PNT sessions in 35-70% of the patient cohort.
Among the seven candidate participants, one patient (KL) volunteered for
the present study. The patient’s lesion profile is shown in the Figure 1C. He was
a 66-year-old right-handed man with chronic nonfluent aphasia who had a
stroke encompassing the left hemisphere 11 years prior to the study and had
previously participated in a transcranial magnetic stimulation (TMS) study in
our laboratory (Hamilton et al., 2010). Written consent was obtained from the

5

patient’s spouse as approved by the Institutional Review Boards of the
University of Pennsylvania and the Moss Rehabilitation Research Institute.

2.2. Experimental Procedure
Phase 1: Behavioral Sessions
Prior to the main fMRI study, the participant completed four behavioral
sessions comprised of overt picture-naming tests performed in a mock MRI
scanner (Figure 1D). Sessions were separated by a gap of two to four weeks. We
employed 3 exemplars (e.g. 3 different pictures of a camel) for each of the 29
candidate items that were selected based on the PNT test-retest data. All images
were in color and were matched for their size and luminance using Photoshop
CS5 (Adobe Inc.). The typicality of the images was ensured by testing several
colleagues at Penn’s Center for Cognitive Neuroscience. A random sequence of
the 87 (29 items x 3 exemplars) picture stimuli was determined by the De Bruijn
sequence (Aguirre, Mattar, & Magis-Weinberg, 2011). The first half of the
sequence (1st-44th trials) was presented in the first block, and the second half of
the sequence (45th-87th trials) was presented in the second block. To match the
number of trials across the block and not to break the random sequence, the last
trial of the first block (44th) was repeated as the first trial of the 2nd block. This
repetition was removed from the time-series prior to data analysis. Another
random De Bruijn sequence was created and presented in the same manner,
totaling 176 trials (44 trials x 4 blocks) split across different blocks. During each

6

block of the test, KL’s verbal responses were recorded using a digital voice
recorder attached to the inside of the mock MRI scanner; these recordings were
later transcribed by the experimenter. Additionally, KL’s head motion was
recorded while he performed the picture-naming task.

Phase 2: fMRI Sessions
Over the course of the 4 behavioral sessions, we identified seven
candidate items that were suitable for the 2nd phase of the study involving fMRI
scanning: “butterfly,” “boot,” “camel,” “closet,” “cow,” “pillow,” and “turkey”
The average accuracy for each of these pictures was approximately 50%. “Closet”
was later replaced with “blanket” because the “closet” images contained
multiple other namable objects (e.g., clothes). We chose “blanket” because this
item was semantically related to “pillow.” Furthermore, for each of the picture
items, we included one additional exemplar (i.e., 4 exemplars per item) to
decrease the repetition of images and to increase the visual variability of
exemplars for each object. This resulted in a total of 28 stimuli for each run of the
fMRI sessions, which were randomly presented using a slow event-related
design (interstimulus interval=12 sec.). As was the case with the behavioral
session, randomization was achieved based on a de Brujin cycle (Aguirre et al.,
2011). There were a total of six functional EPI runs. KL’s verbal response was
monitored via a MRI-compatible microphone and recorded using OptiMRI (v
3.1). A week later, we conducted a 2nd fMRI session in which we replaced

7

“butterfly” and “boot” with “cactus” and “milk.” This was done because KL had
exhibited ceiling accuracy on “butterfly” and “boot” (Figure 3B). We selected
“cactus” and “milk” because they were semantically related to the items “camel”
and “cow,” respectively. The procedure for the 2nd fMRI session was identical to
that of the 1st fMRI session.
MRI data were collected using a 3T Siemens Trio scanner (Siemens
Medical System, Erlangen, Germany) equipped with an 8-channel head coil.
Scanning began with acquisition of a T1-weighted structural volume using a
magnetization prepared rapid acquisition gradient echo (MPRAGE) protocol
[axial orientation, repetition time (TR) = 1620 ms, echo time (TE) = 3.09 ms, flip
angle = 15 º, field of view (FOV) = 187.5 x 250 mm, slices=160, voxel resolution =
0.98 × 0.98 × 1 mm]. Subsequently, 6 runs of blood oxygenation level-dependent
(BOLD) functional MRI scanning were performed (TR = 2500 ms, TE = 25 ms, flip
angle = 90 º, FOV = 234 × 234 mm, 44 slices, voxel resolution = 3 × 3 × 3 mm).
Finally, a B0 mapping sequence was acquired at the end of the scanning (TR =
1010 ms, TE1 = 2.67 ms, TE2=5.28 ms, flip angle = 60º, FOV = 234 × 234 mm, 44
slices, voxel resolution = 3 × 3 × 3 mm).

2.3. fMRI data analysis
Preprocessing: All functional images acquired from the two fMRI sessions
were combined for preprocessing (i.e., a total of 12 runs). We first unwarped data
using the prelude and FLIRT routines from FSL version 5.0.5 (FMRIB Software

8

Library, University of Oxford). These unwarped images were corrected in slicetiming and realigned using SPM8 (Wellcome Trust Centre for Neuroimaging).
We then normalized the lesion images to the MNI space using ANTS (Avants et
al., 2011): First, we acquired the transformation matrix by normalizing
AAL_MNI_V4 atlas to the subjects’ anatomical image (with an additional binary
mask image defining lesioned areas) in the native space. Next, the matrix was
inversely applied to normalization of both anatomical and functional images to
the MNI space (Figure 2A). For complementary univariate analysis, these
normalized images were brought into SPM8 for smoothing with 8mm FWHM
Gaussian kernel.
Multivariate pattern classification (Searchlights): Prior to the main analysis,
unsmoothed images were further processed by applying a high-pass filter (128s
cut-off) and by mean-centering time-courses across the entire runs. For the
purpose of binary classification using a GNB (Gaussian Naïve Bayses) algorithm
(Raizada & Lee, 2013), we extracted time points from a subset of data consisting
of “cow” and “blanket.” (We note that data from the other five items were not
included in order to maintain a balanced data set for the binary classification. See
the behavioral and fMRI results for more details.) These two items were then
labeled as correct and error depending on naming performance and were
collapsed across the objects. The classification was performed only within the
intact tissue of brain areas by creating local searchlight sphere comprised of a
center voxel and neighboring voxels in a 2-voxel radius. For cross-validation, we

9

employed a leave-two-out procedure in which the classifier was trained with (n-1)
number of observations for each condition and tested on the remaining two
observations (one for correct and one for error). The mean accuracy was stored at
the center voxel of the searchlight sphere after cross-validation. Finally, a
MonteCarlo simulation was performed to validate the classification accuracy.
The simulation procedure was identical except that the classifier was trained on
randomly shuffled labels between correct and error trials. This was performed
1000 times and the distribution of accuracies across the 1000 iterations was
obtained for every center voxel of the searchlight sphere. Lastly, significant
voxels were determined by comparing the classification accuracy to the random
distribution at a threshold of significance of P<0.01 (i.e., higher accuracy than the
top 1% of the accuracy distribution).
Multivariate pattern classification (Pattern Similarity): Our main searchlight
analyses revealed two findings (discussed further in the Results below): 1) the
right occipito-temporal cortex was the site with the largest and more robust
clusters, and 2) the subject’s naming performance was notable for the high
prevalence of semantic errors. Therefore, in an exploratory analysis, we
contrasted the pattern of neural activity in the occipito-temporal cortex
associated with a specific frequently-observed semantic error—misnaming “cow”
as “milk”—compared to the patterns of activity associated with correct naming
of “cow” and “milk.” To do this, the time-courses of fMRI data were extracted
within an ROI of the occipito-temporal cluster (a total of 127 voxels)

10

corresponding to the following instances: 1) “cow” correctly named as “cow”; 2)
“cow” incorrectly named as “milk”; and 3) “milk” correctly named as “milk.”
The Pearson-product moment coefficient was first obtained then converted to a Z’
score for every pair-wise comparison among the three instances: Z’(milk for cow
vs. milk for milk), Z’(cow for cow vs. milk for milk), and Z’(cow for cow vs. other
animals for cow ).

Univariate analyses: In addition to the searchlights, we performed a mass
univariate analysis. The fMRI time-courses of each condition were convolved
with a canonical hemodynamic response function (HRF) in order to obtain
estimated responses under the general linear model (GLM) framework in SPM12.
Six motion parameters were also included as regressors. For small volume
correction, we constructed a region-of-interest (ROI) image covering the occipital
and parietal cortex using the Anatomy tool box (Eickhoff et al., 2005). We then
excluded voxels residing on the lesion site from the ROI. Resulting maps were
obtained at the voxel-wise threshold of P<0.001 (uncorrected) and the cluster size
correction of P<0.05 (family-wise error). The cluster size correction was removed
in exploratory analyses.

3. Results
3.1. Behavioral data in the MRI mock scanner

11

The subject’s performance was consistent across 4 visits (48.3%, 49.4%,
51.7%, 48.3% naming accuracy from visit 1 to 4 respectively), indicating no
training effect over the course of the behavioral sessions. While overall
performance was stable, accuracy on particular items varied from one
experiment to another. For example, the accuracy of boot was 50%, 83%, 42%,
and 38% from the visit 1 through 4. As noted above, by averaging accuracies
across the 4 behavioral sessions, we identified the following 7 items that had an
approximately 50% mean accuracy rate: “boot,” “butterfly,” “closet,” “camel,”
“turkey,” “pillow,” and “cow” (Figure 3A). Error type analysis revealed that
majority of errors during the behavioral sessions were nonresponses (60.7%),
followed by semantic (32.1%), and phonemic errors (7.1%) (Figure 3D). Error
types were distributed homogeneously across all exemplars and across the 4
visits.

3.2. Behavioral data in the fMRI scanner
KL’s naming performance improved when naming a more limited
number of pictures during the fMRI sessions (first session: 80%, second session:
81%; see Figure 3B & 3C for accuracies on individual pictures). For both fMRI
sessions, the majority of errors were semantic paraphasias (88% and 97%, for the
first and second sessions, respectively), followed by a few phonetic errors (6%
and 0%) and nonresponses (6% and 3%). Figure 3D shows the percentages of
different errors types combined across the 2 fMRI sessions. Notably, the rate of

12

nonresponses was substantially reduced during phase 2, compared to phase 1.
KL’s naming performance on two items—“boot” and “butterfly”—was at ceiling
(100% and 96%, respectively) during the first fMRI session. Therefore, as
described above, these items were replaced with the items “milk” and “cactus”
in the second fMRI session. By averaging the accuracy across the two fMRI
sessions, we identified two items (“cow” and “blanket”) that yielded balanced
data sets and thus used these items for a pattern-classification test. For both fMRI
sessions, KL made minimal movements during the naming task (Figure 2B).
------------- Table 1 about here -------------

3.3. fMRI data (multivariate searchlights)
Prior to performing a searchlight on data that combined sessions 1 and 2,
we ran a searchlight in each of the two sessions separately to see if the overall
resulting maps were consistent with each other. As can be seen in Figure 4A, the
searchlight yielded similar results, although higher sensitivity was seen in the
second map. Based upon the resulting maps and comparable behavioral
performance, we were reassured that these two data sets could be combined for
purpose of the MVPA searchlight.
The main searchlight analysis on this merged data sets revealed separable
patterns of neural activity between correct and error trials in multiple sites
within intact cortical areas bilaterally (Figure 4B; Table 1). Among the significant
regions, the largest cluster with the highest accuracy was found within the right

13

occipito-temporal cortex, encompassing the lateral occipital complex (LOC),
middle occipital cortex, fusiform gyrus, and inferior temporal cortex.
Additionally, the right superior temporal gyrus (STG), precuneus, insula, and
precentral gyrus exhibited pattern separability in the right hemisphere. In the left
hemisphere, significant clusters were found in cingulate gyrus, postcentral gyrus,
precuneus, LOC, and frontal pole. We repeated the searchlight after excluding
no-response trials to see if this result could potentially be confounded by absence
of motoric activity in that trials. Although the data were slightly unbalanced, the
results were roughly the same as main analysis (Figure 4C), militating against the
possibility of motoric influence.
Then, we examined if the pattern of neural activity was more similar
between “cow” and “milk” when KL erroneously named “cow” as “milk” than
when he correctly named the object. We specifically examined the occipitotemporal region due to its robustness in differentiating correct from incorrect
trials, as discussed above. Pair-wise comparisons of Z’ scores among the three
naming responses in question yielded the following similarity results: Z’(milk for
cow vs. milk for milk) =0.36; Z’(cow for cow vs. milk for milk)=0.09; Z’(cow for
cow vs. other animals for cow)=0.28.

3.4. fMRI data (univariate analysis)
We first examined what areas were generally active during the naming
task. This revealed a large expanse of clusters throughout the remaining tissues

14

in cortical, subcortical, and cerebellar regions (Figure 5A; Table 2). Next, we
compared correct and error trials, which revealed no significant clusters for
either [correct > error] or [error > correct] contrasts. However, when we removed
the secondary cluster-size correction, a sizable cluster emerged within right STS
in the [error > correct] contrast (Figure 5B). To avoid stringent multiple
comparison correction, we repeated this analysis via the small volume correction
(SVC) using a mask image covering only occipital and parietal regions. This
analysis revealed a single cluster located in the upper portion of right
supramarginal gyrus, which was not found by the MVPA searchlight analysis
(Figure 5C, Table 2).

------------- Table 2 about here -------------

4. Discussion
In the present fMRI study, we tested the hypothesis that, in persons with
aphasia, spontaneous errors in naming are due to poor coordination among
neural populations that are engaged in compensatory language processes. That is,
good coordination (i.e., a stable, reliable, efficient representation) may allow for
consistent and accurate access to the object identity, while poor coordination (i.e.,
an unstable, unreliable, and inefficient representation) may lead to compromised
access. Crucially, we hypothesized that this neural coordination manifested itself

15

as spatially differential patterns of brain activity invoked during compensatory
language processing. Our MVPA searchlight revealed significant clusters
exhibiting such a propensity, in areas throughout residual intact cortical tissues
that are known to participate in different stages of naming processes, from visual
recognition to articulation. Notably, none of these regions were found using
standard univariate analysis, which instead yielded small non-significant clusters
in either the right STG or right SMG. Among the MVPA clusters, the right
occipito-temporal cortex—including the fusiform area and LOC—exhibited the
most distinct and robust patterns of activity pertaining to naming outcomes. Our
interpretation of these findings is that that these differences in pattern activation
could relate to our patient’s semantic impairments in naming.

Neural correlates of naming processes
The fact that naming performance is variable in chronic aphasia indicates
that newly reorganized compensatory systems are still unstable while operating
language processes. A number of past neuroimaging studies have delineated the
compensatory language network in patients with aphasia and have
demonstrated the neural correlates of naming retrieval (Turkeltaub, Messing,
Norise, & Hamilton, 2011). However, these reports are mixed with respect to
lateralization. For example, some studies have reported naming-related cortical
activity in residual perilesional areas of the left hemisphere (Fridriksson, 2010,
Léger et al., 2002, van Oers et al., 2010, Szaflarski et al., 2011), while others have

16

identified similar activity in the right hemisphere (Meinzer et al., 2006, PostmanCaucheteux et al., 2009). Some evidence suggests that the compensatory network
can shift dynamically between left and right hemispheres over the course of
aphasia recovery from the acute stage to the chronic stage (Saur et al., 2006).
Together, these seemingly conflicting reports are due to a variety of differences
between the studies (e.g., sample sizes, experimental paradigms, lesion
configurations and the severity of patient’s symptom) and thus may complement
each other to reveal a bihemispheric network of both left and right hemisphere
reorganization underlying language recovery (Turkeltaub et al., 2011).
In addition to differences in lateralization, previous reports are mixed
with respect to the relevance of increased or decreased levels of neural activity
(Hamilton, Chrysikou, & Coslett, 2011). In this investigation, we have also
compared the neural activity associated with correct and error trials using
conventional univariate analysis. Although we did not find any significant
regions at our initial statistical threshold, we found that a right superior temporal
cluster evoked increased activity during the error trials compared to correct trials
under more lenient statistical threshold. We also found a smaller cluster within
the right supramarginal gyrus yielding greater activity during error trials than
during correct trials when SVC was employed using occipito-temporal mask.
This result is consistent with a previous fMRI study reporting that error trials
yielded stronger activity than correct trials within the right hemisphere in

17

patients with chronic aphasia (Fridriksson et al., 2009, Postman-Caucheteux et al.,
2009).
Although it is possible that different loci within the compensatory
language network may possess different neural properties that were detected by
different analysis approaches, we stress that our MVPA searchlight revealed a
number of significant behaviorally-relevant clusters that evoked comparable
levels of neural activity, while univariate analysis, even with relaxed statistical
thresholds, was insensitive to detecting these clusters (Lee, Janata, Frost, Hanke,
& Granger, 2011, Lee, Turkeltaub, Granger, & Raizada, 2012). This confirms our
hypothesis that naming errors could be due to less coordinated regional neural
patterns, which are detected more readily using MVPA than a conventional
univariate approach. To the best of our knowledge, this is the first fMRI study in
which a machine-learning classification scheme was employed for relating the
on-line neural activity patterns to behavioral naming performance in aphasia
(But see Saur et al., 2010 that applied a machine-learning classification to early
fMRI data to predict prognosis of language performance 6 months after stroke).
Our approach and findings differ from previous aphasia neuroimaging
studies in important ways. First, in previous studies, it was often the case that
incorrectly named pictures tended to contain more syllables and were less
familiar than correctly named pictures. Thus, differences in neural activity
associated with naming errors could have been attributable to differences in lowlevel visual and auditory characteristics (Postman-Caucheteux et al., 2009). In the

18

present study, we sought to avoid such confounds by comparing the identical set
of stimuli. That is, these pictures were matched for the visual and phonological
characteristics, but the only differences were naming outcomes (e.g., correct and
error). Secondly, previous studies made comparisons between correct and error
trials that were not balanced; typically, there were more trials for the correct
condition than the error condition (Fridriksson et al., 2009, Postman-Caucheteux
et al., 2009). Like previous studies, we had more correct trials than error trials
overall during the two fMRI experiments, even though the same item used in the
fMRI experiments yielded approximately 50% naming accuracy during the prior
behavioral sessions. As such, we specifically made use of a subset of the data set
(‘cow’ and ‘blanket’) because these were the only data in the fMRI experiments
that were suitable for a balanced comparison. Thirdly, in the previous literature,
data were mostly collected in a single fMRI session. For machine-learning
classification, however, small data sets often yield unsatisfactory results due to
over-fitting (Pereira, Mitchell, & Botvinick, 2009). To overcome this, we ran two
separate fMRI sessions and concatenated the two data sets. We note that this was
an a priori plan devised during the stage of conceptualization of the current
study, not a post-hoc decision made after acquiring the first set of fMRI data.
Given that both behavioral and neural data were comparable between the two
fMRI sessions (80% and 81% of accuracy respectively), we were convinced that
no significant relevant neurological change had occurred over the week between
fMRI sessions. After the behavioral portion of this study (i.e., Phase 1) was

19

complete, we were initially concerned about KL’s tendency toward nonresponses
as this would have made it difficult to label those trials in our planned analysis
during the fMRI sessions (i.e., Phase 2). As such, we encouraged the patient to
name as best as he could before moving forward to the Phase 2. Impressively, KL
did not exhibit a tendency for nonresponses during the Phase 2 and made
naming attempts on nearly all trials.

Functional organization of naming processes implicated by pattern separability
Our MVPA searchlight found separable patterns of neural activity
between correct and error trials in multiple cortical loci that have been
implicated in the neuroimaging literature of overt picture naming (Fridriksson et
al., 2009, Kemeny et al., 2006, Postman-Caucheteux et al., 2009). Significant
clusters within the ventral part of the precentral and mid-portion of right
STG/STS may reflect motoric and auditory differences between correct and error
trials for the same pictures. The cluster in the anterior cingulate cortex is likely
due to error monitoring during the naming task (Carter et al., 1998). Subject KL
indeed reported that he was immediately aware of the errors that he was making
but that he could not prevent himself from making them. Notably, the most
significant and largest cluster was found in the right occipito-temporal cortex
encompassing the lateral occipital complex, fusiform cortex, and mid-occipital
gyrus. A similar result was reported by a previous fMRI study, in which
incorrect trials yielded stronger activity within this region (Fridriksson et al.,

20

2009). Nevertheless, in our data set, the standard univariate analysis did not
yield any difference between correct and error trials. As mentioned above, this
could be due to the fact that the same set of pictures were compared between
correct and error trials.
Error-type analysis revealed that the subject KL mostly made semantic
errors, with frequent confusion of “cow” as “milk” and “pillow” as “blanket.”
The “cow” picture was, at times, erroneously identified as other living objects
such as “camel” or even “turkey.” However, a systematic tendency toward
perseveration errors was not observed. We took advantage of “milk” trials that
were introduced in the second fMRI session in order to compare the neural
similarity within the right occipito-temporal cluster between trials in which KL
erroneously named “milk” instead of “cow,” versus when he accurately named
either “milk” or “cow.” Among three pair-wise comparisons, the most similar
neural patterns were elicited by instances in which the subject incorrectly named
“milk” rather than “cow” and when he correctly named “milk.” Together,
current findings from our main searchlight and similarity analyses suggest that,
for at least some persons with aphasia, semantic naming errors may be the
consequence of eliciting incorrect representations of semantically-related targets
in the right occipitotemporal cortex.
The occipito-temporal cortex is often implicated in neuroimaging studies
of object recognition (Grill-Spector & Malach, 2004) and naming (DeLeon et al.,
2007, Fridriksson et al., 2009, Kemeny et al., 2006). Evidence indicates that it is a

21

site for an intermediate stage of object recognition beyond early visual
processing. Although the LOC is frequently implicated in shape processing (Kim,
Biederman, Lescroart, & Hayworth, 2009), this region also participates in
semantic processing (Connolly et al., 2012, Fairhall & Caramazza, 2013, Kable,
Kan, Wilson, Thompson-Schill, & Chatterjee, 2005) and object naming (Large,
Aldcroft, & Vilis, 2007). The right mid-fusiform gyrus has been implicated in the
categorization of visual objects including faces (Kanwisher, McDermott, & Chun,
1997), and damage to this region results in errors in object recognition (James,
Culham, Humphrey, Milner, & Goodale, 2003, Vandenbulcke, Peeters, Fannes, &
Vandenberghe, 2006).
Intriguingly, the right inferior frontal gyrus did not emerge in the current
study despite the fact that this region has been frequently implicated as playing a
key role in compensatory language processing (Hamilton et al., 2011, PostmanCaucheteux et al., 2009). Of note, our subject had previously participated in
studies in which he received multiple sessions of the TMS on the right IFG and
had shown a substantive and persistent improvement in naming performance
(Hamilton et al., 2010). One possibility is that the neural processes in this region
might have been altered by the previous administration of therapeutic TMS.
Further investigation is warranted in order to extend the current pilot fMRI
study results by employing a larger sample of subjects who are naïve to focused
neuromodulation therapies.

22

Other considerations
There are some limitations that would need to be addressed and
improved in future neuroimaging studies examining the neural correlates of
naming processes in aphasia. First, we employed a slow-event related design for
the purpose of spacing out successive naming trials with a 12 sec ISI. For a
healthy normal participant, the HRF typically peaks at 4-6 sec. and returns to
baseline in around 12 sec. However, the HRF time-course of a chronic aphasic
patient may not correspond well with this assumption. Relatedly, a previous
neuroimaging study reported a delayed temporal profile of HRF in the stroke
patients (Bonakdarpour, Parrish, & Thompson, 2007). In future investigations,
the time-to-peak calculation should be performed prior to selecting time-points
for MVPA classification. Secondly, we were unable to acquire reaction time data
in the current experiment, as it was difficult to align the onset of stimulus
presentation with the onset of recorded voice responses in a precise manner. If
we had the opportunity to evaluate reaction time data, it could be possible to
further characterize both behavioral and neural data pertaining to naming
performance. Lastly, some of the picture items may not have been ideal
exemplars and could potentially have caused confusion (e.g., some blanket
images could have conceivably been mistaken for cushions). Although we tried
to ensure typicality of all images by testing several colleagues in the institute,
future studies should rely on a more systematic method of measuring the
typicality of each image.

23

Conclusion
Variable naming performance in patients with chronic aphasia has
inspired a body of neuroimaging research geared at characterizing the
compensatory language mechanisms that are engaged during the process of
overt naming. The present fMRI study suggests that retrieval of object naming
depends on coordination among regional neural populations within the right
occipitotemporal cortex, which is responsible for correct representation of the
object. Our findings suggest that emerging imaging analysis approaches that
employ novel pattern-recognition and machine-learning algorithms may
ultimately prove superior to conventional neuroimaging analysis for
characterizing certain brain-behavior relationships in aphasia. Thus, while
preliminary, this proof-of-concept work has significant potential implications for
future research linking subject-specific lesions to particular behavioral language
deficits (e.g., semantic or phonetic impairments). This may, in turn, further
facilitate the development and refinement of interventions that optimize
successful performance in a patient-specific manner.

5. References
Aguirre, G. K., Mattar, M. G., & Magis-Weinberg, L. (2011). de Bruijn cycles for neural
decoding. NeuroImage, 56(3), 1293–1300. doi:10.1016/j.neuroimage.2011.02.005

24

Avants, B. B., Tustison, N. J., Song, G., Cook, P. A., Klein, A., & Gee, J. C. (2011). A
reproducible evaluation of ANTs similarity metric performance in brain image
registration. NeuroImage, 54(3), 2033–2044.
doi:10.1016/j.neuroimage.2010.09.025
Bates, E., Wilson, S. M., Saygin, A. P., Dick, F., Sereno, M. I., Knight, R. T., &
Dronkers, N. F. (2003). Voxel-based lesion–symptom mapping. Nature
Neuroscience, 6(5), 448–450. doi:10.1038/nn1050
Bonakdarpour, B., Parrish, T. B., & Thompson, C. K. (2007). Hemodynamic response
function in patients with stroke-induced aphasia: Implications for fMRI data
analysis. NeuroImage, 36(2), 322–331. doi:10.1016/j.neuroimage.2007.02.035
Carter, C. S., Braver, T. S., Barch, D. M., Botvinick, M. M., Noll, D., & Cohen, J. D.
(1998). Anterior Cingulate Cortex, Error Detection, and the Online Monitoring of
Performance. Science, 280(5364), 747–749. doi:10.1126/science.280.5364.747
Connolly, A. C., Guntupalli, J. S., Gors, J., Hanke, M., Halchenko, Y. O., Wu, Y.-C.,
Abdi, H., et al. (2012). The Representation of Biological Classes in the Human
Brain. The Journal of Neuroscience, 32(8), 2608–2618.
doi:10.1523/JNEUROSCI.5547-11.2012
DeLeon, J., Gottesman, R. F., Kleinman, J. T., Newhart, M., Davis, C., Heidler-Gary, J.,
Lee, A., et al. (2007). Neural regions essential for distinct cognitive processes
underlying picture naming. Brain, 130(5), 1408–1422. doi:10.1093/brain/awm011
Eickhoff, S. B., Stephan, K. E., Mohlberg, H., Grefkes, C., Fink, G. R., Amunts, K., &
Zilles, K. (2005). A new SPM toolbox for combining probabilistic

25

cytoarchitectonic maps and functional imaging data. NeuroImage, 25(4), 1325–
1335. doi:10.1016/j.neuroimage.2004.12.034
Fairhall, S. L., & Caramazza, A. (2013). Brain Regions That Represent Amodal
Conceptual Knowledge. The Journal of Neuroscience, 33(25), 10552–10558.
doi:10.1523/JNEUROSCI.0051-13.2013
Fridriksson, J. (2010). Preservation and Modulation of Specific Left Hemisphere Regions
is Vital for Treated Recovery from Anomia in Stroke. The Journal of
Neuroscience, 30(35), 11558–11564. doi:10.1523/JNEUROSCI.2227-10.2010
Fridriksson, J., Baker, J. M., & Moser, D. (2009). Cortical mapping of naming errors in
aphasia. Human Brain Mapping, 30(8), 2487–2498. doi:10.1002/hbm.20683
Grill-Spector, K., & Malach, R. (2004). The Human Visual Cortex. Annual Review of
Neuroscience, 27(1), 649–677. doi:10.1146/annurev.neuro.27.070203.144220
Hamilton, R. H., Chrysikou, E. G., & Coslett, B. (2011). Mechanisms of aphasia recovery
after stroke and the role of noninvasive brain stimulation. Brain and Language,
118(1–2), 40–50. doi:10.1016/j.bandl.2011.02.005
Hamilton, R. H., Sanders, L., Benson, J., Faseyitan, O., Norise, C., Naeser, M., Martin, P.,
et al. (2010). Stimulating conversation: Enhancement of elicited propositional
speech in a patient with chronic non-fluent aphasia following transcranial
magnetic stimulation. Brain and Language, 113(1), 45–50.
doi:10.1016/j.bandl.2010.01.001
James, T. W., Culham, J., Humphrey, G. K., Milner, A. D., & Goodale, M. A. (2003).
Ventral occipital lesions impair object recognition but not object‐directed
grasping: an fMRI study. Brain, 126(11), 2463–2475. doi:10.1093/brain/awg248

26

Kable, J. W., Kan, I. P., Wilson, A., Thompson-Schill, S. L., & Chatterjee, A. (2005).
Conceptual Representations of Action in the Lateral Temporal Cortex. Journal of
Cognitive Neuroscience, 17(12), 1855–1870. doi:10.1162/089892905775008625
Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The Fusiform Face Area: A
Module in Human Extrastriate Cortex Specialized for Face Perception. The
Journal of Neuroscience, 17(11), 4302–4311.
Kemeny, S., Xu, J., Park, G. H., Hosey, L. A., Wettig, C. M., & Braun, A. R. (2006).
Temporal Dissociation of Early Lexical Access and Articulation Using a Delayed
Naming Task — An fMRI Study. Cerebral Cortex, 16(4), 587–595.
doi:10.1093/cercor/bhj006
Kim, J. G., Biederman, I., Lescroart, M. D., & Hayworth, K. J. (2009). Adaptation to
objects in the lateral occipital complex (LOC): Shape or semantics? Vision
Research, 49(18), 2297–2305. doi:10.1016/j.visres.2009.06.020
Large, M.-E., Aldcroft, A., & Vilis, T. (2007). Task-related laterality effects in the lateral
occipital complex. Brain Research, 1128, 130–138.
doi:10.1016/j.brainres.2006.10.023
Lee, Y.-S., Janata, P., Frost, C., Hanke, M., & Granger, R. (2011). Investigation of
melodic contour processing in the brain using multivariate pattern-based fMRI.
NeuroImage, 57(1), 293–300. doi:10.1016/j.neuroimage.2011.02.006
Lee, Y.-S., Turkeltaub, P., Granger, R., & Raizada, R. D. S. (2012). Categorical Speech
Processing in Broca’s Area: An fMRI Study Using Multivariate Pattern-Based
Analysis. The Journal of Neuroscience, 32(11), 3942–3948.
doi:10.1523/JNEUROSCI.3814-11.2012

27

Léger, A., Démonet, J.-F., Ruff, S., Aithamon, B., Touyeras, B., Puel, M., Boulanouar, K.,
et al. (2002). Neural Substrates of Spoken Language Rehabilitation in an Aphasic
Patient: An fMRI Study. NeuroImage, 17(1), 174–183.
doi:10.1006/nimg.2002.1238
Mahmoudi, A., Takerkart, S., Regragui, F., Boussaoud, D., & Brovelli, A. (2012).
Multivoxel Pattern Analysis for fMRI Data: A Review. Computational and
Mathematical Methods in Medicine, 2012, e961257. doi:10.1155/2012/961257
Meinzer, M., Beeson, P. M., Cappa, S., Crinion, J., Kiran, S., Saur, D., Parrish, T., et al.
(2013). Neuroimaging in aphasia treatment research: Consensus and practical
guidelines for data analysis. NeuroImage, 73, 215–224.
doi:10.1016/j.neuroimage.2012.02.058
Meinzer, M., Flaisch, T., Obleser, J., Assadollahi, R., Djundja, D., Barthel, G., &
Rockstroh, B. (2006). Brain regions essential for improved lexical access in an
aged aphasic patient: a case report. BMC Neurology, 6(1), 28. doi:10.1186/14712377-6-28
van Oers, C. A. M. M., Vink, M., van Zandvoort, M. J. E., van der Worp, H. B., de Haan,
E. H. F., Kappelle, L. J., Ramsey, N. F., et al. (2010). Contribution of the left and
right inferior frontal gyrus in recovery from aphasia. A functional MRI study in
stroke patients with preserved hemodynamic responsiveness. NeuroImage, 49(1),
885–893. doi:10.1016/j.neuroimage.2009.08.057
Pereira, F., Mitchell, T., & Botvinick, M. (2009). Machine learning classifiers and fMRI:
A tutorial overview. NeuroImage, Mathematics in Brain Imaging, 45(1,
Supplement 1), S199–S209. doi:10.1016/j.neuroimage.2008.11.007

28

Postman-Caucheteux, W. A., Birn, R. M., Pursley, R. H., Butman, J. A., Solomon, J. M.,
Picchioni, D., McArdle, J., et al. (2009). Single-trial fMRI Shows Contralesional
Activity Linked to Overt Naming Errors in Chronic Aphasic Patients. Journal of
Cognitive Neuroscience, 22(6), 1299–1318. doi:10.1162/jocn.2009.21261
Raizada, R. D. S., & Lee, Y.-S. (2013). Smoothness without Smoothing: Why Gaussian
Naive Bayes Is Not Naive for Multi-Subject Searchlight Studies. PLoS ONE, 8(7),
e69566. doi:10.1371/journal.pone.0069566
Saur, D., & Hartwigsen, G. (2012). Neurobiology of Language Recovery After Stroke:
Lessons From Neuroimaging Studies. Archives of Physical Medicine and
Rehabilitation, Rehabilitation of Neurological Language Disorders, 93(1,
Supplement), S15–S25. doi:10.1016/j.apmr.2011.03.036
Saur, D., Lange, R., Baumgaertner, A., Schraknepper, V., Willmes, K., Rijntjes, M., &
Weiller, C. (2006). Dynamics of language reorganization after stroke. Brain: A
Journal of Neurology, 129(Pt 6), 1371–1384. doi:10.1093/brain/awl090
Schwartz, M. F., Kimberg, D. Y., Walker, G. M., Brecher, A., Faseyitan, O. K., Dell, G.
S., Mirman, D., et al. (2011). Neuroanatomical dissociation for taxonomic and
thematic knowledge in the human brain. Proceedings of the National Academy of
Sciences, 108(20), 8520–8524. doi:10.1073/pnas.1014935108
Szaflarski, J. P., Eaton, K., Ball, A. L., Banks, C., Vannest, J., Allendorfer, J. B., Page, S.,
et al. (2011). Poststroke Aphasia Recovery Assessed With Functional Magnetic
Resonance Imaging and a Picture Identification Task. Journal of Stroke and
Cerebrovascular Diseases, 20(4), 336–345.
doi:10.1016/j.jstrokecerebrovasdis.2010.02.003

29

Thompson, C. K., & Ouden, D.-B. den. (2008). Neuroimaging and recovery of language
in aphasia. Current Neurology and Neuroscience Reports, 8(6), 475–483.
doi:10.1007/s11910-008-0076-0
Turkeltaub, P. E., Messing, S., Norise, C., & Hamilton, R. H. (2011). Are networks for
residual language function and recovery consistent across aphasic patients?
Neurology, 76(20), 1726–1734. doi:10.1212/WNL.0b013e31821a44c1
Van Essen, D. C., Ugurbil, K., Auerbach, E., Barch, D., Behrens, T. E. J., Bucholz, R.,
Chang, A., et al. (2012). The Human Connectome Project: A data acquisition
perspective. NeuroImage, ConnectivityConnectivity, 62(4), 2222–2231.
doi:10.1016/j.neuroimage.2012.02.018
Vandenbulcke, M., Peeters, R., Fannes, K., & Vandenberghe, R. (2006). Knowledge of
visual attributes in the right hemisphere. Nature Neuroscience, 9(7), 964–970.
doi:10.1038/nn1721
Walker, G. M., & Schwartz, M. F. (2012). Short-form Philadelphia naming test: rationale
and empirical evaluation. American Journal of Speech-Language Pathology /
American Speech-Language-Hearing Association, 21(2), S140-153.
doi:10.1044/1058-0360(2012/11-0089)

Figure Legends
Figure 1 A. Contrast of the rationale for conventional univariate analysis versus
MVPA in linking neural activity to behavior. B. Some of the candidate pictures
chosen from an existing PNT test-retest data set. These were presented during
Phase 1 of behavioral sessions. C. Anatomical lesion profile of patient KL. An

30

expansive lesion is shown in the left hemisphere encroaching the frontal, parietal,
and temporal lobes.

Figure 2 A. Normalization output of KL’s anatomical and functional images to
the standard MNI space. B. Head motion profiles of the patient KL during the
two fMRI sessions. The trial numbers across six runs are shown in the x-axis, and
the degree of displacement is shown in the y-axis.

Figure 3 A. Seven objects that were chosen during the Phase 1 of behavioral
naming sessions in the mock scanner. Average accuracy is shown under each
picture. B. The Naming performance on the seven items during the 1st fMRI
session. C. The Naming performance on the seven items during the 2nd fMRI
sessions. D. Error types and percentages are shown in the bar graph for both
Phase 1 (behavioral) and Phase 2 (fMRI), indicating KL’s semantic deficit.

Figure 4 A. Multi-slice view comparing searchlight map on 1st and 2nd fMRI data
set. B. Significant clusters are overlaid in the surface rendering of a normal brain
using workbench (Van Essen et al., 2012). C. Cross-section view depicting the
searchlight result when no-response trials are included in the error condition vs.
when they are excluded.

31

Figure 5 A. Multi-slice view depicting cortical areas that are activated by picture
stimuli. B. The right STS cluster (a total of 75 voxels) exhibits stronger activity
during error trials than correct trials when cluster-size correction is removed. C.
A cluster is found at the right supramarginal gyrus when [error > correct]
comparison is performed via small volume correction (SVC).

32

33

34

35

Table 1. Cortical regions exhibiting differential activity patterns between correct and
incorrect trials
MNI
Coordinates

36

#
voxels

Region name

x

y

z

accuracy

Right Lateral Occipito-temporal Cortex
Occipital Fusiform Gyrus
Intracalcarine Cortex
Lingual Gyrus
Inferior Temporal Gyrus
Supracalcarine Cortex
Lateral Occipital Cortex
Left Subcallosal Cortex
Cingulate Gyrus
Left Postcentral Gyrus
Left Lateral Occipital Cortex
Precuneous Cortex
Left Frontal Pole
Left Lateral Occipital Cortex
Precuneous Cortex
Right Superior Temporal Gyrus
Planum Temporale
Right Superior Parietal Lobule
Right Precuneous Cortex
Left Lateral Occipital Cortex
Right Parietal Operculum Cortex
Insular Cortex
Right Precentral Gyrus

44
31
18
33
41
1
48
-3
-3
-18
-27
-24
-9
-39
0
47
48
27
15
-42
31
33
40

-76
-70
-83
-59
-59
-82
-74
27
37
-44
-65
-58
64
-71
-53
-28
-35
-41
-62
-80
-30
-23
-9

-1
-2
8
1
-3
5
8
-1
-3
50
25
23
11
36
34
-3
15
41
31
-3
22
16
31

0.65
0.64
0.64
0.63
0.63
0.63
0.58
0.63
0.61
0.63
0.63
0.61
0.62
0.62
0.62
0.61
0.61
0.61
0.61
0.60
0.60
0.60
0.60

205

t-stat.

z-stat.

#
voxels

7967

12
11
15
12
26
23
16
13
21
12
15
12

Table 2. Cortical regions identified by the conventional univariate analysis
MNI
Coordinates
Region name

x

y

z

Hemi

55

-3

L

11.31 >7.84

19

18

L

10.84 >7.84

37
-

11
-

L

9.98 >7.84
10.61 >7.84

All > Base

Superior frontal gyrus
Inferior frontal gyrus
Inferior frontal gyrus
Inferior temporal gyrus

15
57
54
45

37

R

289

38

Globus Pallidus

18
21

Cerebellum

21

Cerebellum

9

Lingual gyrus

18

Orbitofrontal gyrus

13
-5
59
65
74

14
17
-3
20
14
10

R

8.55 >7.84
7.55 7.48

R

5.89

5.86

R

5.49

5.46

R

5.46

5.43

0

L

4.64

4.62

4

L

4.5

4.48

18

R

3.4

3.4

R

205

Error >Correct (no cluster corr)
Superior temporal gyrus/sulcus

60

Superior temporal gyrus/sulcus

60

26
20

57

44

75

Error > Correct (SVC)
Supramarginal gyrus

2

Highlights
• MVPA searchlights revealed regional patterns of neural activity associated
with compensatory naming processing in aphasia, which was not detected by
conventional univariate analyses.
• The largest and most robust cluster was found in right occipitotemporal cortex,
which is mirrored by predominant semantic error rates.
• This is one of the first attempts of employing machine-learning techniques to
predict behavioral naming performance using on-line fMRI data in aphasia
studies.

38