You are in: Home » History of the Unit
4. LANGUAGE AND COMMUNICATION PROGRESS REPORT
Overview
The ability to use and understand language is one of the fundamental human capacities. The scientific understanding of language function requires an interdisciplinary enterprise that combines psychological, linguistic, neuropsychological, neurobiological, and computational inputs. The aim of the Language and Communication programme is to study human language in these terms, as a cognitive, computational, and neural system. Within this global programme, we identified four major research themes. These are summarised below, and presented in more detail in the main body of the report.
More generally, the period under review has been one of development, recruitment, and bedding-down. In particular, we have successfully instigated a shift of emphasis towards understanding the neural basis of language processing. The appointment of a senior EEG and MEG specialist (Pulvermüller) and an fMRI specialist (Johnsrude) in the year 2000, has resulted in a vigorous and successful neuro-imaging programme, which, despite only being fully operational for the last 18 months, has already begun to yield valuable new results. These more recent developments have interacted strongly with the behavioural aspects of the programme (led by Marslen-Wilson and by Norris) which have been running for the full period under review. These cross-programme interactions are a major source for the proposals we will make for next quinquennium.
Project L1: Phonological and lexical processing of speech
The focus of this project is on the key early processes in understanding spoken language. We investigate the nature of the transformation undergone by the speech input as it is mapped from early auditory analysis through to representations of phonological form and subsequent access to the mental lexicon. These questions are explored in an interdisciplinary framework, following three main strands.
• A major subcomponent of the project uses fMRI neuro-imaging techniques to probe the brain systems involved in speech analysis. This work looks both at general properties of auditory analysis at the cortical level, and at those systems specifically engaged by speech. Important collaborative work with other groups in Cambridge (Roy Patterson's group in Physiology) and with the MRC IHR in Nottingham and the FIL in London have begun to map out the very specific functional differentiation of cortical areas in and around primary auditory cortex.
• A second component takes a more cognitive and computational approach, examining the role of phonological and lexical factors in the processes of speech segmentation – how the stream of speech is segmented into phonemes, syllables and words. There are two key findings here – the identification of the "Possible Word Constraint", a new language universal principle of speech segmentation, and the demonstration, for the first time, of clear-cut effects of acoustic-phonetic cues in disambiguating onset-embedded words (e.g. cap in captain) that were previously thought to present intractable problems for several classes of word-recognition models.
• The third component continues the development of two leading theories of spoken word-recognition. The Cohort model, long associated with Marslen-Wilson, has evolved into a distributed, connectionist format, with a novel processing architecture which makes distinctive predictions about the role of semantic factors in the word-recognition process. The Shortlist and Merge models, developed by Norris and colleagues, presents new research and theoretical developments clarifying the role of feedback in word recognition and in perceptual adaptation to speech distortions. In a complementary development, we have initiated pioneering neuro-imaging research, using variations in speech intelligibility to map out brain regions involved in mapping from sound to meaning, and potential interactions between them.
Project L2: The structure of lexical representation
The second main project focuses on words, in the mind and in the brain. The form and meaning of words play a critical role in language comprehension and production. We examine here the properties of the mental lexicon – the mental representations of knowledge of words – from cross-linguistic and neuro-psychological perspectives.
• A scientific account of the mental lexicon requires comparisons across languages. Over the last quinquennium we have been able to develop an unique set of systematic empirical comparisons, adding two new language families (represented by Polish and Arabic) to previous studies of Chinese and of French and Italian. This not only reveals a surprising degree of divergence in organising principles across languages, but also documents new morphological phenomena – for example, the highly abstract prosodic morphemes that dominate word formation in Arabic.
• The second major project here takes forward the highly visible debate on the significance of regular and irregular morphology ("words" vs "rules") in the English past tense. In influential work, carried out in collaboration with Tyler (Cambridge), we have amassed considerable new evidence for a neuropsychological double dissociation between the processing of regular and irregular past tense forms in aphasic patients. Further work using behavioural and neuro-imaging (EEG, fMRI) techniques with intact adults, points to a re-interpretation of the regular/irregular contrast in English as primarily morpho-phonological in nature – a conclusion which is supported by further cross-linguistic studies of regularity and irregularity in French and Polish.
Project L3/M4: Short-term memory and speech perception
This third project, joint with the Memory Group, seeks to unite the study of short-term memory (STM) and the study of language comprehension. Speech recognition and comprehension are reliant on temporary storage, and this requires an understanding of the systems involved in the representation and storage of ordered information in STM. Our emphasis here was on computational modelling, supported by empirical work to place additional constraints on theoretical development.
• The key achievement here was the development of a new model of the phonological store component of working memory. This model (the Primacy model) gives a complete and precise account of the fundamental behavioural characteristics of serial recall from STM. An extensive presentation of this model was published in the leading journal in the field.
• The second important aspect of the project was the acquisition of new data that allowed the rejection of popular alternative explanations of STM based on associative chaining or on position-item correspondences.
Project L4: Neurophysiology of language
A priority for this quinquennium was to add the temporal precision provided by EEG-based neuro-imaging to the spatial precision provided by fMRI and PET. By the end of 2000, a new EEG laboratory was built and equipped, and a senior EEG specialist (Pulvermüller) and his team were installed at the CBU. Pulvermüller also brought with him a fruitful collaboration with Ilmoniemi's MEG lab in Helsinki, giving us access to this important new technology. Working within a theoretical framework based on Hebbian concepts of distributed, functionally connected ensembles of neurons, the group is making significant advances in several directions. We highlight two of these here.
• The existence of category-specific brain processes has been a salient issue in recent cognitive neuroscience, and is predicted by Hebbian correlation learning. A series of EEG studies have shown neurophysiological differences between word categories, with, for example, action-words related to different motor areas (e.g. hand vs mouth, as in pick vs lick), showing differential cortical activation in the predicted frontal motor areas. Neuropsychological lesion studies have provided confirmatory results, and preliminary data from MEG and TMS are also encouraging.
• Mismatch Negativity (MMN), a neurophysiological response that reflects automatic cortical processes, has been found in recent EEG and MEG studies to be sensitive to higher-order linguistically related processes. Words elicit a stronger MMN than meaningless pseudowords, and the stems of content words and inflectional affixes elicit MMN's with different latencies and topographies. Striking recent results suggest that MMN responses directly reflect the timing of cortical events linked to word recognition processes at different sites in perisylvian cortex.
Project L1: Phonological and lexical processing of speech
Scientific direction: Norris (50%), Marslen-Wilson (20%), Johnsrude (95%), Gaskell (100%)
MRC-scientists: Davis (80%)
MRC research support: Butterfield (100%), Woods (50%), van Casteren (25%)
The focus of this project is on how the phonetic and phonological properties of language are represented and accessed in the adult system. What is the nature of the transformation undergone by the speech input as it is mapped from early auditory analysis through to lexical representations of phonological form and the generation of phonological perceptual experience? This is explored in an interdisciplinary framework, following three main strands.
Subproject L1.1 employs neuro-imaging techniques in a series of experiments that probe the brain systems involved in speech analysis. This work looks both at general properties of auditory analysis at the cortical level, and at those systems specifically engaged by speech. This was collaborative work with Hall (MRC IHR, Nottingham), Griffiths (FIL, London) and with Patterson and Uppenkamp of the Centre for the Neural Basis of Hearing (CNBH) in the University of Cambridge Department of Physiology. Subproject L1.2 takes a more cognitive and computational approach, examining the role of phonological and lexical factors in the processes of speech segmentation – how the stream of speech is segmented into phonemes and into higher-order syllabic and morphemic units. The third subproject, L1.3, develops contrasting approaches to the functional architecture of the word recognition system, contrasting current versions of the Cohort model (Gaskell, Marslen-Wilson) with the Shortlist and Merge models (Norris). This also includes recent neuro-imaging research, using variations in speech intelligibility to map out brain regions involved in mapping from sound to meaning, and potential interactions between them (Johnsrude, Davis).
L1.1. Imaging auditory cortex: from sound to phonology
The anatomical organization of the auditory system and its onward connections provides a framework within which to explore the functional organization of speech perception. The anatomy suggests several parallel streams of processing extending hierarchically outwards from primary auditory cortical core (on Heschl's gyrus), through belt and parabelt nonprimary auditory areas. Although the anatomy of these areas in humans is still uncertain, they can be expected to be hierarchically organized and functionally differentiable, as they are in macaques (see Rauschecker, 1998; Kaas & Hackett, 2000, for reviews). In preliminary, collaborative examination of the functional properties of these systems, projects L1.1.1 – 3 investigate the general analytic capacities of the auditory system such as pitch extraction, temporal processing, sound localization and auditory grouping. Project L1.1.4 focuses on the transition from the processing of sound to the processing of speech.
L.1.1.1. Pitch based on temporal cues
Pitch extraction is critical for the tracking of prosodic contour (fundamental frequency) in fluent speech. A potential complication in searching for the neural basis of pitch perception is that two separate pitch mechanisms may be involved. The periodic portions of speech, such as vowels, can be described as spectrally shaped harmonic complexes. The lower harmonics of such sounds are resolved by the peripheral auditory system, and changes in their fundamental frequency ("F0", corresponding to the perceived pitch) could be encoded either by changes in the temporal response ("phase locking") of auditory nerve fibres to individual harmonics, and/or by changes in the place of excitation along the basilar membrane. It is also possible to produce a tone with a strong pitch by regularizing the time intervals in a broadband noise, so that one time interval (a few ms) occurs more often than any of the others. When high-pass filtered at frequencies of 500 Hz or above, these regular-interval (RI) sounds (Patterson, 1994) effectively excite all frequency channels in the same way as random noise, with no resolved spectral peaks. The perception of pitch elicited by these stimuli must therefore rely on extraction of regular time intervals of a few ms, rather than extraction of any prominent spectral features, enabling one to test temporal models of pitch perception (eg, Patterson et al., 1994). Together with colleagues from the CNBH and the FIL, Johnsrude has investigated this issue in a series of fMRI studies (Griffiths, Uppenkamp, Johnsrude, Josephs & Patterson, 2001; Patterson, Uppenkamp, Johnsrude & Griffiths, in press). Listeners were presented with random noise bursts, a series of RI sounds with a fixed pitch, and a series of RI sounds with a pitch excursion between successive items. Sounds with a pitch generated greater activation than did the random noise within a confined region on lateral Heschl's gyrus (HG), in a probable auditory belt area. Relative to the fixed-pitch stimuli, the melodic pitch sequence generated a greater response at the lateral-most extremity of HG and on the convexity of the superior temporal gyrus (STG) in the right hemisphere; see Figure L.1. These results confirm that non-primary auditory cortex is important for the analysis of spectrotemporal patterns in sound. Furthermore, the centre of activity shifts anterolaterally into higher-order areas as information must be integrated over time.
Figure L.1 : fMRI response to RI sounds in auditory regions: The top panels show activation on the supratemporal plane: activation data is superimposed on a mean structural image from 9 subjects. The white area in the central panels denotes the average location of Heschl's gyrus as identified using morphological criteria in these subjects. Noise, compared to silence, activates a large auditory area bilaterally (shown in blue). Extraction of pitch based on temporal cues activates an area at the lateral edge of Heschl's gyrus in both hemispheres (red): this is probably an auditory belt region. Finally, introduction of a pitch difference between successive tones produces right-dominant activation, at the lateral extremity of Heschl's gyrus, extending anteriorly and posteriorly along the convexity of the superior temporal gyrus (green/aqua). (See Patterson et al., in press)
.jpg)
L1.1.2. Stimulus complexity
Whereas single-frequency tones may activate focal areas within an auditory field, broadband sounds should activate larger neuronal populations, particularly within the non-primary auditory areas that correspond to belt cortex (Rauschecker & Tian, 1995). Using fMRI, we have reported (Gonçalves, Hall, Johnsrude & Haggard, 2001; Hall, Johnsrude, Haggard, Akeroyd, & Summerfield, 2002) activation in putatative core auditory regions to simple tones, with greater activation in non-primary auditory areas (including lateral superior temporal planum bilaterally) when activation to simple tones was subtracted from that to a harmonic-complex tone of the same pitch. Activation was also greater to frequency-modulated tones than to steady-state tones in these areas. Giraud, Lorenzi, Ashburner, Wable, Johnsrude, Frackowiak, & Kleinschmidt (2000) also reported a significant effect of amplitude modulation in bilateral primary and non-primary areas, strongest in the posterolateral regions of the superior temporal gyrus. These results, suggesting that greater stimulus complexity leads to greater responding in nonprimary areas, is consistent with a hierarchical model of auditory processing.
L1.1.3. Localization of activation foci relative to auditory cortex
An important premise of functional imaging is that activation will follow follow anatomy (Brett, Johnsrude, & Owen, 2002): anatomically homologous areas in the two hemispheres, which are presumably similar in function (such as primary auditory cortex) should show similar activation patterns. Given that auditory stimuli generally yield strong, bilateral auditory cortical activations, this assumption can be tested directly, by examining whether the right-left asymmetry evident in the anatomy of primary auditory cortex (PAC) (see Penhune, Zatorre, MacDonald & Evans, 1996; Rademacher, Morosan, Schormann, Schleicher, Werner, Freund, & Zilles, 2001) is also evident in functional data. Under conditions in which primary auditory cortex is assumed to be activated (in comparing sound conditions to silent baseline conditions, for example), one would expect to observe peak activation more anteriorly in the right auditory region than in the left, if activation does respect the anatomical asymmetry. We compared, across 28 published positron emission tomography (PET) studies of audition, activation foci in the right and left auditory region to the centroids (centres of mass) of auditory cortex in the two hemispheres, using either the probability-weighted centroids of Heschl's gyrus; the location of PAC (Penhune et al., 1996) or the probability-weighted centroids of PAC itself (Rademacher et al., 2001). We found that activation foci were consistently farther away from PAC, falling in nonprimary cortex, in the right hemisphere compared to the left. Moreover, across a wide range of stimulus types, activation was consistently more intense in the right hemisphere compared to the left (Johnsrude, Giraud, & Frackowiak, 2002). These findings are hard to reconcile with conventional neuroimaging assumptions, and may be due to vascular artefact and partial volume effects resulting from spatial smoothing. Whatever their origin, the results suggest that the spatial resolution of PET is insufficient to explore the functional organization of auditory core, belt and parabelt regions. FMRI, in part because of its ability to reveal effects within subjects, may be a better way to study auditory functional organization (Owen, Epstein, & Johnsrude, 2002).
In a recent fMRI study (Patterson et al., in press), we compared auditory activation in a group to activation patterns within individual subjects, to determine whether there were consistent differences between individual listeners in the location of functional activation within auditory cortex, and whether the differences corresponded to differences in the sulcal and gyral morphology of individuals (Penhune et al., 1996). After spatial normalization, variability in gross anatomical landmarks (sulci and gyri) from subject to subject was not large in the auditory region. Furthermore, within subjects, the location of activation foci for a particular stimulus condition was highly consistent. However the location of peak activation for a particular stimulus condition varied substantially across listeners; much more than the anatomy varied. It is established that anatomically specialized brain areas, identifiable on the basis of their microanatomical structure, do not always respect gross anatomical features such as sulci and gyri (Rademacher et al., 2001; see Brett et al, 2002 for a discussion). Our functional results are consistent with this.
To conform to the assumptions of the statistical model used in conventional image analysis, data are spatially smoothed, which confounds extent of activation with intensity of activation. Intense but focal changes in signal are 'smeared out' over adjacent volume elements so that it is difficult to draw conclusions about anatomical specificity. To do so one needs to define anatomical regions of interest, and then explore signal change in unsmoothed data within these regions. This is difficult to do in practice, since brain microanatomy does not respect the gross morphological landmarks observable on structural MR scans (see Brett et al., 2002) for discussion. A promising way to circumvent this problem employs probability maps of cytoarchitectonically defined primary auditory cortex. We have run two preliminary studies exploring this technique (Johnsrude, Cusack, Morosan, Hall, Brett, Zilles, & Frackowiak, 2001; (Johnsrude, Morosan, Hall, Cusack, Brett, Zilles, & Frackowiak, 2001; Johnsrude, Morosan, Cusack, Brett, Ashburner, Zilles, & Frackowiak, 2000), with good results. Further development is planned in section MR2.6 (Methods).
L1.1.4. Separating speech from non-speech
It is commonly assumed that, in the cochlea and the brainstem, the auditory system processes speech sounds without differentiating them from any other sounds. At some stage however, it must treat speech and non-speech sounds differently. In collaboration with Roy Patterson and Stefan Uppenkamp from the CNBH (Cambridge) we used functional MRI to delimit the location of this stage in the auditory pathway by identifying the point where the sound is found to match a specific phonological category. We had previously defined and evaluated a set of synthesised vowel and non-vowel sounds, that were matched for acoustical features but which differ markedly in their similarity to speech (Uppenkamp, Patterson, Johnsrude, Norris, & Marslen-Wilson, 2001). Four of these sounds were used in an fMRI experiment: speech-like with pitch or no pitch, and non-speech-like with pitch or no pitch. Two additional conditions were included as control; these were silence and natural vowels. Sparse imaging with a scan repeat time of 10 sec was used to separate scanner noise and acoustic stimuli in time. When contrasted with silence, all of the sound conditions show very similar activation patterns, centred around Heschl's gyrus bilaterally, with the most prominent peak towards the lateral end of HG. Natural and synthesised vowels, either with or without pitch, did not differ in terms of their activation patterns. There was also no significant activation specific to the (fixed) pitch in either speech or non-speech conditions, indicating that phonological category was a more prominent feature than pitch. There was, however, significantly more activation in the speech-like conditions than in the non-speech conditions, in an area below and posterior to Heschl's gyrus in planum temporale. Whereas this activation is bilateral and symmetric in the group data, single-subject analysis revealed that some listeners show more activation on the right while others show more on the left. This study forms the basis for a more extensive programme laid out in section SL2.
L1.2. Speech segmentation and phonological analysis
The speech signal is continuous, noisy, and variable. It remains a considerable scientific and practical challenge to determine how the listener imposes order on this complex stream of information, and in particular, segments it into appropriate access units at different levels of description. The research summarised here represents three complementary approaches to the use of acoustic-phonetic, phonological, and prosodic cues to lexical segmentation – that is, to the problem of determining the beginnings and ends of words in the speech stream.
L1.2.1. Prosodic cues to lexical segmentation: Bottom-up disambiguation of word-onsets
The lack of acoustic markers of word boundaries in connected speech may create temporary ambiguities between words like cap and the start of longer words like captain. These apparent onset ambiguities have motivated models of spoken word recognition in which lexical competition allows information after the end of an embedded word to assist in identification – typically through competition between non-aligned lexical hypotheses. However, evidence from acoustic-phonetics suggests consistent differences between the syllables in monosyllabic and bisyllabic words. For example, Lehiste (1972) reports significant shortening of the syllable [slip] in words like sleepy and sleepiness. In a series of experiments Davis and colleagues investigated the degree of actual processing ambiguity between embedded words and longer competitors in spoken sentences (Davis, Marslen-Wilson, & Gaskell, 2002). Stimulus sentences for these experiments contained monosyllabic words or frequency-matched bisyllables that contained the monosyllable as the initial syllable (such as the pair cap and captain), placed in a spoken sentence frame ("The soldier saluted the flag with his cap tucked…/with his captain looking…"). An initial gating study showed that duration differences (and possibly other prosodic cues) were able to bias, as appropriate, towards either monosyllabic or bisyllabic interpretations before the end of the syllable. A subsequent series of cross-modal repetition-priming experiments examined in detail the activation of onset-embedded words and longer competitors as the critical syllable and its following context was heard. These showed immediate effects on the relative level of activation of lexical hypotheses according to their compatibility with bottom-up prosodic constraints. This is a significant result which not only underlines the role of fine-grained acoustic detail in the perception and segmentation of connected speech but also demonstrates that onset-embedding may be a less serious computational problem than previous theorists have suggested.
Preliminary simulations illustrate how a recurrent neural network could model the processing of strings containing onset-embeddings when provided with appropriately structured input (Davis, Marslen-Wilson & Gaskell, 2000). Further support for this approach to lexical segmentation and identification comes from simulations that successfully model the relationship between phonological and lexical learning during the development of spoken language comprehension in infants between six and 18 months in age (Davis, in press). The network uses a single learning mechanism and processing resources (hidden units) to learn two different tasks: to predict the upcoming input, and to identify words in the input (requiring lexical segmentation). Rather than competing for processing resources, the combined demands of the two tasks interact to assist learning in the network, leading to more rapid vocabulary acquisition than in networks with the two tasks being assigned separate hidden units. Furthermore, the network shows the expected developmental profile by learning phonological properties of the input before lexical learning, consistent with empirical evidence (Jusczyk, 1999), and by showing gains in the speed of lexical identification during the second year (cf. Fernald, Pinto, Swingley, Weinberg, & McRoberts, 1998).
L1.2.2. The Possible Word Constraint: a new cue to lexical segmentation
This project has concentrated on how listeners make use of cues provided by metrical or phonotactic information, and how these cues interact with the process of lexical competition. It has its origins in earlier work demonstrating that listeners can make use of the onsets of stressed syllables to determine the likely location of word boundaries (Cutler and Norris, 1988). Later work by Norris, McQueen, and Cutler (1995) demonstrated that these effects could be modulated by lexical competition. Research over the last period has extended these findings, and has led to the development of a new model of lexical segmentation.
Norris, McQueen, Cutler, and Butterfield, (1997) presented evidence that listeners make use of what has been labelled the Possible Word Constraint (PWC). Listeners found it easier to detect words in nonsense strings, when extracting the word would leave a residue that could possibly be a real word (e.g. 'see' in 'seeshub', where 'shub' is a phonotactically possible word in English), than when the residue could not be a real word (e.g. 'see' in 'seesh', where 'sh' could not be a word, as it does not contain a vowel). This was modelled by a development of Shortlist (Norris, 1994), in which lexical candidates were penalised if there was no vowel between the end of the candidate and the nearest 'known' boundary in the input, where a 'known' boundary was silence, a phonotactic boundary, or the onset of a strong syllable. This modification also simulated earlier data (McQueen, Norris and Cutler, 1994) on the interaction between lexical competition effects and metrical segmentation (the MSS of Cutler and Norris, 1988) which had previously been simulated by a more complex mechanism. Further work has shown that the PWC is driven by solely by whether or not the 'residue' contains a vowel, and that the identity of the vowel is unimportant. Norris, Cutler, McQueen, Butterfield, and Kearns (2001) showed that the PWC operated even when the vowel in the residue is a schwa, or a lax vowel. Syllables with schwa or lax vowels cannot be real words in English. This suggests that the PWC is a language-universal strategy, and is guided by consideration of what might be a word in any language, not by whether the residue could be a word in the listener's own language. A strategy like the PWC should be of great value in acquiring language as well as perceiving language as it would provide the infant with valuable information as to the likely location of word boundaries. In accord with this, Johnson, Jusczyk, Cutler and Norris (in press) have found evidence that 12-month-old babies behave in accord with the predictions of the PWC.
L1.2.3. Phonological variation in lexical and sentential context
Regular phonological variation – for example, assimilation of place of articulation – is an important aspect of fluent speech production and recognition, and can be used on-line by the listener as a cue to segmentation (Gaskell & Marslen-Wilson, 1996, 1998). We extended this research to look at the case where phonological variation could generate a natural case of lexical ambiguity involving two normally unambiguous words. For example, in running speech, "a quick run picks you up" can become confusable with "a quick rum picks you up". This is because the coronal place of articulation of the final consonant of "run" assimilates to the labial place of articulation of the initial consonant of "pick", yielding a surface form which is closer to "rum" than to "run". We examined the perception of this kind of change in Gaskell & Marslen-Wilson (2001a; see also Gaskell & Marslen-Wilson, 1999b; Gaskell, 2000a, 2000b). The results supported previous research in demonstrating the importance of evaluating phonological alternations in their phonological context, but also demonstrated a strong effect of sentential context on resolution of this kind of ambiguity. This provides a bridging link between models of spoken word recognition and lexical ambiguity resolution.
A second strand of research in this area examined the effects of resyllabification and phonological liaison in the perception of French. The dominant model of speech perception for French emphasises the role of the syllable as a segmentation and access unit. When vowel initial words are embedded in fluent speech the syllable model predicts that natural resyllabification processes such as liaison should impede recognition of these words. Our research demonstrated the opposite effect (Gaskell, Spinelli & Meunier, in press; Spinelli, Gaskell & Meunier, 2000), and showed that, as in English, sensitivity to the phonological context of regular variations in speech is a crucial component of the recognition system. The results suggest that the role of the syllable in models of speech processing for Romance languages should be refined.
L1.3 Architecture of spoken word recognition
A topic that continues to be of central importance to research in spoken word recognition is the overall architecture and structure of the recognition system. For example, can speech recognition be considered to be a series of autonomous and modular processes, or should it be characterised as a highly interactive system? Naturally, this question cannot be answered without also considering the nature of the processes and representations involved in spoken word recognition. The research reported under L1.3.1 and L1.3.2 addressed both the broader architectural issues and more specific questions about the nature of the representations of phonological form. Project L1.3.3 reports promising new research using fMRI techniques to probe the organisation of neural systems involved in speech comprehension.
L1.3.1. Shortlist and Merge: Issues in lexical architecture
A particular issue in the architecture of the speech recognition system is whether pre-lexical (phonemic or phonetic) processing can be influenced by top-down feedback from the lexicon. The dominant view in the literature has been that there must be feedback from the lexicon to prelexical processes. This is a superficially plausible position because there is extensive evidence that listeners' ability to make phonological decisions is influenced by lexical information. Furthermore, many researchers believe that lexical feedback is to be expected because it will help recognition. However, the fact that phonological decisions can be influenced by lexical information does not necessarily imply that there is feedback. The view that feedback will help recognition of speech sounds has been based largely on intuition rather than on any formal analysis of the problem.
The most significant product of this work, which continues a long-standing collaboration with Cutler and McQueen (Max-Planck Institute, Nijmegen) has been an influential Behavioral and Brain Sciences paper (Norris, McQueen, and Cutler, 2000a, 2000b), which argued that speech recognition is a feed-forward process in which pre-lexical processing is not subject to any feedback from lexical processes. The paper also presented a new computational model (Merge) of the process by which listeners perform phoneme identification judgements in speech recognition tasks. In an extensive review of the literature on feedback in speech recognition we pointed out that the idea that lexical feedback should be helpful in speech recognition was misguided. Contrary to the popular view, feedback can not be of any help in on-line word recognition. Additionally, we described how the data cited as evidence for feedback could be explained by a bottom-up model. In order to explain some of the data that appeared to demonstrate feedback we developed a new computational model of phonetic decision making (Norris, McQueen, and Cutler, 1999, 2000a, 200b; Norris, 2001). This new model – Merge, is a development of Shortlist (Norris, 1994). The central feature of Merge is that it explains lexical influences on phonetic decision making in terms of a decision mechanism that combines information from both lexical and pre-lexical levels. We show that there is behavioural, neuropsychological and neuroimaging evidence that the processes responsible for making phonological decisions are not the same as those responsible for prelexical processing in comprehension. We also presented arguments to show that all models must have something analogous to the decision component of Merge. Therefore, by dispensing with lexical feedback, Merge provides a more parsimonious account of the data than alternative interactive models. One further achievement of this paper is that it establishes a set of criteria that specify what would constitute evidence against a modular feed-forward account of spoken word recognition. This will enable researchers to focus on the critical issues distinguishing feed-forward models and feedback models.
Critical data on lexical effects in phoneme categorisation that provided much of the motivation for the development of Merge was reported in McQueen, Norris and Cutler (1999a). That paper examined lexical effects in subcategorical mismatch, that had been studied earlier by Marslen-Wilson and Warren (1994). Our study produced two critical results. First, we confirmed that lexical competitors could have an inhibitory effect on phoneme identification in non-words. This result, combined with other data in the literature, forced us to reject the Race model of phoneme identification proposed by Cutler and Norris (1979). According to the Race model, phoneme identification is performed by a race between lexical and prelexical routes. Phoneme identification in non-words can only be performed by the pre-lexical route, and should therefore not be influenced lexical effects of any sort. A second important finding was that small changes in the experimental procedure could make the inhibitory lexical effects appear or disappear. They could not therefore be an automatic consequence of the speech recognition process as suggested by Marslen-Wilson and Warren. Finally, we presented arguments to show that, contrary to Marslen-Wilson and Warren's claim, the data were consistent with explanations based on lexical competition.
A number of other papers (Cutler, Norris and McQueen, 2000; McQueen, Cutler and Norris, 1999, 2000a, 2000b, in press; McQueen, Norris and Cutler, 1999b, 2001; Norris, 1999,2001; Norris, Cutler and McQueen, 2000) have also reported data, theoretical arguments, and simulations to support the general conclusion that speech recognition is a bottom-up process.
Feedback in perceptual learning
A central part of the Merge argument against lexical involvement in pre-lexical processing is that it cannot help on-line recognition. This does not, however, exclude the possibility that lexical information can be of benefit in learning or modifying phonemic categories. This possibility rests on the theoretical distinction between the immediate on-line activation feedback, as incorporated in TRACE and other interactive-activation models, and the feedback responsible, over the longer term, for learning (see Norris, 1993, for discussion). For example, a standard 'feed-forward' back-propagation network classifies its input in a completely feed-forward manner, but learns by feeding an error signal back down the network. There can be feedback for learning in the absence of feedback for on-line processing.
We have studied this by presenting listeners with stimuli containing ambiguous phonemes - half way between /f/ and /s/ - which can appear either in non-words, or in contexts where lexical information can bias the interpretation of the ambiguous phoneme (e.g. cara?, where interpreting /?/ as /f/ would make the word 'caraf' whereas interpreting it as /s/ would make a non-word). When these ambiguous phonemes appear in words, listeners subsequently identify these phonemes as instances of the lexically determined phoneme (McQueen, Norris, and Cutler, 2001; Norris, McQueen, and Cutler, submitted). The use of lexical information for learning has considerable value in helping listeners adjust to the different phonemic categories used by speakers with different accents or unusual speaking styles. The experimental effects are very large, and emerge over the course of hearing only 20 instances of the ambiguous phoneme. This work suggests that listeners use top-down lexical information when it is useful (in learning) but not when it is not (during on-line perception).
Lexical activation in continuous speech.
Models of continuous speech recognition such as Shortlist and TRACE assume that, during the course of speech recognition, many candidate words are accessed (possibly including several tokens of the same word), and that these candidate words compete with each other to determine the final analysis of the input. There is now extensive empirical support for this claim (e.g. McQueen, Norris and Cutler, 1994). One consequence of this competition process is that when a word like 'trombone' is heard, activation of 'trombone' should inhibit activation of the lexical candidate representing 'bone'. However, some studies in the literature (Shillcock, 1990) have reported that when the word 'trombone' appears in a sentence, 'bone' is activated, as indexed by cross-modal semantic priming. If this result is reliable, it could imply that the theories are wrong. On the other hand, it could imply that semantic priming follows simply from accessing the word 'bone' in the lexicon, even if that word candidate is subsequently inhibited. To examine this we performed a series of cross-modal priming experiments (Norris, Cutler and McQueen, in preparation), using both semantic and identity priming, and using both sentences and isolated words. In semantic priming the comparison was between priming from bone->rib, or trombone->rib. There was no evidence for semantic activation of the embedded word 'bone' in 'trombone' in any of 5 semantic priming experiments, although 'bone' did produce priming when presented as an isolated word. In fact, in sentences we found semantic priming from the word 'bone' itself in only one experiment where we employed a manipulation of contrastive stress. We did find evidence of identity priming in sentences. Most significantly, the priming was facilitatory for 'bone' itself, but inhibitory for 'bone' in 'trombone'. That is, it appears that bone is accessed and then suppressed, exactly as would be expected by lexical competition. That is, identity priming appears to be driven by the activation of lexical candidates, not lexical entries themselves. The absence of semantic priming in sentences is consistent with earlier work at the MRC-APU by Williams (1988), and, indeed, with the failure of 'trombone' to prime 'bone' in previous replications reported by Marslen-Wilson, Tyler, Waksler, & Older (1994)
Phonological priming
In recent years, a number of studies have examined phonological priming effects as a means of investigating the form of the phonological representations underlying spoken word recognition. One puzzling feature of the data has been that lexical decision and naming paradigms produce somewhat different patterns of results. For example, priming by rhyme overlap (cat-mat) is much larger in lexical decision than in naming. In lexical decision only words benefit from priming, whereas in naming both words and non-words benefit. We examined the possibility that much of the phonological priming effect in lexical decision might be strategic (Norris, McQueen and Cutler, 2002). To study this we manipulated the nature of the filler trials in a phonological priming experiment. When we included targets which almost rhymed with their primes (foils, e.g., bulk-SULSH) facilitation for rhyming targets was severely attenuated in lexical decision, but not in naming. Subjects appear to have a bias to respond 'yes' to rhyming trials in lexical decision, and this bias can be attenuated by including foils that discourage this strategy. Nevertheless, there appears to be a residual automatic effect of phonological priming in lexical decision that is similar in size to that observed in naming.
L1.3.2 The Distributed Cohort Model
One of the leading models of spoken word recognition is the Cohort model, which has evolved considerably over the past two decades (Marslen-Wilson & Welsh, 1978; Marslen-Wilson, 1987). More recently, on the basis of arguments made by Marslen-Wilson and Warren (1994), the model has assumed a distributed computational substrate, and has proposed a distinctive processing architecture, where both phonological and semantic representations are seen as the output of the system. Gaskell, working in collaboration with Marslen-Wilson, has taken the lead in implementing a connectionist model of spoken word recognition with these general characteristics. This model (the Distributed Cohort Model or DCM) has a number of significant features. First, it dispenses with the commitment to pre-lexical integration of speech information into segmental or syllabic units found in almost all models of speech perception. Second, it defines lexical activation in terms of an output representation of both lexical content (meaning) and lexical form (phonology), rather than in terms of abstract recognition units. The predictions of this model for the parallel activation of spoken words were analysed in Gaskell & Marslen-Wilson (1999a; 2001b), and are examined experimentally below. The predictions of the DCM with respect to integration of subphonemic cues during lexical access are examined in Gaskell (2000c). The DCM has achieved recognition as one of the principal models of spoken word recognition over the course of the review period, and is gaining empirical support (Gaskell & Marslen-Wilson, 2002; Rodd, Gaskell, & Marslen-Wilson, 2002).
Dynamics of lexical competition during speech comprehension
The distinctive feature of the DCM is the claims that it makes about the detailed structure of lexical competition between co-activated word-candidates during spoken word recognition. In contrast to models that represent competing lexical activations using localist word nodes, the DCM instantiates a model of lexical access where a featural representation of speech is mapped directly on to a distributed lexical representation. This type of representation can only capture multiple lexical candidates as overlapping patterns of activation, in a so-called "blend" pattern (Gaskell & Marslen-Wilson, 1999a, 2001b) This blend will only represent multiple candidates successfully to the extent that they have similar representations. For spoken words, this is true if lexical representations are organised in terms of their phonology – members of the same word-initial cohort. However, if the same words are represented in terms of their semantics, which will vary widely, the blend pattern will be much less successful in representing the properties of individual words. According to the DCM, lexical representations are organised both in terms of phonology and meaning. This means that the effects of competition on parallel activation will vary across these dimensions of the distributed representational space, with competition having much stronger effects, for the same words, on the coherence of semantic representations as opposed to phonological representations.
We tested these predictions in a series of cross-modal priming tasks, where word fragments that varied in ambiguity (i.e. number of competitors) were evaluated in three different priming situations (Gaskell & Marslen-Wilson, 2002). A fragment like "garm" from garment is already relatively unambiguous and compatible with few other possible words. A matched fragment like "capt" from captain is much more ambiguous, and remains ambiguous for longer. These variations in competitor environment had little effect on the strength of priming when the relationship between prime and target was primarily lexical and phonological, as in a straightforward repetition priming task (garm/garment). However, when the relationship was semantic rather than phonological, as in garm/clothes, so that the task probed the ability of the system to capture the specific semantic properties of the prime, then the effects of competition were much stronger. Primes like "garm" or "husb" that had fewer competitors, primed reliably more strongly, and earlier in the word. The ambiguous primes were unable to elicit priming until much later – after the end of the word in fact, when all competitors were ruled out. These contrasting effects follow directly from the DCM representational assumptions, and are not obviously predicted by any of the current competing models.
Semantic competition in lexical access: making sense of ambiguity
A key property of the DCM, in common with a number of other models (e.g., Plaut & Shallice, 1993) is that word recognition involves activation of meaning representations, so that competition between semantic representations plays as important a role in lexical access as does competition between orthographic or phonological form representations (or, indeed, abstract lexical nodes). This means, in turn, as we indicated in the previous section, that activation of dissimilar semantic representations will slow down the recognition process. This should strongly affect ambiguous words like bark, which have two quite unrelated meanings. They should be identified more slowly than unambiguous words like soap. The existing literature, however, contains several reports indicating an ambiguity advantage, with faster response times to ambiguous words in lexical decision tasks (e.g., Azuma & Van Orden 1997; Pexman & Lupker, 1999). This seems to present severe difficulties for theories in which semantic competition is a necessary consequence of their processing architecture.
In recent research, however, we have been able to show that a true ambiguity advantage does not exist, and that previous results were based on a failure to separate out ambiguity due to multiple senses from ambiguity due to multiple meanings (Rodd, 2001; Rodd, Gaskell, & Marslen-Wilson, 2002). Words with multiple senses, unlike words with multiple meanings (such as bark), share a range of separable but similar meanings. The word twist for example has several senses (as in 'twisting your ankle' as opposed to 'twisting the truth'), but they all share a common semantic core. In a series of experiments, using lexical decision to both visually and auditorily presented words, where sense and ambiguity were factorially co-varied, we showed that multiple senses did indeed speed responses, but that multiple meanings had the opposite effect, slowing down responses. Analyses of the stimulus sets used by earlier researchers (e.g., Azuma and Van Orden, 1997) showed that they had confounded sense and meaning, so that sets showing an ambiguity advantage were those where ambiguity was counted in terms of number of senses rather than number of separate meanings.
The importance of this result is not so much that it rules out alternative models (for discussion see Rodd et al, 2002), as that it removes an apparent empirical obstacle to the viability of recognition models based on distributed semantic representation. In new research, furthermore, Rodd has demonstrated that network models are capable of capturing both the detrimental effect of multiple meanings and the facilitatory effect of multiple senses – in the latter case because the network forms broader attractor basins, allowing inputs to settle more quickly into the correct attractor (Rodd, Gaskell, & Marslen-Wilson, 2001; submitted).
L1.3.3. Processing distorted speech: Neuro-imaging evidence for hierarchical processing.
A striking property of speech comprehension is its resilience in the face of the background noise, signal degradation, and variability that pervades everyday speech. The robustness of comprehension reflects both the ability of the system to adapt to variation and the redundant coding of information in spoken language. Most phonetic distinctions are carried by multiple acoustic cues and comprehension can be maintained when any single source of information is lost. In a recent fMRI study, we have capitalised on this informational redundancy and used several forms of distortion to observe the neural response common to partially intelligible speech (Davis & Johnsrude, submitted; Johnsrude, Davis, Dilks, & Turnbull, 2001). By comparing responses to acoustically different but still intelligible speech we could separate those neural systems that process the sounds of speech (which may respond differently to acoustically different forms of distortion) from areas that respond to more abstract, higher-level information that is present in all forms of intelligible speech. Furthermore, we could explore the neural correlates of processes that compensate for distortion, recovering meaning from distorted input.
In this study, three forms of artificial distortion were used, each of which provides a different challenge to the processes of spoken language comprehension: (1) speech was presented in continuous background noise, (2) speech was interrupted by noise or (3) the spectral detail of speech was replaced by noise. Behavioural testing allowed us to develop three levels of intelligibility for each form of distortion, such that sentences could be presented from which participants could repeat around 90%, 60% or 20% of the words correctly. We used fMRI to measure the neural response to sentences presented with these different forms and levels of distortion, as well as clear speech and unintelligible, amplitude-modulated noise. Activation correlated with the intelligibility of the presented sentences in anterior temporal regions bilaterally, in more posterior temporal regions on the left, in the left hippocampus and in the left inferior frontal gyrus (see Fig L2).
Figure L2: Lateral view of brain areas showing a significant response to increasing intelligibility of auditorily presented sentences in Davis & Johnsrude (submitted). Activation is shown superimposed on the mean EPI image across subjects at p < 0.001 uncorrected. Colour scale shows intelligibility-responsive regions which showed a reliable difference between the three forms of distortion (form-dependent activation at p<.001 is shown in red) or no reliable difference between distortions (form-independent activation at p>0.05 in blue).
.jpg)
Differences in the neural response to the three forms of distortion (independent of intelligibility) indicate sensitivity to the acoustic form of the signal. We observed a response in middle portions of the superior temporal lobe, lateral and inferior to primary auditory cortex, which although correlated with intelligibility, also differed between the three types of distortion. This finding provides compelling evidence for hierarchically organised processing of spoken stimuli in lateral regions of the temporal lobe. Within (probable) secondary auditory cortical regions we observe a response to the sound properties of spoken sentences, while anterior and posterior regions of the temporal lobe were insensitive to sound properties, responding only to the intelligibility of spoken sentences, and not their acoustic form.
Comparing brain activation to distorted speech with responses to clear speech and to unintelligible noise revealed a number of areas in which activation was increased when comprehension was more difficult or effortful. These regions were entirely left lateralised, including the temporal lobe, thalamus, and an extensive region of the frontal lobe, including the inferior frontal gyrus and regions of premotor cortex. These activated areas partially overlapped with temporal lobe and frontal lobe regions in which response correlated with intelligibility as well as areas which showed a differential response to the three forms of distortion. These findings are consistent with the recruitment of additional low-level, sound-based processes as well as higher-level, linguistic and/or attentional processes in the comprehension of distorted speech.
Adaptation to noise-vocoded speech in normal listeners
In a related subsequent study, we have used behavioural techniques to lay the foundations for further research into the mechanisms underlying adaptation to speech distortions. An additional interest here was that noise-vocoded speech (Shannon, Zeng, Kamath, Wygonski, & Ekelid, 1995) simulates the temporal information preserved by cochlear implants (CI), and has been used to model the perception of CI-processed speech in normal listeners (Rosen, Faulkner, & Wilkinson, 1999). In this work, we explored how listeners learn to understand noise-vocoded speech: factors such as acoustic or lexical feedback that influence this form of learning may suggest ways to assist the rehabilitation of CI users.
We conducted two experiments in normal listeners, measuring the intelligibility of 6-channel, noise-vocoded sentences, to assess the effect of different training schemes. Comprehension was assessed by measuring the proportion of words from each sentence that listeners were able to report correctly. Across experiments we varied the ordering and the type of feedback that listeners heard during the training period. The most striking results were for training blocks that used "jabberwocky" sentences. These were phonotactically legal nonword sentences matched on length and prosodic contour to a comparison set of normal sentences ("the police returned to the museum" became "cho tekine garund pid ga sumeun"). Listeners pre-exposed to jabberwocky sentences performed no better than naïve (untrained) listeners, suggesting that higher-level word- or meaning-based information is required for this kind of perceptual learning to occur. A proposal for continuing this work is given in section SL5.2. of the future proposals.
L1.3.4 Cortical interactions in processing lexical ambiguity in sentential context
To understand the meaning of a sentence, we need to put together the meanings of its constituent words. This made more difficult by the presence of ambiguous words. For example, in a phrase such as "the loud bark", the listener needs to work out that the noun "bark" refers to the "bark of a dog" and not the "bark of a tree". Processes that select appropriate interpretations on the basis of the context are extremely important; over 80% of the common words listed in dictionaries are given more than one definition (Rodd et al, 2002). In a pioneering study, collaborative with Rodd (CSL, Cambridge), we have begun to investigate the neural substrates of these combination processes (Rodd, Davis, & Johnsrude, 2002).
We used fMRI and sparse imaging to scan volunteers while listening to sentences that contained ambiguous words (e.g. "the shell was fired towards the tank") and matched, unambiguous sentences (e.g. "her secrets were written in her diary"). Comparing brain areas activated by these two types of sentences revealed increased processing for ambiguous sentences in three brain regions: the posterior left middle temporal gyrus, and the left and right inferior frontal gyri. This points to a combined role of frontal and temporal lobe systems in activating, selecting and combining the meaning of words in natural speech comprehension. Current views of the domains of functioning of these areas suggest that the temporal region supports the access and storage of semantic information about individual words, whereas frontal regions support control functions involved in the selection and combination of these meanings.
Project L2: The structure of lexical representation
Scientific direction: Marslen-Wilson (80%), Pulvermuller (5%)
MRC Research support: Boudelaa (100%), Davis (20%), van Casteren (50%), Shtyrov (5%), Hauk (5%), Reid (100%)
Externally funded: Meunier (100%), Zhou (100%)
Students: Reid (100%), Chou (100%), Rodd (100%), Ford (100%)
At the core of the language system is the mental lexicon, the mental representation of the forms and meanings of the words and morphemes in a language. These representations play a central role in processes of language comprehension and production, mediating between the processes of speech analysis and the computation of meaning. Our focus in the first main project here (L2.1) is on the structure of these representations: whether the mental lexicon is organised in terms of whole-word representations, or whether the representation is predominantly morphemic in character. We approach these issues cross-linguistically, in order to investigate a representative range of possible lexical and morphological systems, and report systematic comparisons across a wide range of different language systems, including Arabic, Polish, and Chinese (as well as English).
The second main focus here, in project L2.2, is on the contrast between regular and irregular inflectional morphology in English. This has come to be viewed as a crucial test case for fundamental disputes about the appropriateness of symbolic as opposed to subsymbolic modes of mental computation. Earlier work by Marslen-Wilson and Tyler (1997) demonstrated a neuropsychological double dissociation between the processing of regular and irregular past tense morphology in aphasic patients. The further work reported here, carried out chiefly in collaboration with Tyler and colleagues (Centre for Speech and Language, Experimental Psychology, Cambridge), has strengthened and broadened the basis for these dissociations, working with a variety of brain-damaged populations and using neuro-imaging techniques to study these phenomena in the intact adult brain. In addition, cross-linguistic studies of regularity and irregularity in French and Polish help to put the phenomena for English in an appropriate perspective.
L2.1 Cross-linguistic studies of lexical structure: Word formation and derivational morphology
An adequate account of spoken language comprehension is going to require a convincing definition of the notion "word". To provide such a definition we need a better understanding of the structure of the mental lexicon – what are its basic units of representation and analysis, and how are these organised in relationship to each other? Any attempt to resolve these questions is immediately confronted with the immense variety of lexical arrangements across languages. The research summarised here represents a systematic and on-going attempt to explore and describe this cross-linguistic variation. Do we find evidence for common underlying principles in lexical organisation. The results so far suggest that lexical systems are as notable for their differences as for their similarities.
The starting point is a view of lexical organisation developed initially in the context of English derivational morphology (Marslen-Wilson et al, 1994). Derivational morphology, in a language like English, is the concatenation of a base form (a root or a stem) with one or more derivational affixes. These may be suffixes or prefixes, as in forms like happiness, analysable as {happy} + {-ness}, or rethink, analysable as {re-} + {think}. These derivational processes change the meaning, and often the form-class of the stem, and are generally thought of as generating new lexical items.
Using a variety of immediate and delayed repetition priming techniques, we have developed a view of the English mental lexicon as a dynamic cognitive entity, distinguished by three core properties: (a) We assume that it is morphemically organised, so that the basic unit of representation is the morpheme, and complex words, where synchronically decomposable, are represented in terms of their constituent morphemes; (b) It is combinatorial, indicating that the same morpheme combines with other morphemes across a morphological family, to create the basis both for meaning and for structure. Thus, the morpheme {dark} in darkness is the same lexical and cognitive entity as the {dark} in darkly. Similarly, the {–ness} in darkness is the same as the {–ness} in toughness. (c) Morphemic decomposition is assumed to be dependent on semantic transparency. Complex forms are only represented in morphemic format if they are semantically transparent. The form punishment is semantically transparent and represented as {punish} + {-ment}. The form department, although superficially also complex, is semantically opaque and is represented as a whole form.
The evidence for these claims about English comes from a set of priming effects, the most salient of which are the following:
Stem priming: This is the priming effect, found in masked priming and in immediate and delayed repetition priming (Marslen-Wilson et al., 1994; Marslen-Wilson & Zhou, 1999), between a semantically transparent complex form and its stem, as in prime/target pairs like darkness/dark. Critically, semantically opaque pairs like department/depart do not prime.
Affix priming: This is the priming effect between semantically unrelated prime-target pairs which share the same affix, as in darkness/toughness and rebuild/rethink (Marslen-Wilson, Ford, Older, & Zhou, 1996). This is strongest for productive affixes, and is interpreted as the combinatorial re-use of the same bound morpheme in both prime and target.
Suffix-suffix interference: This is the interference effect observed between semantically transparent pairs sharing the same stem but different suffixes, as in darkness/darkly (Marslen-Wilson et al., 1994; Marslen-Wilson & Zhou, 1999). The absence of priming between these highly semantically and morphologically related pairs is interpreted as inhibition between two affixes competing for linkage to the same stem.
The co-occurrence of these three effects we take to be diagnostic of a decompositional and combinatorial system. How far do we find similar effects in other languages?
L2.1.1. Polish
The first language we report on is Polish, a Slavic language with an exceptionally rich morphological system, and which also, like English, employs a concatenative derivational morphology, combining stems with sequences of prefixes and affixes. Unlike English, essentially all surface forms are morphologically complex, combining a bound stem with one or more suffixes. Thus the word przybiegłam 'I run up' consists of the stem 'bieg-', the derivational-aspectual prefix 'przy-', and the inflectional suffixes '-ł' and '-am', indicating the past tense and the 1st person singular feminine. In a series of studies, chiefly using delayed repetition priming tasks to reduce semantic effects, we have found a profile of results that is very similar overall to English (Reid, 2001; Reid & Marslen-Wilson, 2000; in press).
Stem priming: We find strong stem priming in both simple and complex forms, ranging from pairs like chodz-enie/chodz-i-ć (walking/to walk) to highly complex forms like bajk-o-pis-ar-stwo/pis-a-ć (fable-writing/to write).
Affix priming: We also find priming for pairs sharing the same affix. This covers sets of pairs sharing derivational affixes, as in kuch-arz/piłk-arz (a cook/footballer) sharing the agentive suffix {-arz}, as well as more complex forms like roz-pakow-ywa-ł-em (to unwrap, 1st person sing., masculine, past tense) and roz-wałkow-ywa-ć ('to flatten something using a rolling-pin). These words share a derivational aspectual prefix {roz-} and the secondary imperfective suffix {ywa-}, and show strong priming even with 12 items intervening between prime and target.
Suffix-suffix interference: Polish shows strong interference effects of this type. Pairs like pis-anie/pis-arz (writing/writer) and balon-owy/balon-ik (balloon-like/a little balloon) show no priming at all in delayed repetition, despite their close morphological and semantic relationship.
Semantic transparency: Finally, Polish shows strong effects of semantic transparency. As in English, there is no priming for semantically opaque pairs that historically shared the same stem, as in pairs like jałowiec/jałowy (juniper/futile), with some indication in further tests that strict semantic compositionality may play a stronger role in determining representation than in English.
The profile of results for the different kinds of priming relationships suggest that Polish and English have a great deal of common. Although they are very different languages in many important respects, they both fit an overall template that we interpret in terms of a morphemically organised and combinatorial mental lexicon.
L2.1.2 Arabic
Semitic languages like Arabic and Hebrew possess a complex morphological system that is organised on fundamentally different principles to languages like English, French, and Polish, with their concatenative morphological processes. Semitic languages, in contrast, employ a non-concatenative morphology, where the surface phonetic form is constructed by interweaving two or more abstract morphemes - the consonantal root, carrying semantic information, and a word pattern which specifies the syntactic category and the phonological structure of the surface form. Thus, for example, the Arabic root {ktb} with the semantic value of <writing>, combines with the word pattern {faala}, with the syntactic meaning of 'active verb', to give the surface form kataba, meaning 'write'. These are highly abstract morphemes, that never surface as phonetic forms on their own. The question we investigated was whether these abstract entities function cognitively in ways comparable to stems and derivational morphemes in concatenative morphologies such as English.
Root priming: Using cross-modal and masked priming techniques, we found clear evidence for priming between pairs that shared the same consonantal root (Boudelaa & Marslen-Wilson, 2000a). Thus, for example, the prime /idxaalun/ (inserting) speeds responses to the target /duxuulun/ (entering), where prime and target have in common the root {dxl}. Strikingly, and quite differently from English and Polish, priming is just as strong when the prime is semantically opaque, as in the form /mudaaxalatun/ (interference), which also shares the root {dxl} with the target /duxuulun/, but where the meaning of the form is not synchronically predictable. This preservation of root priming under conditions of semantic opacity shows up consistently across all our experiments, and is also found for Hebrew (e.g., Frost, Forster, & Deutsch, 1997).
In contrast with English, however, a further level of more abstract semantic analysis plays a role in lexical organisation. In addition to the tri-consonantal root, such as {ktb} and {dxl} mentioned above, we also find evidence for more abstract bi-consonantal morphological unit called the etymon. For example, in the form [batara] the core meaning is carried not by the tri-consonantal root morpheme {btr} but by the etymon morpheme {b,t} which surfaces in other forms like [batta] (sever), [batala] (cut off) with the same meaning <cutting>. In a series of priming experiments we show that the etymon can yield the morphological priming effects typically obtained with triconsonantal root morphemes. Two words sharing an etymon facilitate each other even though they do not share a root (Boudelaa & Marslen-Wilson, 2001a)
Word pattern priming: Analogously to affix priming, we find significant effects between pairs that share the same word-pattern but have different roots and different meanings (Boudelaa & Marslen-Wilson, 2000b), as in pairs like /xuduuun/ and /uduuun/ (submission/happening), sharing the word pattern {fuuulun} (with the meaning "deverbal noun, singular"). The absence of priming between forms like /suuunun/ and /uduuun/ (prisons/happening), which have word patterns that are phonologically but not morphologically identical, demonstrates the morphological nature of the effects here, and rules out an account in terms of phonological overlap between prime and target.
These results for Arabic, and the comparable results for Hebrew, suggest strong support for a decompositional, combinatorial system, with abstract morphemes combining to produce the surface form, and being separated out in the process of recognition. The complete absence, however, of a semantic transparency effect in root priming, signals an apparent fundamental difference in the principles underlying the role of morphological combination.
In addition, further experimental analysis of the word pattern revealed that these priming effects were mainly being driven by the CV-Skeleton, a highly abstract structural unit coding the phonological shape of the surface word and its primary syntactic function (McCarthy, 1981, 1982). In three experiments using masked, cross-modal, and auditory-auditory priming we examined the role of the CV-Skeleton as a morphemic unit in the processing and representation of Arabic words. Strikingly, word pairs sharing only the CV-Skeleton primed reliably throughout, with the amount of priming being as large as that observed between pairs sharing the full word pattern (Boudelaa & Marslen-Wilson, submitted). This is important not only because of the abstractness of the psycholinguistic entities thus identified, but also because this is a type of morpheme for which there is no counterpart in languages like English. Although the CV-Skeleton does play a role in English, it does so only as a phonological entity, and does not carry meaning.
Time-course of morphological activation in Arabic and English
In priming studies of morphology, it is an important issue to separate out potential confounds due either to form overlap between prime and target or to semantic overlap. One way of assessing these issues is to use incremental masked priming, where the exposure duration of a masked prime is systematically varied to determine the time-course with which orthographic, morphological, and semantic factors come into play. In earlier work in English, carried out collaboratively with the CSL (Cambridge), we found that morphological structure plays a dominant role in structuring lexical representations at all stages of processing (Rastle, Davis, Marslen-Wilson & Tyler, 2000). Varying SOA over the range 43, 72, and 230 ms, we found morphological effects (for pairs like sadness/sad) at all SOA's. These were just as strong as identity priming effects. Semantic effects did not appear reliably until the longest SOA, while there was some evidence for transient form-based effects at the shortest SOA.
This paradigm can readily be transported into Arabic, where it is important not only to evaluate the time-course with which different factors come into play, but also to probe the morphological analysis processes involved in the analysis of Arabic forms into word-patterns and roots. Accordingly, we varied the morphological (word pattern, and root), orthographic, and semantic relationship between prime and targets over four closely spaced SOA conditions (32, 48, 64, and 80 ms). The results show distinctive patterns of activation for the two morphemes (Boudelaa & Marslen-Wilson, 2001b). Word pattern effects are transient and detectable only at intermediate SOA's (48-64ms). Robust root effects are evident at all four SOA's, and are unaffected by factors of semantic transparency. However, the pure effects of semantics, for the semantically related items, are only seen at the longest SOA, comparable to the English results. Unlike English, however, we do not see good evidence for form-based effects at the earliest SOA. More generally, the patterning of different effects over time, and the separability of word-pattern and root effects, argues strongly that morphological effects in Semitic languages represent distinct structural characteristics of the language, and are not reducible to form or meaning overlap.
L2.1.3 Compounding in Chinese and English
A final set of comparisons involve compounding, a quite different procedure for word-formation, and where the starting point is Mandarin Chinese, rather than English. Compounding is a highly productive means of word formation in both English and Mandarin. Unlike derivational word formation, it does not involve the combination of a stem with an affix, but the linkage of two free stems – as in the English compound houseboat, made up of the two nouns house and boat. The effect of this, in contrast to derivation, is that compounding is not fully compositional or combinatorial in nature. The meaning of a compound is never fully predictable from the meaning of its components - although a snowman is a man made of snow, a milkman is not a man made of milk, and so forth. To know what a compound means, one needs to know what it refers to. The issue, that we addressed first in Mandarin and then in English, is whether this leads to a whole-word, rather than a decompositional and morpheme-based representation of compounds in the mental lexicon.
Compounding in Mandarin takes place in a functionally very different linguistic environment from English. Mandarin has essentially no derivational morphology, so that compounding is its only productive means of word-formation, under conditions where there is considerable pressure due to homophony at the syllabic level. Recent analyses suggest that around 70% of word types in Mandarin are bisyllabic compounds (Institute of Language Teaching and Research, 1986). In a series of collaborative studies with Zhou (Cambridge, Beijing) we addressed the issue of whether a morphemic account or a whole form account was appropriate (Zhou & Marslen-Wilson, 1995; Zhou, Marslen-Wilson, Taft, & Shu, 1999; Zhou & Marslen-Wilson, 2000). Mandarin seemed to be a plausible candidate for a morphemic account, because of the salience of individual morphemes in the spoken language and in the writing system. Extensive research using auditory-auditory repetition priming shows that such an account is not correct. Compounds are represented as separate lexical entries, and not as combinations of their constituent morphemes. This means that Mandarin, unlike the other languages we have studied, does not have a system of word-formation that is decompositional and combinatorial.
This raises the question of how English compounds are represented, which we investigated in a series of cross-modal experiments, looking at the priming relations between transparent (bathroom), opaque (blackmail), and pseudo (shamrock) compounds and their constituent morphemes (Zhou & Marslen-Wilson, 2000). Two results in particular seem hard to handle for a morphemic story. The first is that we did not find priming between the first and second constituents of a compound. Thus bath, for example, does not prime tub. This is quite inconsistent with the view that compounds are represented as strengthened links between their constituent morphemes. The second finding is that shared constituents between transparent morphemes do not lead to priming unless the compounds as a whole are semantically related. Thus headache does not prime headscarf, even though they both transparently contain the morpheme head. In contrast, teacup does prime teapot. This is because these two compounds are strongly semantically related, whereas headache and headscarf are not. This is not consistent with a morphemic, combinatorial story, where the morpheme head is a constituent of headache and headscarf in the same way that punish is a constituent of both punishment and punishable. In summary, compounding in Mandarin and English seem to be remarkably similar, reflecting in the same way the representational consequences of the unpredictability of the meaning of compounds.
Summary
Despite the small sample of languages studied, we are left with a wide range of lexical arrangements. Mandarin Chinese seems to lie at one extreme, with apparently no combinatorial procedures for word-formation, and with a lexicon made up of whole forms, in which compounds and the words that make up these compounds all have separate lexical representations. English has a similar system for compounding, but also has a decompositional system of word-formation and representation, reflecting the different processing requirements of derivational procedures that operate on a combinatorial basis, and that deliver predictable and compositional meanings. In this respect English parallels the broad characteristics of a language like Polish, which has a much richer and more complex morphological system. Both of these languages, nonetheless, share with Arabic (and Hebrew) a combinatorial and decompositional approach to lexical representation. In these Semitic languages, however, morphological representation appears to play a more fundamental structural role, so that no surface form can be produced without some underlying process of morpho-phonological combination. This delivers both the surface form, and its basic syntactic and semantic properties.
These differences in lexical representation, although reflecting only a preliminary sample of languages, and reflecting a single type of experimental approach, nonetheless suggest that unitary models of spoken word access processes may not be possible across languages. Although all lexical systems must share the same underlying cognitive and neural constraints on representation, process, and acquisition, these constraints seem to be sufficiently broad to allow different systems to be constructed on apparently quite different organisational principles.
L2.2 Regular and irregular inflectional morphology: Issues in the architecture of the human language system
A key issue in cognitive neuroscience is the functional and the neural architecture of the systems underlying human language, and whether the organisation of these systems should be characterised in terms of a uniform overall computational and neural process, or whether multiple and distinct underlying mechanisms are involved. A particular empirical focus for this issue has been the irregular and regular forms of the English past tense, which provide a sharp contrast in the demands that they make on processes of language learning, comprehension and production. The regular past tense, formed by adding the regular affix /-d/ to the verb stem [as in jump-jumped; agree-agreed], is the paradigmatic example of a predictable, rule-like process. The irregular past tense, applying to a closed set of about 160 English verbs, represents the converse case of an unpredictable and idiosyncratic process [as in think-thought; make-made], requiring rote learning of each example.
This debate has taken a strongly neuropsychological turn over the past five years, with the publication of several sets of results which point to the underlying dissociability of the neural systems required for the successful production and perception of regular and irregular inflected forms. The pattern that emerges links particular patterns of neuropathology to selective deficits with either type of material. Deficits for the irregulars are associated with damage to L temporal cortex whereas deficits for the regulars are associated with damage to L inferior frontal cortex (LIFC) and underlying structures. The association of lesion site and behavioural deficit suggests that the LIFC is preferentially involved in processing regular inflectional morphology, and supports specialised mechanisms for handling morpho-phonologically complex forms, such as the English past tense (Marslen-Wilson & Tyler, 1997; 1998; Tyler, Randall, & Marslen-Wilson, 2002a; Tyler, deMornay-Davies, Anokinah, Longworth, Randall, & Marslen-Wilson, 2002b).
These claims have not, however, gone unchallenged, and both theoretical and empirical arguments have been adduced to support the contrary, single systems view. Joanisse & Seidenberg (1999) have proposed a unitary computational model in which deficits involving the irregulars reflect damage to semantic systems, while deficits involving the regulars reflect a phonological deficit. Correlational evidence to support this is provided by Patterson, Lambon Ralph, Hodges, & McClelland (2001), showing that patients with a progressive semantic disorder perform less well with irregular forms, while non-fluent aphasics with associated phonological deficits perform less well on the regular forms (Bird, Lambon Ralph, Seidenberg, McClelland, & Patterson, in press). The weight of the current evidence, however, as summarised below, suggests that some degree of underlying dissociation is nonetheless the correct analysis.
L2.2.1 Semantics and the irregular past tense
Marslen-Wilson & Tyler (1997, 1998) were the first to report a correlation between semantic deficits and impaired performance on the English irregular past tense. In an auditory-auditory repetition priming task, two patients with impaired semantic priming (for pairs like cello/violin) showed preserved priming between regular past tense pairs (jumped/jump) but impaired priming for irregular pairs (gave/give). We moved in two directions to evaluate the implications of this result.
In a continuing programme of collaborative research with Tyler (CSL, Cambridge), we probed in more detail the patterns of neuropathology associated with dissociations in performance on the regular and irregular past tense morphology, and their association with semantic impairments (Tyler et al, 2000b). We tested five nonfluent patients, all of whom had extensive LH damage involving L inferior frontal gyrus and underlying structures, and four Herpes Simplex Encephalitis (HSE) patients with semantic deficits who had damage to inferior temporal cortex but where inferior frontal cortex was spared. In a new priming study, extending and replicating the earlier work (Marslen-Wilson & Tyler, 1997), the nonfluent patients showed no priming for the regular past tense but significant priming for the irregulars (whereas controls show priming for both). In contrast, the HSE patients showed significantly impaired performance for the irregulars in an elicitation task (Tyler et al, 2000b). The linkage of distinct patterns of behavioural deficit with disjoint patterns of neuropathology suggest that two separable systems underlie processing of the regular and irregular past tense.
A second set of experiments confronted more directly the implied causal relationship between semantic impairment and deficits with the irregulars, asking whether the underlying relationship between irregular forms and their stems was semantic rather than linguistic and morphological. Two studies with normal adults suggested, however, that the relationship is morphological in both cases – i.e., that gave and give are related because they share a common morpheme, in the same way as jumped and jump, and in contrast to semantically related pairs, such as cello/violin, which do not have a morpheme in common and are lexically quite separate. In a delayed repetition priming experiment, designed to separate semantic effects from morphological effects, priming of regular and irregular pairs is equally well preserved, while semantic priming has dissipated (Marslen-Wilson & Tyler, 1998). In an ERP study, conducted in collaboration with Mark Johnson and Gergo Csibra (Birkbeck, London), we evaluated the patterns of brain activity associated with regular, irregular, and semantic immediate cross-modal priming. The results were unequivocal, with regular and irregular priming patterning together, and both showing left anterior negativities standardly associated with linguistic processing, while semantic primes showed only a centrally distributed N400-type effect (Marslen-Wilson, Csibra, Ford, Hatzakis, Gaskell, & Johnson, 2000). On the basis of these two studies, we interpreted the co-occurrence of semantic deficits and of disrupted access to irregular past tense forms as reflecting a shared dependence on lexical access processes that mediated access to stored whole forms, but not as reflecting a causal relationship between damage to semantic systems and impaired performance on irregular forms.
L2.2.2 Phonology and the regular past tense
The account that we have developed of the English regular past tense attributes the impairment in patients with left inferior frontal damage to specific difficulties with morpho-phonological parsing – i.e., with the segmentation of complex inflected forms, such as the regular past tense, into its morphemic components. The single mechanism account (Joannisse & Seidenberg, 1999) argues that a general phonological processing deficit causes the poor performance with the regular past tense, and does not recognise the possibility of a deficit specific to morpho-phonological parsing. To evaluate the claims made by these different approaches, we developed a speeded same-different judgement task for use with four nonfluent patients with documented difficulties with the regular past tense (Tyler et al, 2002a). We compared patients' ability to detect the difference between the past tense and stem of regular (played/play) and irregular (taught/teach) past tense verbs, as well as matched pseudo-regular and irregular pairs (trade/tray and port/peach). These real word conditions were accompanied by matched sets of non-words. Patients' latencies to the regular past tense real word-pairs were consistently slower than to the phonologically matched pseudo-regular and non-word pairs. To test for a general phonological processing deficit, we conducted several tests of phonological processing ability. The results show that the patients had a range of difficulties in phonological processing, from very mild to severe, which did not correlate with their performance on the speeded judgement task. This pattern of results is inconsistent with the Joannisse & Seidenberg analysis, and supports a specialised morpho-phonological processing mechanism which can be dissociated from other phonological processes and which is used directly in the processing of the regular past tense.
The differential difficulty of the non-fluent patients with the regular past tense comparisons leads to predictions for the performance of unimpaired subjects in neuro-imaging experiments with the same materials. We have begun to evaluate these predictions in fMRI studies, using a sparse imaging technique to allow the test-pairs to be presented free of interference from scanner noise (Tyler, Stamatakis, Post, Randall, & Marslen-Wilson, submitted). Interestingly, activation patterns for the regular past tense comparisons differ from the control conditions in a set of areas that overlap closely with brain areas that are damaged in the patient population – these include left superior temporal and left inferior frontal cortex, and the anterior cingulate. This points to a fronto-temporal network, where temporal lobe lexical access processes may be modulated by specialised frontal systems activated by morpho-phonologically complex input strings.
L2.2.3 Regularity and irregularity in cross-linguistic perspective
The analysis we have developed of the regular/irregular contrast in English focuses on the contrast between the irregular past tenses as phonologically simple forms, that can access stored lexical representations directly, while the regular past tenses, because they are morpho-phonologically complex combinations of stems and affixes, require the involvement of further analysis processes. However, this association of regularity and irregularity with the presence or absence of morpho-phonological complexity, is a historical accident that is specific to English (and perhaps to Germanic – see Clahsen, 1999). In other languages, it is typically the case that both regular and irregular forms require the application of morpho-phonological parsing mechanisms. This predicts that we should not see the same differentiation between regular and irregular forms, in selected processing tasks, that we see in English. Specifically, in cross-modal immediate repetition priming in English (e.g., Marslen-Wilson, Hare, & Older, 1993), we consistently find that irregular past tense forms do not prime their stems (as in gave/give) while strong priming is obtained for regular past tense pairs (as in jumped/jump) . In an earlier study (Orsolini & Marslen-Wilson, 1997),we tested this prediction for Italian, where both regular forms (e.g, prendiamo "we take" from the verb prendere) and irregular forms (such as presero "they took", also from prendere) are morpho-phonologically complex, being composed of a verb stem and one or more inflectional affixes, and where we found equally strong priming for both regular and irregular pairs. We have now extended this to two further languages.
The French language allows a wider range of contrasts than Italian in types of regularity of verbal inflection. As in the English and the Italian studies the subjects heard a spoken prime (such as aimons, 'we love') immediately followed by lexical decision to a visual probe (such as aimer, 'to love'). We contrasted four types of French verbs, varying in the phonological and morphological regularity of their verb form inflection, and ranging from fully regular verbs (aimons/aimer, 'we love/to love) to entirely irregular verbs with idiosyncratic stem alternations (irons/aller, 'we will go/to go'). Even for an alternation like irons/aller, where the irregular stem {ir-} has no phonological relationship to the base form of the verb, the surface form is still morpho-phonologically complex, being made up of the irregular stem plus the regular inflectional affix {-ons}. This marks the first person plural in the same way as in regular forms such as aimons. Morphologically related primes, whether regular or irregular, significantly facilitated lexical decision responses for all verb classes. The same pattern of results was observed in a second experiment using a masked priming paradigm (Meunier & Marslen-Wilson, 2001; in press). These results contrasted with English, where regularly inflected verbs prime their stems but irregular verbs do not. A comparable pattern was also observed for Polish, which allows for extensive contrasts in regularity and irregularity in word formation, both for verbs and for nouns (Reid & Marslen-Wilson, 2001).
These cross-linguistic results are consistent with our basic hypotheses about regularity and irregularity effects in the English past tense, although they need to be evaluated further in appropriate neuropsychological populations (planned research with Polish aphasics is discussed in SL3). The regular/irregular contrast in English does indeed tap into separable underlying processing mechanisms, but only because this contrast, in English, is confounded with morpho-phonological complexity.
L.2.2.4 Cross-linguistic contrasts in morpheme frequency effects; computational and empirical studies
An important theoretical distinction in models of spoken and written language processing is between localist models (e.g., Shortlist, Norris, 1994) and distributed models (such as the DCM; see L1.3.2). An increasingly prominent domain for evaluating these contrasts is the representation and processing of inflectional and derivational morphology (e.g. Plaut & Gonnerman, 2000). A relevant source of evidence here is the effect of word and morpheme frequency on the recognition of morphologically complex words. Previous experimental work in Dutch (Baayen, Dijkstra & Schreuder, 1997) has been argued to be consistent only with a dual-route, localist model in which inflected words are both stored as whole forms and are decomposed into stems and affixes. In recent work, we have developed a distributed connectionist model that can recognise Dutch nouns and verbs in both monomorphemic and inflected forms (Davis, van Casteren & Marslen-Wilson, in press). When tested on a lexical decision task, the network shows the appropriate pattern of frequency effects to simulate experimental data from Dutch, but only when homonymous affixes are included in the training set - such as the Dutch plural affix {-en} which also marks verb infinitives. Competition from the homonymous forms of an inflectional affix means that effects of word-form frequency are more robust for items that are marked with an inflectional affix which serves two distinct morphological functions.
In recent experimental work we have tested the predictions of this distributed processing model when applied to English inflected forms. In an experiment comparing the comparative {-er} and superlative {-est} adjectival endings (Ford, Davis & Marslen-Wilson, in press), we observed significant differences in the processing of these two inflected forms. Items that ended in the homonymous {-er} affix (which is also used for agentive forms like dancer) were processed more slowly, and did not differ in their behaviour from items matched on word-form frequency. We have also explored effects of affix homonymy in the processing of the English {-s} plural ending, which also marks the third person singular form of verbs. Results show stronger effects of lemma frequency than observed in Dutch, suggesting that processing of the English plural is less affected by competition from the verbal inflection. This finding may reflect cross-linguistic differences in the relative frequency of verbal interpretations of the noun plural ending. Further simulations are currently under way to simulate these differences using the distributed model we applied previously to Dutch (Davis et al, in press).
L2.2.5 Electrophysiological indicators of morphological processing
Research has also begun using EEG techniques to probe potential differences in the cortical signatures of stems and inflectional affixes, using the Mismatch Negativity response (MMN – for further details see reports under L4). In a first study addressing this issue we compared the MMN to a verb stem to that elicited by an inflectional affix. The results indicated (a) an earlier MMN to the stem as compared with the suffix and (b) a more frontal distribution of the verb stem-elicited MMN and a more posterior distribution of the suffix-related MMN (Shtyrov & Pulvermüller, 2002a). Distributed source estimates localized the main sources of the verb MMN in inferior frontal cortex and of the suffix MMN in posterior perisylvian and parietal areas. This is consistent with earlier claims that function words and inflectional affixes have a left perisylvian neuronal representation (Pulvermüller, 1995; Pulvermüller, Lutzenberger, & Birbaumer, 1995). Several further experiments are planned to try to unpack these phenomena in more detail (see future proposals in SL3 and SL4)
Additional research is in its preliminary stages, again using MMN techniques, to probe the representation of regular and irregular verb forms in English, and the representation of consonantal roots and word patterns in Arabic (see SL3).
Project L3/M4: Short-term memory and speech perception
Scientific direction: Norris (50%), Page (100%)
MRC Research support: Hall (100%), Woods (50%)
External grant support: Cumming (50%)
Student: Cumming (100%)
The study of short-term memory (STM) and the study of language comprehension have effectively been treated as two quite independent topics of research in cognitive psychology. However, almost all memory researchers believe that STM must play some role in language processing, and there is little doubt that speech recognition and comprehension are reliant on some form of temporary storage. This project was designed as an initial step toward developing an integrated account of the relation between STM and language. The work focussed on two of the most significant links between memory and language; both require the maintenance of representations of order (e.g. of phonemes or words), and both depend on the storage or manipulation of phonological representations. The main emphasis of this project was computational modelling, supported by empirical work to place additional constraints on theoretical development. At the time this project began, there were already well established computational models of speech recognition (e.g. TRACE, Shortlist). In contrast, there had been very little work developing computational models of STM. The research reported here therefore started from the view that any attempt to provide an integrated account of STM and language needed to begin by developing a computational model of STM. In particular, we needed to focus on modelling memory for ordered representations of phonological information.
L3.1 Computational modelling of short-term memory: The Primacy and the SEM models
The most significant achievement of this project over the last report period has been the development of two new computational models of short-term memory (STM): the Primacy model (Page and Norris, 1998) and the Start-End Model (SEM) (Henson, 1998). The Primacy model can be considered to be a computational implementation of the phonological loop component of the Working Memory model of Baddeley and Hitch (1974). The SEM model deals with serial recall at a more general level and includes features designed to explain memory over longer periods than normally supported by the phonological store.
Both of these models were designed to address shortcomings in existing models of STM. Although the Working Memory model has been exceptionally productive, and successfully accounts for much of the central data on STM, it remains limited by the fact that it is a qualitative verbal theory. The primary source of data on STM comes from the immediate serial recall task, yet the Working Memory model has no account of how information from memory can be recalled in the correct order. We took the view that further progress in the field of STM research was dependent on the development of computational theories that would provide quantitative simulations of the data.
An initial limitation on this enterprise was that most of the existing studies in the literature did not describe their data in sufficient detail to provide the necessary insights for modelling. Most reports of STM data simply report serial position curves, but many only report proportions of items or lists correctly recalled. One of our initial tasks was therefore to collect data that could be analysed in greater detail. Some of this work is reported in Henson's thesis, along with procedures for analysis of serial recall data. At the outset of this work we realised that one of the most important constraints on models of STM was data reported by Baddeley (1968). Baddeley found that when subjects were presented with lists with an alternating pattern of phonologically confusable or non-confusable items (e.g. BFDLCY) recall of non-confusable items was unaffected by the presence of the confusable items in the list. This finding, since replicated and extended by Henson, Norris, Page and Baddeley, (1996), has come to be a benchmark test of models of STM. Although other models can provide qualitative simulations of these data, only the Primacy model and SEM can give accurate quantitative simulations. In particular, this data appears completely incompatible with models based on associative chaining. Chaining models must predict that if one of the confusable items (say D) is not recalled, then that item will not be available as a cue for the recall of the following non-confusable item (L). Any increase in errors on confusable items must therefore also increase errors on non-confusable items. Also, this data can only be explained by models using a two-stage recall process, as first proposed in the Primacy model.
The central assumption of the Primacy model is that order is represented by the relative activation levels of list items. The first item in a list is assigned a particular level of activation, and successive items each have activation levels that are lower than the previous activation by some constant amount. Recall is modelled as a noisy choice process, followed by item suppression. That is, zero-mean gaussian noise is added to all item activations and the most strongly activated item is then selected for recall. The activation of that item is then suppressed so that it can not be recalled again. Errors in recall occur because the noise in the selection process can lead to the selection of items in the wrong order. As activations decay, the difference in activation levels between nearby items decreases, and so the possibility of selecting the wrong item increases. Performance therefore declines over the course of the list (the primacy effect). Performance improves for the last item in the list (recency) because that item can only be recalled in the wrong position by being recalled too early. Other list items can be recalled either too early or too late. With some additional assumptions about the nature of the rehearsal process, the model gives very precise simulations of the central characteristics of STM. An additional component of the model is influenced by the phonological relationship between list items, and causes the degradation in performance seen with lists of phonologically similar items (BVPDE etc). The full model can simulate data (Baddeley 1968; Henson, Norris, Page and Baddeley, 1996) that cannot be accurately simulated by any model other than SEM, which incorporated the same assumptions about the mechanism of retrieval from phonological memory. The only other model that can simulate the general pattern of the data is the latest version of the Burgess and Hitch (1999) model. Page and Norris (1998b) have also presented a connectionist implementation of the Primacy model designed to highlight the parallels between recall from STM and models of speech production.
The SEM model is rather more complex than the Primacy model. Whereas the primacy model relies on an activation gradient that decays over the course of a list, the SEM also has a second signal that rises over the course of the list. These two signals provide a two-dimensional retrieval cue. Effectively, this cue codes position relative to the start and ends of the list. Henson (1999) contrasted the predictions of the SEM model with those of positional models by examining protrusion errors in serial recall. Subjects often make errors where an item from the previous list is recalled in the current list. These items frequently appear in the same position as they did in the earlier list. Henson showed that there were frequent protrusions from the end of one list to the end of the next list, even when the lists varied in length. This is to be expected from the SEM model, but is inconsistent with models using position-item associations, where the final items in lists of different lengths would have different positional associations.
Note that the Primacy model was designed specifically as a model of the phonological loop, and between list protrusions operate over time-spans well beyond the duration of the loop. These positional protrusions therefore don't have any implications for our understanding of the loop. However, Page and Henson (2001) have suggested that the Primacy and SEM models might be combined to provide an integrated account of memory for serial order over both the short and medium term.
The data from Henson et al (1996) raised considerable doubt over the plausibility of chaining models. In work originating from Cumming's thesis we turned our attention to an evaluation of positional models. In this research we used the Hebb effect (Hebb, 1961), whereby subjects performance in immediate serial recall improves on a list which is repeated every three or so trials. The representation of the repeating list that is being built up can be probed by presenting a 'transfer' lists that differs in some systematic way from the repeated Hebb list. Cumming, Page and Norris (in press) used transfer lists where alternate list items were identical to the items used in the repeated Hebb lists, while the order of the remaining items was rearranged (e.g. 542716983 - > 643792581, where items in even numbered positions remain unchanged). Contrary to the predictions of theories depending on associations between items and their positions (e.g. Burgess and Hitch, 1992,1999; Brown, Preece and Hulme, 2000), there was no benefit to the transfer lists over control lists consisting of a completely reordered set of items. Although it has been suggested that one of the advantages of positional models is that the build up of position-item associations over time could account for the Hebb effect, these results show that the Hebb effect is not due to position-item associations. Also, Cumming et al. presented simulations using the Burgess and Hitch model to show that it did not simulate the basic Hebb effect after all.
L3.2 Effects of irrelevant speech on STM
Another project has investigated the effects of irrelevant speech on short-term retention of visually presented lists. When participants in an ISR task are presented with speech that they do not have to attend to, their memory performance deteriorates (Colle and Welsh, 1976; Salame and Baddeley, 1982, 1986). Several theories of STM make quite specific claims about the underlying mechanism of the irrelevant speech effect. For example, the Object-Oriented Episodic Record (OOER) theory of Jones (1993) predicts that irrelevant speech can only effect performance if is occurs during rehearsal. Nairne's (1990) feature theory has to predict that the irrelevant speech must be presented at the same time as the stimulus list. In a series of studies (Norris, Page and Baddeley, submitted) we have shown that irrelevant speech still impairs performance even when presented after the end of the stimulus list, during an interval in which subjects are prevented from rehearsal by shadowing digits presented at a rate of one every 500ms. Furthermore, there is no significant retroactive irrelevant speech if subjects perform articulatory suppression during list presentation so as to prevent the visual material being recoded into the phonological loop. The effect therefore appears to depend specifically on information being retained in the loop. The complete pattern of data is inconsistent with both Nairne's feature model, and Jones OOER model, but is consistent with the Working Memory model and, by implication, computational models such and the Primacy model and Burgess and Hitch's position-item model.
Note that all of the computational modelling reported here, and the modelling of spoken word recognition in Shortlist and Merge (L1.3.2) uses localist connectionist models. Page (2000a, 2000b) has recently published an influential Behavioral and Brain Sciences article comparing the merits of localist and distributed representations. In that paper he presents a strong case that localist models are to be preferred over distributed models, both for theoretical reasons, and for their greater consistency with the neurobiological data.
A further series of experiments, still in progress has been investigating several somewhat anomalous results in the STM literature that do not fit neatly into the standard Working Memory model. For example, there are suggestions that the phonological similarity effect is much more long lasting with auditory than visual presentation (Longoni, Richardson, & Aiello, 1993). This would suggest that the similarity effect is not simply dependent on a phonological store that can be driven by visual or auditory input, but also has an acoustic/phonetic component too. We have confirmed and extended this result in more carefully controlled studies comparing the time-course of the phonological similarity effect with visual or auditory presentation. Other work has investigated the question of whether articulatory suppression really does prevent phonological recoding of visually presented material, as assumed in the Working Memory framework. Although suppression consistently eliminates the phonological similarity effect for visual, but not auditory, lists, subjects can nevertheless judge whether two visually presented words are homophones, even when suppressing. This implies that they do have access to phonological representations. We have combined the serial recall task and a homophone judgement task by requiring subjects to detect homophones in a list of words which they have to recall in order. When the homophones appear in adjacent list positions, subjects can perform this task reliably, even when suppressing. Performance declines rapidly when one or two items intervene between the homophones, This suggests that some access to phonological representations is possible in a serial recall task, but this information is only available very briefly. Further work is investigating whether forcing subjects to attend to phonology, by requiring them to perform a homophone detection task, can modulate the phonological similarity effect, both with and without suppression.
In summary, the theoretical work reported here has resulted in the development of two new computational models of short-term memory while the empirical work has generated data that places considerable constraints on the form that any successful model of STM must take. Data from Henson et al (1996) allows us to eliminate a whole class of theories based on associative chaining. Cumming et al (in press) presented data inconsistent with position-item theories. Finally, data on irrelevant speech reported in Norris et al (submitted) poses considerable problems for the feature integration theory (Nairne, 1990; Neath, 2000) and the OOER theory of Jones (1993).
Project L4: Neurophysiology of Language
Scientific Direction: Pulvermüller (90%)
MRC-scientist support: Shtyrov (90%), Hauk (90%)
Student: Harris (100%)
The MRC Cognition and Brain Sciences Unit's project L4 on the neurophysiology of language started with the appointment of Friedemann Pulvermüller in summer 2000, and subsequent appointments of Olaf Hauk and Yury Shtyrov later in the same year. In January 2001, a new 64-channel EEG system was up and running in the newly converted CBU EEG laboratory. Our research currently focuses on methods with high temporal resolution, in particular magnetoencephalography (MEG) and electroencephalography (EEG), to reveal the exact time-course of cortical activity when words and sentences are being processed. Time course information is particularly important in the investigation of language, because language comprehension is a dynamic process in which various types of information (phonological, syntactic, semantic etc.) are being integrated within a fraction of a second. Compared with fMRI, the EEG and MEG methods allow for less precise conclusions on where in the brain activity arises. The future strategy is therefore to complement the exact temporal information provided by EEG and MEG results with data obtained with techniques allowing for precise localization, in particular functional Magnetic Resonance Tomography (fMRI) and Transcranial Magnetic Stimulation (TMS).
The research reported here is framed within a neural model of language function that uses established neuroscientific principles to model language-related spatio-temporal patterns of brain activity. The model aims to spell out phonemes, words, sentences, and their meaning in terms of concrete neuron circuits, taking as a starting-point the concept of Hebbian cell assemblies. These are neuronal networks that form as a consequence of correlation learning in a biological associative memory, the cortex (Pulvermüller, 1999). From a clinical perspective, we hope to convert knowledge about the neurobiological basis of language into concrete proposals for clinical practice, with the aim is of developing and testing scientifically-based treatment approaches for acquired language deficits following brain lesion.
L4.1 Neurophysiological activity reflecting word-category-specific processes
A prediction of a neurobiological model of word processing in the brain is that words referring to actions are cortically realized by distributed neuronal ensembles that include neurons related to the execution of the relevant motor programs (Pulvermüller, 2001). A word such as "(to) kick" should be realized not only by neurons housed in the perisylvian language areas related to the articulation and acoustic perception of the word, but its neuronal representation should incorporate additional neurons involved in the coordination of the movements involved in actual kicking. This is a necessary postulate for any neurobiological language model that takes seriously the principle of association learning. If neurons that frequently fire together also wire together by strengthening their synaptic connections, the disjoint neuronal populations related to the word form and the referent actions should link into a strongly connected neuronal group when word and action frequently occur at the same time or in close temporal succession. The argument made for the leg-related word "(to) kick" can be extended to other action words, for examples hand/arm-related words, such as "(to) pick", and face-related words, such as "(to) lick". These considerations suggest that action words of different types are cortically realized as cell assemblies with different cortical distributions. It is well-known that the motor and premotor cortices in the frontal lobe are organized topographically with the leg representation located dorsal to the arm representation, which, in turn, is dorsal to the representation of the face (He, Dum, & Strick, 1993; Penfield & Rasmussen, 1950). Therefore, the word-related neuron ensembles should be differentially distributed over fronto-central areas (Pulvermüller, 2001). Allthough all word networks would include neurons in the perisylvian language areas, the leg-related words would accordingly be realized as networks that include additional neurons in dorso-medial primary motor and pre-motor areas, hand/arm-word representations would include more inferior action-related neurons, and face word representations neurons in inferior fronto-central areas. The referent actions of action words would be woven into the word-related neuronal ensemble (Pulvermüller, 2001).
According to classical neurological models of language, words are processed in two restricted language centers in the dominant hemisphere, usually the left in right-handers. Modern brain imaging studies, however, showed that various other areas can also become active during word processing. Retrieving word meaning, for example, certainly involves additional areas, although there is, at present, much discussion about their exact definition (for an overview, see Pulvermüller, 1999). Category-specific patterns of brain activity have been found in numerous neurophysiological and metabolic imaging studies (for an overview, see Pulvermüller, 2001), although not all studies have observed category differences over varying tasks and stimulus sets (Devlin et al., 2002). In our recent neurophysiological studies, we found consistently that well-matched words from different categories (function vs. content words, nouns vs. verbs, visually-associated vs. action associated nouns) led to distinct patterns of cortical activation, and that the neurophysiological differences are best explained by semantic differences between word types (Pulvermüller, Assadollahi, & Elbert, 2001; Pulvermüller, Härle, & Hummel, 2000; Pulvermüller, Lutzenberger, & Preissl, 1999; Pulvermüller, Mohr, & Schleichert, 1999).
L4.1.1 Behavioral and EEG studies of action semantics
In behavioral and neurophysiological experiments, we tested the predictions, outlined above, arising from a Hebbian model of word processing. Because the predictions relate associations at the cognitive level (for example, whether a word refers to, and reminds one of, actions performed with the leg) to neurophysiological events in the brain, it was necessary to establish the cognitive properties of the stimulus words used in the neuroimaging studies. This is only possible in behavioural experiments. We asked subjects to judge words in English and in German (in separate studies) with regard to their action associations. The ratings allowed us to pick groups of words primarily associated with one of the three body parts under examination, face, arm/hand, and leg/foot. The words were matched for physical and psycholinguistic variables, such as length, familiarity, word frequency, and imageability. This led to well-matched stimulus groups, which significantly differed in their action associations (face, arm, and leg words). These were used in subsequent imaging studies.
In neurophysiological studies, the EEG was recorded while subjects made lexical decision responses to words shown briefly at fixation. Response times were slightly slower for leg words as compared with the other two word categories. Importantly, event-related brain potentials reliably differed between subcategories of action words. As Current Source Density Analyses (CSDA) indicated, leg words elicited stronger activity (as indicated by stronger outward flow of electrons) at the top of the head, near the cortical representation of the leg, as compared with face-related words. Face words, in contrast, produced stronger activity signs at left-anterior sites, close to the cortical representation of the face. Although these data do not allow for an exact localization of the cortical sources of the word-category-related differences in brain activation, the results argue that subcategories of action verbs related to actions performed with different body parts are neurophysiologically distinct (Pulvermüller, Hummel, & Härle, 2001).
In the earlier experiment, subjects had to respond overtly to the stimulus words. Since differences in response times were found between action word subcategories, differential motor preparation is a possible confound. Therefore, we are investigating putative neurophysiological differences between leg- and face-related words with a different set of stimuli and in a silent reading task in which no motor response is required. To obtain a better estimate of the cortical locus of the word category differences in brain activation, Olaf Hauk calculated Minimum Norm Estimates (MNE). The MNE is a method for solving the so-called Inverse Problem (von Helmholtz, 1853) which selects the unique solution that explains the scalp topography by the least amount of overall current (Hämäläinen, Hari, Ilmoniemi, Knuutila, & Lounasmaa, 1993). Our preliminary MNE of the cortical generators indicate that a central dorsal area including the primary motor and premotor areas representing the leg were more strongly activated for leg words as compared with other word groups, and that an inferior frontal area was most active for face words (Hauk & Pulvermüller, 2002). We take these results as evidence that areas in the frontal lobes are differentially activated by words referring to actions performed with different body parts. The plan is to continue this line of research in the future proposal, project SL4.1.
L4.1.2 When is word meaning reflected in the brain response?
The great strength of neurophysiological data is the exact localization of the effects in time. Time-wise, our results were surprising, because several experiments unequivocally demonstrate that word category differences were present early in the brain response. The earliest meaning-related differences (to frequently repeated word stimuli presented in a memory task) were already observed 100 ms after the onset of visual word stimuli (Pulvermüller, Assadollahi et al., 2001). In this MEG study carried out with Ramin Assadollahi and Thomas Elbert at the University of Konstanz, we found a high correlation between the magnitude of an early magnetic brain wave (latency ~100 ms) elicited by matched written words and our subjects' ratings of the strength with which individual words reminded them of referent perceptions and actions. This suggests that meaning-related aspects of a word are reflected in the brain response already 100 ms after the information in the input allows for word identification.
While some studies suggested a very early onset of the neurophysiological differences between word categories, the latest word category differences we saw in our series of experiments were present at ~200 ms after visual word onset (Pulvermüller, Hummel et al., 2001). Clearly, future research is required to exactly define the point in time where specific brain responses possibly related to the meaning of a word arise. Some of the variance may be accounted for by physical word properties, such as the length of written words and their sound structure and recognition point (Marslen-Wilson & Tyler, 1980) of spoken words (see also L2.3 and L4.3).
The fact that the brain distinguishes early between word categories, at ~100-200 ms, after delivery of the information necessary for word identification, may be important for theories of language in the brain. It is consistent with proposals based on psycholinguistic reaction time experiments according to which access to the lexical and semantic information occur early in word processing (Marslen-Wilson & Tyler, 1980). Our findings speak against the view that the meaning of words is accessed only at around 400 ms after the critical information is presented (e.g, Friederici, 2002). Instead, they suggest that meaning access is an early neurophysiological process occurring near-simultaneously with the access to lexical and phonological information. To further clarify these issues, we plan to look more closely at the specific brain activity patterns elicited by different kinds of action words. With the use of advanced neurophysiological source localization techniques (see MR3.2-4), it should become possible to separate the perisylvian phonologically- and lexically-related activity patterns from those in dorsal motor and premotor areas so that conclusions about the spatio-temporal pattern of activation of phonological and semantic activity will become possible.
L4.1.3 Methodological issues in the investigation of word-evoked brain activity
To draw conclusions about the neurophysiological reflections of word semantics (see L4.1 and SL4.1), it is important to study other properties of words that become manifest in the brain response and which therefore can confound the results of EEG/MEG activity related to word processing. Further, as noted, there is the open question of why some of the word category differences surfaced quite early, whereas in other research, they appeared with longer delays (for an overview, see Assadollahi & Pulvermüller, 2001). In a series of studies, we investigated the influence of the factors of word length and word frequency on the EEG and MEG response. In contrast to earlier studies, we kept the variance in length and frequency of our stimuli minimal. This is important methodologically, because variability in these stimulus properties adds noise to the early ERP responses (P70, N100), which are known to be short-lasting and focal, so that potential early physiological category differences may be masked (for discussion, see Pulvermüller, 1999).
A main finding of our studies (Assadollahi & Pulvermüller, 2001; Hauk & Pulvermüller, submitted) is that word length effects became manifest already around 100 ms after word presentation onset, whereas word frequency modulates the neurophysiological brain response slightly later, at ~150 ms. This makes it clear that early neurophysiological differences between words can be related to word length or frequency. However, the differences between long and short and between common and unusual words were always widespread, making it unlikely that topographically specific differences between subcategories of action words (L4.1.1) are a by-product of differences in length or frequency. Nevertheless, due to the modulation of neurophysiological activity related to word length and frequency, it appears advisable in neurophysiological studies of word-category processes to exactly match the word material for these variables. In contrast to earlier studies (King & Kutas, 1998), we could not replicate a correlation of word frequency with the latency of components of the word-evoked neurophysiological response.
In the context of the debate about latencies of word category effects, we note that long words usually elicited relatively large early neurophysiological responses, whereas short words elicited relatively large responses at a later point in time. This indicates that the mixing of long and short words can account for why some investigations found early word-category differences (100-200 ms; Sereno, Rayner & Posner, 1997) and others only found them later (~400 ms; Polich & Donchin, 1987). A similar confound may occur for the variable word frequency. Some of the latency variance found in earlier studies on category-specific word-evoked activity may be accounted for in terms of physical and psycholingusitic stimulus properties.
L4.2 Specific word category deficits after lesions in the right hemisphere
We have proposed that language units are cortically processed by distributed cell assemblies including neurons in both hemispheres, transcortical cell assemblies (TCA, Mohr et al., 1994; Pulvermüller & Mohr, 1996; see also L4.4). These neurons may be left-lateralized in most individuals in the sense that they include more left- than right-hemispheric neurons, but the networks crucial for processing a word would still bridge the midline. The claim is based on associative memory theory and the fact that the perception of a spoken language unit, for example a word, always leads to correlated activity in both cortical hemispheres. To test the TCA model, we can look at the effect of focal lesions in the hemisphere not dominant for language.The TCA model suggests that a right hemispheric lesion can lead to similar, although less pronounced, category-specific deficits to those reported earlier following lesions to the left dominant hemisphere.
Category-specific deficits arising from lesions in the dominant hemisphere are well-known (Warrington & McCarthy, 1983; Warrington & Shallice, 1984). Lesions in the left temporal lobe are sometimes associated with selective difficulties in processing particular word categories, e.g. animal names, whereas left frontal lesions in many cases impair the processing of action verbs (Daniele et al., 1994). Together with Bettina Neininger, doctorate student at the University of Konstanz, Germany, we investigated whether focal lesions in the right non-dominant hemisphere can also lead to selective difficulties in processing particular types of words.
No overt language dysfunction was found in our population of 18 patients with focal ischemic lesions primarily affecting the right frontal or temporal lobes. Consistent with the model, however, a sensitive psycholinguistic test, a lexical decision task, revealed a double dissociation between word groups. Processing of action verbs was imparied after right-frontal lesions, whereas the processing of nouns primarily characterized by visual associations was degraded after lesion in right temporo-occipital areas (Neininger & Pulvermüller, in press). Neurological control patients did not show similar deficits, nor was there any impairment of the processing of nouns characterized by both strong action and visual associations. It was striking that that small focal lesions, for example in primary and premotor cortex, could lead to a marked behavioral impairment that showed up on the lexical decision task (Neininger & Pulvermüller, 2001). This supports the idea that the neuron networks necessary for word processing have a right-hemispheric component as well.
We plan to follow up on this work by searching for other word-category deficits accompanying specific focal right-hemispheric lesions. In this context, we will use TMS to cause temporary modification of the function of different right-hemispheric areas. The effect of such temporary functional modulation will be looked at with psycholinguistic experiments and the results will be used to further evaluate the TCA model (see future proposal SL4.2).
L4.3 The Mismatch Negativity (MMN) as a tool for investigating language in the brain (Friedemann Pulvermüller and Yury Shtyrov, with Risto Ilmoniemi and Risto Näätänen, University of Helsinki)
The neural systems underlying language function are usually explored in the context of tasks where language stimuli are presented and subjects have to focus their attention on these stimuli and perform a linguistic task. Neurobiological models of word processing and language would suggest that strongly linked networks of neurons are built up that specialize in the detection (and production) of words, or sequences of words, and that the strong neuronal connections defining these neuron ensembles lead to spreading of neuronal activity in the network. This should take place regardless of whether the brain is in an attentive or non-attentive state, and regardless of whether attention is directed toward the language input or some other input. A series of experiments has been performed to explore brain processes triggered by language stimuli when these are not the focus of the subjects' attention.
When words are being presented visually, there is, on the one hand, evidence for implicit word processing (Price, Wise, & Frackowiak, 1996), but, on the other, inattentional blindness for words (Rees, Russell, Frith, & Driver, 1999) has been reported when subjects were instructed to engage in a distracting task. Since, from both ontogenetic and a phylogenetic perspectives spoken language is more basic than written language, we used spoken words and word sequences for our investigations.
The Mismatch Negativity (MMN), a neurophysiological index of the detection of a change in the acoustic environment that can be elicited in the absence of focussed attention (Näätänen, 2001) was recorded to spoken CV syllables. Subjects were instructed to watch a simultaneously presented silent video film and ignore the acoustic stimuli. The critical syllables followed context syllables with which they formed either words or meaningless pseudo-words. The MMN was significantly altered by the context. The same syllable elicited larger MMNs when it terminated a real Finnish word than when it was presented in pseudo-word context (Pulvermüller, Kujala et al., 2001). Further experiments ruled out the possibility that the MMN enhancement to words is due to bi- or trigram frequencies of the phonemes making up the words (Pulvermüller, Kujala et al., 2001). We call this enhancement of the MMN in word context relative to pseudo-word context the lexical enhancement of the MMN (see Figure L3 below). These results show that word-related brain responses can be elicited when subjects do not focus their attention on the language input. The activation of the memory traces for words does not appear to require that subjects engage in a linguistic task.
Figure L3: Words elicited larger magnetic MMNs than pseudowords (left side). This lexical enhancement effect was present at ~150 ms after the stimulus information necessary for word recognition. The locus of the main cortical source, as revealed by calculating the Equivalent Current Dipole, was found in the left superior temporal cortex. Source locus did not distinguish between words and pseudowords. From Pulvermüller, Kujala et al., 2001).
.jpg)
The lexical enhancement effect was originally obtained with Finnish stimuli, and we have since replicated this with English stimuli (Shtyrov & Pulvermüller, 2002b). In a study using monosyllabic English words, deviant and standard CVC stimuli were distinguished by their final phoneme. Word-word and word-pseudoword pairs were included, as in pairs like type-tight and pipe-pite. An item ending in a [t] was always used as the frequent standard stimulus, whereas an item terminating with a [p] was the infrequent deviant stimulus that elicits the MMN. Physical differences between stimuli were minimized by cross-splicing word-initial CV syllables and word-final phoneme sounds so that all standard stimuli shared their final sound and the same was true for the deviants. The unexpected word-final phoneme elicited a larger MMN than the same phonetic signal terminating a CVC pseudoword, thereby replicating a lexical enhancement effect for the English words. An additional result was that the enhancement effect was dependent on the lexical status of the deviant stimulus but not of the standard stimulus (Shtyrov & Pulvermüller, 2002b).
In more recent research (Pulvermüller, Shtyrov, Ilmoniemi, & Marslen-Wilson, submitted) we have even more direct evidence for the sensitivity of the MMN to linguistically and cognitively relevant processing events. The concept of "word-recognition point" is well established in the Cohort Model of Marslen-Wilson, and claims that spoken words can be identified by the listener as soon as the speech input diverges from other possible words in the language. Thus, for example, the fragment "crocod…" might be sufficient to identify the word crocodile, and the recognition point would be set in the region of the /d/. In experiments run in Finnish, and again using the MEG laboratory in Helsinki, we were able to measure the magnetic Mismatch Negativity (MMN) elicited by spoken words, for pairs of words like tuon and tuot. These are forms of the verb tuo 'bring', with either the inflectional ending [-n], meaning "I bring" or with the ending [-t], meaning "you bring". The participants were independently tested to determine their individual recognition points for these pairs, with recognition point being consistently slower for tuot. In the MMN experiment itself, the two words alternated as standard and deviant across conditions.
The results were striking, with the latency and detailed timing of the MMN responses primarily determined by word recognition points (see Figure L4 below). Both ECD (Equivalent Current Dipole) and MCE (Minimum-Norm Current Estimate) calculations of the spatio-temporal properties of the MMN responses showed (a) that these peaks occurred about 100-150 ms after the information in the acoustic input was sufficient for word recognition, and (b) that the peak was significantly delayed for the word (tuot) with a later recognition point. The neural generators for these responses were located in the left superior temporal cortex, which is known to play a role in mapping sound onto lexical meaning. Even more telling was the further result that the recognition-points computed in the gating task for the individual participants correlated significantly (r = 0.66) with the latency of these activity peaks in superior temporal cortex. We conclude that the latency of the magnetic MMN elicited by spoken words reflects an early brain process that underlies the recognition of individual lexical items in individual subjects.
Figure L4. Minimum Current Estimates of cortical sources activated during the presentation of the spoken Finnish words "tuon" and "tuot". Gating experiments revealed average word recognition points for "tuon" at 350 ms and for tuot at 420 ms after word onset. Starting at ~100 ms after the respective recognition points, cortical activation was observed in the left superior temporal lobe, correlated in time with individual subject's recognition points.
.jpg)
This is not only an important result in itself, but also it validates the use of the MMN to probe the higher-level structure of language processing mechanisms in the brain. In future research, we plan to exploit this vigorously, using the MMN to examine more generally the spatio-temporal patterns of cortical activation invoked by linguistic material (see SL3 and SL4 in the future proposal).
L4.4 Constraint-induced aphasia therapy
Scientific development needs to contribute to clinical practice. This project addresses a specific clinical domain where neuroscientific theories of language can potentially become useful, namely aphasia therapy. On this basis we have developed and tested a new aphasia therapy method, called Constraint-Induced Aphasia (CIA) therapy. The principles underlying CIA therapy are (i) to provoke a high correlation of neuronal activity patterns by using a massed practice regime, (ii) to stimulate distributed cortical systems related to the processing of words and their meaning by providing behaviorally relevant multimodal input that mimicks communication in everyday life, and (iii) to prevent the learned nonuse of cortical functions that is frequently observed in individuals suffering from stroke by introducing communication contraints (Taub, Uswatte & Elbert, 2002).
In a randomized and controlled study, we could show that Constraint-Induced Aphasia therapy leads to significant improvements of language performance in chronic aphasics over a period of 10 days of intense treatment (Pulvermüller, Neininger et al., 2001). Comparable improvement was absent in a control group receiving the same amount of conventional language treatment. Changes of language-related brain activation in the course of language therapy were documented using event-related potentials.
We plan to extend this research by looking at the plastic changes related to Constraint-Induced Aphasia therapy using both EEG and fMRI methods. This research will be carried out in collaboration with Bettina Mohr at Anglia Polytechnic University and Elizabeth Warburton and Jean-Claude Baron at the Stroke Unit of Addenbrooke's Hospital Cambridge (see SL4.5).
L4.5 Further research
In a long-lasting collaboration, Bettina Mohr and Friedemann Pulvermüller are investigating the effect of redundant stimulus information on the speed and accuracy of cognitive processes. The redundancy gain, the improvement of processing with redundant information, has been shown to be particularly strong for stimuli that have been learned (words, known faces, common objects), whereas it is reduced or absent for uncommon material for which no cortical representation can be assumed (pseudo-words, unknown faces, non-objects) (Mohr, Pulvermüller, & Zaidel, 1994). The redundancy gain to words was also found to be absent in patients with commissurotomy (Mohr, Pulvermüller, Rayman, & Zaidel, 1994), and recently, we observed that absence of the redundancy gain characterizes one other syndrome for which a dysfunction in interhemispheric interaction has been suggested, namely schizophrenia (Mohr, Heim, Pulvermüller, & Rockstroh, 2001; Mohr, Pulvermüller, Cohen, & Rockstroh, 2000). In a series of studies with healthy subjects, the influence of varying stimulus onset asynchronies on the redundant target effect for words was investigated. The redundancy gain vanished at minimal SOAs (50 ms), but an additional word-specific facilitation effect was found at an SOA of ~150 ms (Mohr & Pulvermüller, 2002). We interpret these results in terms of the activity dynamics of cortical representations.
In a collaboration with Christian Dobel at the Max Planck Institute of Psycholinguistics in Nijmegen, we looked at cortical laterality of neurophysiological processes while aphasics and normal control subjects performed syntactic and semantic tasks. A recent result was that there is strong right-lateralization of brain activity during the task aphasics have most problems with (the syntactic task in our study) whereas laterality of neurophysiological responses to the left lesioned hemisphere was seen during the task where performance was relatively intact (the semantic task) (Dobel et al., 2001). These results support theories of language recovery after stroke postulating that language-related activation of the right hemisphere is not an effective compensatory strategy (Heiss, Kessler, Thiel, Ghaemi, & Karbe, 1999). We have also looked at neurophysiological changes in normal subjects that can be induced by learning and lead to an alteration of word processing (Pulvermüller, Mohr, Schleichert & Veit, 2000).
The collaborations undertaken at the CBU with Robert Carlyon, where the neurophysiological correlates of the continuity illusion were investigated, and with William Marslen-Wilson, where Mismatch Negativity indicators of morphological processing have begun to be investigated, are specified in the project descriptions A3 and L2.2.5, respectively.
AWARDS AND HONOURS
Ingrid Johnsrude was elected to a Non-stipendiary Research Fellowship of Clare Hall College, Cambridge in October 2002. Friedemann Pulvermüller elected to membership of the Rodin Remediation Academy, Stockholm, 2002. William Marslen-Wilson has held a visiting Professorship in the Department of Psychology, Birkbeck College since July 1997, and was installed as a Fellow of the College in March 2000. He was the Wei Lun Visiting Professor at the Chinese University of Hong Kong in April 2000, and gave the annual Drever Lecture at the University of Edinburgh in April 2002. He was elected to an Honorary Professorship of Language and Cognition at the University of Cambridge in September 2002.
PUBLICATIONS
Refereed Journals
Assadollahi, R. & PULVERMÜLLER, F. (2001). Neuromagnetic evidence for early access to cognitive representations. Neuroreport, 12, 207-213.
BISHOP, D.V.M. (1998). Development of the children's communication checklist (CCC): a method for assessing qualitative aspects of communicative impairment in children. Journal of Child Psychology and Psychiatry 39, 879-891.
BISHOP, D. V. M. & BISHOP, S. J. (1998). "Twin language": a risk factor for language impairment? Journal of Speech, Language, and Hearing Research, 41, 150-160.
BISHOP, D.V.M., Bright, P., James, C., BISHOP, S.J. & Van Der Lely, H.K.J. (2000). Grammatical SLI: A distinct subtype of developmental language impairment? Applied Psycholinguistics, 21, 159-18.
BISHOP, D.V.M., CARLYON, R.P., DEEKS, J.M. & BISHOP, S.J. (1999). Auditory temporal processing impairment: neither necessary nor sufficient for causing language impairment in children. Journal of Speech Language and Hearing Research, 42, 1295-1310.
BISHOP, D.V.M., Chan, J., Hartley, J. & Weir, F. (1998). When a nod is as good as a word: form-function relationships between questions and their responses. Applied Psycholinguistics, 19, 415-431.
BOUDELAA, S. & Gaskell, G.M. (2002). A re-examination of the default system for Arabic plurals, Language and Cognitive Processes, 17, 321-343.
BOUDELAA, S. & MARSLEN-WILSON, W.D. (2001). Morphological units in the Arabic mental lexicon. Cognition, 81, 65-92.
BOUDELAA, S., & MARSLEN-WILSON, W.D. (submitted). Abstract morphemes and lexical representation: The CV-Skeleton in Arabic.
BOUDELAA, S., & MARSLEN-WILSON, W.D. (submitted) Morphology, structure, and lexical processing in Modern Standard Arabic
BRETT, M., JOHNSRUDE, I.S & OWEN, A.M. (2002). The problem of functional localization in the human brain. Nature Reviews Neuroscience, 3, 243-249.
Calvert, G.A. & BISHOP, D.V.M. (1998). Quantifying hand preference using a behavioural continuum. Laterality, 3, 255-268.
CARLYON, R.P., DEEKS, J.M., NORRIS, D.G. & BUTTERFIELD, S. (2002). The continuity illusion and vowel identification. Acta Acustica united with Acustica, 88, 406-415.
COX, S.M.L., Stefanova, E., JOHNSRUDE, I.S., Robbins, T.W. & OWEN, A.M. (2002). Preference formation and working memory in Parkinson's disease and normal ageing. Neuropsychologia, 40, 317-326.
CUMMING, N., Page, M.P.A. & NORRIS, D.G. (in press). Testing a positional model of the Hebb effect. Memory.
Curran, E., Sykacek, P., Stokes, M., Roberts, S., Penney, W., JOHNSRUDE, I. & OWEN, A.. (Submitted). Cognitive tasks for driving a brain computer interfacing system: a pilot study.
Cutler, A. & NORRIS, D.G. (1999). Sharpening Ockham's Razor. Behavioural and Brain Sciences, 22, 40-41.
Dale, P.S., Simonoff, E., BISHOP, D.V.M., Eley, T.C., Oliver, B., Price, T.S., Purcell, S., Stevenson, J. & Plomin, R. (1998). Genetic influence on language delay in two-year-old children. Nature Neuroscience, 1, 324-328.
DAVIS, M.H. & JOHNSRUDE, I.S. (submitted). Hierarchical processes in spoken language comphrehension.
DAVIS, M.H., MARSLEN-WILSON, W.D. & Gaskell, M.G. (2002). Leading up the lexical garden-path: segmentation and ambiguity in spoken word recognition. Journal of Experimental Psychology: Human Perception and Performance, 28, 218-244.
DAVIS, M.H., Moss, H.E., Davies, P. & Tyler, L.K. (1999). Spot the difference: investigations of conceptual structure for living things and artifacts using speeded word and picture matching. Brain and Language, 69, 411-414.
DAVIS, M.H., Moss, H.E. & Tyler, L.K. (submitted). Spot the difference: testing the distinguishing features of concrete concepts using speeded word-picture matching.
DAVIS, M.H., VAN CASTEREN, M., MARSLEN-WILSON, W.D. (in press). Frequency effects in the processing of inflectional morphology: a distributed connectionist account. Journal of Linguistics.
Devlin, J.T., Russell, R.P., DAVIS, M.H., Price, C.J., Moss, H.E., Fadili, J. & Tyler, L.K. (2002). Is there an anatomical basis for category specificity? Semantic memory studies in PET and fMRI. Neuropsychologia, 40, 54-75.
Devlin, J.T., Russell, R.P., DAVIS, M.H., Price, C.J., Wilson, J., Moss, H.E., Matthews, P.M. & Tyler, L.K. (2000). Susceptibility induced loss of signal: comparing PET and fMRI on a semantic task. Neuroimage, 11, 589-600.
Dobel, C., PULVERMÜLLER, F., Härle, M., Cohen, R., Koebbel, P., Schoenle, P.W. & Rockstroh, B. (2001). Syntactic and semantic processing in the healthy and aphasic human brain. Experimental Brain Research, 140, 77-85.
Donlan, BISHOP, D.V.M., Hitch, G.J. (1998). Magnitude comparisons by children with specific language impairments: evidence of unimpaired symbolic procession. International Journal of Language and Communication, 33, 149-160.
Endrass, T., MOHR, B. & Rockstroth, B. (2002). Reduced interhemispheric transmission in schizophrenia patients: Evidence from event-related potentials. Neuroscience Letters, 320, 57-60.
Eulitz, E., HAUK, O. & Cohen, R. (2000). Electroencephalographic activity over temporal brain areas during phonological encoding in picture naming. Clinical Neurophysiology, 111, 2088-2097.
Fell, J., HAUK, O. & Hinrichs, H. (2000). Linear inverse filtering improves spatial separation of nonlinear brain dynamics: a simulation study. Journal of Neuroscience Methods, 98, 49-56.
FORD, M., MARSLEN-WILSON, W. D. & DAVIS, M. H. (in press). Morphology and frequency: contrasting methodologies. Journal of Linguistics.
Frost, R., Deutsch, A., Gilboa, O., Tannenbaum, M. & MARSLEN-WILSON, W.D. (2001). Morphological priming: dissociation of phonological, semantic, and morphological factors. Memory and Cognition, 28, 1277-1288.
GASKELL, M.G. (2000). Modeling lexical effects on semantic categorization and semantic effects on word recognition. Behavioral and Brain Sciences 23, 329-330.
GASKELL, M.G. & MARSLEN-WILSON, W.D. (1999). Ambiguity, competition and blending in spoken word recognition. Cognitive Science, 23, 439-462.
Gaskell, M.G. & MARSLEN-WILSON, W.D. (2001). Lexical ambiguity resolution and spoken word recognition: bridging the gap. Journal of Memory and Language, 44, 325-349.
Gaskell, M.G. & MARSLEN-WILSON, W.D. (2002). Representation and competition in the perception of spoken words. Cognitive Psychology, 45, 220-226.
Gaskell, M.G., Spinelli, E., & MEUNIER, F. (2002). Perception of resyllabification in French. Memory & Cognition, 30, 798-810.
Giraud, A-L, Lorenzi, C., Ashburner, J., Wable, J., JOHNSRUDE, I.S, Frackowiak, R., & Kleinschmidt, A. (2000). Representation of the temporal envelope of sounds in the human brain. Journal of Neurophysiology, 84, 1588-1598.
Gonçalves, M., Hall, D.A., JOHNSRUDE, I.S. & Haggard, M.P. (2001). Can meaningful effective connectivities be obtained between auditory cortical regions? NeuroImage, 14, 1353-1360.
Good, C.D., JOHNSRUDE, I.S., Ashburner, J., Henson, R.N.A., Friston, K.J. & Frackowiak, R.S.J. (2001). An optimised voxel-based morphometric study of ageing in 465 normal adult human brains. NeuroImage, 14, 21-36.
Griffiths, T., Uppenkamp, S., JOHNSRUDE, I.S., Josephs, O. & Patterson, R.D. (2001). Encoding of the temporal regularity of sound in the human brainstem. Nature Neuroscience, 4, 633-637.
Hall, D.A., Hart, H. & JOHNSRUDE, I.S (in press) Relationships between human auditory cortical structure and function (Review). Audiology and Neurootology.
Hall, D.A., JOHNSRUDE, I.S., Goncalves, M., Haggard, M.P., Palmer, A.R., Summerfield, A.Q., Akeroyd, M.A. & Frackowiak, R.S.J. (2002). Spectral and temporal processing in human auditory cortex. Cerebral Cortex, 12, 140-149.
HAUK, O., Keil, A., Elbert, T. & Mueller, M.M. (2002). Comparison of data transformation procedures to enhance topographical accuracy in time-series analysis of the human EEG. Journal of Neuroscience Methods, 113, 111-122.
HAUK, O. & PULVERMÜLLER, F. (submitted). Neurophysiological distinction of action words in the frontal lobe: an ERP study using minimum current estimates.
HAUK, O. & PULVERMÜLLER, F. (submitted). Effects of word length and frequency on the human ERP.
HAUK, O., Rockstroh, B. & Eulitz, C. (2001). Grapheme monitoring in picture naming: an electrophysiological study of language production. Brain Topography, 14, 3-13.
HENSON, R.N.A. (1998). Short-term memory for serial order: the start-end model. Cognitive Psychology, 36, 73-137.
HENSON, R.N.A. (1998). Item repetition in short-term memory: Ranschburg repeated. Journal of Experimental Psychology: Learning, Memory and Cognition, 24, 1162-1181.
HENSON, R.N.A. (1999). Positional information in short-term memory: Relative or absolute? Memory & Cognition 27, 915-927.
HENSON, R.N.A. (1999). Coding position in short-term memory. International Journal of Psychology, 34, 430-409.
HENSON, R.N.A., Nott, I.D., D.G., PAGE. M.P.A., BADDELEY, A.D. (1996). Unchained memory: Error patterns rule out chaining models of immediate serial recall. Quarterly Journal of Experimental Psychology, 49a, 80-115.
Herdman, C.M., Chernecki, D., & NORRIS, D.G. (1999). Naming cAsE aLtErNaTeD words. Memory & Cognition, 27, 254-266.
Hill, E.L., BISHOP, D.V.M., & NIMMO-SMITH, I. (1998). Representational gestures in Developmental Coordination Disorder and specific language impairment: Error-types and the reliability of ratings. Human Movement Science, 17, 655-678.
Johnson, E.K., Jusczyk, P.W., Cutler, E.A. & NORRIS, D.G. (in press). Lexical viability constraints on speech segmentation by infants. Cognitive Psychology.
JOHNSRUDE , I.S., Giraud, A.L. & Frackowiak, R.S.J. (2002). Functional imaging of the auditory system: the use of positron emission tomography. Journal of Audiology and Neuro-otology, 7, 251-276.
JOHNSRUDE, I.S., OWEN, A.M., White, N.M., Zhao, W.V. & Bohbot, V. (2000). Impaired preference conditioning after anterior temporal lobe resection in humans. Journal of Neuroscience, 20, 2649-2656.
JOHNSRUDE, I.S., Penhune, V.B. & Zatorre, R.J. (2000). Functional specificity in right human auditory cortex for perceiving pitch direction. Brain, 123, 155-163.
KNOTT, R.A. & MARSLEN-WILSON, W.D. (2001). Does the medial temporal lobe bind phonological memories? Journal of Cognitive Neuroscience, 13, 593-609.
KNOTT, R., PATTERSON, K., HODGES, J.R. (2000). The role of speech production in auditory- verbal short-term memory: evidence from progressive fluent aphasia. Neuropsychologia, 38, 125-142.
Kujala, A., Huotilainen, M., Uther, M., SHTYROV, Y., Monto, S., Ilmoniemi, R.J., & Naatanen R. (submitted). Development of cortical representations that underlie linguistic communication with non-speech sounds: evidence from learning Morse code.
Kujala,T., SHTYROV, Y., Winkler, I., Saher, M., Tervaniemi, M., Sallinen,M., Teder-Salejarvi, W., Alho, K., Reinikainen, K. & Naatanen, R. (submitted). Detrimental effects of long-term noise exposure on the central auditory process in the brain.
LONG, C.J., CARLYON, R.P., McKay, C.M., & Vanat, Z. (in press). Temporal pitch perception: examination of first-order intervals. International Journal of Audiology.
Maguire, E.A., Gadian, D.G., JOHNSRUDE, I.S., Good, C.D., Ashburner, J., Frackowiak, R.S.J. & Frith, C.D. (2000). Navigating induces structural change in the hippocampus of taxi drivers. Proceedings of the National Academy of Sciences in the USA, 97, 4398-4403.
MARSLEN-WILSON, W.D. (2000). What phonetic decision making does not tell us about lexical architecture. Behavioral & Brain Sciences, 23, 337-339.
MARSLEN-WILSON, W.D. (2001). Access to lexical representation: cross-linguistic issues. Language and Cognitive Processes, 16, 699-708.
MARSLEN-WILSON, W., & Tyler, L. K. (1998). Rules, representations, and the English past tense. Trends in Cognitive Sciences, 2, 428-435.
MARSLEN-WILSON, W.D. & Zhou, X. (1999). Abstractness, allomorphy, and lexical architecture. Language and Cognitive Processes, 14, 321-352.
McQueen, J., Cutler, A. & NORRIS, D.G. (1999). Lexical activation produces impotent phonemic percepts. Journal of the Acoustical Society of America, 106, 2296.
McQueen, J.M., Cutler, E.A. & NORRIS, D.G. (in press). Flow of information in the spoken word recognition system. Speech Communication.
McQueen, J., NORRIS, D.G. and Cutler, A. (1998). Lexical influence in phonetic decision making: Evidence from subcategorical mismatches. Journal of Experimental Psychology: Human Perception and Performance, 25, 1363-1389.
Meunier, F., & MARSLEN-WILSON, W.D. (in press) Regularity and irregularity in French verb inflection. Language and Cognitive Processes.
MEUNIER, F. & Segui, J. (1999). Morphological Priming Effect: The Role of Surface Frequency. Brain And Language, 68, 54-60.
MEUNIER, F. & Segui, J. (1999). Frequency Effects in Auditory Word Recognition: The case of suffixed words. Journal of Memory and Language, 41, 327-344.
Micheyl, C., CARLYON, R.P., SHTYROV, Y., HAUK, O., DODSON, T. & PULVERMÜLLER, F. (in press). Neurophysiological correlates of perceptual illusion: a mismatch negativity study. Journal of Cognitive Neuroscience.
Mohr, B., Heim, S., PULVERMÜLLER, F. & Rockstroh, B. (2001). Functional asymmetry in schizophrenic patients during auditory speech processing. Schizophrenia Research, 52, 69-78.
MOHR, B, Landgrebe, A. & Schweinberger, S. (2002). Interhemispheric cooperation for familiar but not unfamiliar face processing. Neuropsychologia, 40, 1841-1848.
MOHR, B., PULVERMÜLLER, F. (in press). Redundancy gains and costs in cognitive processing: Effects of short stimulus onset asynchronies. Journal of Experimental Psychology: Learning, Memory and Cognition.
Mohr, B., PULVERMÜLLER, F., Cohen, R. & Rockstroh, B. (2000). Interhemispheric cooperation during word processing: evidence for callosal dysfunction in schizophrenic patients. Schizophrenia Research, 46, 231-239.
Muller, V., Lutzenberger, W., PULVERMÜLLER, F. & Mohr, B. (2001). Investigation of brain dynamics in Parkinson's Disease by methods derived from nonlinear dynamics. Experimental Brain Research, 137, 103-110.
Neininger, B. & PULVERMÜLLER, F. (2001). The right hemisphere's role in action word processing: A double case study. Neurocase, 7, 303-317.
Neininger, B. & PULVERMÜLLER, F. (in press). Word category specific deficits after right-hemispheric lesions. Neuropsychologia.
NORRIS, D.G., Cutler, A,, McQueen, J.M., & BUTTERFIELD, S. (submitted). Lexical activation in speech comprehension.
NORRIS, D.G., Cutler, A., McQueen, J., BUTTERFIELD, S. & Kearns, R. (2001). Language universal constraints on the segmentation of English. Language and Cognitive Processes, 15, 637-660.
NORRIS, D.G., McQueen, J.M. & Cutler, A. (2000). Feedback on feedback on feedback. Behavioral & Brain Sciences, 23, 299-370.
NORRIS, D.G., McQueen, J. and Cutler, A. (2000). Merging information in speech recognition: Feedback is never necessary. Behavioural and Brain Sciences, 23, 299-370.
NORRIS, D.G., McQueen, J.M. & Cutler, E.A. (2002). Bias effects in facilitatory phonological priming. Memory and Cognition, 30, 399-411.
NORRIS, D.G., McQueen, J.M., & Cutler, A. (submitted). Perceptual learning in speech.
NORRIS, D.G., McQueen, J. M., Cutler, A., & BUTTERFIELD, S. (1997). The possible-word constraint in the segmentation of continuous speech. Cognitive Psychology, 34, 191-243.
NORRIS, D.G., Page, M.P.A. & Baddeley, A. (submitted) A retroactive effect of irrelevant speech.
OWEN, A.M., Menon, D.K., JOHNSRUDE, I.S., BOR, D., Scott, S.K., MANLY, T., Williams, E.J., Mummery, C. & Pickard, J.D. (in press). Detecting residual cognitive function in persistent vegetative state (PVS). Neurocase.
PAGE, M.P.A. (2000). Sticking to the manifesto: response to commentaries. Behavioral & Brain Sciences, 23, 496-512.
PAGE, M.P.A. (2000). Connectionist modelling in psychology: A localist manifesto. Behavioral and Brain Sciences, 23, 443-467.
PAGE, M.P.A. & NORRIS, D.G. (1998). The primacy model: a new model of immediate serial recall. Psychological Review, 105, 761-781.
Page, M. & NORRIS, D.G. (submitted). Irrelevant speech: What needs to be modelled, and how should we model it?
Palva, S., Palva, J.M., SHTYROV, Y., Kujala, T., Ilmoniemi, R.J., Kaila, K. & Naatanen, R. (2002). Distinct gamma-band evoked responses to speech and non-speech sounds in humans. The Journal of Neuroscience, 22 (RC211), 1-5.
Patterson, R. Uppenkamp S. JOHNSRUDE, I.S & Griffiths, T.D. (in press). The processing of temporal pitch and melody information in auditory cortex. Neuron.
PULVERMÜLLER, F. (2000). Syntactic circuits: How does the brain create serial order in sentences? Brain and Language, 71, 194-199.
PULVERMÜLLER, F. (2001). Brain reflections of words and their meaning. Trends in Cognitive Sciences, 5, 517-524.
PULVERMÜLLER, F. (2001). Mutual access and mutual dependence of conceptual components (Comment on Humphreys and Forde). Behavioral and Brain Sciences, 24, 490-492.
PULVERMÜLLER, F. (2002). A brain perspective on language mechanisms: from discrete neuronal ensembles to serial order. Progress in Neurobiology, 67, 85-111.
PULVERMÜLLER, F., Assadollahi, R. & Elbert, T. (2001). Neuromagnetic evidence for early semantic access in word recognition. European Journal of Neuroscience, 13, 201-205.
PULVERMÜLLER, F., Ehlert, C. & Elbert, T. (submitted). Evidence for the relevance of the right frontal lobe for action verb processing.
PULVERMÜLLER, F., Genkinger, B., Elbert, T., MOHR, B, Rockstroth, B., Koebbel, P. & Taub, E. (2001). Constraint-induced therapy of chronic aphasia following stroke. Stroke, 32, 1621-1626.
PULVERMÜLLER, F., Härle, M. & Hummel, F. (2000). Neurophysiological distinction of semantic verb categories. Neuroreport, 11, 2789-2793.
PULVERMÜLLER, F., Härle, M. & Hummel, F. (2001). Walking or talking? Behavioral and electrophysiological correlates of action verb processing. Brain and Language, 78, 143-168.
PULVERMÜLLER, F., Kujala, T., SHTYROV, Y., Simola, J., Tiitinen, H., Alku, P., Alho, K., Martinkauppi, S., Ilmoniemi, R.I. & Naatanen, R. (2001). Memory traces for words as revealed by the mismatch negativity (MMN). NeuroImage, 14, 606-616.
PULVERMÜLLER, F., Mohr, B., Schleichert, H. & Veit, R. (2000). Operant conditioning of left-hemispheric slow cortical potentials and its effect on word processing. Biological Psychology, 53, 177-215.
PULVERMÜLLER, F., Neininger, B., Elbert, T.E., Mohr, B., Rockstroh, B., Koebbel, P. & Taub, E. (2001). Constraint-induced therapy of chronic aphasia after stroke. Stroke, 32, 1621-1626.
PULVERMÜLLER, F., SHTYROV, Y., Kujala, T. & Näätänen, R. (submitted). Word-specific cortical activation in the human brain.
PULVERMÜLLER, F., SHTYROV, Y., Ilmoniemi, R., & MARSLEN-WILSON, W.D. (submitted) Tracking speech comprehension in space and time.
Rastle, K. & DAVIS, M.H. (2002). On the complexities of measuring naming. Journal of Experimental Psychology: Human Perception and Performance, 28, 307-314.
Rastle, K., DAVIS, M.H., MARSLEN-WILSON, W. & Tyler, L.K. (2000). Morphological and semantic effects in visual word recognition: A time course study. Language and Cognitive Processes, 15, 507-537.
REID, A.A., & MARSLEN-WILSON, W.D. (in press) Lexical representation of morphologically complex words: Evidence from Polish. Journal of Linguistics.
RODD, J., Gaskell, M.G. & MARSLEN-WILSON, W.D. (2002). Making sense out of ambiguity: semantic competition in lexical access. Journal of Memory and Language, 46, 245-266.
Rowe, J.B., OWEN, A.M., JOHNSRUDE, I.S. & Passingham, R.E. (2001). Imaging the components of a planning task. Neuropsychologia, 39, 315-327.
Schweinberger, S., Landgrebe, A. & MOHR, B. (2002). Personal names and the right hemisphere: an illusory link? Brain and Language, 80, 111-120.
Scott, S. K., JOHNSRUDE, I. S. (in press). The neuroanatomy of speech perception. Trends in Neurosciences.
SHTYROV, Y. & PULVERMÜLLER, F. (2002a). Memory traces for inflectional affixes as shown by the mismatch negativity. The European Journal of Neuroscience, 15, 1085-1091.
SHTYROV, Y. & PULVERMÜLLER, F. (2002b). Neurophysiological evidence of memory traces for words in the human brain. Neuroreport, 13, 521-525.
SHTYROV, Y., PULVERMÜLLER, F., Naatanen R. & Ilmoniemi R.J. (submitted). Early automatic syntactic processing in the human brain: a magnetoencelographic study.
Spinelli, E., GASKELL, G. & MEUNIER, F. (2000) Traitement du langage parlé: resyllabation, liaison et enchaînment. Actes des XXIIIemes Journées d'Étude sur la Parole. pp.193-196.
Stothard, S.E., Snowling, M.J., BISHOP, D.V.M., Chipchase, B.B. & Kaplan, C.A. (1998). Language-impaired preschoolers: a follow-up into adolescence. Journal of Speech, Language, and Hearing Research, 41, 407-418.
Tyler, L.K., Demornay-Davies, P., Anokinah, R., Longworth, C., Randall, B. & MARSLEN-WILSON, W.D. (2002). Dissociations in processing past tense morphology: neuropathology and behavioral studies. Journal of Cognitive Neuroscience, 14, 79-94.
Tyler, L.K., Randal, B., & MARSLEN-WILSON, W.D. (2002). Phonology and neuropsychology of the English past tense. Neuropsychologia, 40, 1154-1166.
Zhou, X. & MARSLEN-WILSON, W. (1999a). Phonology, orthography, and semantic activation in reading Chinese. Journal of Memory and Language, 41, 579-606.
Zhou, X. & MARSLEN-WILSON, W.D. (1999b). The nature of sublexical processing in reading Chinese Characters. Journal of Experimental Psychology: Learning, Memory and Cognition, 25, 819-837.
Zhou, X. & MARSLEN-WILSON, W.D. (2000a). The relative time course of semantic and phonological activation in reading Chinese. Journal of Experimental Psychology - Learning, Memory and Cognition, 26, 1245-1265.
Zhou, X. & MARSLEN-WILSON, W. (2000b). Lexical representation of compound words: cross-linguistic evidence. Psychologia 43, 47-66.
Zhou, X. & MARSLEN-WILSON, W. (2002). Semantic processing of phonetic radicals in reading Chinese characters. Acta Psychologica Sinica, 34, 1-9.
Zhou, X., MARSLEN-WILSON, W.D., Taft, M. & Shu, H. (1999). Morphology, orthography, and phonology in reading Chinese compound words. Language and Cognitive Processes, 14, 525-565.
Books
PULVERMÜLLER, F. (in press). The neuroscience of language: on brain circuits of words and serial order. Cambridge, UK: Cambridge University Press.
Theses
CUMMING, N. (2001). The Hebb effect: investigating long-term learning from short-term memory.
RODD, J. (2000). Semantic representation and lexical competition: evidence from ambiguity.
REID, Agnieszka. (2001) The combinatorial lexicon: Psycholinguistic studies of Polish morphology. Birkbeck College, University of Lodon.
Chapters & Book Contributions
Cutler, A., McQueen, J.M., NORRIS, D.G. & Syllable, J. (2002). The silly balls roll on. In E. Dupoux (Ed.), Cognition: A critical look. Advances, questions and controversies, in honour of Jacques Mehler. Cambridge, MA: MIT Press.
DAVIS, M.H. (in press). Connectionist modelling of lexical segmentation and vocabulary acquisition. To appear in Quinlan (Ed), Connectionist approaches to developmental psychology. Hove, UK: Psychology Press.
GASKELL, M.G. & MARSLEN-WILSON, W.D. (2000). The perception of assimilated speech. In O.Fujimura, B.D. Joseph & B.Palek (Eds). Item order in Language and Speech, (pp671-687). Prague: Karolinum Press.
Gaskell, M.G. & MARSLEN-WILSON, W.D. (2001). Simulating parallel activation in spoken word recognition, In M.Christiansen & N Chater (Eds), Connectionist Psycholinguistics. Westport. CA: Ablex.
GRAHAM, K.S., JOHNSRUDE, I.S. & Simons, J.S. (in press). The Temporal Lobe. In Encyclopedia of the Neurological Sciences. San Diego: Academic University Press.
Hare, M.L., FORD, M. & MARSLEN-WILSON, W.D. (1999). Ambiguity and frequency effects in regular verb inflection. In Bybee, J. & Hopper, P. (Eds), Frequency and the emergence of linguistic structure. Amsterdam: John Benjamins
JOHNSRUDE, I.S. (2001). Neuropsychological consequences of temporal-lobe lesions. In Harrison, JE & OWEN, AM (Eds) Cognitive deficits in brain disorders, (pp37-58). London: Martin Dunitz Publishers.
Loucas, T., & MARSLEN-WILSON, W.D. (1999). An experimental and computational exploration of developmental patterns in lexical access and representation. In M. Perkins & S, Howard (Eds) New Directions in Language Development and Disorders. New York: Kluwer Academic/Plenum Publishers.
MARSLEN-WILSON, W.D. (1999). Abstractness and combination: The morphemic lexicon. In S. Garrod & M. Pickering (Eds), Language Processing, (pp101-119). Hove: Psychology Press.
Marslen-Wilson, W.D. (in press). The mental lexicon in language comprehension. Oxford International Encyclopedia of Linguistics. New York: Oxford University Press.
Moss, H.E. & GASKELL, M.G. (1999). Lexical semantic processing during speech. In Garrod, S. & Pickering, M. (Eds). Language Processing. (pp.59-99). Hove: Psychology Press.
Muller, M.M. & PULVERMÜLLER, F. (2002). Hochfrequente oszillatorische aktivitaet als indikator kognitiver prozesse im gehirn [High frequency oscillatory activity as indicator of cognitive processes in the brain]. In Enzyklopdie der Psychologie, Biologische Grundlagen der Psychologie (Encyclopedia of Psychology, Biological Foundations of Psychology), (pp87-124). Gottingen: Hogrefe Verlag.
NORRIS, D.G. (1999). Computational Psycholinguistics. In MIT Encyclopedia of the Cognitive Sciences, Wilson and Keil (Eds), (pp168-169). Cambridge, MA: MIT Press.
NORRIS, D.G. and Wise, R. (2000). The study of prelexical and lexical processes in comprehension: psycholinguistics and functional neuroimaging. In M. Gazzaniga (Ed). The New Cognitive Neurosciences. Cambridge, MA: MIT Press.
OWEN, A.M., EPSTEIN, R. & JOHNSRUDE, I.S. (2001). FMRI: Applications in cognitive neuroscience. In P M Matthews, P Jezzard & S M Smith (Eds) Functional Magnetic Resonance Imaging of the Brain: Methods for Neuroscience, (pp311-327). Oxford: Oxford University Press.
PAGE, M. (2000). Creativity and the Making of Contemporary Art. In Strange and Charmed: Science and the Contemporary Visual Arts, (pp104-119). Calouste Gulbenkian Foundation.
PAGE, M., & Henson, R.N.A. (2001). Computational models of short-term memory: Modelling serial recall of verbal material. In Andrade, J (Ed), Working Memory in Perspective, (pp177-198). London: Routledge.
PAGE, M.P.A. & NORRIS, D.G. (1998). Modelling immediate serial recall with a localist implementation of the primacy model. In Grainger, Jonathan (Ed); Jacobs, Arthur M. (Ed) Localist connectionist approaches to human cognition. Scientific psychology series, (pp227-255). Mahwah, NJ, US: Lawrence Erlbaum Associates, Inc.
PULVERMÜLLER, F. (2000). On distributed cell assemblies, high frequencies, and the significance of EEG/MEG recordings. In Miller, R. (Ed), Time and the brain, (pp241-249). Amsterdam: Harwood Academic Publishers.
PULVERMÜLLER, F, (2001). Connectionist models of language processing. In Baltes, P.B. & Smelser, N.J. (Eds), International Encyclopedia of Social and Behavioural Sciences. New York: Elsevier.
PULVERMÜLLER, F. (in press). Imag(in)ing language in the brain. In Frontiers in Neurobiology. An international handbook. Volume 5: Intelligence systems. Rome: Encyclopaedia Italiana Press.
Rastle, K. G. & DAVIS, M. H. (in press). Reading morphologically complex words: Some thoughts from masked priming. To appear in Kinoshita, S. & Lupker, S.J. (Eds.) Masked priming: State of the art. Psychology Press.
REID, A.A, & MARSLEN-WILSON, W.D. (2000). Derivational and inflectional morphemes in the Polish mental lexicon: Preliminary studies. In C. Schaner-Wolles, J.R. Rennison & F. Neubarth (Eds.), Naturally! Linguistic studies in honour of Wolfgang Ulrich Dressler presented on the occasion of his 60th birthday. Torino, Italy: Rosenberg & Sellier.
Zhou, X., & MARSLEN-WILSON, W.D. (1999c). Sublexical processing in reading Chinese. In J.Wang, A.W.Inhoff, & H-C Chen (Eds.), Reading Chinese Script: A Cognitive Analysis. (pp37-63). Mahwah, NJ: LEA.
Conference Proceedings and Published Abstracts
Assadollahi, R. & PULVERMÜLLER, F. (2001). Word length and frequency in early lexical access: neuromagnetic evidence. Society for Psychophysiological Research, Abstracts of the 41st Annual Meeting, Psychophysiology 38: suppl 1, S21.
Assadollahi, R. & PULVERMÜLLER, F. (2001). Reading words in the MEG response. Cognitive Neuroscience Conference, New York, Journal of Cognitive Neuroscience, 12, Suppl, 123.
Assadollahi, R. & Pulvermüller, F. (in press). Signatures of Syntactic Processing in the Neuromagnetic Mismatch Field. Psychophysiology.
BOUDELAA, S. & GASKELL, G. (2000). In search of the minority default: the case of Arabic plurals. Proceedings of the Twenty Second Annual Meeting of the Cognitive Science Society, (pp48-53). Mawhab, NJ: Erlbaum..
BOUDELAA, S. & MARSLEN-WILSON, W.D. (2000). Non-concatenative morphemes in language processing: evidence from modern standard Arabic. In A. Cutler, J.M. McQueen & R. Zondervan (Eds), Proceedings of the Workshop on Spoken Word Access Processes, (pp23-26). Nijmegen, The Netherlands: Max Planck Institute for Psycholinguistics
BOUDELAA, S. & MARSLEN-WILSON, W. (2001), The time-course of morphological, phonological and semantic processes in reading Modern Standard Arabic. Proceedings of the Twenty Third Annual Meeting of the Cognitive Science Society, (pp 138-143). Mahwah,NJ: Lawrence Erblaum Associates.
BOUDELAA, S. & MARSLEN-WILSON, W.D. (2001). Fractionating Arabic morphology: differential time-course of word pattern and root processing. Perspectives on Morphological Processing, From Aix-en-Provence to Nijmegen, (p5). Nijmegen: Max Planck Institute for Psycholinguistics
COX, S.M.L., Andrade, A. & JOHNSRUDE, I.S. (2002). Functional imaging of conditioned preferences. Society for Neuroscience Abstracts, 22, 381.17.
CUSACK, R., CARLYON, R.P., JOHNSRUDE, I.S. & EPSTEIN, R.A. (2001), Functional interaction between left and right auditory pathways at or before auditory cortex demonstrated using fMRI. Society for Neuroscience Abstracts, 27, 512.11.
Cutler, A., NORRIS, D.G. & McQueen, J., Tracking TRACE's troubles (2000). In A. Cutler, J.M. McQueen & R. Zondervan (Eds), Proceedings of the Workshop on Spoken Word Access Processes, (pp63-66). Nijmegen, The Netherlands: Max Planck Institute for Psycholinguistics.
Cutler, A., van Ooijen, B. & NORRIS, D.G. (1999). Vowels, consonants and lexical activation. In J.J. Chala, Y. Hasegawa, M. Chala, D. Granville, & A.C. Bailey (Eds.), Proceedings of the Fourteenth International Congress of Phonetic Sciences (ICPhS). Vol. 3. (pp.2053-2056). Berkeley: UC Berkeley, Dept of Linguistics.
DAVIS, M.H., GASKELL, M.G., & MARSLEN-WILSON, W.D. (2000). Lexical segmentation and ambiguity: Investigating the recognition of onset-embedded words. In A. Cutler, J.M. McQueen & R. Zondervan (Eds), Proceedings of the Workshop on Spoken Word Access Processes, (pp 71-74). The Netherlands: Max Planck Institute for Psycholinguistics.
DAVIS, M.H., Henson, R.N., JOHNSRUDE, I.S. & Rugg, M.D.(2001). Priming effects in single-word reading: an event-related fMR study. Society for Neuroscience Abstracts, 27, 82.3.
DAVIS, MH, JOHNSRUDE IS. (2002). Acoustic and Linguistic Processes in Spoken Language Comprehension. EURESCO Research Conference on the Science of Aphasia, Naples, Italy
DAVIS, M.H., JOHNSRUDE, I.S. (2002). Effortful understanding: An fMRI study of the perception of distorted speech. ICSLP Workshop: Temporal Integration in the Perception of Speech, Aix-en-Provence, France.
DAVIS, M.H., MEUNIER, F., MARSLEN-WILSON, W.D. (2002). Effects of syntactic class and morphological complexity in word reading: an fMRI study. Journal of Cognitive Neuroscience, 14, Suppl, 27.
DOVE, A., Rowe, J., JOHNSRUDE, I.S. & OWEN, A.M. (2001). Involvement of dorsolateral prefrontal cortex in manipulation of items held in working memory. Journal of Cognitive Neuroscience, 13, Suppl, 89.
GASKELL, M.G. (2000). A quick rum picks you up, but is it good for you? Sentence context effects in the identification of spoken words. In A. Cutler, J.M. McQueen & R. Zondervan (Eds), Proceedings the Workshop on Spoken Word Access Processes. Nijmegen, The Netherlands: Max Planck Institute for Psycholinguistics.
Genkinger, B., PULVERMÜLLER, F., Elbert, T.R., Mohr, B., Rockstroh, B., Koebbel, P. & Taub, E. (2001). Constraint-induced therapy of aphasia following stroke. Journal of Cognitive Neuroscience, 13. Suppl, 127-128.
Gonçalves, M., Hall, D., JOHNSRUDE, I.S. & Haggard, M. (2001). Can meaningful effective connectivities be obtained between auditory cortical regions? NeuroImage 13, (6 Suppl.), S130.
Good C., JOHNSRUDE, I.S., Ashburner, J., Friston, K., & Frackowiak, R.S.J. (2000) Voxel based morphometry of 465 normal adult human brains. NeuroImage 10, 607.
Good, C.D., JOHNSRUDE, I.S., Ashburner, J., Friston, K.J. & Frackowiak, R.S.J. (2001). Cerebral asymmetry: a voxel based morphometry study of 465 normal adult human brains. International Society for Magnetic Resonance in Medicine, Glasgow UK, June, 2001.
Greer, M.J., VAN CASTEREN, M., McLellan, S., Moss, H.E., Rodd, J., ROGERS, T.T. & Tyler, L.K. (2001). Mind over matter: the emergence of semantic categories from featural representations. Proceedings of the Twenty Third Annual Conference of the Cognitive Science Society, 386-391. Mahwah,NJ: Lawrence Erblaum Associates.
Griffiths, T.D., Uppenkamp, S., JOHNSRUDE, I.S. & Patterson, R.D.(2001). FMRI study of the mechanism for encoding temporal regularity in the human brainstem. Association for Research in Otolaryngology, Florida, Feb 2001.
Gustard, S., JOHNSRUDE, I.S., Gillard, J. Donovan, T., Bullmore, E, , & Pickard, J. (2001). A functional magnetic resonance imaging battery for preoperative mapping of motor, motor planning and language function in the cortex. British Chapter of the International Society for Magnetic Resonance in Medicine, Cambridge, UK Sept 2001.
Haggard, M. P., Gonçalves, M., Hall, D., JOHNSRUDE, I.S. (2000). Can meaningful effective connectivities be between auditory cortex fields? Issues in subject variability. British Society for Audiology, Keele, Staffs.
Hall, D. & JOHNSRUDE, I.S. (2001). Interpreting acoustical responses to complexity with respect to the cytoarchitecture of the human supratemporal plane. Association for Research in Otolaryngology, Florida, Feb 2001.
Hall, D.A., JOHNSRUDE, I.S., Gonçalves, M. S., Haggard, M. P., Palmer, A. R., & Summerfield, A. Q. (2000). FMRI mapping of responses to spectral and temporal complexity. British Society of Audiology Short Paper Meeting, Keele, Staffs.
Hall, D.A, JOHNSRUDE, I.S., Gonçalves, M.S., Haggard, M.P., Palmer, A.R., Summerfield, Q, Akeroyd, M.A., Frackowiak, R.S.J. (2000) Cortical representations of modulation and bandwidth measured using fMRI. NeuroImage, 10, 701.
HAUK, O. & PULVERMÜLLER, F. (2001). Effects of word length and frequency of the human ERP. Society for Psychophysiological Research, Abstracts of the 41st Annual Meeting, Psychophysiology 38, suppl1, S49.
HAUK, O., HOLZ, G. & PULVERMÜLLER, F. (2001). Foot and mouth in the human word-evoked brain potential. Society for Psychophysiological Research, Abstracts of the 41st Annual Meeting, Psychophysiology 38, suppl1, S49.
HAUK, O., Holz, G. ,& PULVERMÜLLER, F. (2002). ERPs in action: Electrophysiological distinction of different types of action words. 8th International Conference on Cognitive Neuroscience, Porquerolles, France, 107, 33.
HAUK, O. & PULVERMÜLLER, F. (2002). Effects of word length and frequency on the human ERP. 8th International Conference on Cognitive Neuroscience, Porquerolles, France, 107, 34.
HAUK, O., Rockstroh, B. & Eulitz, C. (2002). Grapheme monitoring in picture naming: an electrophysiological study of language production. 8th International Conference on Cognitive Neuroscience, Porquerolles, France, 108, 35.
JOHNSRUDE, I.S., CUSACK, R., Morosan, P., Hall, D., BRETT, M., Zilles, K. & Frackowiak, R. (2001). Cytoarchitectonic region-of-interest analysis of auditory imaging data. NeuroImage 13, 6Suppl., S897.
JOHNSRUDE, I.S., DAVIS, M.H., Dilks, P.J. & Turnbull, I.J. (2001). An fMRI study of speech intelligibility. Society for Neuroscience Abstracts, 27, 949.7.
JOHNSRUDE, I.S., DAVIS, M.H., HERVAIS-ADELMAN, A., Brent, L. (2002). Adaptation to noise-vocoded speech in normal listeners: perceptual learning depends on lexical feedback. Poster presented at the BSA Short Papers Meeting on Experimental Studies of Hearing and Deafness, University of Sheffield, Sept 16th - 17th, 2002.
JOHNSRUDE, I.S., Giraud, A.L., Morosan, P., BRETT, M., OWEN, A.M. & Zilles, K. (2000). Functional Imaging of the auditory system: The use of positron emission tomography. Proceedings of the 4th European Congress of Oto-Rhino-Laryngology Head and Neck Surgery. K. Jahnke & M. Fischer (Eds.) , (pp49-58). Bologna: Monduzzi Editore.
JOHNSRUDE, I.S., Gustard, S., Bullmore, E., Gillard, J., and Pickard, J. (2001). A functional magnetic resonance imaging battery for preoperative mapping of motor, motor planning and language function in the cortex. British Institute of Radiology, London UK, Nov 2001.
JOHNSRUDE, I.S., Morosan, P., BRETT, M., Ashburner, J., Zilles, K. & Frackowiak, R.S.J. (2000). Functional Specialization within three cytoarchitectonically defined primaty auditory cortical areas in humans. Society for Neuroscience Abstracts, 26, 737.15.
JOHNSRUDE, I.S., Morosan, P., Hall, D., CUSACK, R., BRETT, M., Zilles, K., & Frackowiak, R. (2001) Functional specialization within three human primary auditory cortical areas. Association for Research in Otolaryngology, Florida, Feb 2001.
Kearns, R.K., NORRIS, D. & Cutler, A. (2002). Syllable processing in English. Proceedings of the Seventh International Conference on Spoken Language Processing, Denver, September. (pp. 1657-1660).
Kujala, T., SHTYROV, Y., Winkler, I., Saher, M., Tervaniemi, M., Sallinen, M., Teder-Salejarvi, W., Alho, K., Reinikainen, K. & Naatanen, R. (2002). Prolonged effects of long-term occupational noise on the auditory system. Neruscience Finland 2002. Turku: The Brain Research Society of Finland.
Longworth, C.E., MARSLEN-WILSON, W.D. & Tyler, L.K. (2002). Non-fluent aphasics show an abnormal time course of semantic activation from the regular past tense. Journal of Cognitive Neuroscience, 14,suppl1, B55.
Longworth, C.E., Randall, B., Tyler, L.K., MARSLEN-WILSON, W.D. (2001). Activating verb semantics from the regular and irregular past tense. Proceedings of the Twenty Third Annual Conference of the Cognitive Science Society, (pp570-575). Mahwah, NJ: Lawrence Erlbaum Associates.
Loucas, T. & MARSLEN-WILSON, W.D. (1999). The development of spoken word recognition: experimental and computational studies. Proceedings of the Twentieth Annual Conference of the Cognitive Science Society, (pp663-668). Mahwah, NJ:Lawrence Erblaum Associates..
MARSLEN-WILSON, W.D. (2000). The lexicon, morphology, and mental computation. International Journal of Psychology, 35, 219.
MARSLEN-WILSON, W.D. (2000), Organising principles in lexical access and representation? A view across languages. In A. Cutler, J.M. McQueen & R. Zondervan (Eds), Proceedings of the Workshop on Spoken Word Access Processes (pp19-22), Nijmegen, The Netherlands: Max Plank Institute for Psycholinguistics.
MARSLEN-WILSON, W.D., Csibra, G., FORD, M., Hatzakis, H., Gaskell, G. & Johnson, M.H. (2000). Associations and dissociations in the processing of regular and irregular verbs: electophysiological evidence. Journal of Cognitive Neuroscience, Suppl S, 55E.
MARSLEN-WILSON, W.D., OWEN, A., FORD, M., BRETT, M., Pickard, J.D. & Price, C. (2002). Brain mechanisms for past tense processing: Neuro-imaging evidence. Journal of Cognitive Neuroscience,14, Suppl 1, F49.
McQueen, J.M., Cutler, A. & NORRIS, D.G. (2000). Why Merge really is autonomous and parsimonious. In A. Cutler, J.M. McQueen & R. Zondervan (Eds), Proceedings of the Workshop on Spoken Word Access Processes, (pp47-50). Nijmegen, The Netherlands: Max Planck Institute for Psycholinguistics.
McQueen, J.M., Cutler, A. & NORRIS, D.G. (2000). Positive and negative influences of the lexicon on phonemic decision-making. In Proceedings of ICSLP2000 (International Conference on Spoken Language Processing), Vol. 3, (pp.778-781), Beijing, China.
McQueen, J.M., NORRIS, D. & Cutler, A. (1999). The time course of lexical involvement in phonetic categorisation. 137th meeting of the Acoustical Society of America. Journal of the Acoustical Society of America, 105, 1398.
McQueen, J.M., NORRIS, D.G. & Cutler, A. (2001). Can lexical knowledge modulate prelexical representations over time? Proceedings of the Workshop on Speech Recognition as Pattern Classification, (pp9-14). Nijmegen, The Netherlands: Max Planck Institute for Psycholinguistics.
MEUNIER, F. & MARSLEN-WILSON, W.D. (2000), Regularity and irregularity in French inflectional morphology. Proceedings of the Twenty-Second Annual Conference of the Cogntive Science Society, 405 - 410, Mahwa, New Jersey: Lawrence Erblaum Associates.
MEUNIER, F. & MARSLEN-WILSON, W.D. (1999). Inflectional morphology in the French mental lexicon. In A. Vandierendonck, M. Brysbaert & K. Van Der Goten (Eds.), Proceedings of the Eleventh Conference of the European Society for Cognitive Psychology, (p188), Escop/Academia Press.
MEUNIER, F., MARSLEN-WILSON, W.D. & FORD, M. (2000). Suffixed word lexical representations in French. In A. Cutler, J.M. McQueen & R. Zondervan (Eds), Proceedings of the Workshop on Spoken Word Access Processes, (pp31–34). Nijmegen, The Netherlands: Max Planck Institute for Psycholinguistics.
MEUNIER, F. & Segui, J. (2000). Morphological priming effect and allomorphy in French. In H. Kaufman & P. Royle (Eds), Proceedings of the Second International Conference on The Mental Lexicon, p. 20.
MOHR, B. & PULVERMÜLLER, F. (2001). Modulation of word processing by lateralized slow cortical potential feedback: Implications for the reahbilitation of language disorders. Journal of the International Neuropsychological Society, 7, 410.
NORRIS, D.G. (1999). The merge model: Speech perception is bottom-up. Journal of the Acoustical Society of America, 106, 613-L40.
NORRIS, D.G. (2001). A random walk model of decision processes in experiments on spoken word recognition. Proceedings of the workshop on speech recognition as pattern classification, 79-84. Nijmegen, The Netherlands: Max Planck Institute for Psycholinguistics.
NORRIS, D.G., & PAGE, M.P.A. (1997). A localist implementation of the primacy model of immediate serial recall. Reference : In J.A. Bullinaria, D.W. Glasspool & G. Houghton (Eds), Proceedings of the 4th Neural Computation and Psychology Workshop. (pp316-330). London: Springer.
NORRIS, D.G., Cutler, A. & McQueen, J. (2000). The optimal architecture for simulating spoken-word recognition. In C. Davis, T. van Gelder & R. Wales (Eds.) Cognitive Science in Australia, Proceedings of the Fifth Biennial Conference of the Australasian Cognitive Science Society, (p 53).
Palva, S., Palva, J.M., SHTYROV, Y., Kujala, T., Ilmoniemi, R.J., Kaila, K. & Naatanen, R. (2001). Distinct early phase-locked gamma responses evoked by speech and non-speech sounds in humans. Arctic Symposium on mechanisms of memory and memory disorders, p34.
Patterson, R., Uffenkamp, S., NORRIS, D.G., MARSLEN-WILSON, W., JOHNSRUDE, I.S. & Williams, E. (2000). Phonological processing in the auditory system: A new class of stimuli and advances in fMRI techniques. Proceedings of the 6th International Conference of Spoken Language Processing (ICSLP) Beijing, China, Vol. II, 1-4.
PULVERMÜLLER, F. (2001). Neurophysiological correlates of grammatical and semantic word categories. Journal of the International Neuropsychological Society, 7, 426.
PULVERMÜLLER, F. (2001). How and where are words represented and processed in the brain? In J.A.Argente (Ed), Proceedings of the 1st international workshop on language, brain and verbal behavior , New York: Academic Press.
PULVERMÜLLER, F. (in press). Neuronal correlation learning and its implications for the brain mechanisms of language. Proceedings of the 2002 meeting of the Rodin Remediation Academy.
PULVERMÜLLER, F. (in press). Neurophysiology of language. (Introduction to the symposium at the Annual Conference of the Society for Psychophysiological Research, Montreal). Psychophysiology
PULVERMÜLLER, F. (in press): Word categories in the brain: from actions and animals to foot and mouth. Proceedings of the 2002 Annual Meeting of the World Federation of Neurology, Research Group on Aphasia and Cognitive Disorders, Villefranche-sur-Mer, 28.-30.5.2002 (in press).
PULVERMÜLLER, F. & Assadollahi, R. (2001). Length and frequency: word properties in the MEG. Cognitive Neuroscience Conference, New York, Journal of Cognitive Neuroscience.
PULVERMÜLLER, F, Assadollahi R, SHTYROV Y & HAUK O. (2001). Neurophysiologic indicators of word types and serial order. 1st Annual Meeting of the Society for Psychophysiological Research, Psychophysiology, 38, Suppl 1, S17.
PULVERMÜLLER, F. Assadollahi, R., SHTYROV, Y. & HAUK, O. (in press). Neurophysiological indicators of word types and serial order (plenary lecture). Psychophysiology.
PULVERMÜLLER, F., Kujala, T., SHTYROV, Y., Simola, J., Tiitinen, H., Alku, P., Alho, K., Martinkauppi, S., Ilmoniemi, R. & Naatanen, R. (2000). Neuromagnetic and neuroelectric signs of pre-attentive word processing. In Elena Yago, Mari Tervaniemi, and Carles Escera (Eds), Abstracts of 2nd International Congress on Mismatch Negativity and its Clinical Applications, Barcelona: University of Barcelona, 57, June 2000.
PULVERMÜLLER, F. & MOHR, B. (in press). Neurophysiological reflections of language recovery after stroke. Proceedings of the 2002 Annual Meeting of the World Federation of Neurology, Research Group on Aphasia and Cognitive Disorders, Villefranche-sur-Mer, 28.-30.5.2002.
PULVERMÜLLER, F. & SHTYROV, Y. (2001). Automatic grammatical processing as revealed by the mismatch negativity. Society for Psychophysiological Research, Abstracts of the 41st Annual Meeting, Psychophysiology 38: suppl1, S79.
PULVERMÜLLER, F. & SHTYROV, Y. (in press). Matching Language And The Brain Using The Mismatch Negativity. Proceedings of the 8th International Cognitive Neuroscience Conference.
PULVERMÜLLER, F. & SHTYROV, Y. (in press): Automatic grammar processing in the brain: an EEG study. Proceedings of the 8th International Cognitive Neuroscience Conference.
PULVERMÜLLER, F., SHTYROV, Y., HAUK, O. & Ilmoniemi, R. (in press). Word recognition as reflected by the Mismatch Negativity. Proceedings of the 8th International Cognitive Neuroscience Conference.
Randall, B.R & MARSLEN-WILSON, W.D. (1999) The relationship between Lexical and Syntactic processing. Proceedings of the Twentieth Annual Conference of Cognitive Science Society. (pp. 871-876). Mahwah, NJ:LEA.
REID, A.A, & MARSLEN-WILSON, W.D. (2000). Organising principles in lexical representation: Evidence from Polish. Proceedings of the Twenty Second Annual Conference of the Cognitive Science Society. pp. 387-392. Mahwah, NJ: Erlbaum.
REID, A.A, & MARSLEN-WILSON, W.D. (2000). Complexity and alternation in the Polish mental lexicon. In A. Cutler, J.M. McQueen, & R. Zondervan (Eds.), Proceedings of the Workshop on Spoken Word Access Processes, (pp35-38). Nijmegen, The Netherlands: Max-Planck Institute for Psycholinguistics.
REID, A.A. & MARSLEN-WILSON, W.D. (2002). Regularity and irregularity in an inflectionally complex language: Evidence from Polish. Proceedings of the Twenty Third Annual Conference of the Cognitive Science Society, (pp855-860). Mahwah, NJ: Erblaum.
REID A.A., & MARSLEN-WILSON, W.D. (2002). Representation of Polish compounds. In M. Corley (Ed.), Proceedings of the 12th ESCOP and the 18th BPS Cognitive Section Conference, p186. . Edinburgh: European Society for Cognitive Psychology and British Psychological Society.
Rodd, J. M., DAVIS, M. H., JOHNSRUDE, I. S. (in press), Barking up the right tree: an fMRI study of semantic ambiguity. Society for Neuroscience Abstracts, 28.
RODD, J., GASKELL, G. & MARSLEN-WILSON, W.D. (1999). Semantic competition and the ambiguity disadvantage. Proceedings of the Twenty First Annual Conference of the Cognitive Science Society.
RODD, J., Gaskell, M.G. & MARSLEN-WILSON, W.D. (2000). The advantages and disadvantages of semantic ambiguity. Proceedings of the Twenty-Second Annual Conference of the Cognitive Science Society, 405-410. Mahwah, New Jersey: Lawrence Erlbaum Associates.
RODD, J., Gaskell, M.G. & MARSLEN-WILSON, W.D. (2000). Semantic ambiguity in spoken word recognition. In Cutler, A., McQueen, J.M., & Zondervan, R. (Eds), Proceedings of the Workshop on Spoken Word Access Processes, (pp103-106). Nijmegen, The Netherlands: Max-Planck-Institute for Psycholinguistics.
RODD, J., Gaskell, M.G. & MARSLEN-WILSON, W.D. (2000). Why are words with many senses recognised faster? International Journal of Psychology, 35, 27.
RODD, J., Gaskell, G. & MARSLEN-WILSON, W.D. (2001). For better or worse: Modelling effects of semantic ambiguity. Proceedings of the Twenty Third Annual Conference of the Cognitive Science Society, (pp868-868). London, UK: Lawrence Erlbaum Associates.
SHTYROV, Y. & PULVERMÜLLER, F. (2001). Processing of inflectional affix in the human brain as shown by mismatch negativity. Society for Psychophysiological Research, Abstracts of the 41st Annual Meeting, Psychophysiology 38, suppl1, S88.
SHTYROV, Y. & PULVERMÜLLER, F. (2001). Procesing of words and pseudowords in the human brain as shown by mismatch negativity: the role of lexical status of the deviant. Society for Psychological Research, Abstracts of the 41st Annual Meeting, Psychophysiology 38, Supplement 1, S89.
SHTYROV, Y. & PULVERMÜLLER, F. (2002). Memory traces for inflectional affixes as shown by the mismatch negativity. ICON8, 8th International Conference on Cognitive Neuroscience, Porquerolles Island, France, p 111.
SHTYROV, Y. & PULVERMÜLLER, F. (2002). Neurophysiological evidence of memory traces for words in the human brain. ICON8, 8th International Conference on Cognitive Neuroscience, Porquerolles Island, France, p 111.
SHTYROV, Y., PULVERMÜLLER, F., Näätänen, R. & Ilmoniemi, R.J. (2002). Automatic grammar processing in the brain. ICON8, 8th International Conference on Cognitive Neuroscience, Porquerolles Island, France, p 65.
Uppenkamp, S., Patterson, R., JOHNSRUDE, I.S., NORRIS, D.G., & MARSLEN-WILSON, W.D. (2001). Searching for phonological processing in the human brain using natural and synthesized vowels. British Society of Audiology, Oxford UK, Sept 2001
Zhou, X. & MARSLEN-WILSON, W.D. (2000). Processing lexical tones in visual recognition of Chinese. International Journal of Psychology, 35, 42.
EXTERNAL GRANTS
Butterfield, S. (1999-2001). National Science Foundation "Grammatical Agreement in Speech Production: The Cognitive Context". (Project Director: Professor Kathryn Bock, Beckman Institute, University of Illinois at Urbana-Champaign, U.S.A.). £2356.
Johnsrude, I.S. (2001). One-day workshop "Structure-function relationships in human auditory cortex", held June 15, MRC CBU, Cambridge (with Deb Hall, MRC Institute of Hearing Research, Nottingham). £10,000.
Johnsrude, I.S. (2002). International two day workshop on "Neurobioloby of communication: comparative and evolutionary perspectives on receptive language", held Sept 25, 26, MRC CBU, Cambridge (with Matt Davis, MRC CBU, Sophie Scott, UCL, Richard Wise, Imperial College). European Science Foundation. €15,000
Plaistead, K.C. and Davis, M.H. (Jan-March 2001). An investigation of pragmatic deficits in autism Nuffield Foundation: Social Science Small Grants Scheme. £5000.
F. Pulvermüller (PI) Activity Dynamics of Cortical Representations, 1997-2001 ,Deutsche Forschungsgemeinschaft (AZ Pu 97/10-1). ca. 70,000 €
F. Pulvermüller (PI) Physiology of the Association of Word Meaning, 1998-2001. Deutsche Forschungsgemeinschaft (AZ Pu 97/11-1). ca. 70,000 €
F. Pulvermüller (PI) Handbook edition "Neuroscience Of Language", 2001-. Mouton-de Gruyter, Berlin, New York. ca. 20,000 €
F. Pulvermüller (PI) The right hemisphere's role in word processing, 2000--2001. University Konstanz. ca. 40,000 €
F. Pulvermüller (PI of Cambridge part), Stefan Wermter (Coordinator) Mirrorbot: Biomimetic multimodal learning in a mirror neuron-based robot. Cambridge part on Neurophysiology of word processing, 2002--2005. European Science Foundation. ca. 435,000 € (1.7 Mio. € overall)
B. Rockstroh & F. Pulvermüller (PIs) CI Aphasia Therapy, 2001--2002. StiftungZNS. ca. 50,000 €
PRINCIPAL COLLABORATORS
Dr R. Assodhali, University of Konstanz.
Professor H-C Chen, Hong Kong.
Dr N. Cumming, CBU/University of Hertfordshire.
Professor T. Elbert, University of Konstanz.
Professor R. Frost (Jerusalem).
Professor G. Hitch, University of York.
Professor R. Imoniemi, University of Helsinki.
Professor A. Lahiri, University of Konstanz.
Dr R. McCarthy, University of Cambridge.
Dr. F. Meunier, CNRS Dynamique du langage, Lyons.
Dr Leah Pilgrim, University of Cambridge.
Dr. Kathy Rastle, Royal Holloway and Bedford College, University of London, UK.
Professor Risto Näätänen, University of Helsinki.
Dr J Rodd, University of Cambridge.
Dr B Neininger, University of Konstanz.
Dr M. Page, University of Hertfordshire
Dr R. Patterson, University of Cambridge.
Dr Helen Moss, University of Cambridge.
Professor G. Rizzolati, University of Parma.
Professor L. Tyler, University of Cambridge.
Professor X-L Zhou, Beijing.
Professor P.Zwitserlood, Munster.
Other sections in the 1998-2002 report
1. SUMMARY
2. ATTENTION GROUP
3. COGNITION AND EMOTION GROUP
4. LANGUAGE AND COMMUNICATION GROUP
5. MEMORY AND KNOWLEDGE GROUP: DMS SECTION
6. MEMORY AND KNOWLEDGE GROUP: REHABILITATION SECTION
7. METHODS RESEARCH AND INFRASTRUCTURE GROUP