You are in: Home » History of the Unit
LANGUAGE AND COMMUNICATION
General Notes. This material has been scanned from the original typescript. While we have done our best to remove errors, some may well remain. You can access other parts of this particular Progress Report either from the menu at the bottom of this entry, by using your browser's back function, by navigating back to the Unit History Timeline, or by accessing the relevant section of the electronic archive. Reference for this report are indexed by number and these can be found in a dedicated section also accessible from the menu at the bottom of this entry.
Bishop 3.0, Cutler 4.16, T Green 5.0 , Marslen-Wilson 0.92, McQueen 3.0, Nimmo-Smith 0.75, Norris 3.0, K Patterson 4.0, Sellen 0.73, Stark 2.25, Sturdy 3.0, Wright 5.0, Arblaster (HSO) 0.25, Blumenthal (HSO) 1.08, Borning (CO) 0.84, Bright (SO) 0.37, Butterfield (HSO)1.92, Doubleday (SO) 0.34, Duff (SO) 1.0, Hendry (SO)1.83, Hoffner (HSO) 0.5, Lickorish (HSO) 2.5, Milroy (HSO) 1.0, van Ooyen (SO) 2.67, Stacey (SO) 0.42, Strain (HSO) 2.08, Stratfold (SO) 0.17.
Total Person Years: Scientists 35.0; Research Support 17.0
Abstract
Objectives
The Language and Communication Programme combines both theoretical and applied studies aimed at (a) specifying the nature of mental representations of spoken and written language through experiments on normal adults and by computer modelling of psycholinguistic processes; (b) using studies of acquired and developmental language disorders to throw light on the interrelationships between components of language and to identify the neurological basis of such disorders; (c) identifying cognitive and environmental factors that influence people's ability to communicate effectively using a range of natural and technological devices.
Scientific progress and achievements over the past five years
A large part of the work in this programme involves experimental studies of normal adults carrying out psycholinguistic tasks under different conditions, in order to test theoretical predictions and to evaluate the contribution of different psychological processes to communicative behaviour. It is a strength of the programme that it incorporates work on all levels of communicative functioning, ranging from elementary processes of speech perception, through to studies on pragmatics and factors influencing effectiveness of communication in real-life settings. There is interest in both spoken and written language, with most of the scientists in the programme working in both areas. There is a particular focus on specifying the separability of different components of processing, and also on the extent to which there may be trade-off effects both within the domain of language and between language and other domains of cognition. This work on normal performance has both theoretical and applied importance. First, it provides a crucial baseline against which to compare the performance of (i) speakers of other languages; (ii) individuals with acquired or developmental language disorders, and (iii) computer simulations of language processes. These comparisons enable us to learn about the underlying processes involved in communicative functioning, the extent to which they depend on experience with a specific language or are universal, and whether or not they are organised in a modular fashion. Second, it provides an empirically-based rationale for those who are concerned with devising new types of information structure and novel means of communication, and ensuring that people make effective use of those communicative systems that already exist.
Specific scientific achievements
1. Provided the first experimental evidence that spoken word recognition involves inhibition between multiple, simultaneously active lexical candidates, and that the activation of competing candidates is modulated by the rhythmic characteristics of the language.
2. Illuminated, through studies of patients with semantic dementia, the way in which phonological information is stored for speech production and how this interacts with word meaning.
3. Demonstrated a major genetic influence on the aetiology of developmental language disorders, and showed that where two twins are affected with a disorder, they are remarkably similar in the pattern of language impairment.
4. Developed effective "cognitive prostheses" for users of computer-based documents.
5. Developed a formal notation to specify the key structural properties of different types of information structure, enabling progress in the development of systems which can minimize cognitive requirements such as the need to keep track of subgoals.
Future plans for the next five years
Cross-linguistic comparisons provide a powerful means of testing the generality of theories concerning the crucial underlying nature of language processing, and studies of this kind will be continued in the fields of speech perception and written language processing. In addition, we will continue to develop both computational models of psycholinguistic processes and theoretical notations for describing naturalistic information structures. These methods not only provide the discipline of requiring precise specification of the nature of psycholinguistic processes; they also, as has already been demonstrated in studies of both normal and abnormal functioning, generate new insights and lead to novel predictions to guide further experimentation. The connectionist approach to modelling generates a number of testable predictions about the developmental course of normal and disordered language development. The biological basis of language will be investigated both by further genetic studies of developmental language disorders and by functional brain imaging studies in adults. The relationships between language and nonverbal processing will be examined in both neuropsychological and applied contexts. A major theme for future work in the programme is the extent to which 'separate' cognitive processes interact, in the course of development and in on-line communicative behaviour. Both in applied work and in studies of disorders, there is considerable interest in the question of why people sometimes fail to make use of knowledge that they have, and in the role of competing cognitive demands on behaviour. We will consider how both normal and disordered language users vary in their communicative skill when either a secondary task, or the design of a naturalistic communicative device, leads to competition for cognitive resources.
Implications for improving health, health care and wealth creation
There are potentially vast commercial and practical implications of computerised systems that could carry out on-line decoding of spoken or written language. The design of such systems has, however, been limited by inadequate understanding of how humans carry out these operations. Likewise, designers of artefacts such as new computing languages or interactional software often fail to take into account the limitations of the human user; enormous costs can arise from the development of inefficient and inappropriate systems. By specifying what these limitations are and by devising analytical tools that can be applied in a wide range of contexts, we pave the way for a more rational basis for future developments. Work in this programme, where model development is guided by and tested against empirical findings, is of considerable importance for progress in this field.
Work on acquired and developmental language disorders provides a clearer conceptualisation of the causes and nature of these impairments, so that we can give more realistic information about prognosis, devise interventions directed towards amelioration of or compensation for specific deficits, and evaluate the efficacy of specific treatments.
Written information plays a central role in health care. Groups such as the Plain English Campaign stress that poor written communication can lead not just to inefficiency but also accidents. Exhortation to "try harder" is seldom enough; in order to know how/what to write, we must understand what factors create problems for readers and writers. The importance of this aspect of the programme is evidenced by the wide-ranging interest from bodies such as the British Standards Institute.
SPEECH AND LANGUAGE (Butterfield, Cutler, McQueen, Norris)
Introduction
Natural speech contains few reliable cues to the location of word boundaries. Indeed, the largest 'gaps' in speech occur within the words (actually within phonemes) rather than between the words themselves. This presents listeners with a problem that has no parallel in written language: How can we identify the words without first knowing where they begin and end? This is the problem of lexical segmentation. How is it that listeners can recognise the words in continuous speech in the absence of clearly marked word boundaries?
We have tackled this problem on three fronts: empirical studies of lexical segmentation and lexical access, computational modelling of speech recognition, and cross-linguistic studies. This work has shown how lexical segmentation is achieved by competition between multiple overlapping lexical candidates. That is, words beginning at many different points are accessed and these words compete with each other to determine the optimum parsing of the input. This lexical competition process is modulated by the listener's sensitivity to prosodic, or rhythmic, features of the language (the Metrical Segmentation Strategy (MSS) (4.22). In English the rhythmic unit is the foot. In French it is the syllable and in Japanese the mora. Sensitivity to prosodic information is a language specific feature of the recognition process -- different languages have different rhythmic characteristics and we have shown that even fluent bilinguals can only fully exploit a single Metrical Segmentation Strategy.
The success of this enterprise owes much to our work in developing a new connectionist model of speech recognition which incorporates both lexical competition and the Metrical Segmentation Strategy. For example, the effects of lexical competition depend crucially on the structure of the vocabulary of the language. Detailed experimental predictions can be derived only by using a model with a realistically sized vocabulary. The SHORTLIST model (4.54, 4.158) is the first major new computational model of spoken word recognition to appear in almost a decade and is the only model capable of performing appropriate large scale simulations.
Computational work on this model has proceeded in parallel with empirical work on lexical access and segmentation. Experimental work has been guided by the predictions of the model and has acted as a source of constraint in further development of the theory.
A. Computational Modelling of Spoken Word Recognition
A1. The SHORTLIST Model (Norris): SHORTLIST (4.54, 4.158) is a new connectionist model of continuous speech recognition. In contrast to TRACE (McClelland & Elman, 1986), SHORTLIST is completely bottom-up and avoids the duplication of lexical networks which makes the architecture of TRACE so implausible. The central feature of the SHORTLIST model is that recognition involves competition between a small set of words selected on the basis of a purely bottom-up analysis of the input. Lexical segmentation occurs as a consequence of the competition between different candidates starting at different points in the input. In TRACE all words in the lexicon are permanently in competition with all other words. However, the bottom-up nature of SHORTLIST means that it can function with as few as two competitors beginning at each phoneme in the input and that it requires more than 108 times fewer inhibitory connections than TRACE. Not only does this feature of the model have good theoretical motivation but it has the practical benefit of enabling the model to work with a realistically large vocabulary. While TRACE is restricted to using dictionaries of about 1000 words comprised of a subset of the phonemes of English, with SHORTLIST we commonly perform simulations with a dictionary of over 26,000 words using the full inventory of English phonemes.
SHORTLIST builds on ideas developed in the race model of Cutler & Norris (1979) and the checking model of Norris (1986). In effect, SHORTLIST can be seen as a specification of the lexical access component required by both of these earlier models. More recently (4.95) we have also extended SHORTLIST to make it sensitive to prosodic information and to incorporate the Metrical Segmentation Strategy of Cutler and Norris (1988).
A2. Lexical Statistics (Cutler, McQueen): The behaviour of models of spoken word recognition necessarily depends on the structure of the vocabulary. Analyses of the vocabulary (lexical statistics) can therefore help determine the plausibility of different models and highlight particular problems that certain models might face.
Both competition-based models like SHORTLIST and strictly sequential models like the Cohort model (Marslen-Wilson & Welsh, 1978) may encounter problems when the intended word has other words embedded within it (e.g., cam in camel). To assess the magnitude of this problem, a number of analyses were carried out on a large machine-readable dictionary using an interactive dictionary query system developed on the project (4.156). Each syllable and string of syllables within each polysyllabic word in the dictionary was checked against the dictionary. The efficiency of different lexical segmentation strategies was found to depend on the number and location of embedded words (4.113, 4.157).
For example, the efficiency of the Metrical Segmentation Strategy depends in part on the number of weak-initial words with unrelated words beginning from their first strong syllable (e.g. 'vermilion' contains 'million'). 16.5% of weak-initial content words were found to be of this type. But weak-initial content words make up only about 4% of conversational English, so it can be estimated that less than 1% of words normally encountered will be of the 'vermilion' type. The MSS is therefore well-suited to this aspect of vocabulary structure. An alternative segmentation strategy states that words are recognised in a strictly left-to-right manner, and that recognition of one word will locate that word's offset and hence the next word's onset. The success of this strategy depends on the proportion of words which are alike in their initial portions, since any words that are embedded at the onsets of longer words will have to be rejected to prevent postulation of false onsets. 57.5% of all polysyllables have at least one word embedded as their initial syllable. Left-to-right recognition is inappropriate given this feature of the English lexicon. Lexical segmentation based on competition (as in SHORTLIST) will be influenced by the proportion and location of words embedded in other words (see section B) but, unlike strictly left-to-right models, embedding will not disrupt recognition.
B. Empirical Work on Speech Recognition and Segmentation (Norris, Cutler , McQueen, Butterfield)
A central feature of the SHORTLIST model is that recognition takes place by means of lexical competition via lateral inhibition. McQueen, Norris & Cutler (4.47) provided the first experimental evidence that spoken word recognition can involve not only the activation of multiple lexical candidates, but competition between these candidates. A series of three experiments provided evidence of both the lexical competition mechanism proposed by SHORTLIST, and the MSS of Cutler and Norris (1988). McQueen et al. employed the same word-spotting task used by Cutler and Norris (1988). They asked subjects to spot words in spoken bisyllabic nonsense words such as /n´mEs/ and /d´mEs/ (as in 'domestic'). Both of these nonsense words contain the embedded word "mess". However, /d´mEs/ should also activate words like "domestic" and "domesticated". These words should compete with "mess" and make it harder to detect "mess" in /d´mEs/ than in /n´mEs/ where there are no such competitors. This is exactly what McQueen et al. (4.47) found.
Norris, McQueen & Cutler (4.95, 4.161) have extended this finding by showing that competition effects are a function of the number of active competitors. This latter study also demonstrates that there is an interaction between lexical competition and the MSS of Cutler and Norris such that the effects of the MSS are apparent only when there is a large number of competitors beginning at the onset of the strong syllable. A new version of SHORTLIST incorporates the MSS by boosting the activation of candidates beginning at the onset of strong syllables and inhibiting candidates which have strong syllable onsets where none is present in the input. The revised model gives a detailed account of these and earlier results.
The Metrical Segmentation Strategy depends crucially on listeners' ability to distinguish between strong and weak syllables. Listeners could in principle base this distinction either on prosody -- strong syllables are stressed, i.e., are longer, louder and have greater pitch movement than weak syllables -- or on vowel quality -- strong syllables have full vowels, weak syllables have reduced vowels. Results by Cutler and Butterfield (4.18) suggest that the latter distinction, vowel quality, is what listeners use. They examined the crucial case of unstressed but unreduced syllables (as in the first syllable of "condition"). In a rating task, listeners grouped these syllables together with stressed syllables (with which they share vowel quality but not stress) and distinct from reduced syllables (with which they share lack of stress, but not vowel quality).
C. Modularity in Spoken Word Recognition (Norris, McQueen)
A continuing theme of our research has been a concern with issues of modularity and interaction in spoken word recognition. Can spoken word recognition be explained by simple, bottom-up models, or do we need to make provision for more complex interactive theories in which, for example, information at the word level can influence decisions at the phoneme level?
Currently, the strongest evidence against a modular account of phonemic processing in lexical access comes from an elegant study by Elman and McClelland (1989). Two lines of research suggest, however, that their work may not be the fatal blow to modular theories that it first appeared to be. Norris (4.125, 4.160) has shown how simple recurrent networks trained to perform phoneme identification can simulate Elman and McClelland's data even though these networks have no lexical representations and are never taught to identify words. Essentially these networks learn to use preceding phonemic context (i.e., information within the phoneme level rather than top-down information from the lexical level) to assist phoneme identification. In non-connectionist architectures, information about preceding context is typically assumed to be derived solely from lexical information. However, in connectionist learning systems where no lexical information is available, it is difficult to prevent the networks from learning to make use of preceding phonemic context. Ironically, connectionism provides the best arguments for modularity !
A series of experimental studies by McQueen also strengthen the case for modularity. McQueen's PhD work (4.180) examined the way in which lexical knowledge influences decisions about speech sounds. Lexical involvement was measured in two tasks (rhyme monitoring and phonetic categorisation) and the predictions of two models of speech perception were compared. An interactive model, which claims that lexical effects in these tasks result from top-down flow of information from lexical to prelexical levels of processing, was contrasted with an autonomous model, which claims that lexical knowledge cannot influence prelexical processing, and that lexical effects are due to a race between lexical and prelexical phonetic decision procedures.
There were robust lexical effects in the rhyme task: word responses were reliably faster than nonword responses, and there was a significant word frequency effect (4.46). The interactive model predicts that lexical knowledge should influence rhyme decisions to nonwords as well as words, due to the presence of rhyming lexical neighbours; however, consistent lexical effects were only found on words. These experiments also showed that there is a strong strategic component in rhyme decision. In phonetic categorisation lexical effects were highly variable (McQueen, 1989). In particular, lexical involvement depended on stimulus quality. A lexical shift in the categorisation of word-final fricatives was found only after degradation by low-pass filtering (4.45). The reaction time data for this lexical effect supported the predictions of the autonomous model, but not those of the interactive model. One categorisation experiment, however, where lexical involvement was measured in a perceptual compensation for coarticulation process, yielded results which were more supportive of the interactive model. But work performed subsequently has shown (a) that the compensation result can be explained by autonomous models (4.125, 4.160) and (b) that the effect has a postperceptual component.
McQueen's thesis concluded that the results in both tasks were more consistent with the autonomous race model. Further, it was argued that lexical effects in phonetic decision-making depend upon stimulus and task parameters, and that an attentional mechanism is necessary to explain this variability. It was also claimed that attentional processes can be more parsimoniously included in the autonomous model.
More recent simulation work helps gain a more detailed understanding of the behaviour of simple criterion-bias models such as Morton's (1979) logogen model and the checking model of Norris (1986). Rhodes, Parkin and Tremewan (1993) have shown that semantic priming influences signal detection theory measures of sensitivity in visual word recognition. Following an argument presented by Farah (1989), they suggested that this was evidence that semantic information influenced perceptual encoding, and that this represented a violation of modularity. Norris (4.55) shows that, contrary to Farah's claim, measures of sensitivity cannot be assumed to reflect the operation of perceptual encoding. Simulations of the logogen model/checking model demonstrate that modular criterion-bias models in which priming has no effect on perceptual encoding predict the same sensitivity effects which Rhodes at al. take as evidence against modularity. Once again, a proper formal understanding of simple modular models shows that they are far more powerful than first appears.
D. Cross-Linguistic Differences in Processing Speech (Cutler, Norris, Kearns)
The international collaborations involved in this work have been made possible by a grant from the Human Frontier Science Program (HFSP).
The aim of psycholinguistics is not just to understand how humans process English, but how they process language in general. To this end our work has placed great emphasis on empirical and theoretical work covering a range of languages, selected because they have very different characteristics. Our previous work has shown that speech segmentation procedures are fundamentally different in French and in English (Cutler, Mehler, Norris & Segui, 1983, 1986, 1988). In both languages the primary unit of segmentation corresponds to the major rhythmic unit of the language. French has a syllabic rhythm whereas English has a stress based rhythm. Cutler, Mehler, Norris and Segui (4.22) extended these techniques to examine the behaviour of fluent French-English bilinguals in speech segmentation tasks. Even these fluent bilinguals did not simply behave like monolinguals in both of their languages. French-dominant bilinguals showed evidence of syllabic segmentation in French but no evidence of stress-based segmentation in English. The converse was true for English dominant bilinguals. It must be emphasised that these bilinguals appear completely fluent to native speakers of both French and English. Only by asking them which language they would least like to lose were we able to force them to express a preference for one language over another. It appears that bilinguals can develop only one of the possible rhythmically based segmentation procedures. The segmentation procedure of their dominant language is applied to all other languages.
An interesting contrast to French and English is provided by Japanese where the rhythmic unit is the mora, a unit smaller than the syllable. As predicted by our hypothesis that the rhythmic unit is also the unit of segmentation, Otake, Hatano, Cutler and Mehler (4.56) found that Japanese do indeed employ a mora-based segmentation procedure. English and French listeners presented with the Japanese speech materials responded quite differently, suggesting that mora-based processing is specific to Japanese listeners. In fact the French listeners segmented the Japanese speech by syllables, just as they segment French and English by syllables!
Moraic segmentation in Japanese was confirmed in subsequent phoneme-monitoring experiments by Cutler and Otake (4.23). Japanese listeners detected phoneme targets faster when they corresponded to single-phoneme morae (e.g. /n/ in "kanji") than when they corresponded to part of a larger mora (e.g. /n/ in "kana"). Presented with English words, Japanese listeners detected /n/ faster in, for example, "cancel" than in "canopy", suggesting that they were applying their native moraic segmentation strategy to input in a foreign language. This finding therefore added further support to the evidence from the earlier studies with French listeners, who had been shown to apply syllabic segmentation to input in English and in Japanese. All of these results confirm the predictions of the Metrical Segmentation Strategy which claims that the primary unit of segmentation should be the rhythmic unit of the language.
E. Computational Modelling of Reading (Norris)
Norris (4.52) has developed an interactive-activation model of reading aloud. In contrast to other models, this theory gives a detailed quantitative account of both the speed and the accuracy of word naming. The model uses an interactive activation network to combine knowledge of spelling-to-sound correspondences at the levels of words, syllables, rhymes, consonant-vowel units and phonemes. As with the SHORTLIST model, interactive activation is used only to simulate the competition process within a single level of analysis; there is no interactoon between levels. The network simulates reaction-time by the number of network cycles required to exceed the response criterion. Derived from the multiple-levels approach of Shallice, Warrington and McCarthy (1983), this simple model out-performs both the Seidenberg and McClelland (1991) and the Coltheart, Atkins, Curtis and Haller (1993) models in terms of its ability to generalise to nonwords. Additionally, it successfully simulates reaction time and error data from a wide range of experimental studies of speeded word naming, showing effects of word-frequency, regularity and consistency. There are many parallels between this model and the connectionist model studied by K Patterson (see her section of this programme). Whereas Patterson's work has been directed mainly towards accounting for patterns of disordered reading following neurological impairment, Norris' model has concentrated on providing a detailed quantitative account of normal performance.
Perhaps one of the most interesting aspects of the model is that, depending on one's perspective, it can equally well be seen as a single-route, dual-route or analogy theory. Although word naming is performed by a single network (suggesting a single route), the network takes input from distinct sources of lexical and sublexical information (suggesting two routes). The sub-lexical information can however be considered as deriving from lexical representations, in line with the analogy theory of Marcel (1980). This work therefore undermines the usefulness of the central theoretical distinctions which have motivated much of the work in this area. Theoretical distinctions derived from loosely formulated verbal theories appear illusory when applied to explicit computational theories.
FUTURE RESEARCH AND TECHNOLOGY TRANSFER
A. Computational Modelling
A1. Development of SHORTLIST: Much of the work planned for the next few years is based on further study of the lexical segmentation problem. Further developments of the SHORTLIST model will provide the theoretical driving force behind this work. The general form of this research will consist of a series of empirical studies combined with simulation work. Although SHORTLIST is a theory which itself makes a number of interesting predictions, it also provides a useful computational framework for evaluating new theoretical ideas. By integrating the different components of the theory within a single model, we can establish how those components interact and can ensure that the overall theory remains internally consistent. The current version of SHORTLIST is therefore not an end in itself but the starting point for further development of a detailed and explicit computational model of human speech recognition. The sections on "Empirical work on speech recognition and segmentation" and "Cross linguistic studies" both describe further work motivated by SHORTLIST which are likely to lead to further revision and refinement of the model.
A2. The Role of the Metrical Segmentation Strategy in Vocabulary Acquisition: Although the Metrical Segmentation Strategy (MSS) has generally been presented as a strategy for using rhythmic cues to help the mature listener segment the speech stream into words, we have also assumed that the MSS will be of value in vocabulary acquisition. Children should be able to use rhythmic information to discover the location of likely word boundaries. Jusczyk, Cutler and Redanz (4.44) have already shown that infants show a preference for listening to words which begin with a strong syllable. We intend to examine the value of the MSS for discovering word boundaries by using a simple statistical procedure which can learn to identify a proportion of the word boundaries in continuous input (Redlich, 1993) by grouping frequently occurring strings of input which have a high mutual information value. We can then investigate how the performance of this simple algorithm can be improved by providing information about the location of strong syllable onsets or syllable boundaries in general. A similar exercise can be carried out in French to investigate the value of syllabic information in identifying word boundaries.
B. Empirical Work on Speech Recognition and Segmentation
One important line of investigation inspired by the SHORTLIST model will extend our study of the role of syllabic information in lexical segmentation. This work stems in part from the observation that, under certain circumstances, SHORTLIST appears to behave rather differently from people. When subjects perform a word spotting task in which they are asked whether they can hear a word in the spoken nonsense string "wreckub" they invariably report that they hear 'wreck'. In SHORTLIST "wreck" is suppressed by "wrecker" (in Southern British English the final 'r' in wrecker is not pronounced). We believe that human listeners are under a strong constraint to parse the input so that word boundaries coincide with possible syllable boundaries. There is no possible boundary in "wreckub" after "wrecker" because /b/ alone is not a possible syllable. A number of experiments are planned which will investigate this issue in a more systematic manner. For example we can compare items like "dollf" and "dollfeep". We would expect identification of "doll" in "dollfeep" to be easier than in "dollf" because there is a possible syllable boundary in "dollfeep" but not in "dollf". A number of other studies are planned to investigate the role of phonotactic information in this process. If the basic observation proves reliable this result will be interesting for a number of reasons. First, largely as a result of our own work (e.g. 4.22), syllabic information is generally considered to play a less important role in the perception of English than in the perception of languages like French which have a syllabic rhythm. This conclusion is however based on tasks requiring metalinguistic judgements about syllables. In the word spotting task, we can infer the effects of syllabic knowledge from its impact on the difficulty of word identification without requiring subjects to make metalinguistic judgements at the syllabic level. In this task we will therefore be able to compare the use of syllabic information in English and French. We will also be able to employ the same task in Japanese and compare the role of the syllable and the mora in lexical segmentation by contrasting the effects of possible syllable boundaries with possible moraic boundaries. Questions raised by the SHORTLIST model are now providing the driving force behind our cross-linguistic work as well as our work on English word recognition.
From a theoretical standpoint these studies are important because they will shed light on how listeners ensure that their interpretation of an utterance does not leave part of the input unassigned to words. In automatic speech recognition systems this is generally achieved by comparing all possible lexical parsings of the input and reducing the weighting of any parsing which leaves parts of the input unaccounted for. However, such a strategy requires that the entire utterance is considered as a whole and can generate an enormous number of possible parses which must be compared. SHORTLIST, supplemented with a strategic bias against candidates not ending at possible syllable boundaries, can perform a similar function while only considering a handful of alternative local interpretations. In SHORTLIST any unaccounted for sections of the input will always correspond to complete syllables.
C. Modularity in Spoken Word Recognition
Other work derived from the SHORTLIST model involves comparing the predictions of SHORTLIST with those of highly interactive models like TRACE (McClelland & Elman, 1986). For example TRACE accounts for both lexical and phonotactic effects in phoneme identification by the same top-down connections from the lexical to the phoneme nodes. Input which violates phonotactic constraints is harder to process than phonotactically acceptable input because it receives less top-down feedback from the lexical level. Lexical effects and phonotactic effects should therefore always be present together. In SHORTLIST, on the other hand, these two effects are independent. Phonotactic effects are purely a product of information within the phonemic level, whereas lexical effects are due to the availability of phonemic information at the lexical level. According to SHORTLIST we should be able to demonstrate phonotactic effects in phoneme identification which remain constant even when the magnitude of lexical effects is manipulated. According to our own previous research (Cutler, Mehler, Norris & Segui, 1987) it should be possible to manipulate lexical effects by altering the composition of filler items in a phoneme monitoring or phoneme categorisation experiment. When experimental lists contain predominantly monosyllabic nonsense words, lexical effects should be greatly reduced while phonotactic effects remain . If such a manipulation can eliminate lexical effects while preserving phonotactic effects this would provide strong evidence for the bottom up account of phoneme recognition embodied in SHORTLIST and against the top-down account of phonotactic effects given by TRACE.
D. Cross Linguistic Studies
Cross-linguistic studies will continue to be an important component of work in speech recognition. SHORTLIST will be extended to include French and Dutch vocabularies to enable us to simulate a broader range of data. The experimental studies of the role of the syllable in segmentation described under B will include work on French and possibly Japanese as well as English. As mentioned above under A, computational studies of the Metrical Segmentation Strategy will include investigations of the value of syllabic information in acquisition of lexical knowledge in both English and French. This area of work will be strengthened by the appointment of Kearns whose work will concentrate primarily on cross linguistic studies.
SEMANTIC, PHONOLOGICAL AND ORTHOGRAPHIC REPRESENTATIONS OF WORDS (K Patterson)
Summary
The goal of this research is to explore the nature of orthographic, phonological and semantic representations of words, and to understand how these domain-specific representations interact in normal language abilities such as producing names of objects and actions (which is a computation from semantics ∅ phonology), reading words aloud (orthography ∅ phonology), reading silently for comprehension (orthography ∅ semantics), writing a spoken word (phonology ∅ orthography), etc. The principal form of evidence comes from studies of adult neurological patients who were normal language users prior to the onset of brain disease or injury: the patterns of breakdown provide insights to the premorbid organisation of language abilities. These neuropsychological studies are however accompanied by experiments with normal adults, to achieve converging evidence from impaired and normal performance. The principal techniques employed are behavioural experiments involving picture naming, word comprehension, repetition, reading and writing. These behavioural studies are however accompanied by two additional techniques, again to achieve a broader base of evidence: (a) functional brain imaging studies (PET), both of patients -- to observe regions of abnormal brain metabolism, and of normal individuals -- to determine regions of significant brain activation when the subjects are performing specific language tasks; (b) connectionist or neural-net modelling, to attempt simulation of significant aspects of human language behaviour. The principal language studied is, of course, English; the studies on normal and impaired English speakers are however accompanied by experiments with native speakers and readers of Japanese, to distinguish between aspects of our conclusions that are universal and those that are determined by the characteristics of particular languages and writing systems.
A major change in this line of research occurred when Dr J R Hodges moved to the Neurology Department in Cambridge in 1990. Prior to this, work on acquired language deficits at the APU had focussed on patients with cerebrovascular accidents (for example, 4.64, 4.128). Hodges' primary research interest is neurodegenerative diseases; and since 1990, we have been collaborating on studies of Dementia of the Alzheimer Type (DAT) and also of more focal progressive conditions (particularly progressive fluent aphasia or "semantic dementia" and progressive nonfluent aphasia) which disrupt specific components of language and memory. Due to the selective nature of the deficits, studies of progressive aphasia are ideally suited to the theoretical goals described above (exploring the interactions between semantic, phonological and orthographic domains of language representations).
A. Speech Production in Reading and Related Tasks
A1. Semantic Dementia: Semantic dementia has only recently been identified as a specific syndrome (see Warrington, 1975, and Schwartz, Marin & Saffran, 1979, for initial reports of such cases, though not with this sobriquet). We have defined the core features of semantic dementia as follows (4.41): (a) selective impairment of semantic memory, causing severe anomia, impaired single-word comprehension (both spoken and written) and an impoverished fund of general knowledge; (b) relative sparing of other components of speech production, notably syntax and phonology; (c) unimpaired perceptual skills and non-verbal problem solving abilities; (d) relatively preserved day-to-day (episodic) memory; and (e) a disorder of reading known as surface dyslexia (Marshall & Newcombe, 1973).
Given the long-standing interest and expertise at the APU in acquired disorders of reading, the surface dyslexic pattern of reading in semantic dementia has been a natural focus of research. In reading aloud, surface dyslexic patients frequently 'regularise' exception words (e.g. pint is given the more typical pronunciation of _int words in English, as in mint, lint, print, etc). Although this pattern had been reported in other patients with impaired comprehension, we were the first to argue that an intact semantic system is critical for reading of words with an atypical spelling-sound correspondence. We have expanded on this relationship in several ways, demonstrating for example (a) that the severity of surface dyslexia is significantly predicted by the degree of semantic loss (4.62), and (b) that those exception words yielding errors in comprehension tasks (like matching spoken words to pictures) are specifically more prone to misreadings (4.28).
Our next important step was to measure performance by patients with semantic dementia in repeating sequences of 3-4 words; some sequences were composed of words whose meanings the patient still appeared to know, others of words whose meanings had become degraded for that patient. By most theories, immediate repetition is a task that -- like reading aloud, only more so -- requires no semantic involvement. We observed, however, a marked difference in performance on the two types of sequence, with repetition of the now "unknown" words characterised by frequent errors, particularly transpositions of phonological elements -- e.g. the spoken sequence mint, rug repeated back as "rint, mug" (4.61).
On the basis of this pattern of results, we have hypothesised that a word's phonological representation for speech production, which almost certainly consists of linked elements rather than a whole preassembled package, derives a major part of its coherence from interaction with the word's meaning. When meaning deteriorates, this source of 'glue' is lost; the componential structure of the phonological representation is then revealed in repetition errors (mint, rug ∅ "rint, mug"), and in reading errors (pint read aloud to rhyme with "mint") which reflect typical spelling-sound correspondences of word components. A regular word can be read correctly without assistance from meaning because its pronunciation is the sum of its component parts.
A2. DAT: Patients with DAT are also germane to this line of investigation: although they have multiple cognitive deficits including a very prominent loss of day-to-day episodic memory (as would be predicted from the location and extent of brain regions affected; see K Patterson's section in the Memory Programme), deterioration of semantic memory typically becomes a significant feature as the disease progresses. We therefore predicted that reading of lower-frequency exception words should be impaired in DAT, and that the degree of this deficit should correlate with performance on tests of semantic memory. These predictions were confirmed in a group study of 45 patients classified as minimal, mild or moderate DAT (4.60). This finding conflicts with the prevailing belief that reading processes are relatively immune to DAT, and has important theoretical and clinical implications for the common use of the National Adult Reading Test (NART) as a measure of premorbid IQ in dementia.
A3. Normal Reading. Our hypothesis about the importance of communication between meaning and phonology in reading makes the prediction, not foreseen by any other theory, that word meaning should play a role in normal subjects' oral reading specifically for lower-frequency exception words. A recent series of experiments supports this hypothesis (4.97): both the accuracy and the speed of reading single words aloud reveal an interaction between regularity of spelling-sound correspondence and imageability, a semantic variable reflecting the extent to which a word's meaning has sensory properties. In sets of lower-frequency words, abstract exception words (e.g., scarce) were named aloud more slowly than either imageable exception words (soot) or abstract regular words (scribe). Perhaps even more dramatically, the normal skilled readers produced a significant number of regularisation errors only on the abstract exception words. We argue that, because of powerful statistical regularities in the relationship between orthography and phonology, only commonly encountered exception words have a major impact on setting the 'weights' on connections between orthography and phonology; less common exceptions to the rule rely on other parts of the system to assist in settling on the correct pronunciation. Words with imageable referents, in particular, are assisted by communication with semantic memory.
A4. Connectionist Modelling: Thus far, the connectionist modelling work (taking place mainly at Carnegie-Mellon University in Pittsburgh (Prof J McClelland and Dr D Plaut) and at University of Southern California (Prof M Seidenberg)) that forms part of this specific project has centred on building an adequate working simulation of the computation from orthography to phonology, using a back-propagation learning algorithm and a training vocabulary of about 3000 monosyllabic English words (4.58; Plaut & McClelland, 1993; Seidenberg & McClelland, 1989). There are clear parallels between this modelling enterprise and that of Dennis Norris (4.52), but also certain differences of emphasis. Norris' goal has been to simulate detailed (mainly reaction time) characteristics of normal readers' performance in oral reading, whereas this project is designed more to capture patterns of acquired disorders of reading, and also to develop a model of the translation from orthography to phonology that can be combined into a larger framework including semantic representations.
Until semantic representations are implemented, we cannot directly simulate our data on either the impact of word meaning in normal reading or the loss of meaning in semantic dementia. The current model is nevertheless germane to these findings in the following sense. With extensive training, the network can learn to produce correct pronunciations for essentially all trained words, even the low-frequency exception words that -- according to our hypothesis and our data from both normal and impaired readers -- require interaction with semantic representations. "Damage" to this fully trained network does not mimic the pattern of surface dyslexia in semantic dementia especially well, capturing neither the precise form of the frequency-by-regularity interaction nor the predominance of regularisation errors. A good simulation of the patient data is, however, achieved if training is stopped at a somewhat earlier stage. This suggests the following hypothesis: as a child learns to read, the weights on connections from developing orthographic representations to already established phonological representations are adjusted to capture (a) the regularities of spelling-sound correspondences, and (b) very common exceptions to these regularities (words like have and done). As the reading vocabulary grows exponentially, although the single orthography ∅ phonology network probably could (as demonstrated by extensively trained simulations) learn to deal with the complexities of the whole vocabulary, there is no need for it to do so; other parts of the system, in particular connections from word meaning to both orthographic and phonological representations, can boost weak or inconsistent computations of phonology from orthography. We therefore suggest that the similar surface dyslexic reading performance observed in the incompletely trained orthography ∅ phonology simulation and in our patients with semantic dementia both represent reading in the absence of the normal interaction with word meaning.
B. Reading Comprehension and Cross-Language Studies
B1. Single-Word Comprehension in Reading: Another strand of this research concerns the extent to which an adult reader's comprehension of a written word relies on activation of its phonological representation. This perennial but unsettled issue gained a new lease of life in the late 1980's, with a series of studies by Van Orden (e.g. 1987) using a yes/no semantic categorisation task (e.g., is the following word the name of an animal?). Despite knowing the correct spelling of animal names like deer and sheep, subjects make a significant number of categorisation errors to homophones of correct exemplars, both word (e.g., dear) and nonword (e.g., sheap) homophones. This result suggests that phonological codes play such a major role in reading comprehension that they can override conflicting orthographic information. The story, however, is not quite that simple; a recent series of experiments (4.12) demonstrates that the major factor determining the extent of the homophone effect is the orthographic similarity between the false and true homophones. Error rates to homophones of real category exemplars can indeed be strikingly high for orthographically similar words (dear/deer) or nonwords (sheap/sheep); but the error rate is substantially lower, and often not reliably higher than to non-homophonic control items, for orthographically dissimilar homophones (mayor/mare; phocks/fox). These results imply that phonological and orthographic codes interact to yield reading comprehension.
B2. English/Japanese: Our studies of normal adult Japanese readers performing similar reading tasks in Japanese Kanji, a non-alphabetic writing system derived from Chinese characters, have identified significant similarities and differences to results in English. It appears that the basic processes of reading in Japanese -- for example, the way in which orthographic and phonological codes interact to activate semantic representations of words -- hardly differ from English. Thus, for example, the pattern of homophone effects in reading comprehension, including the modulation by orthographic similarity between members of a homophone pair, is virtually identical in English and in Japanese Kanji, despite profound differences in the way that the two orthographies represent the sounds and meanings of their respective spoken languages (4.89). On the other hand, the precise nature of the orthography ∅ phonology computation does differ between these writing systems. Whilst this computation in English and other alphabetic orthographies occurs at both whole-word and sub-word levels, yielding the regularity/consistency effects described earlier, in Japanese Kanji the whole-word level largely dominates processes occurring for component characters. Results supporting this hypothesis from studies of normal readers can be found in 4.50 and 4.98; supporting results from acquired disorders of reading are presented in 4.130.
C. Semantic vs. Syntactic Language Abilities
As mentioned in the thumb-nail sketch of semantic dementia given earlier, marked deterioration of the semantic component of language seems to co-exist with a relative preservation of syntactic skills. We have documented this informally for speech production with assessments of the patients' spontaneous speech, which remains grammatically well-formed as it becomes progressively empty of content words other than very general nouns like "thing" and "piece" and general verbs like "make" and "do". We have also demonstrated syntactic preservation on one standardised test of speech comprehension (TROG, Bishop, 1983). Most dramatically, using on-line comprehension techniques developed by Tyler & Marslen-Wilson (see Tyler, 1992), we have shown that, as compared with monitoring for a specific word in a randomly ordered sequence of words, a patient with profound semantic dementia exhibited completely normal facilitation in a syntactically organised sequence but no further benefit (of the kind shown by normal listeners) in a semantically meaningful sequence (4.42).
D. Functional Brain Imaging
We have been using functional brain imaging techniques (PET) to investigate some of the neuroanatomical correlates of components of language processing. Because brain imaging studies are suited to a rather different type of scientific question, and also because this work is one component of a broader PET language research programme at the MRC Cyclotron Unit in London, the studies to date are not necessarily addressed to precisely the same issues as the behavioural experiments and the connectionist simulations described above. For example, our PET work on spoken word recognition began with a simple demonstration that, when normal subjects listen to familiar spoken words (by contrast with a tape of spoken words played backwards), peak activation occurs in the posterior, superior left temporal lobe -- almost precisely where, a century ago, Wernicke located the centre for speech recognition from results of post-mortem pathology in aphasic patients. In the same PET study (4.43), we observed peak activation for written words (by contrast with words written in 'false fonts', meaningless letter-like forms) in left middle-temporal cortex, not precisely at but near to the classical neurological localisation for written-word recognition. A more novel result comes from a study relating extent of blood-flow change to rate of presentation of spoken words. In virtually all areas of bilateral temporal cortex, bloodflow increased monotonically with an increase in the rate at which subjects heard spoken words (10, 30, 50, 70 or 90 words/minute). The one important exception to this rule was Wernicke's area, which was highly and equally activated at all rates of presentation (4.69). Differential sensitivity to rate of physical stimulation seems to be distinguishing between sensory processing (in primary auditory cortex) and more linguistic processing in the region specialised for language function.
E. Statistics of the Input to the Orthographic Representational System(Sturdy)
How is the visual input to a system that extracts an orthographic representation best described? Collaborations with Watt (Stirling University) attempted to answer this question by using his MIRAGE image processing system, developed at the APU. Technical problems meant that this work has mostly been carried out at Stirling. An alternative approach was developed, using low-level statistical measures extracted from the MRC Psycholinguistic Database. This approach assumes that the visual system extracts information from images only insofar as it is necessary for further processing modules; the work then goes on to explore the usefulness of low-level statistics, such as word length and mass, for disambiguating a target word input. The results indicate that a combination of low-level statistics and word frequency information is surprisingly useful for reducing the candidate set size to small numbers. This result motivated the development of a modification of perhaps the best-known connectionist model of word recognition, the Interactive Activation model of McClelland and Rumelhart. In this modification, information is input first to the word level of representation, and then subsequently to a bigram level of representation, rather than flowing up from a feature level to a letter level and then to a word level. Preliminary results are promising. This work has generated two offshoots: exploration of the MRC Psycholinguistic Database, and exploration of quantitative neuroanatomical constraints on building connectionist models.
PROPOSALS FOR FUTURE WORK
A. Speech Production in Reading and Related Tasks
A1. Semantic Dementia: One major aim in our initial studies of this disorder, reviewed above, has been to assess the status of phonological representations for speech production, using tasks such as word reading and repetition. We have also assessed written language production: all of our semantic dementia cases have been surface dysgraphic in writing as well as surface dyslexic in reading, making phonologically plausible spelling errors (e.g., "X-rays" written as ex-raise!) in both spontaneous writing and writing to dictation. Thus our hypothesis about the role of meaning in maintaining the integrity of representations for output applies to orthographic as well as phonological production. The hypothesis, however, is broader still. In a test known as "object decision" (Humphreys, Riddoch & Quinlan, 1988), a series of drawings -- some correctly drawn real objects and some chimeric combinations of parts of two real objects (e.g., the body of a sheep with the head of a dog) -- is presented to the subject who is asked to judge each drawing as real or not. Patients with semantic dementia are markedly impaired on object decision, which cannot be attributed to any peripheral impairment because, unlike DAT patients, semantic-dementia patients have unimpaired visuo-spatial abilities and perform at a normal level on tasks such as matching two photographs of the same object taken from different angles. The deficit instead suggests to us that high-level structural representations required for recognition as well as production depend on communication with meaning and are therefore impaired in tandem with loss of meaning. In the next phase of this project, we will evaluate this hypothesis with regard to representations required to recognise both spoken and written words, using tasks such as lexical decision, the language equivalent of object decision.
A2. DAT: In our current study, because patients were enrolled at various degrees of severity, we were able to perform useful cross-sectional analyses revealing, for example, a specific deficit in reading exception words and a general deficit in writing that is more pronounced for words with atypical spellings. This is, however, a longitudinal study, and we will soon have six consecutive data points (3 years of 6-monthly test sessions) for about 30 DAT patients from which to assess longitudinal patterns of the relationship between semantic memory and a variety of language tasks: naming, reading, writing, single-word comprehension, syntactic comprehension, etc. This is an extraordinarily rich data base that will enable us to analyse both group effects and single-case patterns; these analyses will occupy a significant amount of time over the next few years. Furthermore, we are continuing to follow the subset of patients who have not deteriorated rapidly.
A3. Normal Reading: We have just embarked on a major study to try to reproduce some significant aspects of surface dyslexia in normal readers, by one or more manipulations of the oral reading task. The manipulations include (i) mixing exception words with nonwords, (ii) forcing the subjects to name exception words at a faster-than-normal rate and (iii) altering the orthographic form of the presented words (e.g., pINT) in a way that may cause the reader to compute pronunciations of segments rather than whole words. The aim is to use evidence from normal readers to help select amongst several competing hypotheses about the nature of the deficit in surface dyslexia and, more generally, to understand the operation of the orthography ∅ phonology computation.
A4. Connectionist Modelling: As indicated above, most of the modelling work thus far has been carried out by our connectionist colleagues in America. We plan to contribute to this aspect of the work over the next few years, initially by doing various tests with the current form of the single implemented procedure (the orthography ∅ phonology network developed by Plaut & McClelland, 1993) and subsequently by helping to develop the fuller system including a semantic component.
A5. Nonfluent Progressive Aphasia: One of the major new lines of research planned for the next 5-year period will be disorders of speech production. The first main focus concerns patients with a disorder known as nonfluent progressive aphasia (or primary progressive aphasia). These patients, in whom the semantic component of language remains relatively intact in the face of severe deterioration of both phonology and syntax, are in some sense the mirror image of patients with fluent progressive aphasia or semantic dementia. We are just starting to investigate detailed aspects of speech production in several patients with this syndrome (Karen Croot, a new research student, will be concentrating on this topic for her thesis research). In the first study, the patients repeat words, read words and name pictures corresponding to the identical set of items. If the disorder is located at the stage of phonological output representations, then the rate of success and the types of errors should be largely independent of language task. If, on the other hand, the deficit involves some more central stage of language processing, then qualitative and/or quantitative aspects of performance should differ across tasks. Our preliminary results from this experiment are not only theoretically revealing but actually ground-breaking, as previous studies of this disorder have provided clinical observations but no experimental data (4.166).
A6. Progressive Anomia: The second focus of our interest in disorders of speech production has been motivated by a longitudinal study of a patient (FM) whom we originally assumed to be at an early stage of semantic dementia, but whose subsequent deterioration has shown a very different pattern. Although her initial performance on our semantic tests was not quite normal, this mild impairment has remained quite stable over a 3-year period, in sharp contrast to a profound decline in any task requiring speech production (naming, category fluency, etc). FM's immediate repetition of words is unimpaired, but she 'loses' the phonological word form rapidly: if required to count aloud for 5 seconds after hearing a spoken word, she can then correctly produce the target word on only about 50% of trials, whereas normal speakers are 100% correct after the same filled delay. Furthermore, it appears that she has a major disconnection between meaning and phonology. For example, in a gating task -- where the subject hears cumulative 50 msec chunks of a spoken word and must try to identify and produce the word -- FM receives no facilitation in the point at which she can produce the target word by seeing a picture of the object whose name she is concurrently hearing (this research is part of Kim Graham's thesis, being completed in 1994). We shall be using data of this kind to evaluate models of speech production, particularly the nature of the link between word meaning and phonological representations.
A7. Progressive Dysgraphia: One of our more striking observations in semantic dementia has been that, in addition to the surface dysgraphia seen in all cases, a subset of patients progressively lose knowledge of how to form letters and sometimes even numbers. The few published cases of such 'apraxic agraphia' have been due to CVA, where it is often difficult to establish the patient's degree of pre-morbid writing skill. In our progressive cases, where production of letter forms was intact at the beginning of the longitudinal study, we have a unique opportunity to study the deterioration of this knowledge and the status of associated abilities.
B. Reading Comprehension and Cross-Language Studies
B1. Single-Word Comprehension in Reading: Following on from the studies reviewed above, we shall continue our investigations of the relative contributions of phonological and orthographic codes in activating meaning for single words. Theorists proposing the primacy of phonology often rely heavily on one result in the literature, Van Orden's (1987) finding that, under conditions of visual masking, the homophone effect in semantic categorisation was no longer modulated by orthographic similarity: normal readers apparently accepted many false homophones followed immediately by a visual pattern mask, even if these were visually dissimilar to the correct word (e.g., "is it a number?" ATE (rather than EIGHT)). Our first attempt to replicate this result, using improved stimulus materials, yielded instead the outcome typical of unmasked semantic categorisation: a high error rate only for orthographically similar homophones. If this orthographic effect obtains in our further planned masking studies, using not only semantic categorisation but also primed word reading, it will provide critical evidence that normal reading comprehension standardly involves orthographic as well as phonological access to meaning.
B2. English/Japanese: My involvement in the project comparing normal reading in English, Japanese and Chinese (with Professor Brian Butterworth, Dr T Wydell and Dr W Yin, University College London) is drawing to a close. I have however planned cross-language neuropsychological studies, to be initiated during a 2-month period as a visiting researcher in Japan in autumn 1994. The first part of this work will focus on Japanese patients with progressive "Gogi" (word-meaning) aphasia, which -- from their neuroanatomical and behavioural pattern -- is almost certainly identical to semantic dementia. Based on our current view of similarities and differences between Japanese and English orthography and phonology, there are some clear predictions about word reading and repetition performance of Japanese patients with this disorder; and Dr Tanabe (Osaka) has kindly agreed to allow me to test several of his cases. The second part will be a broader project comparing disorders of speech production in Japanese and English (in collaboration with Drs I Tatsumi and S Sasanuma of the Tokyo Metropolitan Institute of Gerontology, my host institution in Japan). The most prominent unit of Japanese phonology is the mora (consisting typically of a consonant-plus-vowel, e.g., /ka/), and we predict that mora units will not come apart in speech production errors in the way that English syllables break up into onsets and rimes (cf the repetition errors of our semantic dementia patients discussed earlier, e.g., mint rug ∅ "rint mug").
C. Semantic vs. Syntactic Language Abilities
The separability of semantic and syntactic components of language is highly controversial. Progressive fluent and nonfluent aphasia appear to provide prima-facie evidence of a double dissociation; but like many such apparent dissociations, it will probably turn out not to be so simple. For example, although the fluent (semantic dementia) patients continue to produce largely well-formed phrases as the semantic content of their speech output deteriorates, their phrases often develop a rather steroptyped character which may not reflect full productive syntax. On the other side of the coin, the nonfluent patients may retain good comprehension of single nouns; but full understanding of verbs requires control of argument structure, and may therefore be abnormal for these syntactically impaired patients. A hypothesis to be explored over the next 5 years, for which we have preliminary support, is that, while syntactic competence may be somewhat separable from individual word meaning, it is inextricably linked to phonological abilities.
D. Functional Brain Imaging
The original motivation for embarking on this line of work was to address questions about patients with brain lesions, in particular a persistent issue regarding the role of intact right-hemisphere structures in the residual language capacities of patients with widespread left-hemisphere infarcts. The PET technology of 5 years ago was not, however, geared to the single-case analyses required for such patients. With currently improved levels of sensitivity such that, for a given permissable radioactive dose, significantly more observations can be obtained from a single subject, we are now in a position to do what we originally hoped to do 5 years ago. The broad goal is to assess the hypothesis that, in the normal population of right-handers with left-hemisphere language specialisation, there is a considerable range of right-hemisphere language capacity. Although there is currently little direct evidence for this hypothesis, it seems the most plausible explanation of the substantial variability in expressive and receptive language abilities amongst patients with large left-hemisphere lesions, and also in reading ability amongst pure alexic patients with more restricted left-occipital lesions. To address this hypothesis properly, we (Drs Richard Wise and Cathy Price at the MRC Cyclotron Unit; Dr David Howard at Birkbeck College; and K Patterson) will need to perform PET activation studies with a variety of language tasks in normal adults, pure alexic patients, globally aphasic patients and patients with left-hemisphere lesions roughly similar to the global aphasics but with markedly better post-CVA language abilities.
SPECIFIC LANGUAGE IMPAIRMENT: NATURE AND CAUSES (Bishop)
Introduction
Language development can be impaired if children have inadequate exposure to language or if they suffer from diseases which compromise the biological bases for language learning. For most children with speech and language difficulties, however, there is no obvious cause: hearing is normal, nonverbal intelligence is adequate, there is no physical or emotional disorder that can account for the language problems, and the home language environment seems unremarkable. This is known as specific language impairment (SLI) (see 4.101 for a review). Bishop's research has been concerned with specifying the nature and causes of this mysterious disorder, addressing three complementary questions: (i) what is the etiology of SLI, i.e., why do some children have this problem when others do not? (ii) how many types of SLI are there? (iii) what is the underlying cognitive basis of SLI? What is it about language that makes it so difficult for these children to learn?
A. Etiology of Specific Language Impairment
Background: For many years, etiology has remained a mystery; there is little evidence that perinatal hazard, postnatal brain damage, recurrent middle ear disease or inadequate language stimulation are implicated (Bishop, 1987). More recently, there has been a surge of interest in genetic causes, for two reasons: (i) studies of related developmental disorders, notably autism and developmental reading disorders, have found strong evidence for a heritable basis; and (ii) several studies have reported an elevated risk of SLI among family members of affected children.
A1. Twin Study of SLI: The classic twin study method provides a starting point for investigating genetic influences. Monozygotic (MZ) twins are genetically identical, whereas dizygotic (DZ) twins share, on average, half their genes. By comparing concordance rates (i.e., the percentage of pairs where both twins are affected) for MZ and DZ twins, one can estimate the importance of genetic factors. More recently, analytical methods have been developed for handling quantitative twin data, such as scores on language tests (DeFries & Fulker, 1988). We designate as probands those individuals who meet diagnostic criteria for disorder; by definition, they have scores well below average on language tests, and so the expectation is that scores of their co-twins will regress to the population mean. The difference in the regression for MZ and DZ co-twins provides an index of the contribution of genetic influences to the disorder.
Both qualitative and quantitative methods have been applied to data from 90 same-sex twin pairs, selected because at least one twin met stringent diagnostic criteria for SLI. 165 twin pairs were individually assessed on a battery of language tests and a nonverbal IQ test. Those scoring below the 10th centile on at least one out of four language tests, with a z-score discrepancy of 1.33 points or more between language score and nonverbal IQ, were designated as probands (i.e. affected with SLI).
Principal results from this study are shown in Figure 1 and reported by Bishop, North & Donlan (4.10). In brief, the study provides evidence for a genetic basis for SLI. When children are categorized according to whether or not they meet diagnostic criteria for SLI, pairwise concordance is 54% for MZ twin pairs (N=63 pairs), as compared with 30% for DZ pairs (N=27 pairs). However, this categorical approach to analysis almost certainly underestimates the genetic effects. Many "unaffected" MZ twins from discordant pairs had language scores that were just as poor as those of the co-twin, but without a substantial discrepancy with nonverbal IQ (such cases were coded as "low language"). Others had a history of speech therapy, although no problems could be detected on assessment. These results suggest that the definition of the phenotype used (based on DSM-III-R criteria) is too restrictive (4.8).
Figure 1: Classification of twin pairs according to diagnosis of twin B, where twin A meets criteria for specific speech/language disorder. Hatched area depicts pairwise concordance
.jpg)
Application of the DeFries-Fulker regression procedure threw further light on the question of what is inherited. All four language measures showed strong heritability, with three language tests giving estimates close to 1.0. The heritability estimates have large standard errors, and so interpretation must be cautious, but the data are consistent with the notion that there is an inherited factor that depresses receptive language ability to a mild degree, but which leads to more severe effects on expressive language development.
A2. Data on Other Family Members: Information is available from parental questionnaire concerning family history of SLI. This shows that the proportion of affected relatives in this twin sample is similar to that seen in singleton samples, and substantially higher than for a comparison sample of unaffected twins recruited by similar means. This offers some reassurance that the factors leading to SLI in twins are similar to those seen in singletons.
A3. Comparisons between SLI and Control Twins: Data from parental questionnaires and medical records are currently being analysed to see whether medical problems around the time of birth or in childhood can influence SLI. This analysis involves comparisons between the SLI twin sample and a control sample of 40 unaffected twin pairs recruited by similar means. Given the strong genetic effects found in the twin analyses, one would not expect these factors to exert a major influence on presence/absence of disorder, but the possibility remains that the severity or profile of impairment might be affected. In fact, most of the results are resoundingly negative; factors such as birth weight, Apgar score (an index of the condition of the infant at birth), middle ear disease, and seizures are all unrelated to language outcome. However, one intriguing and unexpected result emerged: there was an excess of toxaemia in the mothers of twins with SLI, and this could not be accounted for in terms of correlates of toxaemia such as social class or gender. Toxaemia is a poorly understood disease of pregnancy in which there is high blood pressure, fluid retention and protein in the urine. At first glance it looks as if maternal toxaemia could influence the developing foetus. However, in many cases of toxaemia, only one of the twins developed a language disorder. The question arises as to why two twins who are both exposed to toxaemia should differ in their outcome. It is possible that toxaemia is a marker for immunological abnormalities associated with language disorder, rather than a direct risk factor.
B. Is There a Distinct Subgroup of Children with Semantic-Pragmatic Difficulties?
Background: One of the difficulties confronting anyone studying developmental language disorders is that it is unclear whether we are dealing with a single condition or a group of different disorders. The clinical manifestations of SLI can be extremely variable. Some children have problems predominantly with production of speech sounds; others make many grammatical errors but appear to understand normally; cases have been described of children with severely restricted comprehension, and others of children who speak fluently in complex sentences but who give tangential answers to questions (Bishop & Rosenbloom, 1987).
Although most specialists recognise that there is considerable variation from child to child, attempts to devise a classification of SLI have not been successful. One problem is that multivariate classification methods depend crucially on the measures entered into the analysis. Standardized tests are not sensitive to all aspects of clinical presentation; many of the features that have been described by clinicians as important for distinguishing between subtypes are difficult to elicit in a formal assessment (4.8). This is particularly true for the pattern of disorder known as "semantic-pragmatic disorder", which Rapin (1987) describes as typified by: fluent speech with adequate articulation; verbosity; comprehension difficulties for the meaning of verbal messages; tendency to interpret messages quite literally; tendency to respond to just one or two words in a sentence; perseveration; use of circumlocutions, semantic paraphasias and lack of semantic specificity; impairment in the ability to take turns and maintain a topic in discourse. None of these clinical features could be identified simply by looking at the score on an existing language test. The clinical impression is that children with this pattern of deficits are qualitatively distinct from other language-impaired children, and may have more in common with autistic children. It is however difficult to verify this impression in the absence of any objective criteria.
Bishop and Adams (1991, 1992) were able to demonstrate pragmatic impairments in experimental tasks for children showing this profile, but the division between subtypes of language-impaired children was not as clear as had been expected. However, analysis of conversational data revealed a range of problems that could be identified reliably and which distinguished subgroups. Children with the clinical features of semantic-pragmatic disorder tended to initiate topics rather than respond to the interlocutor's overtures, and they were poor at matching the content of their message to the needs of the listener. They also tended to drift off topic and sometimes gave tangential answers to questions. Bishop and Adams (1991) concluded that attempts to elicit these behaviours in experimental contexts failed because the concrete context and explicit structure of the task allowed the child to compensate for underlying problems. It is precisely in open-ended situations, when there are no clear rules to follow and no environmental props, that the pragmatic difficulties of these children become most apparent.
These studies suggested that to identify and understand semantic-pragmatic disorder, one needs to devise methods for studying abnormalities of conversational behaviour in naturalistic contexts. This methodological shift is no light undertaking. Analysis of natural language samples is time-consuming, and the lack of experimental control makes interpretation of abnormal phenomena ambiguous. Coding open-ended data poses major problems of inter-rater reliability. Unless this step is taken, however, the characterisation and measurement of pragmatic difficulties will remain elusive.
B1. Devising a Clinical Procedure for Conversational Analysis: A project to develop a clinical procedure for conversational analysis was initiated in 1990, in collaboration with Catherine Adams at Manchester University. Development of the analysis is a cyclical procedure: in order to achieve reasonable inter-rater reliability, precise coding criteria must be specified, evaluated, and then revised, in an iterative fashion. The point has now been reached where there is a coding system that captures salient aspects of conversational behaviour, and which is extensively documented in a self-teaching manual. The analysis involves coding conversations, utterance by utterance, and provides a number of summary indices including measures of conversational assertiveness and responsiveness, ability to contribute to topic development, and overt indicators of difficulties experienced by the interlocutor. The transcription conventions have been shown to have good reliability when learned by a novice working through the manual. Inter-rater agreement for coding of conversational acts and topic development is around 80%. This should be further improved in the latest version of the coding system, which has been extensively revised to give more specific guidelines in problem areas. The project has one more year to run, during which further checks on reliability will be carried out with a new research assistant who has not been involved in developing the system. Conversations have been video-recorded and transcribed for 20 language-impaired children, half of whom meet criteria for semantic-pragmatic disorder, as well as 50 control children. These data will be used to assess the validity of the system in distinguishing both between language-impaired and control children, and between subtypes of language-impaired children.
In the course of pilot work on the coding system, a small study was carried out to investigate factors influencing one particular symptom of semantic-pragmatic disorder, "verbosity". Previous work suggested that the impression of verbosity arose not so much because children said a great deal, but because they generated utterances that were initiating rather than responsive, opening up new topics, asking questions of the adult, and generally taking the lead in conversation. This is unusual behaviour in school-aged children confronted with an unfamiliar adult. Bishop, Hartley and Weir (4.9) considered how contextual factors affected conversational behaviour in such children. One possibility was that children with semantic-pragmatic disorder might take the conversational lead in order to avoid having to answer questions posed by an adult. However, just the opposite was found: when the adult adopted a less controlling style of conversation in a toy play setting, these children were even more assertive. The verbose behaviour seen in certain children thus appears to reflect a disinhibited conversational style rather than a strategic response to a taxing and stressful situation. This interpretation meshes well with recent neuropsychological accounts of autism and related conditions that propose impairments of frontal-limbic systems involved in inhibitory processing (4.6).
C. The Nature of Grammatical Impairments in Language-Impaired Children
Background: For many language-impaired children, the most striking problems are in use of grammatical morphology. There is a tendency to omit inflectional endings (such as past tense -ed) and function words such as auxiliaries. In recent years, there has been an explosion of theory and research on this topic (see Bishop, 1992), stimulated partly by linguists who have proposed that these children may lack a specific module of an innate language-learning device (e.g. Gopnik & Crago, 1991). This view has been contrasted most strongly with that of Leonard and his colleagues (e.g. Leonard et al, 1992), who have argued that perceptual limitations may impede learning of morphological rules, especially in languages where morphological endings are neither very varied nor very salient, such as English. Both these theories maintain that SLI children lack knowledge of grammatical rules. This would seem to predict, however, that children should show consistent patterns of behaviour -- either omitting inflectional endings altogether, or using them in a haphazard, unsystematic fashion, or simply using them in specific learned contexts. For instance, the child might learn that "cats" refers to a group of animals, without having any recognition of the regular relationship between cat-cats, dog-dogs, etc.
C1. Grammatical Analysis of SLI: Bishop (4.7) analysed speech samples from 12 SLI children with severe grammatical problems and found that their production of grammatical morphology did not fit any of the predicted patterns. Thus although omissions of inflections were common, children did produce morphological endings such as past tense -ed fairly frequently, and when they did so it was nearly always appropriate. Furthermore, variability in production of inflections was not word-specific: thus the same word would be produced correctly inflected in some contexts, but (wrongly) uninflected in others. This led to a search for factors that might account for the variability in performance. In fact there already exists a body of research that is relevant to this issue, although it has tended to inhabit a theoretical vacuum, unrelated to mainstream studies. This is work on "linguistic trade-offs", which has demonstrated that complexity at one level of language processing (e.g. phonology) can influence accuracy of performance at another (e.g. grammatical morphology). This line of work suggested that limited processing capacity might be affecting the ability of SLI children to carry out the operations necessary to compute and retrieve morphological endings. If so, accuracy should perhaps be affected by the complexity of the message they were attempting, and the amount of material already formulated. Tentative support for this notion was found in the corpus of data studied by Bishop (4.7). The likelihood of producing the correct morphological ending decreased with serial position of a word in the utterance.
FUTURE PROPOSALS
A. Etiology and Prognosis of SLI
A1. Twin Study: The principal aim is to home in on a clearer definition of the phenotype for SLI, and to find markers for the phenotype that will distinguish teenagers and adults with a past history of SLI from those without such history. Subsidiary aims are (i) to use genetic data to test the notion that SLI is a qualitatively distinct disorder, as opposed to simply a quantitative departure from normality; and (ii) to use twin data to test the theoretical basis for associations between language disorder, literacy problems and abnormal motor development and lateralization.
In the past few years it has been increasingly recognised that methods from behaviour genetics can answer questions that go far beyond simple estimation of heritability. Twin data can throw light on the definition of the phenotype, and on causal relationships between correlated impairments. For instance, we know that impaired motor performance is common in children with SLI: is this a manifestation of the same underlying condition, or a coincidental phenomenon? An extension of the DeFries-Fulker method enables us to see whether a common genetic factor can account for co-morbidity between two disorders. If the same genetic factors lead to both language and motor deficits, then poor language performance in the proband should predict poor motor scores in the co-twin, with the extent of impairment depending on the strength of genetic relationship. This method will be used to examine the nature of the relationship between SLI and (a) slow motor performance on a tapping task (cf. 4.68); (b) laterality on the tapping task (cf. Bishop, 1990); (c) reading and spelling disability (see Bishop & Adams, 1990a); and (d) measures of phonological short-term memory (Gathercole & Baddeley, 1990).
There are two important reasons for retesting the existing sample of twins. First, the data obtained so far point to further measures that will illuminate our understanding of the phenotype; second, by retesting children after an interval, we can assess notions about how the presentation of disorder changes with age (see 4.8).
In terms of further measures, now that a genetic basis for SLI has been demonstrated, the search is on to find the best marker for the phenotype. Collaborative work is planned with Professor Paula Tallal of Rutgers University to look at heritability of performance on a task of rapid auditory processing. Tallal has for many years argued that the linguistic manifestations of SLI are secondary to a more fundamental temporal processing deficit, such that the auditory perceptual system has limited temporal resolution and so cannot adequately process rapidly changing or brief signals. In more recent work, she has proposed a neurobiological basis for this disorder, linking it to work suggesting analogous problems in handling transient information in the visual system in reading-disabled children. Her theoretical orientation contrasts sharply with that of researchers such as Gopnik, who argue for specific syntactic deficiencies, and with hypotheses put forward by Gathercole and Baddeley (1990) and by Bishop (4.7), who suggest that SLI may be due to fundamental deficits in components of a working memory system. It will be of particular interest to devise tests which pit these hypotheses against one another.
In addition to studies of the existing twin sample, Bishop has been asked to collaborate in a new large-scale longitudinal twin study which will be carried out by Professor Robert Plomin, Professor Michael Rutter and Dr Emily Simonoff at the Institute of Psychiatry in London. This study aims to investigate the heritability of mild mental impairments, and will also provide an opportunity to use genetic data to validate the distinction between specific and global developmental disorders involving language. A large national cohort of twins will be screened starting at 2 years of age to identify those with significant developmental delays, with affected cases being subjected to more detailed investigation.
One factor emphasised by the twin study that has just been completed is the way in which the presentation of SLI can change with age. On the basis of medical and speech therapy records, it seems that many children continued to show significant improvement as they progressed into adolescence. Bishop will collaborate with Professor Maggie Snowling and Dr Carole Kaplan of Newcastle University in a follow-up of a sample of SLI children originally studied at the age of 4 years by Bishop and Edmundson (1987). The sample is now aged 15 to 16 years; by studying them we will be able to specify more closely the long-term prognosis of SLI, and document how patterns of impairment change over time.
A2. Family Studies: It is usually assumed that a single dominant gene could not be responsible for causing SLI because pedigree data do not fit a Mendelian pattern of inheritance. That was the case in the twin study, where many twins with SLI did not have an affected parent. However, caution is needed. As noted above, the presentation of SLI changes with age, and heritable forms of disorder may resolve as children grow older. In future research on SLI, we aim to study cases of "resolved SLI" to evaluate proposed markers of residual problems; these should enable us to re-define the phenotype to include cases of transient problems. Tallal is currently involved in family studies of SLI, which provide evidence that the temporal processing deficit can be demonstrated in adult relatives of SLI children, who do not have obvious indications of language difficulties. The temporal processing test is, therefore, a promising marker of the phenotype, although it is not the only one we shall be using. In general, the best candidate markers are novel tasks which subjects have not had an opportunity to practice, such as repetition, reading or writing of nonsense words.
A3. Comparisons with Control Twins: The DeFries-Fulker method of analysis provides an estimate of group heritability, i.e. the extent to which genetic factors can explain a language deficit. This may differ from conventional estimates of heritability of a trait (h2), which estimates the genetic contribution to individual differences within the normal range. If there is a single factor, such as a major gene, that depresses test scores in a minority of individuals, then these two heritability estimates could differ substantially. One may draw an analogy with height: variations in height in the normal population have moderate heritability and appear to be influenced by a wide range of genes; however, certain forms of dwarfism are caused by a single major gene that is of very rare occurrence. Thus the factors that cause severe limitations of stature are different from those that influence height in the normal range. A comparison between group heritability (i.e. role of genetic influences in causing impairment) and conventional individual heritability will help to determine whether a developmental disorder simply corresponds to the tail of the normal distribution of ability, or whether it is a qualitatitively distinct disorder with a specific etiology. The very high estimates of group heritability that we obtained for our language measures suggest that the causes of disorder are different from the causes of normal variation; but, to answer this question properly, we need to investigate heritability of the same language abilities in the normal population. To this end, a sample of twins of similar age and nonverbal ability will be recruited and given the same language assessments.
B. Experimental Studies of Children with Pragmatic Difficulties
The in-depth analysis of conversations that has been entailed by work so far has suggested a number of hypotheses about the underlying nature of pragmatic problems. These are not mutually exclusive, and there is some suggestion that different problems may predominate in different children. More systematic testing of these hypotheses will be carried out using experimental measures.
B1. Differentiating Explanations of Semantic-Pragmatic Disorder: A popular view is that pragmatic difficulties arise in children who have limitations of social cognition, akin to, but less severe than, those seen in autistic children (Bishop, 1989). Bishop and Adams (1989) noted that a common source of problems was a failure to match the message to the listener, so that a child would either tell a conversational partner something they already knew, or would fail to provide information that was crucial for understanding. Both characteristics tended to co-exist in the same child, suggesting a fundamental problem in understanding how much shared knowledge the interlocutor had. If this does reflect a problem with social interaction, we would expect to be able to demonstrate two kinds of associated impairment: (i) abnormalities of nonverbal communication (e.g. in the use of gaze) and (ii) problems with tasks that involve "theory of mind". To date, however, attempts to demonstrate such problems using experimental tasks have been largely unsuccessful; as well as the referential communication task used by Bishop and Adams (1991), there are unpublished data on a theory of mind task, and results from an undergraduate project looking at use of eye contact in conversational settings. In none of these studies have any deficits been shown in children with semantic-pragmatic disorder. This has led to the formulation of alternative explanatory hypotheses (see below). Before ruling out defective social cognition as an explanation, however, we need to see whether children who seem insensitive to the interlocutor's needs in conversation do have difficulty with more complicated tasks testing theory of mind.
A more linguistically-based hypothesis proposes that conversational problems may arise because the child uses low-frequency words and expressions without fully understanding their meaning and appropriate use. For example, Bishop and Adams (1989) described a boy who seemed to use "because" as a general purpose connective, and other children use discourse markers such as "well" and "by the way" in a haphazard fashion at inappropriate points. This gives their conversation a superficially adult appearance, but confuses the interlocutor, who, for instance, will anticipate some kind of topic shift after hearing "by the way". We will determine whether children who show this kind of behaviour have problems in judging acceptability of utterances that include these kinds of expressions in appropriate and inappropriate contexts.
One conversational feature that does not seem to be explicable in either linguistic or social terms is the tendency to drift off topic. This is an elusive problem that has been very difficult to pin down, illustrated in the following example;
The child's final turn has a clear link to the content of the preceding utterance, but there is no link to what may be termed the global topic that has been defined by the preceding stretch of conversation (which may be loosely characterised as "parties"). This suggests that the child has problems in building a mental model that incorporates new information into a structure containing all the salient prior information. This account has something in common with Caplan et al's (1990) proposed explanation for "loose associations" and "illogical thinking" in the connected speech of schizophrenic children who manifest frank thought disorder. They suggested that limited processing capacity and/or distractibility lead to difficulties in keeping track of topic in a conversation. Problems in inhibiting irrelevant thoughts may lead to intrusion of unrelated material into the conversation. Although these conversational features are far more severe in schizophrenia than in the language-impaired subjects studied by Bishop and colleagues, they seem qualitatitively similar, suggesting that it would be worthwhile looking more closely at attentional processes and ability to inhibit irrelevant responses in children who show this pattern. We are developing a task which is designed to assess facilitatory and inhibitory attentional effects, adapting a paradigm from mainstream experimental research on attention (including work by Duncan: see his section in the programme on Attention and Cognitive Control) to make it suitable for young subjects.
B2. A Checklist for Identifying Semantic-Pragmatic Problems: Pilot work has been done to develop a checklist, which takes 23 areas of communicative functioning which are not easily assessed by conventional methods. Included are several symptom areas which are characteristic of semantic-pragmatic disorder, such as ability to mesh conversation with that of a partner, conversational assertiveness, nonverbal communication, prosody, and tendency to give tangential responses. For each area, a teacher choses from five descriptions the one which is most like the child. Inter-rater reliability will be assessed with the help of staff at schools for language-impaired children. If these behaviours can be measured reliably, this will provide a more solid and objective background against which to evaluate findings from conversational analysis, as well as indicating whether the symptoms of semantic-pragmatic disorder do group coherently.
B3. Application of the Conversational Analysis Procedure to Other Clinical Groups: Once we have an analytical method that is both reliable and valid in identifying pragmatic difficulties, this will be a valuable tool for investigating conversational competence in other clinical groups who have been described as having communicative problems that are not easy to quantify with existing assessments (e.g. head-injured patients with frontal lobe impairments; schizophrenic patients; individuals with fragile X syndrome or Asperger's syndrome). Informal links are already established with Dr David Skuse of the Institute of Child Health in London, who is keen to apply this method to girls with Turner's syndrome. These children have average verbal abilities, with poor visuospatial skills. Recent work has documented problems with social relationships in this population. The typical picture is of a child who presents as sociable and unusually mature in interactional style, but who has difficulties in making friends. A couple of cases seen in collaboration with Dr Skuse had oddities of communicative style when analysed in detail, particularly in terms of inappropriate use of adult-like discourse markers (see B1 above). It is predicted that conversation analysis procedure would discriminate girls with Turner's syndrome from a control group and would throw light on their social impairment.
C. Experimental Studies of Grammatical Deficits in Children and Adults
C1. Manipulating the Conditions Under Which Grammatical Errors Occur: The aim is test the notion that grammatical deficits reflect limited processing capacity by manipulating the amount of verbal and nonverbal material that has to be processed in a task, and observing the effects on the ability to produce appropriate morphological endings. If children with SLI have not learned grammatical rules, then morphological deficits should be apparent in all situations. If one can manipulate error rates, this would be evidence against a 'lack of competence' account. The principal motivation for these studies comes from previous work on children with SLI. However, similar issues arise in the study of adults with acquired aphasia, where it has been suggested that agrammatic symptoms may reflect performance limitations rather than a lack of linguistic competence (Hesketh & Bishop, 1994), and it is intended to extend this work to investigate the limited-capacity hypothesis. Similar test materials and methods would be appropriate for individuals with developmental and acquired language disorders. Subjects will be presented with sentences that have either a correctly inflected verb, or an obligatory inflection omitted, or an inflection wrongly added. The task is (i) to repeat the sentence and (ii) to make a grammaticality judgement. It is predicted that accuracy of production will decrease with serial position of the inflected item in the utterance, in line with the pattern seen in spontaneous speech data. Grammaticality judgement is often regarded as an index of "competence" rather than "performance", but it does require the subject to generate and evaluate a grammatical representation of what has been heard; it is therefore possible that, if parsing took place unusually slowly, or representations decayed very fast (see also Baddeley et al, submitted), then one would see problems similar to those observed in production, with the language analyser developing an ever-increasing backlog of material as sentence length increased. Other variables that would be predicted to influence accuracy if a performance limitation was implicated are (i) rate of presentation; (ii) inter-item interval; and (iii) word frequency of the inflected item (on the assumption that low frequency verbs would take longer to access than high frequency words). In addition, there is the question of whether any capacity limitation is restricted to language, or whether performance on this type of task could be enhanced or impaired by requring the subject to perform a secondary, nonverbal task that demanded attention (e.g. pressing a button when a light appeared).
C2. Syntactic Processing in Children with Moderate Hearing Loss: It is often argued that SLI must involve dysfunction of an innate grammatical module, because the grammatical difficulties of these children are so selective and severe. This logic is, however, faulty. A relatively peripheral auditory processing deficit could, in principal, alter the course of grammatical development in just this way (Bishop 1992). To evaluate this notion, tests of syntactic comprehension and production will be administered to children with partial hearing loss to see if similar and selective grammatical deficits can arise purely as a result of peripheral auditory deficits.
STRATEGIES FOR READING AND WRITING (Stark, Wright)
Introduction
One goal of this research is to understand how document design influences the strategies that people adopt when reading work-related materials or information provided for the general public. Readers draw upon a wide range of cognitive resources, including attention, comprehension and memory processes. They also plan actions and monitor their execution (e.g. in order to follow written instructions). A premise underlying this research is that we need to know what strategic activities readers engage in when working with written materials, and whether these activities are helpful or not, before we discover how writers can best support readers through judicious information design. The scientific issue concerns how these diverse cognitive activities are integrated by readers.
People are often economical in their reading behaviour, choosing not to attend to information on the page in front of them. The influence of pictures on such attentional strategies is unclear because most research on pictures in text has examined the effects on comprehension and memory. We have explored the attentional effects of including two kinds of pictures in documents: diagrams offering an overview of the text and line drawings referring to textual details. By controlling when readers encountered pictures during reading we have been able to investigate the cognitive consequences of over-ruling the reader's chosen strategy, and so could assess the benefits of encouraging readers to adopt different ways of integrating text and graphics.
Written instructions pervade both working and private life yet they are often presented in a way that makes them difficult to follow. Some of the reasons for poor instructions are organisational, others lie in the cognitive processes of written communication. A second goal of our research is to understand the cognitive processes of creating written communications. It is known that writers may not appreciate the information needs of their audience, but it is unclear whether this blindspot is restricted to certain kinds of content. We have extended this research in two directions, examining writers' choices about both content and presentation. By manipulating the potential for referential errors in the environment where the instructions would be carried out, we were able to explore writers' presuppositions about readers' knowledge. We also compared readers' preference for words or pictures with writers' choice of representational form when giving route directions. A difference between these choices implies that inexperienced instruction-givers do not consider the full range of design options when writing.
Currently most of the research being done on information design focuses on human-computer interaction. In contrast, many of our studies address issues relating to the strategies people use for reading printed materials. We exploit electronic documents as a convenient research tool and benefit from the advantage that the research can also address design issues relating to the use of computer-based documents. There is a practical need for this. An international survey showed that researchers in the behavioural sciences are increasingly doing part of their work at home, enabled by information technology (4.77). We have therefore examined reading strategies with the new kinds of document structures and the additional forms of reader-support that are possible for computer-based documents (4.85, 4.137).
A. Strategies of Attending when Texts have Pictures(Wright, Lickorish, Milroy)
A1. Attending to Graphic Overviews: The two main issues addressed in this series of experiments were (a) Did people separate or integrate the reading of text and graphics? (b) Did the strategy adopted enhance or impair comprehension and retention? The text was presented on a computer screen and readers only needed to click with the mouse on a label on the screen to access the graphic, which was a verbally labelled box diagram. Most people studied the graphic before and/or after but not during their reading of the text, suggesting that they anticipated cognitive costs from accessing the graphic overview. However, repeating the graphic throughout the text rather than leaving it to be accessed by choice helped readers incorporate the information from the graphic while building the gist of the text, and resulted in better scores on a subsequent quiz. This finding calls into question an explanation of readers' strategies being the result of the cognitive costs of integrating text and graphics. This lack of cost was also found when graphics were interspersed through the text summarising the information up to that point: study time was reduced but quiz scores were still high. In order to check whether reluctance to study the full overview diagram was caused by the amount of detail it contained , the gradually growing graphic was made available to readers, but they still seldom accessed it as an organising aid while reading (4.81). When the text was accessed from the diagram, instead of the other way around, we found that reading times and quiz scores were almost as good (4.175). These studies show that readers' understanding can be improved if they integrate information from a graphic overview while building the gist of a text, but that people may not have the appropriate control/access strategies for doing this (4.174).
A2. Attending to Graphic Definitions: We examined readers' strategies for attending to glossary information while reading. It was hypothesized that people's willingness to interupt their reading would be influenced by the way the explanation was signalled within and accessed from the text (e.g. either implicit or perceptually cued) and also by the representational form (verbal/pictorial) of the explanation (4.11). People read through several texts in which they could click on any unfamiliar words and have the meaning appear in the margin in a form that was either entirely verbal or verbal+pictorial. We found that readers' meta-understanding was a crucial determinant of their access strategy. Implicit cues were adequate for novel words but not for partially known words, where salient flagging of the clickable items significantly increased the frequency with which readers accessed the explanations. The inclusion of graphics had no significant effects, but when the explanation included a line drawing this led to more re-reading of the text, suggesting that comprehension processes associated with higher levels of discourse processing may have been hampered by the inclusion of the picture. Studies are under way to extend this work to animated graphics (see proposals for future work). Although readers could maintain their fixation on the text when verbal definitions were presented auditorily, this too resulted in more re-reading of the text. It is concluded that there are cognitive costs associated with requiring readers to integrate across the modality of representation (words/pictures) and across the sensory modality of input (visual/auditory). These findings motivate the reading strategy people adopted for the studies reported in A1. Since most graphics will amplify details rather than provide overviews, readers may over-generalise a single strategy for integrating text and graphics.
A3. Providing Diagrams: Given that readers do not attend to graphics while reading, it is plausible that neither do writers while writing. This series of experiments examined whether non-professional writers would add graphics to text, and if they did, whether these would be as overviews or as details (4.88). The task was to explain to a stranger how to cross the writer's home town on foot. It was found that these authors rarely included a diagram of the route to be taken, whether writing a letter to a friend or drafting a design for the back page of a leaflet. Yet these authors could draw route diagrams and always included a diagram when assembling the message from preformed elements. Route instructions that included sketch maps were rated as more usable than instructions without graphics; yet even after doing this rating task, people continued to write instructions without including sketch maps. These data show that the knowledge people have as readers is not necessarily accessed when they write. It is suggested that this particular writing task may be speech driven, with the writers imagining themselves talking to a stranger.
B. Search Strategies and Memory when Problem Solving (Wright, Lickorish, Milroy, Stark)
B1. Memory Demands in Find and Compare Tasks: When readers need to compare details in different parts of a document, they must remember both where to go and what they have already found. Providing readers with optional memory aids is an unobtrusive way of assessing the differential memory demands within such tasks. This research was prompted by our discovery that in a find-and-compare task, readers' choice of procedure for moving around a document varied with the content and/or organizational structure of the text (4.84). In a series of experiments designed to clarify the determinants of readers' navigation choices, we examined the adequacy of five models which differed in predicting whether readers would select the procedure having: (i) the fewest actions, (ii) the smallest memory demands, (iii) the greatest ease of learning, (iv) the simplest overall heuristic, (v) the fastest cognitive computation times. In our first study the best predictions of navigation strategy for various search tasks came from the cognitive computation times (an analysis first proposed by Card, Moran and Newell, 1983). When the navigation procedure that had been chosen most often was modified, the analysis of cognitive computation times predicted readers would choose an alternative navigation procedure, but they did not. Encouraging readers to use the rejected navigation procedure showed that it gave faster solution times and did so without increasing use of the retrospective memory aids. Across this series of experiments the use of memory aids was strongly predicted by the length of the navigation procedure chosen (4.85). That is to say, search strategies determined memory demands, not the other way round. Further studies showed that people's navigation choices were strongly influenced by perceptual factors (4.86). This series of studies has seen the development of a powerful new paradigm for exploring readers' meta-memory, through the use of on-line memory aids. Our data suggest differences between retrospective and prospective memory in this kind of reading task. This research also raises issues about the kinds of cognitive support that can, and need to, be provided for people working with computer-based documents (4.137). Wright has summarised the support readers need when undertaking different kinds of search tasks within documents (4.133) and has overviewed the design implications arising from the cognitive demands of readers needing to move from place to place within electronic texts (4.138).
B2. Memory and Rhetorical Assignments within Non-Linear Texts: When people are reading texts in which additional verbal information can be summoned into view, this additional information may be accorded the status of an aside or it may gain salience from the act of summoning it. Stark showed that when people were searching texts for items having a specified combination of target features, this separation of the information sources improved readers' memory for the material, compared with leaving all details in the main text. Far from subordinating the information, the use of a pop-up window seemed to add emphasis (4.74), although readers' performance was disrupted if the summoned information occluded the main text (4.75). Several of the general design issues surrounding the use of non-linear documents have been discussed (4.136, 4.170, 4.172). Stark has provided a detailed analysis of the computer-based tools that can assist problem solvers when faced with taking decisions in multi-attribute contexts (4.132).
B3. Impact of Computers on Creativity and Problem Solving: For people inexperienced in using computers, the requirement to work with computer-based information may be analogous to imposing a secondary task. While this may only slow down routine activities, it may seriously hamper creative problem solving. This has implications for documentation as writers increasingly keyboard their own materials instead of passing manuscripts to typists. Wright examined the effects on creativity when students of architecture undertook a five-day design project working with computers and also by traditional means. No decrement was found for the computer-based system although the pattern of work changed and students worked in longer stretches when using the computer. The data pointed to some prerequisite skills for success in computer-aided design. Students with high scores on a copy-drawing task and on a spatial thinking task gained high marks for their computer designs (4.129, 4.167). This suggests that the basic cognitive skills needed for creative 3-D design are not changed by shifting the design medium.
C. Following and Giving Directions (Wright, Lickorish)
C1. Avoiding Ambiguity in Verbal Instructions: The findings from our previous research on instructions have been applied to computer documentation (4.134) and to the design of instructions for the general public (4.139). We have carried out a new series of experiments exploring writers' choices about sequencing information and their awareness of the need for increasing precision when giving instructions in contexts where the potential for ambiguity is varied (4.80). People were asked to write instructions for modifying a typescript. It was found that few writers provided an overview and many left readers to do a considerable amount of problem solving to work out where the instructions should be applied. When the typescript was modified so that the potential for ambiguity was increased, writers responded by framing their instructions more precisely but many still remained ambiguous. Writers' choices about sequencing information have theoretical significance relating to the integration of different cognitive processes. It was found that most writers first mentioned the general location (e.g. the second paragraph), then the action to be performed (underline Tuesday), and then gave the precise location where the action was to be carried out (e.g. on the third line). This ordering reflects a compromise between supporting comprehension through a canonical declarative word order, and helping readers formulate a procedural action plan for which the reverse order is needed. It is concluded that giving verbal instructions is a linguistic skill that not all adults have perfected; and writers easily make faulty presuppositions about their readers .
C2. Wayfinding Inside Buildings: People often have difficulties locating destinations within modern building complexes such as hospitals. Because sign-posting needs to answer readers' questions, a survey was undertaken to discover what destinations people were looking for when entering the outpatients department in a local hospital. It was found that one third were looking for a place defined by a person's name (e.g. Mr Smith's clinic), one third were looking for a place defined by medical treatment (e.g. eye clinic) and one third were looking for an architecturally defined place (e.g. room G12). This heterogeneity made signposting in the hospital more difficult than it need be. The problem could be reduced by making one of these categories much more salient in the letter of appointment (4.87).
Signposting within a building can be supported by hand-held maps generated from office desktop publishing facilities. In order to examine the relevance of such maps to a hospital outpatients' department, we devised an experimental procedure with similarities to a Treasure Hunt. It was found essential to modify the first map we produced because of misinterpretations by readers (4.87). This supports the view that empirical evaluation is an essential component of information design. Comparison of people navigating with and without the map showed that map users did not double back on their tracks so often, which suggests they were less often lost. However, they did not reach their destinations any faster because they took time to plan their route; and this they considered to be time well spent (4.82). These findings urge caution in assuming that fastest performance is best where information design is concerned. There is a need for greater understanding of how readers interact with written materials and the value people place on being able to use the information in the ways that they want.
D. Translating Research into Practitioner Domains (Wright)
One indication of the impact of this research on practical issues is the range of occasions during the past five years when we have been asked to relate research findings to specific practitioner domains (e.g. that of the professional technical writer - 4.187). Requests from academic groups outside psychology have resulted in keynote conference papers on the topic of quality in documentation (for the Canadian Centre for Research in the Writing Process (4.135) and the Applied Linguistics Department at the University of Twente in Holland (4.140)); and on instructions for the public (for the Design Engineering Department of the University of Delft - 4.139). Papers on information design were requested by professional groups such as the Printing Industry Research Association (4.169, 4.189). The Association for Clinical Research in the Pharmaceutical Industry invited a chapter on the design of Case Report Forms (4.83). In addition several workshops were held for professional communicators (e.g. University Computer Support Services; clinical researchers, etc). Membership of committees convened by the British Standards Institute has offered another means of bridging the gap between laboratory and practice. Substantial contributions have been made to seven British Standards relating to documentation. Also several papers have been republished in edited books; this too helps to disseminate the research beyond the boundaries of academe (Wright, 1977; 1986; 1988).
FUTURE PROPOSALS
Overview
A major problem in understanding reading and writing strategies is in knowing which psychological processes will be the major determinants of performance such as following instructions or taking decisions. Our work on the influence of design factors on readers' behaviour has shown that predicting the interplay among processes such as attention, comprehension and memory requires situating the cognition within the task constraints and the affordances of the information environment. Our proposed research will maintain its focus on the three subdomains of (a) texts with graphics, (b) documents that are searched and (c) material conveying procedural instructions. Within each subdomain, however, the theoretical focus will be shifted by changing the task demands. For texts having graphics the shift will be towards comprehension and memory processes. Although this is a major focus within the research literature, new issues arise concerning the strategic trade-offs readers make between access and comprehension processes when the adjunct materials (whether verbal or pictorial) are animated.
Readers engage in many kinds of search but the few models of document searching that currently exist are concerned with students extracting information from expository text. We propose to examine the relevance of these models to adults filtering information in order to take decisions about multidimensional problems. One of the key issues is the extent to which the order in which searchers consider the various attributes introduces biases into their decision-making. A related issue is whether features of the interface, such as the availability of data-marking and data-collecting tools, can change search strategies.
The theme of reading and writing instructions will build on our previous work but will focus on how readers create action plans. One issue to be addressed is whether readers create a mental representation of the actions to be performed that is independent of the symbolic form in which the instructions are given. By contrasting verbal and pictorial materials, questions about the cognitive comparability of these representations can be answered. In addition we will examine people's ability to comply with instructions on how to write, for example by following a model of the text structure required. This research could be considered an empirical contribution to the theme of translating research into practitioner domains.
A. Comprehension and Memory when Reading Graphics and Text
Background: We have already shown that, during reading, people easily integrate verbal adjuncts to a text but choose not to integrate pictorial information. We propose to explore the role of verbal/pictorial representations in determining these alternative strategies. The scientific question is whether building discourse understanding from a multimodal text has cognitive costs, and whether these comprehension costs are offset by enhanced retention of the material. It is important to extend this research to procedural information because instructions differ from narrative text in three critical respects: (a) readers can deal with the text in piecemeal fashion and may not need to construct higher level discourse representations; (b) the material read will not need to be remembered for long if the instructions are to be implemented immediately; (c) the precise details of the text are of crucial importance. Animating graphic instructions enables the dynamic features of the procedure to be shown. However, there could be disadvantages if readers' strategic options for reflection and review are constrained by the animated display. This expansion of the research will link themes A and C.
Proposal: The convergent objectives of this research will be realised in separate series of studies, each with more limited aims: (1) To ascertain whether the formation of qualitatively different mental representations is one of the causal factors underlying the reluctance of readers to integrate text and graphics while reading. (2) To determine whether the differential advantages of overviews compared with graphics of textual details will apply to animated graphics. It is hypothesised that animation will increase readers' willingness to attend to the graphics but will decrease their retention of the main text for both categories of graphic. (3) To explore how readers cope with procedural instructions given as animated graphic sequences. It is predicted that problems of readers wanting to reflect and review will arise only with longer procedural sequences.
A range of covariate measures will be taken in order to assess the comparability of the independent treatment groups and to enable post-hoc stratification of the data. The graphic adjuncts to the texts will vary in the proportion of verbal and pictorial information they contain, and across different groups will vary in amount of animation. In order to investigate the interplay among attention, comprehension and memory processes, the comparisons will be replicated with different ways of accessing the graphic materials and with graphic displays that either do or do not occlude the text. Presentation will be self-paced and people will be free to re-read the text and graphics if they wish. In all studies the main measures of performance will be (i) reading pattern - i.e. the frequency of switching between the text and the graphics, (ii) time to complete the task, (iii) time spent studying the text, (iv) time spent studying the graphic, and (v) performance on immediate and delayed retention tests. In addition, for the procedural instructions, the reading pattern will include switching between the instructions and the domain in which the instructions must be carried out, and retention will also be indexed by errors and memory lapses in carrying out the instructions.
B. Attending to Multiple Criteria when Searching
Background: Our interest in readers' search strategies has been motivated by its usefulness in highlighting some of the concurrent memory demands involved. Previously, in tasks having well-defined search targets, we examined the prospective and retrospective memory demands when several items had to be found and then compared on a single attribute; but there are many other kinds of search task. We propose to examine searches involving multiple criteria (e.g. someone looking for a GP who is in a small practice that is fairly near their home and which has ample parking space). Here people reduce the memory problems by attending to successive subsets of criteria. This can be done in several ways and different search strategies may lead to different kinds of errors. Little is known about the strategies people adopt when searching documents for multidimensional targets, nor whether their search strategies are malleable and influenced by the perceptual characteristics of the display or by whatever aids to searching are available.
There already exists evidence that people search multi-attribute arrays more rapidly if these are iconic rather than verbal, but the generality of this finding could be limited by the structure of the document. Recent work by May and Barnard has provided a model of the interaction between the features of icons and the visual structure underlying the interface of a software application (May, Barnard & Blandford, 1993; May, Barnard, Boecker & Green, 1990; May, Tweedie & Barnard, 1993). This can be extended to highly formatted text structures such as tables. Readers' search strategy may also be influenced by the document tools that they have available for highlighting items for further consideration or removing items that need not be considered further. The use of such tools relates to the design of cognitive prostheses supporting attentional and memory processes. This overlap of the areas of search and decision-making and problem solving promises to be a fruitful conjunction for highlighting the strategic control of several cognitive processes.
Proposal: One objective will be to investigate how adequately existing models of readers searching prose texts (e.g. Guthrie, 1988) will apply to people "filtering" information in order to take a multi-dimensional decision. A second objective is to determine the extent to which readers' search strategy depends on situational factors such as the symbolic representation of the search features (verbal/iconic), the document structure (table/list), or the filtering tools available.
A novel research procedure will need to be invented in which people can be presented with electronic documents containing multi-dimensional information about familiar entities, such as houses or shrubs, and be asked use this information to find the item that best meets a particular set of requirements. Special-purpose, computer-based tools will be made available to assist searchers, and will differentially assist in excluding non-contenders or highlighting potential targets. A range of covariate measures, including verbal and pictorial memory, will be taken in order to assess the comparability of the different treatment groups and to enable post-hoc stratification of the data. Search strategy will be revealed by the pattern of tool-use (i.e. whether searchers drop items out of the attended set or highlight items into this set). The speed and accuracy of solving the problems will indicate which strategies are the more efficient.
C. Following and Giving Directions
Background: Having already established that people are not very successful at giving written instructions, we propose to shift the research focus from writing processes to reading processes, or more specifically to the creation of plans for action on the basis of what has been read. Most research on verbal directions has focused on the linguistic characteristics of discrete instructions involving only one or two steps. As a consequence very little is currently known about people's reading strategies when following a lengthy sequence of instructions. Here meta-memory processes could play a crucial role, since the readers' success will depend in part on knowing how much to try to remember. Remembering too little means returning often to read the instructions; attempting to remember too much risks errors.
The need to communicate instructions across language boundaries has increased the use of pictorial instructions, but there are also issues about how easily people can remember a pictorial sequence compared with a verbal description of the same procedure. In discussing executive control functions, Barnard has pointed out that the representation adopted by learners can have a crucial effect on how the content is processed and thereby on what is learned (Lee, 1993). If readers construct a mental model of the procedure to be carried out, then alternative presentations of instructions (verbal/graphic) may influence the ease with which the model can be created, but once built there should be no residual differences in ease of retention or application. On the other hand, if people's mental representations vary with how the instructions are given, this may have a range of performance consequences for speed, accuracy, confidence, etc. There could be important differences in the mental representations people choose to create for visuo-spatial and visuo-temporal instructions. Green found that for people trying to understand the temporal contingencies represented in programming languages, visual notations similar to circuit diagrams were less helpful than verbal representations (4.36, 4.37). For visuo-spatial contingencies the reverse might be true.
There are contexts where instructions can be given by providing readers with a model of the output required rather than a detailed sequence of steps for achieving that output. Organisations that have documentation house styles often use models in this way. We propose to explore factors that may influence the ability of people to match a model of an effective communication.
Proposal: This research will have three major aims, each realised in a separate series of experiments. One aim is to ascertain whether readers are sufficiently aware of the meta memory demands of instructions to segment them into appropriate chunks when creating action plans. If it is found that readers' segmentation strategies are inappropriate, then design remedies will be sought - e.g. by providing visual segmentation cues on screen. A second objective is to establish whether, for instructions that are to be carried out immediately, a common mental representation for action plans is created irrespective of the symbolic form of the instructions (verbal/graphic). The third objective is to explore whether people can generate appropriate action plans to write in a way that follows a textual model. Models differing in their display features and in their underlying organisation will be used. We predict that, because it enhances attentional control, people will be able to follow a model more successfully when they have had the opportunity to contrast this with alternative textual models.
In order to study the issues of segmentation and representation we will develop a computer based two-window task, one window for the instructions and another for the "machine" on which the instructions must be carried out. Readers will summon the instructions step by step, switching at will between the reading and acting windows. The time spent and the activities undertaken in the reading window (e.g. requesting the next procedural step or reviewing previous steps) will indicate the segment size within action plans. Success in carrying out the instruction provides an index of readers' understanding and memory of the instructions.
The studies of the mental representations adopted for action plans will employ an interference paradigm. If readers retain many of the surface features of the instructions, then doing a verbal task such as unscrambling proverbs after reading the instructions and before carrying them out should be more harmful to verbally presented instructions than to graphic ones. Conversely a visual rotation task, such as the manikin test, should be more disruptive when instructions are pictorial rather than verbal. Appropriate control conditions will show if readers create action plans to minimise interference from the intervening task.
The studies exploring whether textual models offer a way of enhancing the quality of written directions will use written exemplars differing in their underlying structures and in their display features. People will be asked to follow these models when organising information extracted from a neutral source. In addition, people will be asked to assess the suitability of a range of models for particular audiences and purposes. These assessment data will address the issue of the relation between sensitivity as a reader and empathy as a writer. Measures of attitudes to writing and writing abilities will ensure the comparability of the different treatment groups. The dependent measures will include the closeness of the writer's style to the model and the adequacy of the details extracted from the information source.
D. Translating Research for Practitioners
There are no signs that the demand for "translation" activities will decrease. Certainly BSI activity continues. The proposed research on people's use of documentation models contributes directly to this theme.
NOTATIONAL STRUCTURES AND INFORMATION ARTIFACTS (Green)
Introduction
Much communication takes place in other ways than through the medium of unconstrained natural language; artificial systems, such as charts, diagrams, tables, and notations are in wide use, forming a class of 'information artifacts' whose purpose is to represent and to communicate. With the growth of computer-based technology have come artifacts which can store and manipulate information, such as personal calculators, data-bases, word processors, systems for CAD (computer-aided design), etc. As is only too obvious, there is still much to be done to help designers avoid putting unrealistic demands on users' abilities. Wealth creation can come, in this context, from improving the competitive edge of British and European design.
Quite apart from their immediate practical relevance, artificial systems are interesting vehicles for study because they can be refashioned (unlike natural language), because they combine linguistic and graphical elements, because they are small enough to allow formal analysis and therefore to allow experimental manipulations, and because they engage several aspects of human performance such as subgoal tracking, the perception of structure (= parsing, when the material is linguistic), and trade-off decisions between alternative strategies. To understand the determinants of behaviour, especially the trade-off decisions, it is necessary to be able to describe the cognitively meaningful properties of artifacts.
Structural features called 'cognitive dimensions of notations' are the corner¬stone of Green's approach, each dimension being one structural feature that is more or less independent of the rest. These 'dimensions' are applicable to every kind of information artifact – not just spreadsheets, but also to CAD systems, programming languages, outliners, etc. Indeed, they would also be applicable to non-interactive information structures, such as timetables, music scores, and the complex patterns of technical prose.
There are two aims to this analysis. On the theoretical side, before we can apply laboratory-based psychology to wider contexts, we need a way to relate the wider contexts to the purified tasks studied in the laboratory. The theoretical aim is therefore to develop a well-defined system of analysis that will provide metrics on, for instance, the number of subgoals that must be created in order to perform certain operations, and to use such analyses to test the applicability of psychological models. In so doing, it has become apparent that existing psychological models need to be extended to deal with the different properties of artificial systems as against natural ones (for example, parsing models need to be extended to deal with programming languages rather than natural language).
The practical aim is to provide a framework of terms that is readily comprehensible to domain specialists, so that a person who used, say, architectural design systems would quickly understand how the dimensions were to be interpreted in that domain. Ideally, these dimensions would be recognised as names for problems or concepts that had a familiar feel, even if they had not been explicitly recognised before.
It is easiest to discover the relevant 'cognitive dimensions' by comparing several different information artifacts; just focusing on a single artifact fails to reveal enough ways in which it might have been different. By way of example we contrast the familiar spreadsheet with a recent development, the box-and-wire style of 'visual program'. The dimensions have been grouped under four headings relating to reading, changing, learning, and testing the validity and usefulness of the framework as a whole.
.jpg)
A. Reading Notations
It is a truism of cognitive psychology that people assign structures to text, scenes, etc., and act on the perceived structure; but the realms of artificial notations (and especially mixed representations, where textual elements conspire with graphical) have been little explored. Research in this area has intrinsic interest – can a single parsing model cover such a wide span, or must we posit different cognitive processes for different notations? Research in this area also has very direct application to the development of notations and working environments. Indeed, the general problem of how structure is assigned is one of the most intriguing questions in psychology.
A1. Role Expressiveness and Structure Parsing: The first research question is how to design a notation that is 'role-expressive', i.e. easy to parse into higher-order structures.
A1.1 Parsing notations: A good model of parsing would tell us how to design for easier parsing. The particular problems of parsing visually-presented partly-graphical representations are that parsing can start anywhere, may have very long range dependencies (although usually rather simple types of dependency), has no analogue to the verb phrase, and can proceed by developing islands of structure in any convenient order. This is unlike parsing spoken natural language, the paradigm for most psycholinguistic models of parsing.
Evidence from many sources shows that programs in conventional programming languages are schematised into structures such as that shown below, which bind together several possibly-discontiguous components, and there is every reason to suppose that other notations are similarly schematised into appropriate structures. Green and Borning (4.33) used an extended form of unification parsing to develop a model capable of parsing short programs in Pascal and Prolog into such schemas.
Sum: = 0; addup2( 0, 0 ).
for J: = 1 to N addup2( N, A ) :-
begin read( X ), N1 is N-1, addup2( N1, B ),
read X; A is B + X.
Sum := Sum + X
end
Underlined components illustrate a 'running-total' plan in Pascal (left) and Prolog.
A unification parser has many advantages as a model of parsing visually-presented material, since it is not tied to word order and since it can incorporate non-lexical information such as indenting and colour cues. Green and Borning predicted that Prolog would be harder to parse than Pascal because its syntax contains fewer keywords, delimiters, and other equivalents of closed-class morphemes. Preliminary experiments by Green and Duff confirmed these predictions: highly experienced Prolog and C programmers were shown programs in their 'native' language from which small pieces had been snipped, and were timed as they matched the excerpts with the gaps. Equivalent programs for three algorithms were used, each with three levels of complexity and with snips made at equivalent points in the semantic structure; identifiers were suppressed to avoid lexical cues, so that subjects were forced to answer by parsing for meaning. Results showed that Prolog subjects took longer and that the increase of time with complexity was greater for Prolog. Notation designers should therefore include cues to help readers identify higher-level structures.
A2. Easing the Parsing Operations: Comprehending graphics versus text: Despite the conventional wisdom that graphical notations are easy to understand, graphical representations are not necessarily better than textual ones. Green, Bellamy and Petre compared comprehensibility of compound conditional expressions in two forms of textual notation and two forms of 'box-and-wire' graphical notation, derived from LabVIEW, a commercial graphical programming language in use at the APU. Subjects in one group were familiar with LabVIEW and had at least some exposure to conventional programming languages, another group were professional users of a very similar graphical notation. Short fragments of programs were presented and subjects answered either a 'forwards' or a 'backwards' question (forwards: what will this program do if X, Y, Z are true? backwards: this program did that, which of X, Y, Z must be true?). All subjects, whatever their experience, performed faster with textual than with graphical notation. Forwards/backwards differences depended on the type of information structure, as predicted. A serious difficulty with the graphical notation is that it requires a version of perceptual maze-following, in which pointers to choice-points have to be stacked up in working memory. Alternative schemes need to be created (4.36, 4.37)
Pointer structures and comprehension: Certain artificial languages also possess 'pointer' or 'indirect' forms of reference, in which an operator is applied not directly to an object but to a second object to which the first object refers. It is widely believed that this is difficult, at least for less experienced users. Green and Baurén modelled the process in a production system language and showed experimentally that similar effects could be demonstrated in a non-programming context. Their analysis was that the over-riding problem in comprehending pointers is that one fragment of the mental representation must be rewritten twice (once to 'dereference' the pointer – i.e. look up its value – and once to dereference the pointee). Their experiment modelled pointer comprehension with a task in which subjects had to mentally replace substrings found in larger strings, following a set of rewriting rules; difficulty was exacerbated when the same fragment was rewritten twice, supporting the basic analysis. Further experiments will be needed to complete this work, but the implication is that notation designers should avoid pointer structures except for expert use. (4.142)
A3. Improving Role-expressiveness
A3.1 Understanding object hierarchies: According to Green's approach, a good notation allows easy parsing for the relevant higher-order mental structures ('role-expressiveness'). 'Object-oriented' programming systems offer an opportunity to test this hypothesis. These systems model the world as objects which interact via messages. Thus, the 'world' could contain a tiny eco-system – cat, mouse, cow, milk, grass, cheese – and messages could pass between these objects to set up a food-chain. It has been claimed that this approach has cognitive benefits over conventional programming.
Typically, however, object-oriented systems only allow the programmer to view the program according to a strict hierarchical structure, a sort of taxonomy. Although in the programmer's conception of the program, objects of one class may interact with objects of other classes to form a coherent substructure such as a food chain, there is no way to represent that conceptual structure; it has to be deduced from the individual messages. Comprehending the program as a whole is made much more difficult. One solution is to add a 'description level' allowing experienced programmers to externalise their own knowledge of a complex program, and to provide means for browsing this description. This prospect has been expounded in detail (4.35) with a cognitive rationale for the various suggestions.
Green led a 3-year JCI-funded project to create a 'Cognitive Browser' as a component of an object-oriented programming environment being constructed at UCL. For technical reasons, the UCL site was forced to modify original proposals, which made it impossible to produce a working version of the Cognitive Browser in the time available. The team nevertheless carried out in-depth studies of the implementational and architectural aspects, together with a wide variety of empirical investigations into cognitive representations of programs, the process of program design, and the utility of graphical representations of programs and of hierarchical classification structures. (Much of Green's work during the present reporting period was related to this project, but has been organised under topic headings here.)
Davies et al. showed that the object hierarchy was by no means the only determinant, often not even the major determinant, of mental structures. Blumenthal (Green's RO) showed that the form of external representation played a significant component in the elicited structure. These studies confirmed the main hypothesis, that programmers' mental representations are much richer than the code representations of the programs (4.25, 4.144).
A3.2 Role-expressiveness in spreadsheets: When it became clear that a working cognitive browser would not become available in time for experimental use, a small-scale system, 'CogMap' (cognitive mapper) was built by Hendry and Green. CogMap replicated the main functionality of the Cognitive Browser proposal in the less demanding domain of spreadsheets. Heavy spreadsheet users were recruited and requested to describe in detail the workings of an actual spreadsheet which they had constructed. All their assertions about the roles of components were successfully translated into the CogMap system, thus demonstrating the adequacy of its representational system (4.38). These interviews also revealed much about how spreadsheet programmers developed personal conventions to represent non-programming facts about their code (where different types of data came from, which parts described different aspects of the enterprise, etc.). Spreadsheet programmers evidently choose to trade-off increased role-expressiveness against increased demands on working memory to keep track of how their conventions are being applied (4.40). Similar interview studies by Green and Petre of professional electronics designers using CAD found very similar usage of personal conventions, as a form of secondary notation, showing that role-expressiveness was relevant to a very different design domain (4.65, 4.66).
B. Changing Structures
B1. A Meta-Notation to reveal Viscosity and Hidden Dependencies (Green, Benyon, Petre): Some information structures are easier to update than others. For instance, updating all the cross-references and section numbers in such a text as this can be very lengthy work. In the cognitive dimensions framework, Green calls this effort viscosity ('resistance to local change'). Differing types of viscosity, examples, and methods of treatment are described in (4.29). In a highly viscous system, one task goal may spawn many subgoals which have no direct relation to the main task. As the work of Robertson and Duncan shows, keeping track of subgoals may be a near-impossibility for some people, so knowing how to create systems which reduce or avoid them would be highly desirable.
The problem is to expose this and other structural properties in some other way than by tedious verbal descriptions, and to obtain a workable measure. By considering information structures as a specialised form of data-base, Green was able to apply a modified and extended form of entity-relationship modelling, a well-understood computer science technique. The resulting meta-notation, called entity-relationship modelling for information artefacts (ERMIA), can be used to represent many types of information structure in a single, easily-understood form, which can be displayed graphically or in symbolic form (the latter allowing manipulation by logic programming languages such as Prolog) (4.30). The resulting 'structure maps' display very clearly some of the key structural properties: how many search steps are required, how many components must be changed to achieve a given transformation (=viscosity), locations of hidden dependencies, etc.
C. Learning
C1. Consistency & Regularity: Language systems are not arbitrary collections of idiosyncratic rules; they have a degree of internal coherence, known as linguistic regularity. In the previous reporting period Green presented research on modelling the user's knowledge of consistency in simple 'action languages' such as word processors and spreadsheets using 'task-action grammar', a form of two-level grammar in which one level, the rule schema, unpacked into a collection of closely-related rules. This model was predictively successful but too complex to be theoretically convincing or practically useful.
During the present reporting period, advances in connectionist research have produced impressive models of regularity effects in developmental psycholinguistics, suggesting that similar models could be applied to the related world of artificial languages. Green and Doubleday used a very straightforward back-propagation architecture to assess regularity of an artificial language, using as a metric the number of training trials required to reach a criterion performance at which all the inputs, such as 'delete word', generated values sufficiently close to 0 or 1 at the output nodes, which were interpreted as command keys such as Control + G. They were able to predict the results of all published experiments on regularity effects in artificial languages, except those in which the system being modelled included any substantial degree of syntactic order, for which more sophisticated architecture will need to be used. Their technique also revealed patches of local inconsistency, which could be helpful to designers.
This result raises the possibility of 'plug-in' evaluations for designs for information artifacts. Instead of requiring a specialist to construct a two-level grammar from which consistency can be determined, a simple description of the inputs and outputs could be plugged into a network and evaluated automatically. If the network took too long to learn the language, one could predict that humans would also have difficulty (4.34).
D. Validity & Usefulness
D1. Testing the Cognitive Dimensions Framework as a Whole: Questions to address are whether the framework is reasonably complete and whether it agrees with the evidence on human performance. Many sources of evidence have been pressed into service over the reporting period: conventional laboratory studies, observational studies, expert opinion, and formal analytical methods.
With regard to the issue of completeness, a particular domain of information artifacts has been studied in depth, namely the design of programming languages. Green and Petre carried out a comparative analysis of 5 such languages, contrasting various attributes (textual or graphical, 'neat' or 'scruffy', 'deep' or 'shallow'). The languages were assessed on each of the cognitive dimensions and comparisons made with all extant user studies on each language. This large project revealed both the lack of serious data regarding most usability aspects of languages and environments, and the lack of 'extensible' theory to apply; for example, parsing theories do not extend to these circumstances, and there is no unified theory of 'hard mental operations', only of special cases like self-embedding. Even in the better-researched area of program design, we could find little dealing with the role of visibility and accessibility of one part of the design while working on the next part. Nevertheless, some usable data exist, and straw tests of some dimensions, such as viscosity, were devised; results were highly encouraging, both as regards the framework as an analytical tool and as a way to communicate to non-specialists (4.92).
To be useful in practice, the framework needs to be comprehensible to domain specialists with no prior knowledge of cognitive psychology. To test this Green collaborated with Modugno, a computer scientist at Carnegie-Mellon. Independent evaluations of a visual programming environment were found to agree well, and inspired several design improvements (4.48).
D2. Knowledge Dissemination: Psychologists frequently complain that their knowledge is not used in design. In the case of software, Green's analysis of the present state of knowledge dissemination (4.118) argues that the agendas, publishing practices, and incentive structures of cognitive psychologists and software engineers differ so greatly as to make dissemination particularly difficult. A survey of recently published texts on software engineering revealed almost no penetration of cognitive psychology, although folk psychology was frequently invoked to explain decisions about the design of programming languages and programming environments. Moreover, design decisions were made in isolation, with no awareness of trade-offs and dependencies. The role of the cognitive dimensions framework is to serve as a form of 'discussion tool', allowing shared understanding between the two communities, and helping to take analysis to a deeper level than the usual list of features. This role can only be served if the framework is widely disseminated. Green has therefore started to seek opportunities to publish in outlets that will be read by practitioners of software engineering and information design (4.31, 4.67, 4.93, 4.96).
FUTURE PROPOSALS
Green's work has exposed areas where cognitive psychology offers the opportunity to contribute in new ways, and has extended the areas of applicability of existing theory in interesting and fruitful ways. In addition it has shown that we can discern the trade-offs between different cognitive dimensions. For these reasons, the main plan for the coming five years is to extend this line of work and to formulate a unified account of principles of notational design.
Five major targets need to be attained. The metanotational analysis of structure needs to be taken forward so that each dimension can be given a formal definition, conflicts and overlap can be reduced, and where possible, metrics can be devised; areas which have received little or no attention to date need to be researched; the trade-offs between dimensions need to be studied; the framework needs to be tried out in a fresh domain; and the ideas need to be disseminated to the relevant communities.
A. Metanotational Description of Interdependencies
In the immediate future, the ERMIA project with Benyon and Petre will be taken forward, to decompose the notions of viscosity and hidden dependencies as completely as possible. The targets for the project include testing the usability of the notation by developing a teaching package and evaluating it with Open University students, comparing their between-student agreement in identifying viscosity and other structural properties. Green is also working on applying the notation to understanding the 'emergent' properties of graphs. The classic analysis by Bertin (1981) distinguishes different levels of using a data graph, and maintains that the purpose of a graphic is to support discovering overall relationships; despite the fact that he presents many examples of redrawing graphics to present overall relationships better, however, he has no underlying theory. Structural analysis identifies the mapping relationships and the conditions that must hold in order to make overall relationships perceptible.
A very simple underlying theory is probably adequate: low-level data components must map onto graph components without 'chasm' or 'fan' traps; the graph components must meet perceptual conditions for emergent properties such as clusters; and those emergent patterns must map onto high-level data relationships. The advance is not in the theory but in the development of a description in which the graphical elements are described in the same language as the data they represent, and the perceptual conditions can be stated in the same language too.
B. Areas Needing Research
Certain parts of the framework, such as propensity to errors, have been well enough researched in other contexts to need little special effort except that of making the literature available in a compact form. Other areas need more work, and two stand out at present.
B1. Imposed Look-Ahead: Sometimes, even in quite simple cases, users need to look ahead to know what to do. The ordinary pocket calculator forces users to think ahead, to foresee the need for parentheses. On the other hand, the notation for rewriting text strings in Unix-based regular expression editors, although grisly and unreadable, has the virtue that it does not force look-ahead. Needless to say, far more complex and potentially expensive examples can be cited.
The analysis of imposed look-ahead requires understanding of both the notational structure (and its environment of use) and the user's mental representation. If these two are not congruent, the cognitive process will generate entities in a different order from that required by the notation. Gray and Anderson (1987) give a detailed analysis of the look-ahead problem for novices constructing Lisp conditionals. How serious the consequences are will depend on the degree to which the environment controls the order of actions: if, for instance, notational components must be entered in strict left-to-right order (as in the calculator), the user will have to restart expressions when components are left out; whereas in more forgiving environments, it will be possible to edit the partly-built structure. Repairing errors is a separate theme, however.
The aim of research in this area is to predict the look-ahead problem and to investigate the resources used to cope with it by practised users. The research will require usable descriptions of the notational structure, the transforms allowed by the environment, and the order constraints imposed; these are mainly analytical problems. Developing Green and Benyon's ERMIA notation will go some way to providing the structural description of the notation, but the permissible transforms and the order constraints need a different approach. Various techniques are available for eliciting mental representations; making a choice will be deferred until a domain has been chosen. Research of this type needs subjects who are experienced users of some system in a domain that can be satisfactorily analysed, but there are many possible domains where experienced users are available, including CAD design and software engineering. It is also hoped to build experimental domains in which varying degrees of lookahead are required and varying degrees of order constraint are imposed. This will require programming manpower.
B2. Role-Expressiveness: To extend parsing models into the domain of visual programming, the first step must be to investigate the cognitive representation of visual programs, about which little is known. For text-based languages a variety of methods have been used: priming, line-by-line revelation with guessing at what comes next, free search using a peephole to see in what order experts scrutinised the text, debugging with controlled cues for semantic or syntactic constructs, and inevitably recall of legal programs versus scrambled ones. Free search has high face validity for our purposes. Robertson et al. (1990) were able to segment their subjects' search traces into 'episodes' that related well to surface-level structural features within the code, usually data flow, and concluded that these corresponded to cognitive constructs at the 'program meaning' level. An alternative interpretation is that their results are a surface phenomenon, not a deep one, and that the effects are caused by the subjects' need to cope with creating links between partially hidden dependencies. (Overlooking the need to process such surface complexities bedevils cognitive psychology in various ways.) Repeating the study using visual programs will test their theory, because data flow is a perceptually manifest dependency in visual program, rather than a purely symbolic one: however, other types of surface dependency can be introduced. We would predict that relative novices will be responsive to surface dependencies, while experts will also be guided by the 'meaning' of the program in deciding which part of it to study next. A successful visual parser would model the expert's behaviour, as it built up a model of the program.
The unification-based parsing investigated by Green and Borning was on the right track, but the system developed could only handle small program fragments. A very promising line is unification-based interesting-corner parsing using a generalisation of chart parsing, such as the version developed by Mellish et al (1992). Their parser has been applied to parsing travel information, using excerpts from police road incident logs as the input material, which has much in common with the structure of a program (long range dependencies, proper names, and the need for an approximate semantic interpretation in order to locate interesting corners). However, there is no graphical element to their parser, nor have they sought to discover the determinants of ease-of-parsing.
Research in this area will proceed in collaboration with one or more experts in computational linguistics. We shall adapt an existing model for which some degree of psycholinguistic support already exists; derive testable predictions from it, applicable to mixed-medium notations; then test these predictions on experienced users.
C. Trade-Off Relationships and Performance Effects
The framework describes a number of ways in which a notation that has an undesirable position on one dimension can be improved, but at the cost of changing its position on another dimension. Unwanted viscosity can be improved by introducing more abstractions, for example, so that many low-level actions are replaced by one high-level action; but the management of abstractions creates new problems of lookahead (which abstractions will be needed?) and of hidden dependencies (what will happen if a given component is changed?). Increasing role-expressiveness can frequently increase viscosity, and so on.
If these manipulations can be shown to have the predicted trade-off effects, it will strongly support the framework. With the growth of interest in 'rational analysis', led by Anderson (1993), this will be a timely investigation, for there will be increasing interest in how cognitive costs determine behaviour.
A very useful lead has been given by O'Hara and Payne (1994, submitted) who showed that in simple tasks (Tower of Hanoi and sliding-block puzzles), users adopted more planning and less situation-based action as the effort per operation was increased. One of their manipulations corresponded closely to an increase in viscosity. Sliding-block puzzles are very restrictive, however, and the proposal is to develop environments for more open-ended tasks.
A likely area to investigate is hypertext. Hypertext systems typically have rather high viscosity, many hidden dependencies, no imposed lookahead, and few abstractions. It is not difficult to change that profile. The literature contains many studies of the impact of hypertext on reading and writing (reviewed by Charney, 1994), but the contribution of 'structure-management' has not been separated out – an oversight that may well vitiate many of the conclusions so far drawn in comparisons between different writing environments.
D. Studying a Fresh Domain
The framework needs to be tested in a domain which is unfamiliar and where a variety of different notations and environments are available. Moreover, we wish to distinguish between routine design (creating a variant of a familiar pattern) and creative design. According to the underlying theory of the framework, these two cases will have different profiles of cognitive dimensions (lookahead acceptable in the former, absolutely barred in the latter, for instance, if existing research on design in familiar and unfamiliar contexts is correct). A domain will be chosen where systems for both types of design have been successfully produced, and the systems will be compared to discover whether they differ in the predicted ways. Preliminary discussions have taken place with the Martin Centre of the Cambridge University Dept. of Architecture, which pursues research into CAD systems for creative architectural design.
E. Dissemination
In the practical realm, the overall target of the approach is a body of knowledge on how to design artificial languages in the light of cognitive psychological principles, presented in a form usable by practitioners. Green will be giving tutorials at workshops and conferences and writing monographs and articles in practitioners' journals.
It is not easy to explain the different dimensions in the abstract, less still their trade-offs. Green therefore intends to build small demonstration devices, exemplifying various possibilities (e.g. two versions of a bibliographic system, one with high viscosity and low abstractions, one the opposite). Preliminary versions have already been built for some examples, using very basic tools, but more sophisticated software building kits are now available.
TECHNOLOGY-MEDIATED COMMUNICATION (Sellen)
Introduction
Recent technological advancements have resulted in vast improvements, both technically and financially, for the development of videoconferencing systems. Along with these advancements comes a renewed interest in such technology as a way of enabling people to meet from remote locations. This technology could have a profound impact on the way we work and socialize, and has been suggested for a range of different purposes, from support for working at home, to making medical expertise available to remote geographical areas. The renewed interest in video for communication is reflected by the growing number of experimental video systems or "media spaces" that exist in research laboratories around the world. Sellen was involved in the design of one such media space at the University of Toronto (Mantei et al., 1991). Since arriving in Cambridge, she has been working in collaboration with Rank Xerox EuroPARC which also has its own media space (4.72).
Sellen's research has explored the ways in which communicative behaviour might be altered when mediated by such technology, usually by comparing it to behaviour in a face-to-face situation. This work has focussed on two aspects of communication: the effects of mediation on conversational processes (turn-taking, for example), and the ways in which people alter their behaviour when working together through video-link technology. These theoretical findings have practical implications for the design of technology, but they also shed light on the kinds of cues that are used in communication, and the cognitive processes that underlie collaborative work.
A. Speech Patterns and Conversational Process
Part of the promise of video technology is the possibility of being able to simulate, for remote participants, some of what people share when they meet in the same physical space. What we have yet to understand is the extent to which the visual channel can provide these benefits. Conversely, we have yet to appreciate how video-mediated interaction may differ fundamentally from sharing the same physical space, including the possibility that it may significantly alter conversational behaviour. Two experiments were carried out (4.71, 4.73) to compare face-to-face and audio-only conversations to those held over three kinds of videoconferencing systems. One videoconferencing system was specifically developed to support selective gaze in a multi-party conversation (Sellen, Buxton & Arnott, 1992). Particular attention was paid to effects of the medium on the surface structure of conversation, measured by automatically tracking the on-off patterns of speech by the participants (i.e., turn-taking and the like).
In terms of the speech measures, only the face-to-face condition showed any significant differences from any other condition, giving rise to more interruptions and fewer formal handovers of the floor than the technology-mediated conditions. The results suggest that mediated conversations are more formal and less interactive than face-to-face conversations because participants feel more disengaged from the situation. In this respect, the audio-only and video systems examined in these studies were equivalent. However, additional analyses revealed that visual access in mediated conversations, and the particular design of the different videoconferencing systems, had important effects on subjects' behaviour and opinions. For example, while the system that supported selective gaze showed no benefits in terms of turn-taking behaviour, it did afford conversational acts such as asides and parallel conversations, which the other videoconferencing systems did not.
B. Collaborative Work in a Media Space
A second phase of the research benefitted from collaboration with ethnomethodologists Christian Heath and Paul Luff at EuroPARC. This marked a shift in emphasis from systems which support face-to-face conversation to systems which support collaborative work. Heath and Luff have emphasized the limitations of designing media spaces around face-to-face conversation, pointing out that much of collaborative work is done by people jointly focussing on physical artifacts and shared workspaces, by monitoring one's co-workers in relation to their activity, and by monitoring one's co-workers in the periphery of the field of vision. This led us to suggest ways in which we might reconfigure media spaces to support these aspects of collaboration (4.154). Heath and Luff have also demonstrated that this ability to monitor activity in the periphery, and to achieve the normal performative impact of gesture and gaze, is undermined when mediated by video technology.
Taken together, these findings motivated us to try various ways of expanding access to the remote space by building some experimental systems and observing their benefits and drawbacks in collaborative tasks (4.72). In the first study, we built an experimental system in which four switchable cameras were deployed in each of two remote offices, and observed participants using the system to collaborate on two tasks. The new views allowed increased access to task-related artifacts; indeed, users preferred these views to more typical "face-to-face" ones. However, problems of establishing a joint frame of reference were exacerbated by the additional complexity of multiple cameras, and the disparate views they offered. In the second study, we provided multiple monitors to coincide with the multiple cameras offering co-workers continuous, multiple views of each other. This improved the transition from one view to another, and resulted in more frequent glances at the face-to-face view during conversation; but establishing a joint frame of reference was still a major difficulty. One practical result of these studies is the identification of important features which must be embodied in the technology. From a theoretical perspective, the studies emphasize the importance of a shared physical environment in establishing shared understanding. This "common ground" is necessary for smooth and efficient communication.
FUTURE PROPOSALS
Although work with experimental video systems over the last five years has led to a number of implications for design of this kind of technology, all of the studies thus far have taken place within the confines of the laboratory. The research has now reached the point where findings from the laboratory must be tested in field settings. Currently, with funding from Xerox, Sellen and colleagues at EuroPARC are negotiating the installation of a remote video link between designers of photocopiers at Rank Xerox's site in Welwyn Garden City, U.K., and the manufacturers of these machines in Venray, Holland. The installation of this link will offer a chance to: (1) observe and collect data on behavioural interaction over this link, and (2) implement a range of design ideas. For example, one plan is to provide multiple views (such as a document camera, a face-to-face camera, and a wide angle camera) and allow users at both sites to switch their view on the remote site. This will provide important data about which views are used and when they are used. In addition, blanket videotape recordings of interaction will provide data on conversational interaction over extended periods of time, making it possible to see whether people adapt their behaviour to the technology in the long term. These kinds of observation are obviously difficult to carry out in the laboratory, and will speak to the generalizability of such studies.
PUBLICATIONS (Excluding work done prior to arrival at APU)
Edited Books
4.1. Hoc, J-M., GREEN, T.R.G., Samurçay, R. & Gilmore, D.J. (Eds.). (1990). The Psychology of Programming. London: Academic Press.
Refereed Articles
4.2. Beardsworth, E. & BISHOP, D.V.M. (1994). Assessment of long-term verbal memory in children. Memory, 2, 129-148.
4.3. BELLAMY, R.K.E. (1990). A psychology of programming for design. In D. Diaper, D. Gilmore, G. Cockton & B. Shackel (Eds.), Human-Computer Interaction - INTERACT '90 (pp. 1005-1006). Amsterdam: Elsevier Science Publishers, B.V.
4.4. BELLAMY, R.K.E. & Carroll, J.M. (1992). Re-structuring the programmer's task. International Journal of Man-Machine Studies, 37, 503-527.
4.5. Bird, J. & BISHOP, D.V.M. (1992). Perception and awareness of phonemes in phonologically impaired children. European Journal of Disorders of Communication, 27, 289-311.
4.6. BISHOP, D.V.M. (1993). Annotation: Autism, executive functions and theory of mind: A neuropsychological perspective. Journal of Child Psychology and Psychiatry, 34, 279-293.
4.7. BISHOP, D.V.M. (in press). Grammatical errors in specific language impairment: Competence or performance limitations? Applied Psycholinguistics.
4.8. BISHOP, D.V.M. (in press). Is specific language impairment a valid diagnostic category? Genetic and psycholinguistic evidence. Philosophical Transactions of the Royal Society B.
4.9. BISHOP, D.V.M., Hartley, J. & Weir, F. (1994). Why and when do some language-impaired children seem talkative? A study of initiation in conversations of children with semantic-pragmatic disorder. Journal of Autism and Developmental Disorders, 24, 177-197.
4.10. BISHOP, D.V.M., North, T. & Donlan, C. (in press). Genetic basis of specific language impairment: Evidence from a twin study. Developmental Medicine and Child Neurology.
4.11. Black, A., WRIGHT, P., Black, D. & Norman, K. (1992). Using dictionary information: Some factors influencing whether readers will check the meanings of unknown words in a text. Hypermedia, 4, 145-169.
4.12. Coltheart, V., PATTERSON, K. & Leahy, J. (in press). When a ROWS is a ROSE: Phonological effects in written word comprehension. Quarterly Journal of Experimental Psychology.
4.13. CUTLER, A. (1993). Phonological cues to open- and closed-class words in the processing of spoken sentences. Journal of Psycholinguistic Research, 22, 109-131.
4.14. CUTLER, A. (in press). Segmentation problems, rhythmic solutions. Lingua.
4.15. CUTLER, A. (in press). The perception of rhythm in language. Cognition.
4.16. CUTLER, A. & BUTTERFIELD, S. (1990). Durational cues to word boundaries in clear speech. Speech Communication, 9, 485-495.
4.17. CUTLER, A. & BUTTERFIELD, S. (1991). Word boundary cues in clear speech: A supplementary report. Speech Communication, 10, 335-353.
4.18. CUTLER, A. & BUTTERFIELD, S. (1992). Rhythmic cues to speech segmentation: Evidence from juncture misperception. Journal of Memory and Language, 31, 218-236.
4.19. CUTLER, A., KEARNS, R., NORRIS, D. & Scott, D.R. (1993). Problems with click detection: Insights from cross-linguistic comparisons. Speech Communication, 13, 401-410.
4.20. CUTLER, A., MCQUEEN, J., & ROBINSON, K. (1990). Elizabeth and John: Sound patterns of men's and women's names. Journal of Linguistics, 26, 471-482.
4.21. CUTLER, A. & Mehler, J. (1993). The periodicity bias. Journal of Phonetics, 21, 103-108.
4.22. CUTLER, A., Mehler, J., NORRIS, D. & Segui, J. (1992). The monolingual nature of speech segmentation by bilinguals. Cognitive Psychology, 24, 381-410.
4.23. CUTLER, A. & Otake, T. (in press). Mora or phoneme? Further evidence for language specific listening. Journal of Memory and Language.
4.24. CUTLER, A. & Scott, D.R. (1990). Speaker sex and perceived apportionment of talk. Applied Psycholinguistics, 11, 253-272.
4.25. Davies, S.P., Gilmore, D.J. & GREEN, T.R.G. (in press). Are objects that important? The effects of expertise and familiarity on the classification of object-oriented code. Human-Computer Interaction.
4.26. Franklin, S., Howard, D. & PATTERSON, K. (1994). Abstract word meaning deafness. Cognitive Neuropsychology, 11, 1-34.
4.27. Franklin, S., Howard, D. & PATTERSON, K. (in press). Abstract word anomia. Cognitive Neuropsychology.
4.28. GRAHAM, K.S., Hodges, J.R. & PATTERSON, K. (1994). The relationship between comprehension and oral reading in progressive fluent aphasia. Neuropsychologia, 32, 299-316.
4.29. GREEN, T.R.G. (1990). The cognitive dimension of viscosity: A sticky problem for HCI. In D. Diaper, D. Gilmore, G. Cockton & B. Shackel (Eds.), Human-Computer Interaction - INTERACT '90 (pp. 79-86). Amsterdam: Elsevier Science Publishers, B.V.
4.30. GREEN, T.R.G. (1991). Describing information artifacts with cognitive dimensions and structure maps. In D. Diaper & N. Hammond (Eds.), People and Computers VI: Proceedings of the HCI '91 Conference (pp. 297-315). Cambridge: Cambridge University Press.
4.31. GREEN, T.R.G. (in press). The cognitive dimensions of information structures. Technical Communication.
4.32. GREEN. T.R.G. & Hoc, J-M. (1991). What is cognitive ergonomics? Le Travail Humain, 54, 291-304.
4.33. GREEN, T.R.G. & Borning, A. (1990). The generalized unification parser: Modelling the parsing of notations. In D. Diaper, D. Gilmore, G. Cockton & B. Shackel (Eds.), Human-Computer Interaction - INTERACT '90 (pp. 951-957). Amsterdam: Elsevier Science Publishers, B.V.
4.34. GREEN, T.R.G. & Doubleday, A. (in press). Linguistic regularity effects and cognitive processes. In Proc. 7th European Conf. on Cognitive Ergonomics.
4.35. GREEN, T.R.G., Gilmore, D.J., BLUMENTHAL, B.B., Davies, S. & Winder, R. (1992). Towards a cognitive browser for OOPS. International Journal of Human-Computer Interaction, 4, 1-34.
4.36. GREEN. T.R.G., Petre, M. & BELLAMY, R.K. (1991). Comprehensibility of visual and textual programs: A test of Superlativism against the 'match-mismatch' conjecture. In J. Koenemann-Belliveau, T.G. Moher & S.P. Robertson (Eds.), Empirical Studies of Programmers - 4th Workshop (pp. 121-146). Norwood, New Jersey: Ablex Publishing Company.
4.37. GREEN, T.R.G. & Petre, M. (in press). When visual programs are harder to read than textual programs. In G.C. van der Veer, M.J. Tauber, S. Bagnarola & M. Antalovits (Eds.), ECCE-6: Proceedings of the 6th European Conference on Cognitive Ergonomics (Lake Balaton, Hungary, 1992).
4.38. HENDRY, D.G. & GREEN, T.R.G. (1993). CogMap: A visual description language for spreadsheets. Journal of Visual Languages and Computing, 4, 35-54.
4.39. HENDRY, D.G. & GREEN, T.R.G. (1993). Spelling mistakes: How well do correctors perform? In S. Ashlund, K. Mullet, A. Henderson, E. Hollnagel & T. White (Eds.), INTERCHI 93 Adjunct Proceedings (Conference on Human Factors in Computing Systems) (pp. 83-84). ACM/IFIP.
4.40. HENDRY, D.G. & GREEN, T.R.G. (in press). Creating, comprehending, and explaining spreadsheets: A cognitive interpretation of what discretionary users think of the spreadsheet model. Int. J. Human-Computer Studies.
4.41. Hodges, J.R., PATTERSON, K., Oxbury, S. & Funnell, E. (1992). Semantic dementia: Progressive fluent aphasia with temporal lobe atrophy. Brain, 115, 1783-1806.
4.42. Hodges, J., PATTERSON, K. & Tyler, L. (in press). Loss of semantic memory: Implications for the modularity of mind. Cognitive Neuropsychology.
4.43. Howard, D., PATTERSON, K., Wise, R., Brown, W.D., Friston, K., Weiller, C. & Frackowiak, R. (1992). The cortical localization of the lexicons: Positron Emission Tomography evidence. Brain, 115, 1769-1782.
4.44. Jusczyk, P.W., CUTLER, A. & Redanz, N.J. (1993). Infants' preference for the predominant stress patterns of English words. Child Development, 64, 675-687.
4.45. MCQUEEN, J.M. (1991). The influence of the lexicon on phonetic categorisation: Stimulus quality in word-final ambiguity. Journal of Experimental Psychology: Human Perception and Performance, 17, 433-443.
4.46. MCQUEEN, J.M. (1993). Rhyme decisions to spoken words and nonwords. Memory and Cognition, 21, 210-222.
4.47. MCQUEEN, J.M., NORRIS, D. & CUTLER, A. (1994). Competition in spoken word recognition: Spotting words in other words. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20, 621-638.
4.48. Modugno, F.M., GREEN, T.R.G. & Myers, B. (in press). Visual programming in a visual domain: a case study of cognitive dimensions. In People and Computers IX: Proc. BCS HCI Conference.
4.49. Monsell, S., PATTERSON, K., Graham, A., Hughes, C. & MILROY, R. (1992). Lexical and sublexical translation of spelling to sound: Strategic anticipation of lexical status. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18, 452-467.
4.50. Morton, J., Sasanuma, S., PATTERSON, K. & Sakuma, N. (1992). The organization of the lexicon in Japanese: Single and compound kanji. British Journal of Psychology, 83, 517-531.
4.51. Nix, A.J., Mehta, G., Dye, J. & CUTLER, A. (1993). Phoneme detection as a tool for comparing perception of natural and synthetic speech. Computer Speech and Language, 7, 211-228.
4.52. NORRIS, D. (in press). A quantitative multiple-levels model of reading aloud. Journal of Experimental Psychology: Human Perception and Performance.
4.53. NORRIS, D.G. (1990). How to build a connectionist idiot (savant). Cognition, 35, 277-291.
4.54. NORRIS, D. (in press). Shortlist: A connectionist model of continuous speech recognition. Cognition.
4.55. NORRIS, D. (in press). Signal detection theory and modularity: On being sensitive to the power of bias models of semantic priming. Journal of Experimental Psychology: Human Perception and Performance.
4.56. Otake, T., Hatano, G., CUTLER, A. & Mehler, J. (1993). Mora or syllable? Speech segmentation in Japanese. Journal of Memory and Language, 32, 258-278.
4.57. VAN OOYEN, B., CUTLER, A. & NORRIS, D.G. (1992). Detection of vowels and consonants with minimal acoustic variation. Speech Communication, 11, 101-108.
4.58. PATTERSON, K.E. (1990). Alexia and neural nets. Japanese Journal of Neuropsychology, 6, 90-99.
4.59. PATTERSON, K.E. (1990). Basic processes of reading - Do they differ in Japanese and English? Japanese Journal of Neuropsychology, 6, 4-14.
4.60. PATTERSON, K., GRAHAM, N. & Hodges, J. (1994). Reading in Alzheimer's type dementia: A preserved ability? Neuropsychology, 8, 395-407.
4.61. PATTERSON, K., GRAHAM, N. & Hodges, J.R. (1994). The impact of semantic memory loss on phonological representations. Journal of Cognitive Neuroscience, 6, 57-69.
4.62. PATTERSON, K. & Hodges, J.R. (1992). Deterioration of word meaning: Implications for reading. Neuropsychologia, 30, 1025-1040.
4.63. PATTERSON, K. & Vargha-Khadem, F. (1991). Neuropsychological observations on the affinity between reading and phonological abilities. Mind and Language, 6, 141-145.
4.64. PATTERSON, K. & WILSON, B. (1990). A ROSE is a ROSE or a NOSE: A deficit in initial letter identification. Cognitive Neuropsychology, 7, (5/6), 447-477.
4.65. Petre, M. & GREEN, T.R.G. (1990). Where to draw the line with text: Some claims by logic designers about graphics in notation. In D. Diaper, D. Gilmore, G. Cockton & B. Shackel (Eds.), Human-Computer Interaction - INTERACT '90 (pp. 463-468). Amsterdam: Elsevier Science Publishers, B.V.
4.66. Petre, M. & GREEN. T.R.G. (1992). Requirements of graphical notations for professional users: Electronic CAD systems as a case study. Le Travail Humain, 55, 47-70.
4.67. Petre, M. & GREEN, T.R.G. (1993). Learning to read graphics: Some evidence that 'seeing' an information is an acquired skill. Journal of Visual Languages and Computing, 4, 55-70.
4.68. Powell, R. & BISHOP, D.V.M. (1992). Clumsiness and perceptual problems in children with specific language impairment. Developmental Medicine and Child Neurology, 34, 755-765.
4.69. Price, C., Wise, R., Ramsay, S., Friston, K., Howard, D., PATTERSON, K. & Frackowiak, R. (1992). Regional response differences within the human auditory cortex when listening to words. Neuroscience Letters, 146, 179-182.
4.70. Price, C., Wise, R., Watson, J., PATTERSON, K., Howard, D. & Frackowiak, R. (in press). Brain activity during reading: The effects of exposure duration and task. Brain.
4.71. SELLEN, A.J. (in press). Remote conversations: The effects of mediating talk with technology. To appear in Human-Computer Interaction.
4.72. Gaver, W., SELLEN, A., Heath, C. & Luff, P. (1993). One is not enough: Multiple views in a media space. Proceedings of INTERCHI '93, Amsterdam, April 24-29.
4.73. SELLEN, A.J. (1992). Speech patterns in video-mediated conversations. Proceedings of SIGCHI '92, Monterey, CA, May, 1992.
4.74. STARK, H.A. (1990). Pop-up windows and memory for text. In D. Diaper, D. Gilmore, G. Cockton & B. Shackel (Eds.), Human-Computer Interaction - INTERACT '90 (pp. 67-72). Amsterdam: Elsevier Science Publishers, B.V.
4.75. STARK, H.A. (1990). What do readers do to pop-ups, and pop-ups do to readers? In R. McAleese & C. Green (Eds.), Hypertext: State of the Art (pp. 2-9). Oxford: Intellect Ltd.
4.76. WILSON, B. & PATTERSON, K. (1990). Rehabilitation for cognitive impairment: Does cognitive psychology apply? Applied Cognitive Psychology, 4, 247-260.
4.77. WRIGHT, P. (1990). Homework: An international comparison of behavioural researchers' use of computers for work at home. In M. Feeney & K. Merry (Eds.), Information Technology and the Research Process (pp.130-145). London: Bowker-Saur.
4.78. WRIGHT, P. (1991). Information design: An informal review of the past decade. Information Design Journal, 6/3.
4.79. WRIGHT, P., Holborn, D. & Stokes, B. (1993). A British view of document design: A conversation with Patricia Wright. Issues in Writing, 5, 116-134.
4.80. WRIGHT, P. & Hull, A.J. (1990). How people give verbal instructions. Applied Cognitive Psychology, 4, 153-174.
4.81. WRIGHT, P., Hull, A.J. & Black, D. (1990). Integrating diagrams and text. The Technical Writing Teacher, Vol. XVII, 244-254.
4.82. WRIGHT, P., Hull, A.J. & LICKORISH, A. (1993). Navigating in a hospital outpatients' department: The merits of maps and wall signs. Journal of Architectural Planning and Research, 10, 76-89.
4.83. WRIGHT, P. & Lawrence, G. (1994). Design of case report forms for clinical trials. In J. Lloyd & A. Raven (Eds.), ACRPI Handbook of Clinical Research (pp. 209-235). London: Churchill Medical Communications.
4.84. WRIGHT, P. & LICKORISH, A. (1990). An empirical comparison of two navigation systems for two hypertexts. In R. McAleese & C. Green (Eds.), Hypertext: State of the Art (pp.84-93). Oxford: Intellect Ltd.
4.85. WRIGHT, P. & LICKORISH, A. (in press). Menus and memory load: navigation strategies in interactive search tasks. International Journal of Human-Computer Studies.
4.86. WRIGHT, P., LICKORISH, A. & MILROY, R. (1994). Remembering while mousing: The cognitive costs of mouse clicks. SIGCHI Bulletin, 26, 41-45.
4.87. WRIGHT, P., LICKORISH, A., & Hull, A.J. (1990). The importance of iterative procedures in the design of location maps for the built environment. Information Design Journal, 6, 67-78.
4.88. WRIGHT, P., LICKORISH, A., Hull, A.J. & UMMELEN, N. (in press). Graphics in written directions: Appreciated by readers but not writers. Applied Cognitive Psychology.
4.89. Wydell, T.N., PATTERSON, K.E. & Humphreys, G.W. (1993). Phonologically mediated access to meaning for Kanji: Is a Rows still a Rose in Japanese KANJI? Journal of Experimental Psychology: Learning, Memory, and Cognition, 19, 491-514.
Submitted
4.90. GREEN, T.R.G. & HENDRY, D.G. Spelling correctors and spelling mistakes. (Manuscript submitted to Transactions of Office Information Systems).
4.91. GREEN, T.R.G., Davies, S.P. & Gilmore, D.J. Contributions of cognitive psychology to HCI. (Manuscript submitted to Zeitschrift fur Psychologie).
4.92. GREEN, T.R.G. & Petre, M. (a) Cognitive dimensions as discussion tools for programming language design. (Manuscript submitted to Human Computer Interaction).
4.93. GREEN, T.R.G. & Petre, M. (b) Cognitive dimensions of visual programming environments. (Manuscript submitted to IEEE Computer)
4.94. Hesketh, A. & BISHOP, D.V.M. (submitted). Agrammatism and adaptation theory. (Manuscript submitted to Aphasiology)
4.95. NORRIS, D., MCQUEEN, J.M. & CUTLER, A. Competition and segmentation in spoken word recognition. (Manuscript submitted to Journal of Experimental Psychology: Learning, Memory, and Cognition)
4.96. Petre, M. & GREEN, T.R.G. Using graphical representations requires skill, and graphical readership is an 'acquired' skill. (Manuscript submitted to Journal of AI in Education).
4.97. STRAIN, E., PATTERSON, K. & Seidenberg, M.S. (1994). Semantic effects in single-word naming. (Manuscript submitted to Journal of Experimental Psychology: Learning, Memory and Cognition)
4.98. Wydell, T.N., Butterworth, B. & PATTERSON, K. (1994). The inconsistency of consistency effects in reading: The case of Japanese Kanji. (Manuscript submitted to Journal of Experimental Psychology: Learning, Memory and Cognition)
Invited Chapters and Commentaries
4.99. BELLAMY, R.K.E. & Gilmore, D.J. (1990). Programming plans: Internal or external structures. In K.J. Gilhooly, M.T.G. Keane, R.H. Logie & G. Erdos (Eds.), Lines of Thinking: Reflections on the Psychology of Thought, Vol. 2 (pp.59-71). Chichester: John Wiley & Sons Limited.
4.100. BISHOP, D.V.M. (1992). The biological basis of specific language impairment. In P. Fletcher & D. Hall (Eds.), Specific Speech and Language Disorders in Children: Correlates and Outcomes (pp. 2-17). London: Whurr Publications.
4.101. BISHOP, D.V.M. (1994). Developmental disorders of speech and language. In M. Rutter, L. Hersov & E. Taylor (Eds.), Child and Adolescent Psychiatry, 3rd edition (pp. 546-568). Oxford: Blackwell Scientific Publications.
4.102. Botelho da Silva, T. & CUTLER, A. (1993). Ill-formedness and transformability in Portuguese idioms. In C. Cacciari & P. Tabossi (Eds.), Idioms: Processing, Structure, and Interpretation (pp. 129-143). Hillsdale, N.J.: Erlbaum.
4.103. CUTLER, A. (1990). Exploiting prosodic probabilities in speech segmentation. In G.T.M. Altmann (Ed.), Cognitive Models of Speech Processing: Psycholinguistic and Computational Perspectives (pp.105-121). Cambridge, MA: MIT Press.
4.104. CUTLER, A. (1990). From performance to phonology: Comments on Beckman and Edwards's paper. In J. Kindston & M. Beckman (Eds.), Between the Grammar and Physics of Speech (pp.208-214). Cambridge: Cambridge University Press.
4.105. CUTLER, A. (1991). Linguistic rhythm and speech segmentation. In J. Sundberg, L. Nord & R. Carlson (Eds.), Music, Language, Speech and Brain (pp. 157-166). Houndsmill, Basinstoke: Macmillan Press.
4.106. CUTLER, A. (1991). Processing constraints of the native phonological repertoire on the native language. In Y. Tohkura, E. Vatikiotis-Bateson & Y. Sagisaka (Eds.), Speech Perception, Production and Linguistic Structure (pp. 275-278). Tokyo: IOS Press.
4.107. CUTLER, A. (1991). The production and perception of word boundaries. In Y. Tohkura, E. Vatikiotis-Bateson & Y. Sagisaka (Eds.), Speech Perception, Production and Linguistic Structure (pp. 419-425). Tokyo: IOS Press.
4.108. CUTLER, A. (1992). Psychology and the segment. In G.J. Docherty & D.R. Ladd (Eds.), Papers in Laboratory Phonology II: Gesture, Segment, Prosody (pp. 290-295). Cambridge: Cambridge University Press.
4.109. CUTLER, A. (1992). Why not abolish psycholinguistics? In W.U. Dressler, H.C. Luschützky & O.E. Pfeiffer (Eds.), Phonologica 1988 (pp. 77-87). Cambridge: Cambridge University Press.
4.110. CUTLER, A. (in press). Marked and unmarked segmentation strategies? In H.C. Nusbaum & J.G. Goodman (Eds.), The Transition from Recognizing Speech Sounds to Recognizing Spoken Words. Cambridge, MA: MIT Press.
4.111. CUTLER, A. (in press). Prosody and the word boundary problem. In J. Morgan & K. Demuth (Eds.), Signal to Syntax. Hillsdale, N.J.: Lawrence Erlbaum Associates.
4.112. CUTLER, A. (in press). Spoken word recognition and production. In J.L. Miller & P.D. Eimas (Eds.), Speech, Language and Communication. Volume 11 of E.C. Carterette & M.P. Friedman (Eds.), Handbook of Perception and Cognition. New York: Academic Press.
4.113. CUTLER, A.& MCQUEEN, J.M. (in press). The recognition of lexical units in speech. In B. de Gelder & J. Morais (Eds.), From Spoken to Written Language. Cambridge, MA: The MIT Press.
4.114. GREEN, T.R.G. (1990). Limited theories as a framework for human-computer interaction. In D. Ackermann & M.J. Tauber (Eds.), Mental Models and Human-Computer Interaction 1 (pp. 3-39). Elsevier Science Publishers B.V. North-Holland.
4.115. GREEN, T.R.G. (1990). Programming languages as information structures. In J-M. Hoc, T.R.G. Green, R. Samurcay & D.J. Gilmore (Eds.), The Psychology of Programming (pp. 117-137). London: Academic Press.
4.116. GREEN, T.R.G. (1990). The nature of programming. In J-M Hoc, T.R.G. Green, R. Samurcay & D.J. Gilmore (Eds.), The Psychology of Programming (pp. 21-44). London: Academic Press.
4.117. GREEN, T.R.G. (1991). User modelling: the information-processing perspective. In J. Rasmussen, H.B. Andersen & N.O. Bernsen (Eds.), Research Directions in Cognitive Science: A European Perspective, Vol. 3: Human Computer Interaction (pp. 27-57). London: Lawrence Erlbaum Associates.
4.118. GREEN, T.R.G. (in press). Why software engineers don't listen to what psychologists don't tell them anyway. In D. Gilmore & R. Winder (Eds.), User-Centred Requirements for Software Engineering Environments. North-Holland: Elsevier.
4.119. MCQUEEN, J.M. (in press). Processing versus representation: A reply to Ohala and Ohala. In B. Connell, A. Arvaniti & I. Watson (Eds.), Papers in Laboratory Phonology IV. Cambridge: Cambridge University Press.
4.120. MCQUEEN, J.M. & CUTLER, A. (in press). Morphology in word recognition. In A.M. Zwicky & A. Spencer (Eds.), A Handbook of Morphology. Oxford: Blackwell.
4.121. Mehler, J. & CUTLER, A. (1990). Psycholinguistic implications of phonological diversity among languages. In Massimo Piattelli Palmarini (Ed.), Cognitive Science in Europe (pp. 119-134). GOLEM Monograph No. 1.
4.122. Mendelsohn, P., GREEN, T.R.G. & Brna, P. (1990). Programming languages in education: The search for an easy start. In J-M. Hoc, T.R.G. Green, R. Samurcay & D.J. Gilmore (Eds.), The Psychology of Programming (pp. 175-200). London: Academic Press.
4.123. NORRIS, D.G. (1990). A dynamic-net model of human speech recognition. In G.T.M. Altmann (Ed.), Cognitive Models of Speech Processing: Psycholinguistic and Computational Perspectives (pp.87-105). Cambridge, MA: MIT Press.
4.124. NORRIS, D. (1991). Connectionism: A case for modularity. In D.A. Balota, G.B. Flores d'Arcais & K. Rayner (Eds.), Comprehension Processes in Reading (pp.331-343). Hillsdale, N.J.: Lawrence Erlbaum Associates.
4.125. NORRIS, D.G. (1992). Connectionism: A new class of bottom up model. In R.G. Reilly & N.E. Sharkey (Eds.), Connectionist Approaches to Language Processing (pp. 351-371). Hove, Sussex: Lawrence Erlbaum Associates.
4.126. PATTERSON, K. (1992). Phonology in reading. The Psychologist, 5, p. 314.
4.127. PATTERSON, K. (1994). Reading, writing and rehabilitation: A reckoning. In M.J. Riddoch & G.W. Humphreys (Eds.), Cognitive Neuropsychology and Cognitive Rehabilitation (pp. 425-447). Hove: Lawrence Erlbaum Associates.
4.128. PATTERSON, K. & MARCEL, A.J. (1992). Phonological ALEXIA or PHONOLOGICAL Alexia? In J. Alegria, D. Holender, J. Junça de Morais & M. Radeau (Eds.), Analytic Approaches to Human Cognition (pp. 259-274). Amsterdam: Elsevier Science Publishers B.V.
4.129. Penz, F., Bourne, M. & WRIGHT, P. (1992). Tools for design: A controlled experiment comparing computer work with traditional hand drawings. In F. Penz (Ed.), Tools for Design (pp. 3-14). Longmans.
4.130. Sasanuma, S. & PATTERSON, K. (in press). The consequences of impaired comprehension for word reading in Japanese and English. In B. de Gelder & J. Morais (Eds.), From Spoken to Written Language. Cambridge, MA: MIT Press.
4.131. Schiele, F. & GREEN, T.R.G. (1990). HCI formalisms and cognitive psychology: The case of task-action grammar. In M. Harrison & H. Thimbleby (Eds.), Formal Methods in Human-Computer Interaction (pp. 9-62). Cambridge: Cambridge University Press.
4.132. STARK, H.A. (1991). Support for multi-attribute choice in OIS: From decision rules to decision tools, and back again. In A.A. Verrijn-Stuart, H.G. Sol & P. Hammersley (Eds.), Support Functionality in the Office Environment (pp. 159-179). Elsevier Science Publishers B.V. (North-Holland).
4.133. WRIGHT, P. (1990). Hypertexts as an interface for learners: Some human factors issues. In D. Jonassen & H. Mandl (Eds.), Designing Hypermedia for Learning (pp. 169-184). Berlin: Heidelberg: Springer-Verlag.
4.134. WRIGHT, P. (1991). Designing and evaluating documentation for IT users. In B. Shackel & S. Richardson (Eds.), Human Factors for Informatics Usability (pp.343-358). Cambridge: Cambridge University Press.
4.135. WRIGHT, P. (1991). Quality: An excellent idea? In Proceedings of the First Conference on Quality in Documentation (pp. 9-24) (Waterloo, Canada, October 1991). University of Waterloo: The Centre for Professional Writing.
4.136. WRIGHT, P. (1991). Review of The Society of Text: Hypertext, hypermedia, and the social construction of information, by E. Barrett (Ed.), MIT Press, 1989. BCS HCI Newsletter, No.16 (April), 17-19.
4.137. WRIGHT, P. (1992). Cognitive overheads and prostheses: Some issues in evaluating hypertexts. In Hypertext '91. Proceedings of Third ACM Conference on Hypertext (pp. 1-12). (San Antonio, Texas, Dec. 15-18 1991), ACM.
4.138. WRIGHT, P. (1993). To jump or not to jump: Strategy selection while reading electronic texts. In C. McKnight, A. Dillon & J. Richardson (Eds.), Hypertext: A Psychological Perspective (pp. 137-152). Chichester: Ellis Horwood.
4.139. WRIGHT, P. (1994). Enhancing the usability of written instructions. In H. Zwaga & T. Boersema (Eds.), Public Graphics: Visual Information for Everyday Use. Utrech.
4.140. WRIGHT, P. (in press). Quality or usability? Quality writing provokes quality reading. In M. Steehouder, C. Jansen & R. Verheijen (Eds.), Quality in Documentation. Amsterdam: Editions Rodopi.
Conference Proceedings
4.141. ALLERHAND, M., BUTTERFIELD, S., CUTLER, A. & Patterson, R. (1992). Assessing syllable strength via an auditory model. In Proceedings of the Institute of Acoustics, Vol. 14, part 6 (pp. 297-304).
4.142. Baurén, M., GREEN, T.R.G. & Petre, M. (1992). From PLOP to PLITH: A cognitive model of programming pointer problems. Psychology of Programming Interest Group Annual Workshop, Paris, 1992.
4.143. BELLAMY, R.K.E. & Carroll, J.M. (1990). Case-based reuse. In Proceedings of the ACM SIGPLAN Symposium on Object-Oriented Programming Emphasizing Practical Applications. New York.
4.144. BLUMENTHAL, B. (1993). Expert programmers' assessments of robustness, stability, and sources of expertise in the programming process. Proc. East-West Int. Conf. on Human-Computer Interaction. Moscow.
4.145. BUTTERFIELD, S. & CUTLER, A. (1990). Intonational cues to word boundaries in clear speech? In Proceedings of the Institute of Acoustics, (November 1990), 12, (pp. 87-94).
4.146. CUTLER, A. (1991). Prosody in situations of communication: Salience and segmentation. In Proceedings of the XIIth International Congress of Phonetic Sciences, Vol. 1 (pp. 264-270), (Aix-en-Provence, France, Aug 19-24, 1991).
4.147. CUTLER, A. (1993). Language-specific processing: Does the evidence converge? In G. Altmann and R. Shillcock (Eds.), Cognitive Models of Speech Processing: The Sperlonga Meeting II (pp. 115-123). Hillsdale, N.J.: Lawrence Erlbaum Associates.
4.148. CUTLER, A. (1993). Segmenting speech in different languages. The Psychologist, 6, 453-455.
4.149. CUTLER, A. & BUTTERFIELD, S. (1990). Syllabic lengthening as a word boundary cue. In R. Seidl (Ed.), Proceedings of the 3rd Australian International Conference on Speech Science and Technology (pp. 324-328), (Melbourne, November 1990). Canberra: Australian Speech Science and Technology Association.
4.150. CUTLER, A. & Fear, B. (1991). Categoricality in acceptability judgments for strong versus weak vowels. In Proceedings of the ESCA Workshop on Phonetics and Phonology of Speaking Styles: Reduction and Elaboration in Speech Communication (pp. 18-1 - 18-5), (Barcelona, Catalonia, Spain 30 Sept-2 Oct, 1991).
4.151. CUTLER, A., KEARNS, R., NORRIS, D. & Scott, D.R. (1992). Listeners' responses to extraneous signals coincident with English and French speech. In Proceedings of the 4th Australian International Conference on Speech Science and Technology (pp. 666-671). Canberra: Australian Speech Science and Technology Association.
4.152. CUTLER, A., NORRIS, D. & VAN OOYEN, B. (1990). Vowels as phoneme detection targets. In Proceedings of the International Conference on Spoken Language Processing, Vol. 1 (pp. 581-584), Kobe, Japan.
4.153. CUTLER, A. & Robinson, T. (1992). Response time as a metric for comparison of speech recognition by humans and machines. In Proceedings ICSLP 92 - International Conference on Spoken Language Processing, Volume 1 (pp. 189-192). Edmonton: University of Alberta.
4.154. Heath, C., Luff, P., & SELLEN, A.J. (In press). Reconfiguring media space. Proceedings of the British Telecom POTS to PANS Symposium, March, 1994.
4.155. HENDRY, D., GREEN, T.R.G., Gilmore, D. & Davies, S. (1992). Improving the communicability of spreadsheet designs: annotating with descriptive tags. Psychology of Programming Interest Group Annual Workshop.
4.156. MCQUEEN, J.M. & Briscoe, E.J. (1991). A computational tool for examining lexical segmentation in continuous speech. In Eurospeech 91: Proceedings of the 2nd European Conference on Speech Communication and Technology, Vol. 2 (pp. 697-700). Genova, Italy, 24-26 September.
4.157. MCQUEEN, J.M. & CUTLER, A. (1992). Words within words: Lexical statistics and lexical access. In Proceedings ICSLP 92 - International Conference on Spoken Language Processing, Volume 1 (pp. 221-224). Edmonton: University of Alberta.
4.158. NORRIS, D. (1991). Rewiring lexical networks on the fly. In Eurospeech 91: Proceedings of the 2nd European Conference on Speech Communication and Technology, Vol. 1 (pp. 117-120). Genova, Italy, 24-26 September.
4.159. NORRIS, D. (1991). The constraints on connectionism. The Psychologist: Bulletin of the British Psychological Society, 4, 293-296.
4.160. NORRIS, D. (1993). Bottom-up connectionist models of interaction. In G. Altmann and R. Shillcock (Eds.), Cognitive Models of Speech Processing: The Sperlonga Meeting II (pp. 211-234). Hillsdale, N.J.: Lawrence Erlbaum Associates.
4.161. NORRIS, D., MCQUEEN, J. & CUTLER, A. (in press). Competition and segmentation in spoken word recognition. In Proceedings of the 1994 International Conference on Computer Speech and Language Processing (Yokohama, Japan).
4.162. NORRIS, D., VAN OOYEN, B. & CUTLER, A. (1992). Speeded detection of vowels and steady-state consonants. In Proceedings ICSLP 92 - International Conference on Spoken Language Processing, Volume 2 (pp. 1055-1058). Edmonton: University of Alberta.
4.163. VAN OOYEN, B. (1993). Detection of vowels and consonants by human listeners: Effect of minimising auditory memory load. In Eurospeech '93: Proceedings of the 3rd European Conference on Speech Communication and Technology (pp. 1507-1510), (Berlin, September, 1993).
4.164. VAN OOYEN, B., CUTLER, A. & Bertinetto, P.M. (1993). Click detection in Italian and English. In Eurospeech '93: Proceedings of the 3rd European Conference on Speech Communication and Technology (pp. 681-684) (Berlin, September, 1993).
4.165. VAN OOYEN, B., CUTLER, A., & NORRIS, D. (1991). Detection times for vowels versus consonants. In Eurospeech 91: Proceedings of the 2nd European Conference on Speech Communication and Technology, Vol. 3 (pp. 1451-1454). Genova, Italy, 24-26 September.
4.166. PATTERSON, K., CROOT, K. & Hodges, J.R. (1994). Speech production: Insights from a study of progressive aphasia. In Proceedings of the 3rd International Conference on Spoken Language Processing (Yokohama, Japan).
4.167. Penz, F., Bourne, M. & WRIGHT, P. (1990). Tools for design: A controlled experiment comparing computer work with traditional hand drawings. In Proceedings of International Symposium on Computers in Architecture. Cambridge: Dept. of Architecture, University of Cambridge, p.1.
4.168. THURSTON, P. & NORRIS, D. (1991). A comparison of two compression functions used for noisy vowel detection with back-propagation networks. In Eurospeech 91: Proceedings of the 2nd European Conference on Speech Communication and Technology, Vol. 2 (pp. 995-998). Genova, Italy, 24-26 September.
4.169. WRIGHT, P. (1990). Technical reading and technical writing: Relevant research for technical authors. In TecDOC '90 (pp. 1-5), (Proceedings of the 8th International Weekend Conference on Technical Communication and Documentation), London: Consert.
4.170. WRIGHT, P. (1990). What's specific about user interfaces of hypertext systems? It's what you can do, and the way you can do it. In A. Rizk, N. Streitz & J. Andre (Eds.), Hypertext: Concepts, Systems and Applications (pp. 358-359) (Proceedings of the European Conference on Hypertext, INRIA, France, November 1990). Cambridge: Cambridge University Press.
4.171. WRIGHT, P. (1991). Reading for use. The Psychologist, 4, p.74.
4.172. WRIGHT, P. (1991). What is Hypertext? Euro-Documentation, 1, 2-3.
4.173. WRIGHT, P. (1992). Information Design the linking agent. In Conference Proceedings of PIRA/RSA Design Festival, 2nd International Conference, 1-3 April, 1992. p1-11.
4.174. WRIGHT, P. & Hull, A. (1990). When do readers read pictures? In Preseedings - Forum 90 (pp. 115-119), Stockholm, Sweden.
4.175. WRIGHT, P. & UMMELEN, N. (1990). Given a diagram, when do readers read text? Postharvest - Forum 90, Box 38, S-124 21 Bandhagen, Sweden. p81.
4.176. Young, D., Altmann, G.T.M., CUTLER, A. & NORRIS, D. (1993). Metrical structure and the perception of time-compressed speech. In Eurospeech '93: Proceedings of the 3rd European Conference on Speech Communication and Technology (pp. 771-774), (Berlin, September, 1993).
Technical Reports and Theses
4.177. CUTLER, A. & BUTTERFIELD, S. (1991). Speech segmentation under difficult listening conditions. Project supported by IBM UK Scientific Centre, October 1988-January 1991. Final Report. Cambridge: Applied Psychology Unit, (pp.28), April.
4.178. CUTLER, A., NORRIS, D.G. & MCQUEEN, J.M. (in press). Modelling lexical access from continuous speech input. Annual Report, dokkyo International Center, Tokyo.
4.179. GREEN, T.R.G., Winder, R., Gilmore, D.J., Davies, S.P., HENDRY, D. & Joly, G. (1992). Designing a cognitive browser for object-oriented programming. AISB Quarterly, Newsletter of the Society for the Study of Artificial Intelligence and Simulation of Behaviour, Autumn, No. 81, 17-20.
4.180. MCQUEEN, J.M. (1991). Phonetic decisions and their relationship to the lexicon. Unpublished PhD Thesis, University of Cambridge.
4.181. PATTERSON, R., NORRIS, D., & CUTLER, A. (Eds.). (1990). Auditory/Connectionist Techniques for Speech (ACTS) Periodic Progress Report No. 1, and associated Annexe. CEC: Brussels, June 1990.
4.182. PATTERSON, R.D., NORRIS, D. & CUTLER, A. (Eds.). (1991). Auditory/Connectionist Techniques for Speech. (ACTS) Periodic Progress Report No. 2 and associated Annexes. Brussels: CEC.
4.183. PATTERSON, R.D., NORRIS, D. & CUTLER, A. (1992). Auditory/Connectionist Techniques for Speech. (ACTS) Periodic Progress Report No, 3. Brussels: CEC, July.
Dissemination
4.184. BISHOP, D.V.M. (1993). Cumulative Index to Journal of Child Psychology and Psychiatry 1960-92. Oxford: Pergamon Press.
4.185. CUTLER, A. (1992). Cross-linguistic differences in speech segmentation. MRC News, September, No. 56, 8-9.
4.186. CUTLER, A. (1992). Perception of speech: Psycholinguistic aspects. In W. Bright (Ed.), International Encyclopedia of Linguistics, Vol. II (pp. 181-183), Oxford: Oxford University Press.
4.187. WRIGHT, P. (1990). The uses of research on writing. The Manual, 3, pp. 5 and 15.
4.188. WRIGHT, P. (1991). What is Hypertext? The Manual, 4, p. 3.
4.189. WRIGHT, P. (1992). Information design: The linking agent. Desktop Publishing Commentary, 8, 6-11.
4.190. WRIGHT, P. (1992). Cognitive aspects of information handling. Subsumed into Task Force 2: The Technological Imperative. Original submissions available from British Library Document Supply Centre, Boston Spa, Yorkshire.
4.191. WRIGHT, P. (1992). Review of Hypertext in Context by C.McKnight, A. Dillon and S.Richardson. International Journal of Man-Machine Studies, 37, 389-391.
4.192. *WRIGHT, P. (1994). Presenting technical information: a review of research findings. In D. Oborne (Ed.), Ergonomics and Human Factors. Cheltenham, Glos: Edward Elgar. [Reprinted from 1977]
4.193. *WRIGHT, P. (1994). Phenomena, function and design: does information make a difference? In D. Oborne (Ed.), Ergonomics and Human Factors. Cheltenham, Glos: Edward Elgar. [Reprinted from 1986]
4.194. *WRIGHT, P. (1994). Communicating with the user. In D. Oborne (Ed.), Ergonomics and Human Factors. Cheltenham, Glos: Edward Elgar. [Reprinted from 1988]
*Originally published outside the reporting period, recently republished to make it available to a wider audience.
REFERENCES TO OTHER WORK
Anderson, J.R. (1993). Rules of the Mind. Erlbaum.
Bertin, J. (1981). Graphics and Graphic Information Processing. (Trans. W. J. Berg.) New York: de Gruyter.
BISHOP, D.V.M. (1987). The causes of specific developmental language disorder ("developmental dysphasia"). Journal of Child Psychology and Psychiatry, 28, 1-8.
BISHOP, D.V.M. (1989). Autism, Asperger's syndrome and semantic-pragmatic disorder: Where are the boundaries? British Journal of Disorders of Communication, 24, 107-121.
BISHOP, D.V.M. (1989). Test for the Reception of Grammar (2nd edition). Manchester: University of Manchester.
BISHOP, D.V.M. (1990). Handedness, clumsiness and developmental language disorders. Neuropsychologia, 28, 681-690.
BISHOP, D.V.M. (1992). The underlying nature of specific language impairment. Journal of Child Psychology & Psychiatry, 33, 1-64.
BISHOP, D.V.M. & Adams, C. (1989). Conversational characteristics of children with semantic-pragmatic disorder. II. What features lead to a judgement of inappropriacy? British Journal of Disorders of Communication, 24, 241-263.
BISHOP, D.V.M. & Adams, C. (1990). A prospective study of the relationship between specific language impairment, phonological disorders and reading retardation. Journal of Child Psychology and Psychiatry, 31, 1027-1050.
BISHOP, D.V.M. & Adams, C. (1991). What do referential communication tasks measure? A study of children with specific language impairment. Applied Psycholinguistics, 12, 199-215.
BISHOP, D.V.M. & Adams, C. (1992). Comprehension problems in children with specific language impairment: Literal and inferential meaning. Journal of Speech & Hearing Research, 35, 119-129.
BISHOP, D.V.M. & Rosenbloom, L. (1987). Classification of childhood language disorders. In W. Yule, M. Rutter (Eds.), Language Development and Disorders. Clinics in Developmental Medicine (double issue), nos. 101-2. (pp. 16-41) London: Mac Keith Press.
Caplan, R., Foy, J.G., Asarnow, R.F. & Sherman, T. (1990). Information processing deficits of schizophrenic children with formal thought disorder. Psychiatric Research, 31, 169-177.Coltheart, M., Curtis, B., Atkins, P. & Haller (1993). Models of reading aloud: Dual-route and parallel-distributed-processing approaches. Psychological Review, 100, 4, 589-608.
Card, S.K., Moran, T.P. & Newell, A. (1983). The Psychology of Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum Asociates Inc.
Charney, D. (1994). The impact of hypertext on processes of reading and writing. In S.J. Hilligoss & C.L. Selfe (Eds.), Literacy and Computers. New York: Modern Language Association.
CUTLER, A., Mehler, J., NORRIS, D. & Segui, J. (1983). A language specific comprehension strategy. Nature, 304, 5922, 159-160.
CUTLER, A., Mehler, J., NORRIS, D. & Segui, J. (1986). The syllable's differing role in the segmentation of French and English. Journal of Memory and Language, 25, 385-400.
CUTLER, A., Mehler, J., NORRIS, D. & Segui, J. (1987). Phoneme identification and the lexicon. Cognitive Psychology, 19, 141-177.
CUTLER, A. & NORRIS, D.G. (1988). The role of strong syllables in segmentation for lexical access. Journal of Experimental Psychology: Human Perception and Performance, 14, 113-121.
CUTLER, A. & NORRIS, D.(1979). Monitoring Sentence Comprehension. In W.E. Cooper and E.C.T. Walker (Eds.). Sentence Processing: Psycholinguistic Studies. Presented to Merrill Garrett, Erlbaum.
CUTLER, A. & NORRIS, D. (1988) The role of strong syllables in segmentation for lexical access. Journal of Experimental Psychology: Human Perception and Performance, 14, 113-121.
DeFries, J.C., Fulker, D.W. (1988). Multiple regression analysis of twin data: Aetiology of deviant scores versus individual differences. Acta Geneticae et Medicae Gemellologiae, 37, 205-216.
Elman, J. & McClelland, J. (1989). Cognitive penetration of the mechanisms of perception: Compensation for coarticulation of lexically restored phonemes. Journal of Memory and Language, 27, 143-165.
Farah, M. (1989). Semantic and perceptual priming: How similar are the underlying mechanisms? Journal of Experimental Psychology: Human Perception and Performance, 15, 188-194.
Gathercole, S.E. & Baddeley, A.D. (1990). The role of phonological memory in vocabulary acquisition: A study of young children learning new names. British Journal of Psychology, 81, 439-454.
Gaver, W., Moran, T., MacLean, A., Lövstrand, L., Dourish, P., Carter, K., and Buxton, W. (1992). Realizing a video environment: EuroPARC's RAVE system. Proceedings of CHI'92 (Monterey, California, 3 - 7 May). ACM, New York, pp. 27 - 35.
Gopnik, M., & Crago, M. (1991). Familial aggregation of a developmental language disorder. Cognition, 39, 1-50.
Gray, W. & Anderson, J.R. (1987). Change-episodes in coding: When and how do programmers change their code? In G.M. Olson, S. Sheppard & E. Soloway (Eds.), Empirical Studies of Programmers: Second Workshop. Ablex.
Guthrie, J.T. (1988). Locating information in documents: An examination of a cognitive model. Reading Research Quarterly, 23, 178-199.
Humphreys, G.W., Riddoch, M.J. & Quinlan, P.T. (1988). Cascade processes in picture identification. Cognitive Neuropsychology, 5, 67-104.
LEE, W.O. (1993). Adapting to interface resources and circumventing interface problems: Knowledge development in a menu search task. In J.L. Alty, D. Diaper & S.P. Guest (Eds.), People and Computers VIII: Proceedings of the HCI '93 Conference (pp. 61-77). Cambridge: Cambridge University Press.
Leonard, L.B., Bortolini, U., Caselli, M.C., McGregor, K.K., & Sabbadini, L. (1992). Morphological deficits in children with specific language impairment: The status of features in the underlying grammar. Language Acquisition, 2, 151-179.
Mantei, M. M., Baecker, R. SELLEN, A. J., Wellman, B., and Buxton, W. (1991). Experiences in the use of a media space. Proceedings of SIGCHI '91, New Orleans, LA., pp. 203-208.
MARCEL, A.J. (1980). Surface dyslexia and beginning reading: A revised hypothesis of the pronunciation of print and its impairments. In M. Coltheart, K. Patterson and J.C. Marshall (Eds.)., Deep Dyslexia pp 227-258, London: Routeledge and Keegan Paul.
Marshall, J.C. & Newcombe, F. (1973). Patterns of paralexia: A psycholinguistic approach. Journal of Psycholinguistic Research, 2, 175-199.
MARSLEN-WILSON, W.D. & Welsh, A. (1978). Processing interactions and lexical access during word-recognition in continuous speech. Cognitive Psychology, 10, 29-63.
MAY, J., BARNARD, P.J. & BLANDFORD, A. (1993). Using structural descriptions of interfaces to automate the modelling of user cognition. User Modelling and User Adapted Interaction, 3, 27-64.
MAY, J., BARNARD, P.J., Boecker, M. & GREEN, A.J. (1990). Characterising structural and dynamic aspects of the interpretation of visual interface objects. In ESPRIT '90 Conference Proceedings (pp. 819-834), Brussels (November 1990), Dordrecht: Kluwer Academic Publishers.
MAY J., TWEEDIE, L. & BARNARD, P.J. (1993). Modelling user performance in visually based interactions. In J.L. Alty, D. Diaper & S.P. Guest (Eds.), People and Computers VIII: Proceedings of the HCI '93 Conference (pp. 95-110). Cambridge: Cambridge University Press.
McClelland, J. & Elman, J. (1986). The TRACE model of speech perception. Cognitive Psychology, 18, 1-86.
MCQUEEN, J.M. (1989). The use of lexical knowledge in phonetic categorisation. In J.P. Tubach & J.J. Mariani (Eds.)., Proceedings of Eurospeech '89, Vol. 2, 581-584. Edinburgh: CEP Consultants Ltd.
Mellish, C., Allport, D. et al. (1992). The TIC Message Analyser. University of Sussex Cognitive Science Research Paper 225.
NORRIS, D. (1986) Context effects without priming. Cognition, 22, 93-136.
O'Hara, K. & Payne, S.J. Cost of operations affects planfuless of problem-solving. (School of Psychology, University of Wales, Cardiff.) (Manuscript submitted for publication)
Plaut, D.C. & McClelland, J.L. (1993). Generalization with componential attractors: Word and nonword reading in an attractor network. In Proceedings of the 15th Annual Conference of the Cognitive Science Society. Hillsdale, NJ: Erlbaum.
Rapin, I. (1987). Developmental dysphasia and autism in pre-school children: characteristics and subtypes. In Proceedings of the First International Symposium on Specific Speech and Language Disorders in Children. (pp. 20-35). London: AFASIC.
Redlich, A.N. (1993). Redundancy reduction as a strategy for unsupervised learning. Neural Computation, 5, 289-304
Rhodes, G., Parkin, A.J., & Tremewan, T. (1993). Semantic priming and sensitivity in lexical decision. Journal of Experimental Psychology: Human Perception and Performance, 19, 154-165.
Robertson, S.P., Davis, E.F., Okabe, K. & Fitz-Randolf, D. (1990). Program comprehension beyond the line. In D. Diaper, D. Gilmore, G. Cockton & B. Shackel (Eds.), Human-Computer Interaction – INTERACT '90. Elsevier.
Schwartz, M.F., Marin, O.S.M. & Saffran, E.M. (1979). Dissociations of language function in dementia: A case study. Brain and Language, 7, 277-306.
Seidenberg, M.S. & McClleland, J.L. (1989). A distributed, developmental model of word recognition and naming. Psychological Review, 96, 523-568.
SELLEN, A.J., Buxton, W.A.S. & Arnott, J. (1992). Using spatial cues to improve desktop video conferencing. 8 minute videotape. Presented at CHI '92, Monterey, CA. Toronto: Dynamic Graphics Project, Computer Systems Research Institute, University of Toronto.
SHALLICE, T., Warrington, E.K., & McCarthy, R (1983). Reading without semantics. Quarterly Journal of Expermental Psychology, 35A, 111-138.
Tyler, L.K. (1992). Spoken Language Comprehension. Cambridge, Mass: MIT Press.
Van Orden, G. (1987). A ROWS is a ROSE: Spelling, sound and reading. Memory and Cognition, 15, 181-198.
Warrington, E.K. (1975). Selective impairment of semantic memory. Quarterly Journal of Experimental Psychology, 27, 635-657.
WRIGHT, P. (1977). Presenting technical information: A survey of research findings. Instructional Science, 6, 93-134.
WRIGHT, P. (1986). Phenomena, function and design: Does information make a difference. In D.J. Oborne (Ed.), Contemporary Ergonomics 1986 (pp.1-18) (Proceedings of the Ergonomics Society 1986 Annual Conference, Durham, England 8-11 April 1986). London: Taylor & Francis.
WRIGHT, P. (1988). Communicating with the user. In M. Heaton & M. Sinclair (Eds.), Designing End-user Interfaces (pp.123-129). (State of the Art Report 15:8). Maidenhead, Berks: Pergamon Infotech Ltd.
Collaborations
Bishop
UK based
Adams - Audiology, Manchester
Snowling - Psychology, Newcastle
Kaplan - Psychiatry, Newcastle
Plomin, Rutter, Simonoff - Institute of Psychiatry, London
Outside UK
Tallal - Behavioral Neuroscience, Rutgers University
Green
UK based
Gilmore - Psychology, Nottingham
Winder - Computer Science, UCL
Benyon, Petre - Educational Technology, Open University
Brna - Artificial Intelligence, Edinburgh
Richens - Architecture, Cambridge
Outside UK
Hoc - CNRS, Paris
Modugno, Myers - Computer Science, Carnegie-Mellon
Bellamy - Computer Science, IBM, NY
Norris
UK based
Briscoe - Computational Linguistics, Cambridge
Outside UK
Cutler, McQueen - Max Planck Institute for Psycholinguistic Research, Nijmegen
K Patterson
UK based
Hodges - Neurology, Cambridge
Butterworth, Wydell, Yin - Psychology, University College London
Tyler, Howard - Psychology, Birkbeck College, London
Wise, Price - MRC Cyclotron Unit, London
Outside UK
McClelland, Plaut - Psychology, Carnegie-Mellon
Seidenberg - Psychology, University of Southern California