Copyright © 1999 Massachusetts Institute of Technology
In PSYCHOLINGUISTICS, computational models are becoming increasingly important both for helping us understand and develop our theories and for deriving empirical predictions from those theories. How a theory of language processing behaves usually depends not just on the mechanics of the model itself, but also on the properties of the linguistic input. Even when the theory is conceptually simple, the interaction between theory and language is often too complex to be explored without the benefit of computer simulations. It is no surprise then that computational models have been at the center of some of the most significant recent developments in psycholinguistics.
The main area of contact with empirical data has been made by models operating roughly at the level of the word. Although there are active and productive efforts underway to develop models of higher-level processes such as syntactic parsing (Kempen and Vosse 1989; McRoy and Hirst 1990; Marcus 1980) and discourse (Kintsch and van Dijk 1978; Kintsch 1988; Sharkey 1990), the complexity of these processes makes it harder to derive detailed experimental predictions.
The neighborhood activation model (Luce, Pisoni, and Goldinger 1990) gives a computational account of isolated word recognition, but only TRACE (McClelland and Elman 1986) and Shortlist (Norris 1994a) have been applied to the more difficult problem of how words can be recognized in continuous speech, where the input may contain no reliable cues to indicate where one word ends and another begins. Both of these models are descendants of McClelland and Rumelhart's (1981) connectionist interactive activation model (IAM) of VISUAL WORD RECOGNITION. The central principle of both models is that the input can activate multiple word candidates, represented by nodes in a network, and that these candidates then compete with each other by means of inhibitory links between overlapping candidates. Thus the spoken input "get in" might activate "tin" as well as "get" and "in," but "tin" would be inhibited by the other two overlapping words. TRACE and Shortlist represent opposite positions in the debate over whether SPOKEN WORD RECOGNITION is an interactive process. In TRACE there is continuous interaction between the lexical and phonemic levels of representation, whereas Shortlist has a completely bottom-up, modular architecture. They also differ in their solution to the problem of how to recognize words beginning at different points in time. TRACE uses a permanent set of complete lexical networks beginning at each point where a word might begin. In Shortlist the network performing the lexical competition is created dynamically and contains only those candidates identified by a bottom-up analysis of the input. On a purely practical level, at least, this has the advantage of enabling Shortlist to perform simulations with realistically sized lexicons of twenty or thirty thousand words. Shortlist has also been extended (Norris, McQueen and Cutler 1995) to incorporate the metrical segmentation strategy of Cutler and Norris (1988), which enables the model to make use of metrical cues to word boundaries.
The most significant nonconnectionist model of spoken word recognition has been Oden and Massaro's (1978) fuzzy logical model of perception (FLMP). FLMP differs from TRACE and Shortlist in that it can be seen as a generic account of how decisions are made on the basis of information from different sources. FLMP itself has nothing to say, for example, about the competition process vital for the recognition of words in continuous speech in both TRACE and Shortlist. Comparisons between FLMP and TRACE in terms of their treatment of the relationship between lexical and phonemic information led to a major revision of the IAM framework and the development of the stochastic interactive activation model (Massaro 1989; McClelland 1991).
IAMs and spreading activation models have also been predominant in the area of LANGUAGE PRODUCTION (Dell 1986, 1988; Harley 1993; Roelofs 1992; see also Houghton 1990). Dell's model is designed to account for the nature and distribution of speech errors. It takes as its input an ordered set of word units (lemmas), representing the speaker's intended production, and produces as its output a string of phonemes that may be corrupted or misordered. In Dell (1988) the model consists of a lexical network in which word nodes are connected to their constituent phonemes by reciprocal links and by a word shape network that reads out successive phonemes in the appropriate syllable structure. The main effect of the reciprocal links from phonemes to words is to give the model a tendency for its errors to form real rather than nonsense words. Whether the production system really does contain these feedback links has been the topic of extensive debate between Dell and Levelt. Levelt, Roelofs, and Meyer (forthcoming) describe the latest computational implementation of the WEAVER ++ model, which is a noninteractive spreading activation model designed to account for an extensive body of response time (RT) data on production as well as speech error data.
The most controversial connectionist model of language has been the Rumelhart and McClelland (1986) model for acquisition of the past tense of verbs. Conventional linguistic accounts of quasi-regular systems such as the past tense assume that the proper explanation is in terms of rules and a list of exceptions. Rumelhart and McClelland modeled the acquisition of the past tense using a simple pattern associator that mapped the phonology of verb roots (e.g., kill, run) onto their past tense forms (killed, ran). They claimed that their model not only explained important facts about the acquisition of verbs, but that it did so without using linguistic rules. The model was fiercely criticized by Pinker and Prince (1988). Later models by MacWhinney and Leinbach (1991) and Plunkett and Marchman (1991, 1993), using backpropagation with hidden units, rectified some of the technical deficiencies of the original model, and claimed to give a more accurate account of the developmental data, but the debate between the connectionist and symbolic camps continues. Recently reported neuropsychological and neuroimaging data suggest a neuroanatomical distinction between mechanisms underlying the rule-based and non-rule-based processes (e.g., Marslen-Wilson and Tyler forthcoming).
A parallel set of arguments has surrounded models of reading aloud. The relationship between spelling and sound is another example of a quasi-regular system where backpropagation networks have been used to give a unitary account of the READING process rather than incorporating spelling-to-sound rules and a list of exception words (e.g., yacht, choir) not pronounced according to the rules (Seidenberg and McClelland 1989; Plaut et al. 1996). The more traditional two-process view is represented by Coltheart et al. (1993), while an interactive activation model by Norris (1994b) takes an intermediate stance.
Coltheart, M., B. Curtis, P. Atkins, and M. Haller. (1993). Models of reading aloud: Dual-route and parallel-distributed-processing approaches. Psychological Review 100:589-608.
Cutler, A., and D. Norris. (1988). The role of strong syllables in segmentation for lexical access. Journal of Experimental Psychology: Human Perception and Performance 14:113-121.
Dell, G. S. (1986). A spreading-activation theory of retrieval in sentence production. Psychological Review 82:407-428.
Dell, G. S. (1988). The retrieval of phonological forms in production: Tests of predictions from a connectionist model. Journal of Memory and Language 27:124-142.
Dijkstra, T., and K. de Smedt, Eds. (1996). Computational Psycholinguistics. London: Taylor and Francis.
Harley, T. A. (1993). Phonological activation of semantic competitors during lexical access in speech production. Language and Cognitive Processes 8:291-309.
Houghton, G. (1990). The problem of serial order: A neural network model of sequence learning and recall. In R. Dale, C. Mellish, and M. Zock, Eds., Current Research in Natural Language Generation. London: Academic Press, pp. 289-319.
Kempen, G., and T. Vosse. (1989). Incremental syntactic tree formation in human sentence processing: A cognitive architecture based on activation decay and simulated annealing. Connection Science 1:273-290.
Kintsch, W. (1988). The role of knowledge in discourse comprehension: A construction-integration model. Psychological Review 95:163-182.
Kintsch, W., and T. A. van Dijk. (1978). Towards a model of text comprehension and production. Psychological Review 85:363-394.
Levelt, W. J. M., A. Roelofs, and A. S. Meyer. (Forthcoming). A theory of lexical access in speech production. Brain and Behavioural Sciences.
Luce, P. A., D. B. Pisoni, and S. D. Goldinger. (1990). Similarity neighborhoods of spoken words. In G. T. M. Altmann, Ed., Cognitive Models of Speech Processing: Psycholinguistic and Computational Perspectives. Cambridge, MA: MIT Press, pp. 122-147.
MacWhinney, B., and J. Leinbach. (1991). Implementations are not conceptualizations: Revising the verb learning model. Cognition 40:121-157.
Marcus, M. P. (1980). A Theory of Syntactic Recognition for Natural Language. Cambridge, MA: MIT Press.
Marslen-Wilson, W. M., and L. K. Tyler. (Forthcoming). Dissociating types of mental computation. Nature.
Massaro, D. W. (1989). Testing between the TRACE model and the Fuzzy Logical Model of Perception. Cognitive Psychology 21:398-421.
McClelland, J. L. (1991). Stochastic interactive processes and the effects of context on perception. Cognitive Psychology 23:1-44.
McClelland, J. L., and J. L. Elman. (1986). The TRACE model of speech perception. Cognitive Psychology 18:1-86.
McClelland, J. L., and D. E. Rumelhart. (1981). An interactive activation model of context effects in letter perception: 1. An account of the basic findings. Psychological Review 88:375-407.
McRoy, S. W., and G. Hirst. (1990). Race-based parsing and syntactic disambiguation. Cognitive Science 14:313-353.
Norris, D. (1994a). Shortlist: A connectionist model of continuous speech recognition. Cognition 52:189-234.
Norris, D. (1994b). A quantitative multiple-levels model of reading aloud. Journal of Experimental Psychology: Human Perception and Performance 20(6):1212-1232.
Norris, D., J. M. McQueen, and A. Cutler. (1995). Competition and segmentation in spoken word recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition 21:1209-1228.
Oden, G. C., and D. W. Massaro. (1978). Integration of featural information in speech perception. Psychological Review 1985:172-191.
Pinker, S., and A. Prince. (1988). On language and connectionism: Analysis of a parallel distributed processing model of language acquisition. Cognition 28:73-193.
Plaut, D. C., J. L. McClelland, M. Seidenberg, and K. E. Patterson. (1996). Understanding normal and impaired word reading: Computational principles in quasi-regular domains. Psychological Review 103:56-115.
Plunkett, K., and V. Marchman. (1991). U-shaped learning and frequency effects in a multi-layered perceptron: Implications for child language acquisition. Cognition 38:43-102.
Plunkett, K., and V. Marchman. (1993). From rote learning to system building: Acquiring verb morphology in children and connectionist nets. Cognition 48:21-69.
Roelofs, A. (1992). A spreading-activation theory of lemma retrieval in speaking. Cognition 42:107-142.
Roelofs, A. (1996). Serial order in planning the production of successive morphemes of a word. Journal of Memory and Language 35:854-876.
Rumelhart, D. E., and J. L. McClelland. (1986). On learning the past tense of English verbs. In McClelland and Rumelhart, Eds., Parallel Distributed Processing: Explorations in the Microstructure of Cognition. Cambridge, MA: MIT Press.
Seidenberg, M., and J. L. McClelland. (1989). A distributed developmental model of word recognition and naming. Psychological Review 96:523-568.
Sharkey, N. E. (1990). A connectionist model of text comprehension. In D.A. Balota and G. B. Flores d'Arcais, Eds., Comprehension Processes in Reading. Hillsdale, NJ: Erlbaum, pp. 487-514 .
Copyright © 1999 Massachusetts Institute of Technology