Copyright © 1999 Massachusetts Institute of Technology
The term ecological validity refers to the extent to which behavior indicative of cognitive functioning sampled in one environment can be taken as characteristic of an individual's cognitive processes in a range of other environments. Consequently, it is a central concern of cognitive scientists who seek to generalize their findings to questions about "how the mind works" on the basis of behavior exhibited in specially designed experimental or diagnostic settings. This concern was provocatively expressed by Urie Bronfenbrenner (1979), who complained that too much of the study of child development depended on the study of children in strange circumstances for short periods of time, in contrast with the ecologies of their everyday lives.
Discussions of the problem of ecological validity first came to prominence in cognitive research in the United States owing to the work of Egon Brunswik and Kurt Lewin, two German scholars who emigrated to the United States in the 1930s. Other important sources of ideas about ecological validity include Roger Barker (1978), whose work on the influence of social setting on behavior retains its influence to the present day, and J. J. GIBSON (1979), who argued that the crucial questions in the study of perception are to be resolved not so much by an attention to the perceiver as by the description of how the environment in particular everyday life arrangements "affords" a person perceptual information; the issue was given further prominence in Ulric Neisser's influential Cognition and Reality in 1976.
Brunswik (1943) proposed an ECOLOGICAL PSYCHOLOGY in which psychological observations would be made by sampling widely the environments within which particular "proximal" tasks are embedded. Brunswik's overall goal was to prevent psychology from being restricted to artificially isolated proximal or peripheral circumstances that are not representative of the "larger patterns of life." In order to avoid this problem, he suggested that situations, or tasks, rather than people, should be considered the basic units of psychological analysis. In addition, these situations or tasks must be "carefully drawn from the universe of the requirements a person happens to face in his commerce with the physical and social environment" (p. 263). To illustrate his approach, Brunswik studied size constancy by accompanying an individual who was interrupted frequently in the course of her normal daily activities and asked to estimate the size of some object she had just been looking at. This person's size estimates correlated highly with physical size of the objects and not with their retinal image size. This result, Brunswik claimed, "possesses a certain generality with regard to normal life conditions" (p. 265).
Lewin proposed a "psychological ecology," as a way of "discovering what part of the physical or social world will determine, during a given period, the 'boundary zone' of the life space" of an individual (1943: 309). By life space, Lewin meant "the person and the psychological environment as it exists for him" (p. 306). He argued that behavior at time t is a function of the situation at time t only, and hence we must find ways to determine the properties of the lifespace "at a given time." This requirement amounts to what ethnographers refer to as "taking the subject's point of view." It seeks to unite the subjective and the objective.
If one agrees that understanding psychological processes in terms of the life space of the subject is important, following the logic of Lewin's argument, Brunswik's approach was inadequate. His experimental procedures did not allow him to observe someone fulfilling a well-specified task in a real-life environment; rather, it amounted to making experiments happen in a nonlaboratory environment. His procedures, in Lewin's terminology, changed the subject's life space to fit the requirements of his predefined set of observation conditions.
Ulric Neisser (1976) also pointed out marked discontinuities between the "spatial, temporal, and intermodal continuities of real objects and events" and the objects and events characteristic of laboratory-based research as a fundamental shortcoming of cognitive psychology, going so far as to suggest, "It is almost as if ecological invalidity were a deliberate feature of the experimental design" (1976: 34).
Urie Bronfenbrenner's (1979, 1993) advocacy of ecologically valid research has greatly influenced the study of cognitive social development (Cole and Cole 1996). There are, he writes, three conditions that ecologically valid research must fulfill: (1) maintain the integrity of the real-life situations it is designed to investigate; (2) be faithful to the larger social and cultural contexts from which the subjects come; (3) be consistent with the participants' definition of the situation, by which he meant that the experimental manipulations and outcomes must be shown to be "perceived by the participants in a manner consistent with the conceptual definitions explicit and implicit in the research design" (1979: 35).
Note that there is a crucial difference between Lewin, Neisser, and Bronfenbrenner's interpretations of how to conduct ecologically valid research and the procedures proposed by Brunswik. Neisser, Bronfenbrenner, and others do not propose that we carry around our laboratory task and make it happen in a lot of settings. They propose that we discover and directly observe the ways that tasks occur (or don't occur) in nonlaboratory settings. Moreover, in Bronfenbrenner's version of this enterprise we must also discover the equivalent of Lewin's "life space," for example how the task and all it involves appear to the subject.
Two decades ago, the idea that such discovery procedures are possible was quite widespread among researchers who used experimental procedures and were cognizant of questions of ecological validity. Herbert Simon was echoing common opinion when he asserted that there is
a general experimental paradigm that can be used to test the commonality of cognitive processes over a wide range of task domains. The paradigm is simple. We find two tasks that have the same formal structure (e.g., they are both tasks of multi-dimensional judgment), one of which is drawn from a social situation and the other is not. If common processes are implicated in both tasks, then we should be able to produce in each task environment phenomena that give evidence of workings of the same basic cognitive mechanisms that appear in the other. (1976: 258)
However, a variety of contemporary research indicates that the requirements for establishing ecological validity place an enormous analytical burden on cognitive scientists (see Cole 1996: ch. 8-9 for an extended treatment of the associated issues). Once we move beyond the laboratory in search of representativeness, the ability to identify tasks is markedly weakened. Failure to define the parameters of the analyst's task or failure to insure that the task-as-discovered is the subject's task can vitiate the enterprise. This point was made clearly by Schwartz and Taylor (1978). Their particular interest was the representativeness of standardized achievement and IQ tests, but their specification of the issues involved has broad applicability in cognitive science. They queried, "Does the test elicit the same behavior as would the same tasks embedded in a real noncontrived situation? . . . Even to speak of the same task across contexts requires a model of the structure of the task. In the absence of such a model, one does not know where the equivalence lies (p. 54)."
As Valsiner and Benigni (1986) point out, standardized cognitive experimental procedures are meant to embody closed analytic systems (the point of the experimental/test procedures being to achieve precisely this closure). Consequently, attempting to establish task equivalence in order to generalize beyond the experimental circumstances amounts to imposing a closed system on a more open behavioral system. To the degree that behavior conforms to the prescripted analytic categories, one achieves ecological validity in Brunswik's sense. Yet a variety of research (reviewed in Cole 1996) has shown that even psychological tests and other presumably "closed system" cognitive tasks are more permeable and negotiable than analysts ordinarily take account of. Insofar as the cognitive scientist's closed system does not capture veridically the elements of the open system it is presumed to model, experimental results systematically misrepresent the life process from which they are derived. The issue of ecological validity then becomes a question of the violence done to the phenomenon of interest owing to the analytic procedures employed (Sbordone and Long 1996).
Barker, R. (1978). Ecological Psychology. Stanford, CA: Stanford University Press.
Bronfenbrenner, U. (1979). The Ecology of Human Development. Cambridge, MA: Harvard University Press.
Brunswik, E. (1943). Organismic achievement and environmental probability. The Psychological Review 50:255-272.
Cole, M. (1996). Cultural Psychology: A Once and Future Discipline. Cambridge, MA: Harvard University Press.
Cole, M., and S. R. Cole. (1996). The Development of Children. 3rd ed. New York: Freeman.
Gibson, J. J. (1979). An Ecological Approach to Visual Perception. Boston: Houghton-Mifflin.
Lewin, K. (1943). Defining the "field at a given time." Psychological Review 50:292-310.
Sbordone, R. J., and C. J. Long, Eds. (1996). Ecological Validity of Neuropsychological Testing. Delray Beach, FL: GR Press/St. Lucie Press, Inc.
Schwartz, J. L., and E. F. Taylor. (1978). Valid assessment of complex behavior. The Torque approach. Quarterly Newsletter of the Laboratory of Comparative Human Cognition 2:54-58.
Simon, H. A. (1976). Discussion: Cognition and social behavior. In J. S. Carroll and J. W. Payne, Eds., Cognition and Social Behavior. Hillsdale, NJ: Erlbaum.
Copyright © 1999 Massachusetts Institute of Technology