Copyright © 1999 Massachusetts Institute of Technology
Binding is the problem of representing conjunctions of properties. It is a very general problem that applies to all types of KNOWLEDGE REPRESENTATION, from the most basic perceptual representations to the most complex cognitive representations. For example, to visually detect a vertical red line among vertical blue lines and diagonal red lines, one must visually bind each line's color to its orientation (see Treisman and Gelade 1980). Similarly, to understand the statement, "John believes that Mary's anger toward Bill stems from Bill's failure to keep their appointment," one must bind John to the agent role of believes, and the structure Bill's failure to keep their appointment to the patient role of stems from (see THEMATIC ROLES). Binding lies at the heart of the capacity for symbolic representation (cf. Fodor and Pylyshyn 1988; Hummel and Holyoak 1997).
A binding may be either static or dynamic. A static binding is a representational unit (such as a symbol or a node in a neural network) that stands for a specific conjunction of properties. For example, a neuron that responds to vertical red lines at location x, y in the visual field represents a static binding of vertical, red, and location x, y. Variants of this approach have been proposed in which bindings are coded as patterns of activation distributed over sets of units (rather than the activity of a single unit; e.g., Smolensky 1990). Although this approach to binding appears very different from the localist (one-unit-one-binding) approach, the two are equivalent in all important respects. In both cases, binding is carried in the units themselves, so different bindings of the same properties are represented by separate units. In a static binding, the capacity to represent how elements are bound together trades off against the capacity to represent the elements themselves (see Holyoak and Hummel forthcoming). In an extreme case, the units coding, say, red diagonal lines may not overlap at all with those representing red vertical lines.
Dynamic binding represents conjunctions of properties as bindings of units in the representation. That is, representational units are tagged with regard to whether they are bound together or not. For example, let red be represented by unit R, vertical by V, and diagonal by D, and let us denote a binding with the tag "+." A red diagonal would be represented as R + D and a red vertical as R + V. Dynamic binding permits a given unit (here, R) to participate in multiple bindings, and as a result (unlike static binding), it permits a representation to be isomorphic with the structure it represents (see Holyoak and Hummel forthcoming).
Dynamic binding permits greater representational flexibility than static binding, but it also has a number of properties that limit its usefulness. First, it is not obvious how to do dynamic binding in a neural (or connectionist) network. The most popular proposed binding tag is based on temporal synchrony: if two units are bound, then they fire in synchrony with one another; otherwise they fire out of synchrony (cf. Gray and Singer 1989; Hummel and Biederman 1992; Hummel and Holyoak 1997; Milner 1974; Shastri and Ajjanagadde 1993; von der Malsburg 1981). Although controversial (see Tovee and Rolls 1992), there is evidence for this type of binding in biological nervous systems (see König and Engel 1995). A more important limitation of dynamic binding is that it is impractical as a basis for binding in long-term MEMORY. For example, we may remember where we parked our car last Tuesday, but it is unlikely that the neurons representing our car have been firing in synchrony with those representing our parking space continuously since then. (The memory might be coded by, say, synaptic links between those neurons, and those links may have been created at the time we parked the car, but such links do not constitute dynamic bindings in the sense discussed here; see Holyoak and Hummel forthcoming.) A third limitation of dynamic binding is that it requires more attention and WORKING MEMORYthan static binding (see Hummel and Holyoak 1997; Stankiewicz, Hummel and Cooper 1998; Treisman and Gelade 1980; Treisman and Schmidt 1982). Although there is no theoretical limit on the number of conjunctive units (i.e., static bindings) that may be active at a given time, there are likely to be strong limits on the number of distinct tags available for dynamic binding. In the case of synchrony, for example, only a finite number of groups of neurons can be active and mutually out of synchrony with one another. Attention may serve, in part, to control the allocation of this finite dynamic binding resource (see Hummel and Stankiewicz 1996).
To the extent that a process exploits dynamic binding, it will profit from the isomorphism between its representations and the represented structures, but it will be demanding of processing resources (attention and working memory); to the extent that it binds properties statically, it will be free to operate in parallel with other processes (i.e., demanding few resources), but the resulting representations will not be isomorphic with the represented structures. These properties of static and dynamic binding have important implications for human perception and cognition. For example, these (and other) considerations led Hummel and Stankiewicz (1996) to predict that attended object images will visually prime both themselves and their left-right reflections, whereas ignored images will prime themselves but not their reflections. In brief, the reason is that dynamic binding (of features into object parts and object parts to spatial relations) is necessary to generate a left-right invariant structural description from an object's image (Hummel and Biederman 1992), and attention is necessary for dynamic binding (Treisman and Gelade 1980); attention should therefore be necessary for left-right invariant structural description. Stankiewicz, Hummel, and Cooper (1998) tested this prediction and the results were exactly as predicted. Apparently, the human visual system uses both static and dynamic codes for binding in the representation of object shape, and these separate codes manifest themselves, among other ways, as differing patterns of priming for attended and ignored object images. Similar tradeoffs between the strengths of static and dynamic binding are also apparent in aspects of human memory and thinking (cf. Hummel and Holyoak 1997).
Fodor, J. A., and Z. W. Pylyshyn. (1988). Connectionism and cognitive architecture: a critical analysis. In S. Pinker and J. Mehler, Eds., Connections and Symbols. Cambridge, MA: MIT Press, pp. 3-71
Gray, C. M. and W. Singer. (1989). Stimulus specific neuronal oscillations in orientation columns of cat visual cortex. Proceedings of the National Academy of Sciences USA 86:1698-1702.
Holyoak, K. J., and J. E. Hummel. (forthcoming). The proper treatment of symbols in connectionism. In E. Dietrich and A. Markman, Eds., Cognitive Dynamics: Conceptual Change in Humans and Machines. Cambridge, MA: MIT Press.
Hummel, J. E., and I. Biederman. (1992). Dynamic binding in a neural network for shape recognition. Psychological Review 99:480-517
Hummel, J. E., and K. J. Holyoak. (1997). Distributed representations of structure: a theory of analogical access and mapping. Psychological Review 104:427-466.
Hummel, J. E., and B. J. Stankiewicz. (1996). An architecture for rapid, hierarchical structural description. In T. Inui and J. McClelland, Eds., Attention and Performance XVI: Information Integration in Perception and Communication. Cambridge, MA: MIT Press, pp. 93-121.
König, P., and A. K. Engel. (1995). Correlated firing in sensory-motor systems. Current Opinion in Neurobiology 5:511-519.
Milner, P. M. (1974). A model for visual shape recognition. Psychological Review. 81:521-535.
Shastri, L., and V. Ajjanagadde. (1993). From simple associations to systematic reasoning: a connectionist representation of rules, variables and dynamic bindings. Behavioral and Brain Sciences 16:417-494.
Smolensky, P. (1990). Tensor product variable binding and the representation of symbolic structures in connectionist systems. Artificial Intelligence 46:159-216.
Stankiewicz, B. J., J. E. Hummel, and E. E. Cooper. (1998). The role of attention in priming for left - right reflections of object images: evidence for a dual representation of object shape. Journal of Experimental Psychology: Human Perception and Performance 24:732-744.
Tovee, M. J., and E. T. Rolls. (1992). Oscillatory activity is not evident in the primate visual cortex with static stimuli. NeuroReport 3:369-372.
Treisman, A., and G. Gelade. (1980). A feature integration theory of attention. Cognitive Psychology 12:97-136.
Treisman, A. M., and H. Schmidt. (1982). Illusory conjunctions in the perception of objects. Cognitive Psychology 14:107-141.
von der Malsburg, C. (1981). The correlation theory of brain function. Internal Report 81-2. Göttingen, Germany. Department of Neurobiology, Max-Plank-Institute for Biophysical Chemistry.
Gallistel, C. R. (1990). The Organization of Learning. Cambridge, MA: MIT Press .
Copyright © 1999 Massachusetts Institute of Technology