Cultural Consensus Theory

Cultural consensus theory is a collection of formal statistical models designed to measure cultural knowledge shared by a set of respondents. Each respondent is given the same set of items designed to tap the respondents' shared knowledge. The data consist of a respondent-item matrix containing each respondent's answers to each of the items. An appropriate cultural consensus model provides estimates of each respondent's competence (knowledge) as well as an estimate of the culturally correct answer to each item. When the theory was developed in the mid-1980s it was motivated by the observation that when an anthropologist goes to a new culture and asks questions, neither the answers to the questions nor the cultural competence of the respondents is known. It has since been applied to a number of research questions, for example, folk medical beliefs, judgment of personality traits in a college sorority, semiotic characterizations of alphabetic systems, occupational prestige, causes of death, illness beliefs of deaf senior citizens, hot-cold concepts of illness, child abuse, national consciousness in Japan, measuring interobserver reliability, and three-way social network data.

Consensus theory uses much of the accumulated knowledge of traditional psychometric test theory without assuming knowledge of the "correct" answers in advance. Traditional test theory begins with respondent-item "performance" data (i.e., items' scores as "correct" or "incorrect"), whereas consensus theory begins with "response" data (items coded as responses given by the respondent, for example, "true" or "false," without scoring the responses). The different models of the theory depend on the format of the questions, for example, true-false, multiple choice, or ranking. Anthropology is the prototypical social science that can use such a methodology; however, research in other areas of social and behavioral science, such as cognitive psychology, social networks, and sociology, can also benefit from its use.

Cultural consensus theory fits into the category of information-pooling methods in which one has answers from several "experts" to a fixed body of "objective" questions. The goal is to aggregate rationally the experts' responses to select the most likely "correct answer" to each question, and also to assess one's degree of confidence in these selections. Cultural consensus theory provides an information-pooling methodology that does not incorporate a researcher's prior beliefs about the correct answers or any prior calibrations of the experts, and instead, it estimates both the respondents' competencies and the consensus answers from the same set of questionnaire data.

A central concept in the theory is the use of the pattern of agreement or consensus among respondents to make inferences about their differential knowledge of culturally shared information represented in the questions. It is assumed that the sole source of correspondence between the answers of any two respondents is a function of the extent to which the knowledge of each is correlated with (overlaps) this shared information. In other words, when responses are not based on shared information they are assumed to be uncorrelated. More formally, the model is derived from a set of three basic assumptions that are elaborated appropriately for each question format:

Assumption 1: Common Truth. There is a fixed answer key applicable to all respondents.
Assumption 2: Local Independence. The respondent-item response random variables satisfy conditional independence (conditional on the correct answer key).
Assumption 3: Homogeneity of Items. Each respondent has a fixed competence over all questions.

In some contexts Assumption 3 is replaced with a weaker one, monotonicity, that allows them to differ in difficulty: Basically, monotonicity says that respondents who have more competence on any subset of questions will have more competence on all subsets.

Formal process models have been derived for the analysis of dichotomous, multiple-choice, matching, and continuous item formats. Informal data models have also been developed for rank order and interval level formats. The theory has also been extended to the analysis of multiple cultures by relaxing the first axiom. In this situation each respondent belongs to exactly one culture, but different cultures may have different answer keys.

For very small sets of respondents (six or fewer), iterative maximum likelihood estimates of the parameters can be obtained by existing methods. For example, in the true-false case, the consensus model is equivalent to the two-class latent structure model with the roles of respondents and items interchanged; thus known estimation methods for that model can be used. For other situations, new estimation methods have been developed and assessed with Monte Carlo data. The theory enables the calculation of the minimal number of respondents needed to reconstruct the correct answers as a function of preselected levels of mean cultural competence of the respondents and levels of confidence in the reconstructed answers. It is also possible to estimate the amount of sampling variability among respondents and thus identify "actual" variance in cultural competence.

The theory performs better than does using a simple majority rule to reconstruct the answer key, especially in cases where there are small numbers of respondents with heterogeneous competence. The success of cultural consensus theory as an information-pooling method can be traced to several factors: (1) it is normally applied to items that tap high concordance cultural codes where mean levels of consensus are high; (2) the theory allows the differential weighting of the respondents' responses in reconstructing the answer key; and (3) the theory uses precise assumptions derived from successful formal models in test theory, latent structure analysis, and signal detection theory. The model has been subjected to extensive testing through simulation and Monte Carlo methods.

Additional links

A. Kimball Romney Consensus Theory Page

-- A. Kimball Romney and William H. Batchelder

Cultural Consensus Theory

See also

Additional links

Further Readings