DEFINITIONAL VALIDITY: THE CONCEPT AND A0 CASE STUDY

BY

STEPHEN CHILTON



An earlier version of this essay was presented at the 1985 Annual Meeting of the International Society of Political Psychology, Washington, D.C. I am indebted to Jerry Gaus, Judith Gillespie, Richard Hudelson, Don Kurtz, Beth Lau, Dana Ward, and several anonymous journal reviewers for their encouragement and/or intelligent commentary. I remain, of course, solely responsible for any lapses of judgment.


DEFINITIONAL VALIDITY: THE CONCEPT AND A CASE STUDY

ABSTRACT

All researchers implicitly make three well-known validity claims for their research: of "internal validity", of "external validity", and (implicitly) of substantive importance ("substantive validity"). However, researchers also need to examine a less well-known validity claim: that the research's methods and measures conform to researchers' implicit claims for the substantive significance of their work ("definitional validity"). This article sets forth the claim of definitional validity and explains its use by application to two fields: the measurement of economic inequality and the measurement of belief constraints. xx on inequality. The article argues that the belief regularities contemplated by correlational measures of constraint are not consonant with the substantive importance constraint is held to have for democracy. These critiques are not new; the contribution of this article is to recognize the claim of definitional validity and demonstrate how it organizes and clarifies the significance of otherwise diverse criticisms.




DEFINITIONAL VALIDITY: THE CONCEPT AND A CASE STUDY

Social-scientific research derives its claim to significance from four successive validity claims - four inferences whereby the immediate research methods, data and measure acquire significance as shedding light on issues of acknowledged ethical consequence. Three of these claims are familiar: the claims of internal validity; of external validity; and of substantive significance, which we will call substantive validity, both for parallelism's sake and because it is indeed a validity claim. The claim of definitional validity, however, is not a familiar one, even though theorists already implicitly evaluate research in terms of it. The purpose of this paper is to define this claim clearly, to show its logical relationship to the claims of internal, external and substantive validity, and to apply this concept to the case of constraint research to show how the claim is tested and how the failure to redeem the claim can undercut an entire research tradition.

Beyond the above discussion of the four validity claims and the associated explanation of their relation to one another, the purpose of this article is to elaborate the unfamiliar claim of definitional validity by evaluating the claims to it made by belief systems research, particularly that using correlational constraint measures. The point of the article is not to develop new attacks on constraint research but instead to show how earlier, diverse criticisms all flow coherently and straightforwardly from the evaluation of definitional validity. Indeed, almost all of the critiques appearing in this evaluation have been made previously, and it would be strange if we found many new ones: constraint research has been subjected to nearly thirty years of theoretical scrutiny, and such scrutiny has always been conducted with at least a background recognition that methods must match their research's substantive significance. This essay simply brings this recognition to the foreground, and constraint research happens to provide good examples.

Note that the claim to definitional validity is made, implicitly or explicitly, by all research, not just constraint research. Constraint research happens to be an exceptionally good example of where previous theoretical critiques can almost all be gathered under the one rubric of analyses of definitional validity.


1. Social Research's Four Validity Claims

The four inferences whereby social-scientific research sets claim to significance are shown in Figure 1 below.(1)



Figure 1: Research projects' four validity claims

Observational methods and analytic measures

|

|

| INTERNAL

| VALIDITY

|

Empirical findings & interpretations applied

solely within the domain of the study

|

|

| EXTERNAL

| VALIDITY

|

Conclusions about society in general

|

|

| DEFINITIONAL

| VALIDITY

|

Relevance to issues believed by the

researcher to be of ethical consequence

|

|

| SUBSTANTIVE

| VALIDITY

|

Relevance to issues of ethical consequence

To start with, researchers must establish that their choice of data collection and data analysis methods supports their empirical findings and interpretations of what is happening in the immediate experimental setting. This issue of internal validity concerns whether the outcome of the process being studied is a consequence of the explanatory variable(s) or of some other variable or process. In other words, are there alternative explanations for the results within the boundaries of the research study itself?

The next question is whether the interpretations can be generalized to some intended domain of generalization beyond the research project. This issue of external validity asks whether the experimental results or interpretations are likely to apply outside the immediate context of the experiment. Do the findings represent a general law, or only one highly contingent on the specific research context?

Skipping over the third validity claim, we take up the fourth claim of substantive validity: that the results are of substantive significance. The claim here is that we want to know these results for reasons beyond the sheer accumulation of curious facts. In other words, researchers would like to claim that the results make some important difference to the way we view (or act within) the social world.

The claim of definitional validity mediates between the first two claims and the fourth claim. The claim to definitional validity states that the researcher's measures are consonant with her substantive intent. The researcher wishes to assert that her findings actually address the questions she sees as substantively significant. Her judgment of substantive importance may be idiosyncratic, but others' agreement is not the issue: the issue of definitional validity only concerns whether the interpretations speak to her sense of substantive value.

There is, of course, a metaethical debate over the truth status of moral claims: whether they have objective truth, intersubjective truth, or no truth at all beyond individual opinion. People who hold that moral claims have no truth value at all can simply ignore the final step of substantive validity. The real focus of this paper (the issue of definitional validity) is not affected, however.

But suppose that we accept that claims to substantive validity do have some truth status: why then should we be concerned with definitional validity? Why attend to what the researcher feels is important about the research instead of dealing directly with the claims of real substantive validity? The answer is that even if claims to substantive importance have some truth status, they very often remain unsettled for a long time. Claims to definitional validity, on the other hand, can be evaluated rather straightforwardly (as the present essay shows), involving a dispute only over the hermeneutic interpretation of what the researcher believes the substantive significance of her findings are. In other words, the evaluation of definitional validity does not negate that of substantive validity, but it can be much easier to handle.

To summarize, of these four claims, only that of substantive validity projects the methodologist into directly moral argumentation. The first two claims depend only on logic and empirical judgment, and while the claim of definitional validity may require a hermeneutic analysis of the text in order to infer the researcher's substantive concerns, it does not require a review of the actual significance of those concerns.(2)

To see concrete examples of these four validity claims, let us look at Cary Covington's (1988) recent work on presidential coalitions. Briefly, his "study examines roll-call votes in the House of Representatives identified by the White House as 'critical' motions during the period 1961-67 to determine the extent to which cross-pressured but supportive Democrats supported the presidents' positions by means other than voting for them. ... Findings demonstrate that ... cross-pressured supportive Democrats abstain [a form of 'alternative support'] on key votes more often than consonantly pressured supporters..." (Covington 1988:3). The study raises the four validity claims, as exemplified by sample claims listed below:

Sample Claims of Internal Validity: Cross-pressured Representatives abstain, switch, and pair to some extent because of Presidential pressure. To state this in another way, there is no variable that partials out the observed relationship in this dataset between the importance Presidents attached to different bills and the likelihood of cross-pressured Representatives providing alternative modes of voting support.

Sample Claims of External Validity: The period 1961-1967 is representative of the American national political process generally. They apply to Republican as well as Democratic administrations.

Sample Claims of Definitional Validity: The measure of cross-pressure really measures cross-pressure; ditto for the measures of "critical motions" and "alternative support". In particular, "alternative support" really is support for the President's position to some degree and "critical motions" can be chosen by the President independent of constraints of party and ideological balance.

Sample Claims of Substantive Validity: The findings show that our political system gives scope to individual action, so that concepts of skill and will remain important for political actors beyond such impersonal constraints as party dominance, ideological balance, and presidential popularity.(3)

If we look at standard social science methodology articles and texts, we observe that methodologists concentrate primarily on internal validity, markedly less on external validity, and little at all on definitional validity. Issues of substantive validity are considered to be the concern of political theory, and are virtually unconsidered in methodological discussion.(4)

This pattern of attention is a natural consequence of the general social-scientific division between issues of fact and value (and avoidance of the latter). Methodologists can handle most issues of internal (and, to a lesser degree, external) validity by reference to accepted standards of logical inference -- i.e., the canons of the scientific method. Issues of whether findings are truly of substantive significance, on the other hand, can be resolved only by arguments involving ethics, for which we have as yet no settled standards. Given the very different character of these two domains of discourse, and given the settled nature of one relative to the other, it should be no surprise that most methodological attention has been to the former domain, where relatively firm conclusions are possible and much work still remains to be done. It is clear that issues of definitional validity are important ones, however. If research methods do not support any sense that the findings have substantive significance, then we have no reason to pay attention to the findings, regardless of the methods' internal and external validity.

Under what circumstances is this scrutiny of definitional validity possible? It is obviously possible whenever there is no dispute about the substantive consequence of the research question. If everyone agrees on the nature of our concern about a certain issue, then we methodologists are on firm ground when we ask whether the methods and measures yield conclusions addressing that concern. But even when the research's substantive concerns are disputed, we can still ask whether the methods yield conclusions addressing the researcher's concern. We need not deal with the issue of substantive validity -- that is, about whether the researcher's concern is of real moral consequence. We are always entitled to ask whether the researcher's methods and moral intentions form a consistent whole; we need not get trapped in debates about what has true substantive consequence.

One difficulty with the evaluation of definitional validity is that researchers frequently do not state explicitly what claims of substantive significance they are making for their research. Working in an existing tradition of inquiry, they customarily take their research's significance for granted. To evaluate definitional validity, then, we will often have to infer the nature of the significance intended, and our evaluation will be conditioned upon our having gotten the sense of significance right.


2. Belief Systems Research Asserts Substantive Significance

Mainstream analyses of democratic politics commonly view it as the politics of belief systems: a process by which policy preferences and the beliefs embedding them are translated into political and finally governmental action. Political debates involve the clash of beliefs over both the factual correctness of various assertions and the virtue of adopting various policies. The outcomes of these debates are correspondingly affected by the distribution and organization of beliefs about such assertions and policies.

Ideally, the citizens of a working democracy would have rational belief structures: accurate in assessment of the world (realism); containing no internal contradictions (logical consistency); and smoothly linking means to ends (goal rationality).(5) Unfortunately, citizens (including researchers!) have no objective way to assess belief systems' realism, logical consistency, and goal rationality. Belief systems theorists accordingly pay most of their attention to the looser demands of what we call ideology. Quite reasonably, researchers hold that if they can't determine rationality, they must settle for what is feasible: the determination of ideological coherence. Belief systems are ideologically coherent when they are "particularly elaborate, close-woven, and far-ranging" (Campbell et al. 1960:192). Ideology "links particular actions and mundane practices with a wider set of meanings" (Apter, 1964:16).

Belief systems theorists argue the substantive importance of belief systems structure in two major ways.(6) First, ideology facilitates democratic politics. In a society where only a limited number of ideologies are present, political leaders can readily search for and attract support, whereas in a society with no ideologies or with thousands, each new political issue requires the laborious construction of completely new alliances. Further, a unidimensional variation of ideologies is especially helpful for party groupings and electoral appeals. Early researchers like V. O. Key, Jr. (1961) and Campbell et al. (1960) emphasize the importance of belief systems research for an understanding of our political system.(7)

Second, structured belief systems also are seen as desirable for individual political actors, not just for the political system as a whole. Ideology gives coherence and meaning to political issues, enabling individuals to grasp and react to new issues more readily. Even though logic may not apply to belief systems, ideology purports to provide the same assurances as logic: that goals are not self-contradictory and that means are appropriate and efficient. In short, ideology is supposed to facilitate citizens' ability to act effectively in political contexts.

The focus on individuals is especially marked in the seminal works of Campbell et al. (1960) and Converse (1964), most explicitly in the latter. Converse lauds "the idea organization that leads to constraint" [his term for ideological structure], because it "permits [the actor] to locate and make sense of a wider range of information from a particular domain than he would find possible without such an organization. ... The efficiency of such a yardstick in the evaluation of events is quite obvious" <214: bracketed page numbers refer to Converse, 1964>. "In our estimation, the use of [ideologies] betokens a conceptual grasp of polities that permits a wide range of more specific ideo-elements to be organized into more tightly constrained wholes. We feel, furthermore, that there are many crucial consequences of such organization: With it, for example, new political events have more meaning, retention of political information from the past is far more adequate, and political behavior increasingly approximates that of sophisticated 'rational' models, which assume relatively full information" <227>. When Converse discusses his coding of the respondents' answers as "broadly philosophical" <222>, a footnote indicates his preference for ideological responses: "In all candor, it should probably be mentioned that a teacher grading papers would be unlikely to give passing marks to more than 20% of the attempted definitions.... We made an effort, however, to be as generous as possible in our assignments" <257: emphasis added>. In summary, Converse takes a directly evaluative stance toward the ideological structure of belief systems: they make "adequate" the retention of political information, and the political actor becomes more "efficient" in "making sense of" or "giving meaning to" the political environment. Such structure is "sophisticated," "rational," and -- the academic emerging -- will receive "passing marks".


3. Measuring Belief System Structure through Correlational Constraint

Belief systems research has two potential standards by which to judge how ideologically structured people's beliefs are (Barton and Parsons, 1977). The first approach, applicable to individuals considered singly, employs standards of logic or "psycho-logic". Such standards are implicitly applied when researchers content-analyze subjects' open-ended responses, looking for the presence of coherent approaches to issue, party, candidate or other evaluations. Originally, this content analysis was simply a matter of finding key words or classifiable statements. In recent years, however, researchers have come to employ the more sophisticated methods associated with Piagetian psychology (e.g., Ward, 1982; Rosenberg, Ward, and Chilton, 1989) and schema theory (e.g., Axelrod, 1973; Fiske and Linville, 1980; Conover and Feldman, 1984).

This essay, however, is concerned not with this first approach to assessing ideological structure but rather with a second general approach: the "constraint" or "model-fitting" approach. This second approach, applicable only to people considered in groups, employs standards derived from the group as a whole. In these methods, response patterns are collectively fit to a statistical model of what an ideologically structured population looks like. Greater aggregate fit then implies more ideological structure. Several models have been used:

Applying the above methods to the U.S. population, constraint researchers have obtained such findings as: elites are more ideologically constrained than the population at large (e.g., Converse, 1964); and the population has become more ideologically constrained of late (e.g., Nie and Andersen, 1974).


4. Correlational Constraint Measures Have No Definitional Validity

These conclusions have been vigorously attacked on many grounds.(9) The present essay evaluates the constraint methods in terms of their definitional validity, not their internal, external, or substantive validity. This evaluation has three parts. First, we interpret the constraint research literature to determine what claims it makes for substantive significance. This was the burden of the last section, which concluded that belief systems research claimed that its strictures on both individual beliefs and group patterns of beliefs made democratic politics easier. Second, we must elucidate the nature of the moral positions constituted in the methods and measures used to assess the constraint concept. Since something of substantive significance is being measured, we may ask what "good" that measurement implies and whether we believe that version of goodness. Third, we evaluate these moral positions against the claims of substantive consequence. These last two steps are the burden of this section.

Because the mean correlation method is the simplest to describe, and because it implies the same metaethical premises as more sophisticated methods, it will be the focus of the critique.

The constraint approach makes no direct judgments on the content of beliefs or ideological positions at issue, so to a first approximation it takes no ethical position.(10) Converse (and Campbell et al. 1960) specifically note that they accept ideological terms from any tradition (socialist, etc.). Insofar as the constraint approach treats all ideologies even-handedly, it resists any direct attack on its ethical premises.

The approach does, however, hold that we can evaluate belief systems in terms of whether group belief configurations are constrained. This is a metaethical position: a criterion by which one may judge the form or structure of ethical positions.(11) "Constraint" is not a statement of what beliefs are good but rather a criterion for recognizing how beliefs should be related to one another: a "good" belief system must be "constrained".

The constraint approach goes no farther than the position that constraint is only one virtue of a belief system. There may well be other ethical or metaethical criteria that distinguish among equally constrained belief systems, but clearly constraint is important. The possibility of such other factors has not been a prominent feature of constraint theorists' defense of their work, however.

There are several methods of measuring constraint, but the mean correlation method has the greatest practical simplicity, avoiding the need for complex computations, subjective coding of responses, or follow-up surveys. So, for example, Converse uses the data from seven questions administered to a 1958 sample of American adults, computes tau-gamma correlation coefficients among the questions, and takes as his measure of constraint the mean absolute value of these correlations.(12) The higher the mean absolute correlation, the more constrained the belief system. A mean of 1 indicates a completely constrained system, since all questions intercorrelate perfectly; a mean of 0 indicates no constraint, since no question correlates with any other.(13)

The constraint criterion evaluates aggregates and thus cannot be applied directly to the belief system of an individual: philosophically, the method interprets actors' beliefs only in the context of other actors'; mathematically, correlations cannot be computed for single individuals. How are aggregates evaluated? They are not evaluated in terms of their adherence to a morally "correct" position. They are not evaluated even in terms of their adherence to any of a range of correct positions (e.g., positions ranging from liberal to conservative). Instead, the aggregates are evaluated in terms of the extent to which their beliefs lie along a straight line in multidimensional belief-space.(14)

Individuals' beliefs are not evaluated by this method, but it is a reasonable conclusion, consistent with the overall method, that this approach deems an individual's beliefs good to the degree that they adhere to, rather than deviate from, the straight line in belief-space defined by this and all other actors' beliefs. The constraint measure therefore creates two evaluations: it explicitly measures the goodness of an aggregate's constellation of beliefs, and it implicitly measures the goodness of each actor's belief system.

Casting the evaluation methods in these terms, we can see immediately several difficulties. For aggregates, the constraint measure fails to recognize obvious good or evil in several circumstances:

It might be argued that, unusual as these examples appear, such unidimensional belief structures do have the virtue of making more orderly democratic processes possible. This argument assumes, however, that democracy requires a linear variation of beliefs. That assumption is both unproven and highly suspect. Unidimensional variation of beliefs may simplify political coalition-building, but it is a long step to the conclusion that democracy needs much cognitive simplicity. Constraint theorists offer no compelling reason to weight simplicity so heavily in the evaluation of societies.(17)

But perhaps constraint is really meant to apply to individuals. After all, are we not concerned that political actors grasp in an integrated fashion the complex issues of the day? Regardless of any collective unidimensionality, will it not be hard to construct truly democratic politics when the demos cannot think rationally? Let us turn, then, from constraint's measure of group virtue to its implied measure of individual virtue. Two arguments reveal the arbitrary nature of the constraint measure applied to an individual's beliefs:

.

. .

.. .

. .. .

. . .

. .

. . X

.

.





Figure 2: "Bad" outlier X lowers the level of constraint



.

. .

.. .

. .. .

. . .

. .

. .

.

.









X



Figure 3: "Good" outlier X raises the level of constraint and alters its direction


5. The Focus on Definitional Validity Organizes Previous Critiques

It is fantastic that such a seemingly natural measure as mean correlation could produce such monstrosities. Even if researchers were happy with what correlational constraint measures, its measure of subgroups would still have no relation to its measure of their combination. And if researchers were not happy with correlational constraint and so used it only as one criterion out of possibly many, it would not aid the measurement of the sort of ordered belief structure intended. In short, correlational constraint has no significance as defined, and whatever meaning it does have in a particular context applies only as long as the context remains fixed. Belief structure is too central a concept of political analysis to be associated with such a flawed measure.

Our focus on definitional validity has clarified where corrections must be made. We have seen that the problems of the constraint measure arise from its Durkheimian premises that knowledge, rationality, and moral obligation are determined by group agreement. The examples advanced above to demonstrate the absurd consequences of this position all draw on the contrast between the individual's beliefs and the configuration of the group's beliefs. While social actors' beliefs are certainly affected by others' beliefs, and while a rational actor's belief system will take cognizance of those other beliefs, it is not reasonable to define ideological coherence solely in terms of belief configurations defined primarily by others. Studies of belief structure must accordingly employ concepts and analytic methods based on individuals, not groups.(20) Thus evaluating definitional validity is of immediate theoretical use.


6. The Status of Definitional Validity

The implications of definitional validity go deeper than simply the organization of critiques, however. it is a fundamental aspect of research logic, and it carries us beyond the traditional fact-value distinction.

(1) Definitional validity is an implicit claim of every research project. Even though it is not as well known as the claims of internal, external and substantive validity, any research project always already asserts that its methods and operationalizations support its normative intentions. For the most part researchers implicitly acknowledge this claim by developing measures consistent with their normative concerns. Thus scrutiny of definitional validity is always appropriate but usually can be taken for granted. In this sense, the validity problems of correlational constraint are unusual, but in principle there is no difference between scrutinizing its claims and those of any other research program.

(2) Definitional validity also mediates between claims of fact and claims of value. Definitional validity connects the purely methodological issues of internal and external validity with the purely normative issues of substantive validity. This intermediate role of definitional validity may explain why it has not previously been distinguished: because of the traditional split in social science between fact and value, connections between them are ignored. If empirical researchers openly state their claims to the substantive significance of their work, they become vulnerable to attacks on the substantive validity of their position. If normative analysts openly state their reliance on empirical presuppositions, they become vulnerable to empirical falsification. Neither side has an incentive to cross the middle ground. But surely our concern for the substantive significance of our work should be the centerpiece of social research, not locked away in its theoretical dungeons. Direct examination of the definitional validity of our research offers us a greater meaningfulness to our work, at the cost only of opening up our positions to attacks on their substantive validity - that is, at the cost only of admitting that social science is rooted in moral concerns.


BIBLIOGRAPHY


FOOTNOTES

0. Amartya Sen did a number of works on the moral premises of measures of inequality, which I mean to use as a second example of (an attack on a concept's) definitional validity. Here are some references Jerry Gaus gave me to get me started:

1. These four claims are related to, but are not identical with, those presented in Fischer (1985). Fischer's "technical verification" combines the claims of internal and external validity, while his "situational validation", "systemic vindication" and "rational social choice" are all subsumed here under the claim of substantive validity. As is usual in social-scientific literature, the claim of definitional validity is taken for granted and thus ignored.

2. Such hermeneutic analysis need be undertaken only when the researcher is unavailable, unable, or unwilling to explicate her research's claims to substantive validity.

3. I discovered this claim by the simple expedient of calling up Professor Covington and asking what he thought the significance of his research was. If he had not been available, an analysis of this work (and his works as a whole, and related works, etc.) would have been required to infer his purpose.

4. An exception is the standard caution made in methods texts that statistical significance does not imply substantive significance. The text's claim applies to the field of methodology; it is less true of the field of policy analysis, where many analysts (e.g., Fischer 1985; DeLeon 1987) have emphasized the role of normative claims made by different evaluators (DeLeon 1986). The increasing social-scientific attention to value relevance arises from the increasing delegitimation of dominant social-scientific paradigms. Formerly, researchers in the (say) pluralist tradition could carry out research whose normative intention, while present, was implicit in their tradition and did not have to be spelled out. (One need not claim, as some critics do, that the failure to spell it out is a deliberate mystification.) The advent of nontraditional perspectives has resulted in a spate of analyses which differ from those of more traditional work and thus have to address more explicitly their normative intentions. What we are observing in this efflorescence of normative analysis is not an increase in normative concern but rather the development of an alternative normative position.

5. These three criteria motivate the discussion in Campbell et al. (1960:190) of "functional relationships" among beliefs.

6. England (1984) discusses several justifications usually given for ideological thought, including some not given here.

7. These justifications no longer appear in belief systems work, but this is probably due to the technical concerns becoming more important, not the substantive concerns becoming rejected or unimportant.

8. Despite its name, this method still fixed individuals in a shared configuration space. The "individual differences" lie only in the different weights different respondents attach to the various dimensions of this common space.

9. For example, Bishop, Tuchfarber, and Oldendick (1978) attack the external validity of Nie and Andersen's (1974) findings by arguing that the supposed increase in ideological constraint may be due to changes over time of the survey questions employed. See also the many criticisms made by Lane (1974); Marcus, Tabb, and Sullivan (1974); Bennett (1975); Barton and Parsons (1977); and Ward (1982).

10. Only "to a first approximation", because the survey questions employed are more relevant to some ideological positions than others. This is not an issue of definitional validity, however. The researchers' question selection constitutes a moral claim that these are [a representative sample of] the only questions that matter (and, by implication, that the associated ideological positions are the only positions that matter). This claim does not conflict with the claims of definitional significance, so Ward's (1982) criticism of it is not of its definitional validity but instead of its substantive validity - that is, an attack on the claim itself. Ward's argument thus passes out of the domain of methodology and into the domain of moral discourse.

Lest this be taken as a rejection of Ward's argument rather than a clarification of it, let me say that I agree with it.

11. Thus in Kohlberg's (1981) work, "universalizability" is such a criterion; it demands that moral principles be justifiable from the position of any moral agent. At issue is not any particular set of justice principles but rather the method of judging such principles.

12. Converse (1964:227-229). A similar procedure was applied to a parallel sample of 1958 Congressional candidates, using somewhat different question wordings. As mentioned earlier, the comparability of results between the two samples is attacked by Bishop, Tuchfarber, and Oldendick (1978). One might also attack the validity of the various statistical manipulations; Converse (1964:258n) himself suggests one possible objection. However, the point of the present paper is that the results from both samples are meaningless because of faulty conceptualization of belief structure.

13. Note that, because of the correlation measure used, Guttman-scale questions have correlations of 1 - as do linear-relationship questions, of course.

14. Actually, the tau-gamma correlation is not quite as easily interpretable as this, employing a notion of concordant and discordant rank-order pairs. Still, the Pearson product-moment correlation image serves well enough for the following points. Further, mean inter-item correlation measures are closely related to measures of single-factor structure; the discussion here is phrased in terms of the latter rather than the former.

15. Barton and Parsons (1977) also make this point.

16. Barton and Parsons (1977) also make this point.

17. The last of the examples above also shows that it might be more useful to look first at belief structures for their receptiveness to democracy, and only afterwards for their unidimensionality. Under the circumstances given, the hapless villagers will clearly never experience democratic government.

18. This aspect of Durkheim's position is discussed by Abrams (1983) in the context of the latter's broader analysis of social science's conflict over how to conceptualize the relationship between human action and institutional constraints. Kohlberg (1981:esp. pp. 106-114) presents a succinct argument against Durkheim's position. Note that my claim is not that group consensus is irrelevant to moral discourse but rather that it cannot in itself be the ultimate moral authority.

19. This critique is not as general as the previous ones, because it depends on a phenomenon peculiar to the Pearson product-moment correlation. Use of a different measure could in some circumstances obviate this critique. However, note that most of the constraint analyses, including those using factor-analytic/structural equation models, depend on product-moment correlations.

20. Currently, Piagetian psychology and schema theory appear to be the only approaches with an individual-level view of belief structure.




Page URL: http://www.d.umn.edu/~schilton/Articles/Def_vali.html
Page Author: Stephen Chilton
Link to Home Page: www.d.umn.edu/~schilton/index.html
Last Modified: November 27, 1999
Mail suggestions and comments, especially re. typos and other errors, to: schilton@d.umn.edu
Honor Roll of Proofreaders and Colleagues: www.d.umn.edu/~schilton/Honor.html
This way to the UMD homepage
This way to the Political Science Department home page

The University of Minnesota is an equal opportunity educator and employer.
Copyright © 1999 Regents of the University of Minnesota. All rights reserved.