Sociology 2155: Outline 13

I. Experiment as a research method. Examples: Test taking and coffee drinking (text, pp. 111-112), Aronson/Mills study of effects of an initiation. Background to each: practical concerns and student habits vs. testing a theory (cognitive dissonance)

A. Characteristics of true experiments

1. Two or more comparison groups: a treatment group and a control group or sometimes, two or more different treatment groups. (e.g. "notorious" Zimbardo prison experiment--the prisoners and the guards)

2. The experimenter introduces variation in the independent variable before assessment of change in the dependent variable

3. Random assignment to the treatment and comparison groups. This is what allows you to equalize the effects of outside (including unknown) factors. From the point of view of establishing causality, this is the single most important advantage of experiment as a research method because it eliminates the possibility that the causal connection(s) we find is(are) really spurious.

Example: what would have happened to the Zimbardo prison experiment if research subjects had been able to choose whether they wanted to be prisoners or guard?

4. Experimental control: each run of the experiment follows the same carefully monitored script. e.g., the Milgram obedience to authority experiement... the lab assistant running the experiment had just 3 or 4 scripted phrases he could use if the research subject didn't want to go on with administering the shocks

5. Pretesting and post-testing... in C/S's hypothetical coffee drinking experiment, each group writes an essay before and after the coffee intervention is introduced.

B. Two varieties

1. Laboratory: far more common, especially in psychology, but also used in sociology

2. Field experiments... e.g., the effects of bumper stickers on the behavior of car drivers. Most evaluation research, which we'll cover next week, is a variation of field experiments.

C. Quasi-experimental design.

1. Nonequivalent control group designs (aggregate matching). Subjects are not randomly assigned to treatment and comparison group. Instead there's an effort to match groups on characteristics that the researcher thinks may be relevant. For example, what if we wanted to study the effects large-scale civic engagement programs on colleges?

2. Before and after designs. We have a pretest and a post-test but no comparison groups. Same example as above.

Notice that as I begin to talk about quasi-experimental design, I begin to move from pure experiments into evaluation research. Sociologists/criminologists don't do a lot of lab experimentation, but evaluation research is an area of growing importance. e.g., Jeff Maahs evaluation of the St. Louis Country Drug Court. We'll spend much of next week on evaluation research.

D. Threats to the internal validity of quasi-experiments

1. Selection bias, resulting in nonequivalent comparison groups

2. Differential attrition (dropping out or even dying) resulting in nonequivalent comparison groups

3. Endogenous change, particularly a problem with before-and- after designs not involving comparison groups

a. the effects of pretesting itself (notice, this is no problem for true experiments, since all the comparison groups take the pretest, if there is one)

b. maturation

c. regression toward the mean

4. Historical events, again a problem for before-and-after designs

5. Treatment misidentification... placebo effect (experimental group in an anti-racism experiment believe their racial attitudes should "improve") or Hawthorne effect (attention itself changes behavior or attitudes, quite apart from the content of the intervention)

E. Generalizability (external validity)... very rare for experiments to include a statistically random sample of the general population... in fact, college students are the most frequent experimental subjects... can address this issue by intentionally recruiting diverse populations to an experiment (Milgram obedience to authority experiment: men and women, on campus at Yale, in an industrial area of New Haven)... remember the Sherman and Berk research on the effects of arrest on battering

F. Ethics of Experiments

1. Relative balance of societal and/or disciplinary benefit vs. possible harm to individuals

2. Honesty about the purposes of the experiment? Be as honest as you can, but generally accepted that deception is sometimes necessary. See statement the American Sociological Association at the top of p. 130.

3. Debriefing and follow-up. Find out whether your subjects had concerns or suspicions and help them to understand the purpose and utility of your research.

Aronson: Treat your research subjects as colleagues in the scientific enterprise... Solicit and value their input.

G. Strengths and weaknesses of experiment

1. Weakness: T o what extent will your results generalize to other populations (sometimes called external validity). Particularly for laboratory experiments, this is the biggest question... Philip Zimbardo's simulated prison at Stanford University certainly produced some interesting dynamics, but how similar is it to a real prison environment? Likewise for the Milgram experiments in which subjects were induced to give shocks, many of them all the way up to XXX: Extreme Danger. How similar was this to the situation of a soldier being ordered to shoot civilians, as in the My Lai massacre during the Vietnam War?

2. Strength: Very good at establishing causal relations between variables. In fact, probably the only definitive way to do this, since for every other research method there are at least some lingering doubts about spuriousness. Remember the criteria: association, time order, nonspuriousness--How does an experiment address each of these? "the gold standard for testing causal hypotheses," as C/S put it

H. Compare and contrast survey research and experimental research in terms of internal and external validity... "the great strength of...."

I. Data analysis. Experiments, like surveys, are a form of quantitative research. Can be analyzed using cross tabulations, measures of association (which you will learn in 3155), and tests of significance (in particular, chi square, which you will learn in 3155).

Example: Back to Judson and Mills experiment with initiation (hypothetical data). Let's look at some hypothetical data from that experiment. N=63

 

  No initiation Mild initiation Severe initiation Totals
Rated group discussion dull 80% 60% 40% 60%
Rated mildly interesting 20% 30% 40% 30%
Rated very interesting 0% 10% 20% 10%
  100% 100% 100% 100%

 

II. Video: " Constructing Social Reality"

III. Exam Two