Three Preliminary Issues
I. Causality versus Correlation
Is at the root of scientific method (Babbie says observation of change)
If you know what causes something - you can predict what will happen
To determine causality the phenomenon must be repeatable.
II. Proof versus Support
Scientists don't PROVE that one thing causes another
They DISPROVE that something else causes it
How many times do you have to do something for it to be proven?
Takes only one time to disprove something.
III. Theory versus Empirical Reality
Surveys test empirical reality
Experiments test theory.
The Experiment is the method that directly concerns itself with causality.
It has to do with THEORY - not empirical situations.
It is the only method where the researcher can unambiguously attribute a change in the dependent variable to the independent variable.
Experiments are good for hypothesis testing
Experiments are explanatory - not descriptive
A Major Advantage of Experiments is Internal Validity
You can draw cause and effect conclusions. It minimizes the effects of extraneous causes because it is a controlled environment
It lends itself to logical rigor
A Major Problem is external validity - generalizing the results
In the real world, causes are not isolated but interact.
Additive effect - The effect of two causes together may be much more than the effect of the two causes added together.
Artificiality
Some notable Sociology experiments
Asch - conformity - the length of lines
Haney, Banks and Zimbardo - prisoners and guards - mock prison setting
This was abandoned after 3 weeks (not 15 weeks)
Milgram - shock people - influence of authority figures
School - colors
Bank Wiring -
Important elements of experiment:
1) Randomized individuals, groups, organizations = unit of experiment
2) Random assignment
3) Pretest/Posttest -- Measure prior to treatment and after treatment.
Respondents may figure out the purpose of the study, chge ansrs.
4) Experimental and Control Group -- Hawthorne Effect - the group improves because of the attention it is given not the treatment
How Do you Select Subjects?
1) Randomization - use a random sample. But this requires at least 200 people (two groups of 100 each). This is too many for most experiments.
2) Quota matching - Recruit more subjects than you need, then group them according to specific characteristics. Put half in each group, so that you have two group nearly alike. This also takes a fairly large sample. The more characteristics that you use the bigger the sample will have to be.
3) Random assignment - randomly assign subjects to experimental or control group.
Reasons for Questioning External Validity (Smith)
A. Experimenter Interference - the experimenter may bias the
results
B. Participant may Interfere with the results
1. Selection - the motivation for volunteering (good grades) may
make them more enthusiastic.
2. Participants may try to "Psych" out the experimenter
C. Treatment problems
Is the experiment situation realistic or artificial?
D. Measurement Problems
2) There is a learning process between pretest and posttest
The solution is double blind (participant and measurer don't know which is which)
And triple blind (participant, measurer, and experimenter)
1. History - a major event occurs during the experiment may change responses.
2. Maturation - people naturally change over time
3. Testing - the fact of testing someone once changes the way they will respond 2nd time
4. Instrumentation - different tests are used pre and post test
5. Statistical Regression - Those with extreme scores are likely to change
i.e. short people have taller children/very tall people have shorter children
6. Selection bias - comparison of subjects is meaningless unless it is comparable (control and experiment groups)
7. Causal time order - ambiguity
8. Diffusion or imitation of treatment - control group is compromised by info
9. Compensation - experimenter feels sorry and gives the control group more care
10. Compensatory rivalry - the control group works harder because of no treatment.
11. Demoralization - the respondents may give up
A particular concern in experiments is leakage. Do people in the control group find out about the treatment from the others?
Solomon Four Group Design addresses this problem.
To use this design you need 2x as many subjects as otherwise. You end up with four groups: Experimental group 1
Control group 1
Experimental group 2
Control group 2
Both group 2s don't receive any pretest.
Experimental group 1 Pretest(1) treatment Posttest(2)
Control group 1 Pretest(3) Posttest(4)
Experimental group 2 treatment Posttest(5)
Control group 2 treatment Posttest(6)
The results you want are as follows:
2 > 1
3 = 4
2 > 4
5 > 6
Quasi Experiments
Sometimes a researcher may do an experiment and for one reason or
another the situation doesn't lend itself to the full scientific procedure.
The results are not fully reliable, but in some situations they may yield
some useful information.
1. One Shot Case Study - there is no posttest.
2. One group pre/post test there is no control group
3. Static group comparison - there is no pretest
4. Focus groups = this is mainly for marketing. It is not a strict
scientific experiment, but a group gathered together to check the response
to a certain product.
5. Natural experiments - the experimenter does not have control
over the conditions in a natural experiment. Nevertheless, some useful
information can come from them. For instance, the reaction of the people
in cities in wartime = those that were bombed versus those that were not
bombed.