75-90% of all social science research is done with surveys.

1980 there were 28 million telephone survey interviews conducted.

There are more than 70 million homes surveyed each year..

Use of surveys is the most used and most abused data collection method.

History

Surveys were first used as far back as ancient Sparta in the 6^th century BC.

In the 19^th -20^th century, nations used surveys to record citizen reaction to political and social issues

From 1916-1932 the Literary Digest accurately predicted the presidential elections

They bombed in 1936 and 1948 - the surveys were not done scientifically.

During the Depression the government used probability samples to study unemployment and attitude toward government programs.

After WWII former government officials now in academia began using surveys to study attitudes and trends.

Today even the Census Bureau asks subjective questions.

Some survey organizations: National Opinion Research Center (NORC)

Survey Research Center - U of Mich

Roper Gallup

Not all surveys are legitimate

Some collect personal data to sell

Others want to sell you a product

Politicians may slant questions to get support for their programs.

A. Types of Surveys
1. Interview - Information is collected through conversation.

a. Schedule of questions is used by the interviewer. The same words, the same order, same tone of voice is used for all interviews. No stimuli is given to the respondent.

b. Non directive interview is different. The interviewer is passive and only directs the topics. The respondent has freedom of expression.
> A good interview sheet will include pronunciation.

>Always state the condition before the alternative ("Aside from…., what is the situation…?")

>If something is being quantified, specify the units - miles, minutes, hours, etc.

>Interviewer needs to be trained

>Do NOT be confrontational. If things start to heat up, leave the topic and come back to it later.

>Develop a pace that is not too fast or too slow.

>Avoid phrases that bias the answers "I suppose that you feel…"

>Never accept proxy answers

>Use card list for multiple choice answers - It's hard for respondent to remember all the choices and thus be able to weigh them equally.

>Make sure that "Don't know" is an option

>Vary the order of responses so that the expected answer is not always first or last.

In Depth Interviews

Average cost is $300. Per interview.

The interviewer shouldn't feel uneasy about collecting info or the respondent won't answer honestly.

Interview should be balanced between fact and feeling

Equal parts facts, opinion, feeling

Don't agree, or disagree, or give advice.

People will avoid showing emotion in our culture today.

Signs - Objectify the issue

Deny feelings

Use "we" or "you"

Cry or act embarrassed

Leave saying "excuse me"

Telephone Interviews

They cut costs considerably

People are less suspicious = so there is increase response rate

It is easy to work with a hired interview staff - to change directions

Multilingual flexibility

BUT: Only 90% of adults can be reached by phone - in some places 82%, in Ozarks 30% of the households have phones

Unlisted numbers are a problem - in Chicago 50% of the phones are unlisted.

Some houses have 2 or more phones

Interviews are often broken off in the middle.

CATI = Computer assisted Telephone Interview

B. Questionnaire - self-administered survey. It needs to have clear written directions.

a. Language

Wording is the main problem in surveys. 58-74% of the problems are improper wording or faulty interpretation.

>Use simple sentence structure - 20 words max per sentence

>Language should be geared to the respondents' level. If you are surveying college educated persons, then the language level is higher. Words should be keyed to the lowest level of respondent

FOG index = (M+S) x 0.4 (in a 100 word sample)

M= mean number of words per sentence

S = number of words with more than 2 syllables (no -es or -ed endings)

The resulting number indicates the grade level of the language.

Gallup, Harris, NORC aim at about a 12^th grade level

>Avoid: Leading questions

Implied alternatives

Double barreled questions = two questions in one

Loaded words and public figures that will bias the answers.

>Synonymous Meanings - Make sure that what you mean is what the respondents mean.

Head of household for whites means male bread earner. For Puerto Ricans it means landlord.

"Would you favor or oppose a law that requires a person to obtain a police permit before he or she can buy a gun?" the majority are in favor.

Change it to "before he or she can fire a gun?" the majority are opposed.

Split Ballot Technique - one half of the respondents are asked the question one way, the other half are asked the other way to check for bias. This requires twice as many respondents.

b. Conceptual Level

1. Memory intensive questions are difficult and hence, answers are unreliable.

("How many times in the past 5 years did you go to the dentist?") It is better to make a shorter time period.

2. What is the level of Abstraction? Is it abstract or concrete? Abstract answers and questions are more difficult.
3. Is the question hypothetical or actual? Hypothetical are more difficult.

 
c. Frame of Reference - "Did you work for pay or profit last week?" Housewives under report their work. Their frame of reference is professional work, and their work is not professional, so they don't include it. "Seen any good movies lately?" Reference is popular definition of good.

>Recognize what you are assuming in the questions you ask

>Know all the issues

>Questions should be meaningful to respondents - they should know all the words.

Socialized medicine = 50% didn't know what it was but answered anyway.

>Know what population you want to survey

>Know how precise you want the answers to be.

1. Open ended

Data is hard to systematize

Answers may be ambiguous

It makes the respondent think harder

It is easier to design

It avoids restricting the thought on the subject

You have to set the frame or you will get too many answers. Indicate how many thoughts on a subject you want.
 

2. Closed ended

Harder to design

Respondent can answer without thinking

It is less ambiguous

You can suggest both sides of the issue

The data can be more easily analyzed

The data is consistent
 

3. 2-Way Questions

Don’t use "or not"

State the pros and cons so you don't bias the answers

Becareful not to over simplify the situation

Have a don't know (no opinion) category

If there is a middle ground, add it

Make sure the choices are mutually exclusive

Sometimes add a "sometimes" category

Avoid polarized alternatives
 

4. Multiple Choice

1. Multiple indicators increases reliability. So have more than one question for each issue you want to measure

1. Open ended question to measure the awareness of the subject
2. Open ended question to measure a general attitude
3. Closed ended question to measure the specific attitude
4. Open ended question to measure the reason for the attitude
5. Closed ended question to measure the intensity

0. Order of questions. Put the easy to answer questions - and interesting attitude questions that they will WANT to answer first.

Put riskier questions and controversial questions in the middle

Put easy questions last - to release tension.

1. Use transitions - don't jump from subject to subject.

"Now that we have covered …., another kind of question is…"

2. Truth issues - How do you know if they are telling the truth?

Try not to offend the respondent

Try not to ask leading questions, or in any way bias their answers

Try to provide less embarrassing ways to answer sensitive questions

Randomized Response Technique (RRT)

For instance: Have you had sex with a prostitute?

If no, and if you get tails when flipping a coin, answer no.

If yes, and you get heads or tails, if no and you get heads, answer yes.

The proportion of the sample that didn't have sex with a prostitute is the number of No answers times 2. Some researchers think that this leads to greater truth telling. It needs a larger sample. Still some may answer no to avoid suspicion.

3. Operationalize the variable

Think carefully about what questions will stand for the concept that you want to measure

4. Know the extent of the negative or positive views. Perhaps you need more categories
5. Get more detail rather than less. You can always leave out detail, but it is hard to get more once the survey is done.
6. Always include age, gender, race, education
7. Will the data be Nominal, Ordinal, Interval or Ratio?

1. Surveys should look attractive. If the survey is sloppy (spelling errors, poor grammar, complex sentences) the respondent is less likely to treat it seriously.
2. Use attractive and easy to read fonts, be neat, clean paper
3. It should be spread out and uncluttered. Don't abbreviate or squeeze
4. Use boxes if you can - otherwise parentheses. Alternative is to circle the number on a list of numbered items.
5. Contingency questions should be indented = even put in a box. Number them differently so it is clear that they are not the same items.
6. Matrix answers (the same set of answers will do for a series of questions) such as Strongly agree, agree, undecided, disagree, strongly disagree. This can lead to a response set - a pattern of agreeing or disagreeing.
7. Sometimes the order is important = particularly for the less educated
8. Instructions should be clear and simple
9. Do a pretest of at least 10 people to test for mistakes.

Include a self-addressed, stamped envelope

Include payment - even $1. Increases the response rate

Include a good letter of explanation, so respondent wants to comply

Make the survey take only about 15 minutes to complete

Do a follow up mailing

A 50% response rate is ok. 70% is excellent

All surveys should be anonymous.

Verify the answers

Review the codes

Check a sample selection for "unavailables" and others who did not answer - are they a high percentage?

Check the non-response rate

Be aware of changes in staff - check for systematic changes in data

Bias = people may answer to please the interviewer, or to look respectable

Memory decay = answers are unreliable

Respondents may be negative, feel the subject matter is not important, or have no data

There could be a demographic bias