Question: Compare and contrast the benefits of three methods used in questionnaire development: cognitive testing, focus groups, and coding interviewer and respondent behavior during field pretests.

1) Compare and contrast the benefits of three methods used in questionnaire development: cognitive testing, focus groups, and coding interviewer and respondent behavior during field pretests. By “benefits” is meant the acquisition of information about weaknesses in the design of the questionnaire that can be repaired prior to the main survey data collection. Name at least three ways the methods differ in their benefits.

2) For each of the following scenarios, list one pretesting technique that would most directly address the problem at hand, and state why that technique would be useful.

a) You are beginning to draft questions for a new survey, and you need to know how your target population thinks and talks about the survey topic—what words they use, how they define those terms, and so on. What technique should you employ?

b) Your primary concern for a survey about to go into the field is that the interviewer-respondent interaction be as standardized as possible. You found during your own very informal testing (administering the questionnaire to a handful of coworkers) that the interaction was somewhat awkward—you were sometimes interrupted with an answer before you had read all of the response categories to the respondent, other times you were asked to repeat questions, and in some cases you were asked what certain words meant (although no standard definition was available). What would be the best way to address this concern in regard to standardization of interviewer-respondent interaction during a pretest?

c) You are very concerned about potential comprehension and recall problems for a questionnaire you have been asked to finalize for data collection. You will ultimately conduct a large-scale “dress rehearsal” field pretest several months from now, but something must be done before then to improve the questionnaire. Some questions have ambiguous wording, and could easily be interpreted in different ways by different people; other questions ask for information that seems rather difficult to recall for most people. In addition, you suspect that there may be different problems for various subgroups of the population (e.g., those with low levels of education, those from different ethnic groups, etc.). What should you do?