While each question format requires attention to specific factors, there are general rules which apply to nearly all questions. The following list is not exhaustive, and is intended to help create questions which will be interpreted to mean the same thing by different respondents. For more discussion of these issues, review the sources listed below.
Clarity of language
The language used in a questionnaire should not be haphazard, and in many cases should be partially dependent upon the characteristics of the target population. There are three language factors to consider which will improve the clarity of your questionnaire: the use of jargon, the use of confusing or ambiguous words, and the use of overly formal language.
- Jargon: The unintentional use of jargon, or more often abbreviations, is an easy error to both commit and correct. With regard to abbreviations, it is best to spell out the entire word and follow it with a parenthetical abbreviation prior to exclusively using the abbreviation. Jargon can be more difficult to discern, and is more dependent upon the target population. For example, while full time equivalence (FTE) is a generally understood concept in the industry of education, it is a less well known phrase to those outside the industry.
- Confusing and ambiguous words: As with jargon, it can be easy to overlook the use of confusing or ambiguous words. A commonplace example is the use of a defining term that seems clear and definitive on the surface, but that is in fact very vague. Examples include asking respondents to list their “income” or to indicate if they engage in an activity “regularly.” Each of these terms could be interpreted differently by respondents, which will ultimately call the results of your questionnaire into question.
- Formal language: As a rule, it is best to avoid using overly complex language. Nardi (2003, 71) gives an excellent example of this, with the sample question “Does the student newspaper obfuscate the truth?” While some populations will be able to accurately define the term “obfuscate,” others may not. If a population is unable to consistently provide an accurate definition for a term used in the question text, the results of the question are less useful.
Biased questions
Though often created unintentionally, biased questions are fairly commonplace, particularly when questionnaires focus on controversial topics such as matters of public policy. It is particularly important to be vigilant about these types of errors, as they can result in an adverse reaction in a respondent, causing their perception of an entire questionnaire to be altered.
- Leading questions: Leading questions are those that contain language intended to guide (i.e. bias) a respondent towards a particular response. Prompting respondents with phrases such as “Do you agree that….” are subtle ways of leading them towards agreement (or disagreement). A commonly used direct approach to leading a respondent involves the use of prestige or rank in the question text. Citing the opinion of medical doctors, the opinion of a president, or some other “expert,” and then asking respondents if they agree with the expert, will increase the likelihood that those inclined for or against your expert will provide a response based solely on that predisposition.
- Loaded and emotional questions: Whereas leading questions rely upon the stated or unstated opinions of others to bias the views of a respondent, loaded questions rely on “loaded,” non-neutral, emotionally laden language to influence the opinion of respondents. Neuman (2007, 172) gives an example from a theoretical questionnaire, asking respondents “Should the mayor fix the pot-holed and dangerous streets in our city?” By using the loaded term “dangerous,” the researcher has introduced bias into the question and increased the likelihood that respondents will agree that the roads require attention.
Double-barreled questions
Another common error in phrasing, double-barreled questions are those that ask two questions in the context of one. Ultimately, this undermines the results, as it is impossible to tell which portion of the question the respondent was referring to. Take the following question as an example: “Are you currently satisfied or dissatisfied with the diversity and quality of the food offered in the Caf over the past academic year.” This question simultaneously asks the student to rate the quality and diversity of the food offered. The researcher would be unable to determine if a student was satisfied with the quality, but dissatisfied with the diversity, or vice versa.
Always or never options
As a rule, neither of these terms should be used in questionnaires; rarely do people always or never do anything. Instead use phrases such as “almost always” or “almost never.”
Further Reading
Nardi, P.M. (2003). Doing survey research: A guide to quantitative methods. Boston, MA: Allyn and Bacon.
Neuman, W. L. (2007). Basics of social research: Qualitative and quantitative approaches (2nd ed.). Boston, MA: Allyn and Bacon.
Rea, L.M. & Parker, R.A. (2005). Designing and conducting survey research: A comprehensive guide (3rd ed.). San Francisco, CA: Jossy-Bass.
Suskie, L.A. (1996). Questionnaire survey research: What works (2nd ed.). Tallahassee, FL: Association for Institutional Research
Weathington, B.L., Cunningham, C.J.L., & Pittenger, D.J. (2010). Research Methods for the Behavioral and Social Sciences. Hoboken, NJ: John Wiley & Sons.