Constructing effective close-ended multiple choice questions requires the researcher to make a series of intentional choices. Topics reviewed here include taking time to ensure response options are mutually exclusive, that the options presented are exhaustive, and the the response interval sizes are consistent.
Mutually exclusive categories: In situations where respondents are asked to choose one option from a list of responses, they must be presented with choices that are truly and completely distinct from one another. If an answer does not have only one possible response category, the rule of mutual exclusivity has been broken, and the validity of the data is undermined. Take for example this hypothetical scale on age: a) 15-20, b) 20-25, c) 25-30, d) 30-35, e) 35-40. If a respondent is 30 years old, in which category do they belong? By featuring age 30 in more than one option, the question is no longer mutually exclusive, as the respondent could select either option “c” or “d” and still be correctly relaying their information. Instead, this scale provides a single option for each age: a) 15-20, b) 21-25, c) 26-30, d) 31-35, e) 36-40.
Exhaustive lists: When presenting a list of responses from which to choose, it is important that the list be as exhaustive as possible. When sufficient attention is not paid to this factor, researchers run the risk of collecting inaccurate or misleading data; respondents might either skip the question entirely or simply select an option because it is the least incorrect of those listed. In some cases, producing an exhaustive list will be easy, such as asking alumni to select the year in which they graduated. Oftentimes, however, ensuring a list is exhaustive requires the review of relevant literature in the field. In some cases, a lack of knowledge about the subject might render an open-ended question a more suitable choice for the researcher.
Number of response options: When creating multiple choice questions, it is important to keep in mind the amount of detailed information the respondent is able to process at once. This becomes particularly important when the response options are asking the respondents to compare options against each other, or to choose an option that is “most” or “least” applicable; these tasks can become overly taxing if the response options are too numerous. The appropriate number is dependent upon the subject and the population being sampled. One possible solution is the creation of several questions to lighten the load of any one question.
Interval size of response options: Whenever the response options featured are numerical intervals (such as age or income), it is important to pay close attention to the size of those intervals. Where possible, the intervals should be as consistent as possible, as is the case with the 4 year age intervals used in Table 2. However, the researcher should always be sensitive to the population being surveyed, and try to ensure that the intervals created will fit the population and not return a skewed result. For example, the age questions above are ill-suited for use on a population such as the St. Olaf College student population, where nearly all the respondents would select the first two response options.
Further Reading
Nardi, P.M. (2003). Doing survey research: A guide to quantitative methods. Boston, MA: Allyn and Bacon.
Neuman, W. L. (2007). Basics of social research: Qualitative and quantitative approaches (2nd ed.). Boston, MA: Allyn and Bacon.
Rea, L.M. & Parker, R.A. (2005). Designing and conducting survey research: A comprehensive guide (3rd ed.). San Francisco, CA: Jossy-Bass.
Suskie, L.A. (1996). Questionnaire survey research: What works (2nd ed.). Tallahassee, FL: Association for Institutional Research
Weathington, B.L., Cunningham, C.J.L., & Pittenger, D.J. (2010). Research Methods for the Behavioral and Social Sciences. Hoboken, NJ: John Wiley & Sons.