Response Rate Factors

Obtaining an adequate response rate requires being attentive to more than just the percent of the sample who have completed the survey. Factors such as the motivation of the sample population, method of administration utilized, and representativeness of the population are important to take into consideration.

Completion rate (quantity): This ratio of the number of completed questionnaires to the number of distributed questionnaires is what most people call to mind when considering response rates. Despite this similarity, it is important to note that a completion rate is different than a response rate, though they are often used interchangeably. A completion rate is concerned with the number who complete a given percentage of a questionnaire – usually around seventy to eighty percent; response rates, alternatively, count everyone who responded, regardless of whether they completed the questionnaire. As it is generally true that the representativeness of the data increases as the number of respondents completing the questionnaire increases, it is certainly important to take this ratio into consideration. In general, completion rates should ideally be greater than sixty or seventy percent, with fifty percent being the lowest acceptable value.

Representativeness of sample population (quality): Unless a researcher is distributing a questionnaire to an absolutely homogenous sample population, there is reason to look beyond the simple completion rate to determine if the response rate is sufficient. Consider the data in the examples below. While the questionnaire from example one gathered more responses than the questionnaire in example two, the results collected from example one are unlikely to be as representative of the population as are the results from example two; the disparity between the demographics of the population and the respondent groups is greater in example one. In this case the quality of the information gathered in example two will likely be of more use to the researcher, despite the fact that the quantity of the information (completion rate) is lower.

Nonresponse bias:  Rather than focusing upon the identity of those who have completed the survey, nonresponse bias considers the identity of those who have chosen to not respond. This can become a particularly salient issue when the questionnaire broaches sensitive or potentially embarrassing topics. If a specific segment of a sample chooses not to answer questions, it introduces a bias into the results that are collected. The significance of that bias increases when (a) the nonresponse is widespread among a segment of the sample and (b) the responses of the segment are theoretically likely to have a large impact on the results. In order to gauge the extent of potential nonresponse bias, a researcher must have a good working knowledge of the characteristics of the sample population.

Identity of sample population:  The extent to which a sample population is motivated to complete a questionnaire can have an important impact on the proportion that follow-through and complete the questionnaire. As a result, consulting any existing literature that addresses the issue of response rates for specific populations is advisable. For example, while institutional-related surveys of college students can often achieve response rates at-or-above seventy percent, institutional-related surveys of alumni often struggle to exceed a response rate of twenty percent.

Method of administration: Just as certain populations are more likely to respond, certain administration techniques are likely to generate higher response rates. Generally speaking, those that require face-to-face contact with respondents generate the highest response rates. Web-based questionnaires, as well as those administered via telephone and on paper (though not by mail), can produce high response rates, though usually not as high as those administered face-to-face. Questionnaires administered by mail often generate low response rates.

Further Reading

Suskie, L.A. (1996). Questionnaire survey research: What works (2nd ed.). Tallahassee, FL: Association for Institutional Research