The concept of validity is concerned with the extent to which your questionnaire measures what it purports to measure, and is often rephrased as “truthfulness,” or “accuracy.” The concept is analogous to using the wrong instrument to measure a concept, such as using a ruler instead of a scale to measure weight.
Determining the accuracy of a question involves examining both the validity of the question phrasing (the degree to which your question truly and accurately reflects the intended focus) and the validity of the responses the question collects (the degree to which the question accurately captures the true thoughts of the respondent). While perfect question validity is impossible to achieve, there are a number of steps that can be taken to assess and improve the validity of a question.
Face validity: Collecting actionable information often involves asking questions that are commonplace, such as those querying the respondent about their age, gender, or marital status. In such instances, one means of lending validity to a question is to rely on the collective judgment of other researchers. If the consensus in the field is that a specific phrasing or indicator is achieving the desired results, a question can be said to have face validity.
Content validity: Related to face validity, content validity also relies upon the consensus of others in the field. It differs from face validity in that content validity relies upon an exhaustive investigation of a concept in order to ensure validity. Nardi (2003, 50) uses the example of “the content of a driving test.” Determining the preparedness of a driver is dependent on the whole of a drivers test, rather than on any one or two individual indicators. In this way, the driving test is only accurate (or valid) when viewed in its entirety.
Criterion validity: Criterion validity relies upon the ability to compare the performance of a new indicator to an existing or widely accepted indicator. Whereas face validity encouraged the adoption of existing indicators, criterion validity uses existing indicators to determine the validity of a newly developed indicator. Criterion validity can be broken down into two subtypes: concurrent and predictive validity.
- Concurrent validity: If the widely accepted indicator is currently (concurrently) available, and the results of the new indicator can be compared against the existing indicator, then concurrent validity can be established. By assuming that both indicators measure the same phenomenon, concurrent validity allows us a means by which to determine whether or not our new indicator measures what we believe it measures.
- Predictive validity: If an indicator can be shown to reliability predict a future outcome, it can be said to have predictive validity. This type of validity is restricted to situations where the indicator and the outcome are distinct from each other, while still measuring the same concept. As an example, Neuman (2007, 119) uses the Scholastic Achievement Test (SAT), which purports to measure “the ability of a student to perform in college.” For the SAT to be said to have a high degree of predictive validity, those who score well on the test must also perform well in college. Should the relationship between these two be inconsistent, the SAT would be said to have low predictive validity.
Construct validity: Similar in some ways to content validity, construct validity relies on the idea that many concepts have numerous ways in which they could be measured. A researcher can choose to utilize several of these indicators, and then combine them into a construct (or index) after the questionnaire is administered. As with content validity, construct validity encourages the use of multiple indicators to increase the accuracy with which a concept is measured.
Other Online Resources
Test Validity and Reliability (AllPsych Online)
Reliability and Validity (University of South Florida – Florida Center for Instructional Technology)
Exploring Reliability in Academic Assessment (University of Northern Iowa College of Humanities and Fine Arts)
Further Reading
Nardi, P.M. (2003). Doing survey research: A guide to quantitative methods. Boston, MA: Allyn and Bacon.
Neuman, W. L. (2007). Basics of social research: Qualitative and quantitative approaches (2nd ed.). Boston, MA: Allyn and Bacon.
Suskie, L.A. (1996). Questionnaire survey research: What works (2nd ed.). Tallahassee, FL: Association for Institutional Research