ValidityDoes the survey measure what it is intended to measure?
In a general sense, validity is the degree to which a test measures what it is intended to measure. According to Messick (1989) it is “the degree to which evidence and theory support the interpretations of test scores entailed by the proposed uses of tests.” So validity is about the inferences we make from the evidence, and not a property of the instrument itself. Below are seven forms of validity and links to NSSE's corresponding studies.
Response process validity
Do respondents understand the questions to mean what we intend them to mean?
Response process validity is the extent to which the actions and thought processes of test takers or survey responders demonstrate that they understand the construct in the same way it is defined by the researchers. There is no statistical test for this type of validity, but rather it is observed through respondent observation, interviews, and feedback. NSSE’s A Guide to Contextualizing Your NSSE Data: Cognitive Interviews and Focus Groups is a step-by-step guide designed to help institutions develop this type of richer understanding of their NSSE data.
Do the survey questions cover all possible facets of the scale or construct?
Content validity is the extent to which a measure represents all facets of a given scale or construct. Like face validity, there is no statistical test, but rather we rely on experts to determine whether or not the instrument measures the construct well.
How well does this group of items measure the theoretical concept?Construct validity is the extent to which a measure correlates with the theorized construct that it purports to measure. The measure is intended to operationalize the concept by gathering observable details that reflect the underlying phenomenon.
Do the questions measure the construct in the same way that others have measured it?
Concurrent validity refers to the degree to which a construct correlates with other measures of the same construct that are measured at about the same time.
Does the scale correlate in predicted ways with outcome measures or other forms of engagement?Predictive validity is the extent to which a score on a scale or test predicts scores on some criterion measure in expected ways.
- Connecting the dots
- Retention and degree progress
- Single institution example study
Known groups validity
Do the results of various subgroups match those from other studies?Known groups validity is the extent to which a measurement is sensitive to differences and similarities in various groups (e.g., men and women, students in various programs of study, or students enrolled at different types of institutions) which are established in other studies.
Consequential validityAre the survey results interpreted and used in ways that improve undergraduate learning?
Consequential validity is established by evidence of the intended and potential consequences of the instrument, such as to improve the undergraduate experience, both inside and outside of the classroom