The COVID-19 pandemic disrupted education around the world in 2020, and the impact continues to this day. The 2020 NSSE administration was underway at more than 600 colleges and universities as they abruptly transitioned to online delivery. Institutional users reached out to express understandable concerns about showing sensitivity to students, how the change in operations would affect survey responses and how to interpret data from students who responded before and after the disruption.
With the survey still in the field, NSSE’s research staff promptly set to analyzing data collected through May 1 to examine these pressing concerns, and the results were reassuring. Key findings included the following:
- About nine out of ten respondents had completed the survey by March 18, before most institutional disruptions.
- We found no evidence of a decline in overall response rates. In fact, average institutional response rates increased slightly between 2019 and 2020. Even comparing same-institution response rates between 2020 and a previous administration, we found a modest uptick.
- Because most NSSE questions ask students to generalize over the full school year, or in some cases over their entire experience, we found no evidence of a systematic change in post-disruption responses.
- We observed an understandable exception to this pattern regarding the number of spring term courses taken online: Starting around mid-March, the proportion of current courses taken online increased notably—evidence that students were answering the survey with care.
- We also compared NSSE Engagement Indicator (EI) scores for pre- and post-disruption respondents, benchmarked against the pattern for prior years. Results generally indicated minimal differences. Interestingly, we found a slight post-disruption increase in certain EI scores such as Quality of Interactions and Supportive Environment.
These findings notwithstanding, we asked 2020 participating institutions to designate a disruption date and offered to exclude post-disruption respondents from institutional reports and comparison groups on request. Only a handful of institutions elected to exclude respondents.
(For more information, see the May 18, 2020 webinar recording available on our Webinars page.)