Documenting Pandemic-Era Conditions

Although the 2020-21 academic year is deserving of an asterisk for its unprecedented circumstances, it remains important to document the conditions for education and to invite student and faculty input on the quality of their experience. Institutions participating in NSSE and FSSE can learn a lot from their data. To support institution’s 2021 data use, we sponsored a webinar, “COVID Times and Student Engagement: Using and Interpreting NSSE 2021 Results” (available for viewing anytime) and reflect on some of the concerns raised by our campus contacts here.

The use of student engagement and other assessment results is critical as we work through pandemic-era conditions, and as we strive to simultaneously advance equity and ensure high-quality learning for all students.

Participating Institutions Find New Ways to Use NSSE Data

The variation among students during the pandemic prompted many participating institutions to examine their results based on student populations of interest. Washington State University mentioned that their first-generation and Pell-eligible students struggled more so they took a closer look at these students’ results. Other institutions noticed large shifts in engagement behaviors including more students indicating that they worked more hours for pay off campus and spent more hours caring for dependents than in the past. Variations among student populations are important to document and to identify where follow-up interventions might be needed with particular student populations.

Photo courtesy of Lewis & Clark College

Some participating institutions were understandably apprehensive about reviewing their 2021 results, worrying about anticipated drops in scores or how results would be interpreted (or misinterpreted). Attaching a “collected during COVID-19” note to the datafiles and summary reports is warranted, but should not flag the data as unusable. That said, concern that unprecedented circumstances make results aberrant from past years, or that comparisons among institutions with different course modalities or in states with varied pandemic protocols render peer comparisons useless, are valid. Yet, institutions like Lewis and Clark College commented that these data are necessary to the routine process of iterative assessment to make determinations of quality, effectiveness, and mission fulfillment required of accreditation.

Uncertainty about the influence of pandemic-induced shifts in instructional modality and in particular, if institutions in comparison groups were similar in terms of course modality, is worthy of consideration. Washington State University noticed that among their 21 peer comparison institutions only two were like them, fully remote during Spring 2021. They wondered if their standard peer comparison groups were appropriate given these new ranges of instructional modality. Some institutions requested more customized analyses, forming more refined groups based on knowledge of similar course modality, while others chose to focus on multi-year trends, noting where results were stable or saw a downturn, and then asking if pandemic circumstances explained results that dipped.

Some institutions mentioned their focus on NSSE’s four new effective teaching items, added as an expanded measure of teaching effectiveness during the pandemic to be more responsive to online learning and reflective of innovative pedagogical practice. Institutions can compare their results to the aggregate findings in this Annual Results story, Students: Positive Perceptions of Faculty Teaching. One salient data point this year is that 56% of students indicated their faculty taught in ways that students preferred to learn. This result might prompt reflection among faculty on the flexibility experienced by students, and the value of carrying forward the instructional practices associated with this perception.

Indiana University Purdue University Indianapolis (IUPUI) administered the Coping with COVID Topical Module and found these results illuminated a better understanding of students’ mental health. These results helped IUPUI move the needle on some internal mental health initiatives and to reinforce the positive things that faculty can do to help students. In addition, in the year prior, IUPUI convened a task force to examine university policies with a diversity mindset. Results from the Inclusiveness & Engagement with Cultural Diversity Topical Module, and in particular the ability to break out individual items by race, ethnicity, sexual orientation and gender identity, contributed to IUPUI’s diversity study. Finally, IUPUI looked to their NSSE perceived gains results to examine concerns about perceptions of learning loss during the pandemic. They mapped these items to their institutional learning outcomes, and noticed a decline in some areas that were also identified as dropping in secondary education and higher education during the mostly remote experience. The perceived loss in learning is troubling and NSSE results allowed them to try and understand this phenomenon a little better.

Recommendations for Contextualizing Results

Overall, despite the varied and unusual circumstances during which NSSE 2021 was administered, these data are valuable to institutional assessment efforts. Here are some recommendations for contextualizing results:

  • Review NSSE 2021 results with an eye toward insights about how the institution successfully stretched, where things fell short, and for clues to the dimensions of undergraduate education that deserve attention moving forward;
  • Read students’ open-ended comments for unique perspectives on the pandemic, what worked, and who deserves appreciation;
  • Examine results by student populations, such as first-generation students, students working more than 20 hours per week off campus, or others who may have had different pandemic-era learning experiences;
  • Place notes and caveats on NSSE 2021 data and reports, but know that these data are useful for routine assessment and accreditation;
  • Focus on NSSE’s four new effective teaching items, and explore these results with faculty to prompt reflection on instructional flexibility and the potential for carrying forward the teaching practices associated with these perceptions;
  • Value ways the results might be relevant to current initiatives on campus, such as diversity and equity efforts, attention to mental health, and perceptions of learning loss during the pandemic.

The use of student engagement and other assessment results is critical as we work through pandemic-era conditions, and as we strive to simultaneously advance equity and ensure high-quality learning for all students. Improvement efforts at colleges and universities are more likely to succeed when they emerge from a shared understanding of the evidence and priorities for action. We look forward to working with NSSE participating institutions to advance this imperative.

Evidence-Based Improvement in Higher Education

Center for Postsecondary Research
Indiana University School of Education
201 N. Rose Avenue
Bloomington, IN 47405-1006
Phone: 812.856.5824