NSSENSSENSSE
Revised: July 1, 2020
The Webinar, "2020 Follow-Up: Using Results Collected During The COVID-19 Pandemic," recorded on June 2, 2020 is available for viewing. NSSE project staff facilitated this webinar to provide guidance about NSSE 2020, including information about our preliminary analyses to explore the effect of the disruption, customization options, and to begin thinking about approaches to using results.
Analyses of 2020 data collected through April 30 were conducted in response to concerns about the impact of the COVID-19 pandemic that disrupted operations at nearly all participating institutions. Bottom line for institutions completing the Report Form and considering whether to exclude post-COVID respondents from NSSE reports: NSSE advises against excluding post-COVID respondents from reports.
For information, visit the following:
COVID VARIABLE: NSSE will include a "covid" variable in your 2020 data file that identifies students who completed the survey after the disruptions might plausibly have affected survey responses. In May 2020, institutions were asked to indicate this date on NSSE’s online Report Form at the same time we collected information about comparison group customization and other reporting essentials.
OPTION TO EXCLUDE POST-COVID RESPONDENTS: In addition, institutions had the option to exclude post-disruption respondents from standard reports. We also collected this choice in the Report Form used to customize comparison groups in May.
DO YOUR OWN ANALYSIS: With your data in hand, use the covid variable to compare results that arrived before and after the disruption, and compare those findings to NSSE’s analysis. Assessment activities like NSSE and FSSE may matter even more in a year of turbulence, and we are hopeful that the data collected will help you examine the impact of the disruption on students' experiences and inform future planning.
Survey operations continued, and the NSSE survey remained open until May 22. As of March 18, 598 (96%) of 620 institutions had already launched and 573 (92%) had sent all but one or two of their scheduled reminder messages. Because many students completed the survey prior to disruptions of normal operations, these responses should be unaffected.
Curious about your NSSE results in the time of COVID-19? We have a blog post for you. Take a look at a new post by @Jillian_Kinzie on what you can learn from your Spring 2020 results. #NSSE2020
The Webinar, "NSSE & FSSE 2020: Guidance on COVID-19 Disruptions," recorded on March 26, 2020 is available for viewing. NSSE and FSSE project staff facilitated this webinar to provide guidance about NSSE and FSSE 2020 and exchange ideas among users.
NSSE staff are now turning attention to preparing reports for participating institutions. Data files will be posted to NSSE’s Institution Interface in mid-July and Institutional Reports will be delivered in early August.
Participating institutions are encouraged to consider approaches to explore their 2020 results to examine what the disruption revealed about their campus or in students’ learning experiences and how this information could be used to inform educational design moving forward. A blog post on NSSE Sightings offers some ways to think about using results.
Because Indiana University and the Center for Postsecondary Research have also experienced coronavirus-related disruptions, NSSE staff are working remotely. The best means of contact is by email. While we are making every effort to respond promptly, please understand that it may take somewhat longer than usual.
Please email staff directly or email NSSE@indiana.edu, which is monitored daily.
For survey administration questions please email your Project Services team.
View the Webinar recording from Thursday, March 26 "NSSE & FSSE 2020: Guidance on COVID-19 Disruptions".
View the Webinar: "NSSE & Coronavirus 2020: Preliminary Analysis Results and Recommendations" recorded on May 18, 2020. Click here for the full recording, or here for a brief summary of the analysis.
View the Webinar, "2020 Follow-Up: Using Results Collected During The COVID-19 Pandemic," recorded on Tuesday, June 2, 2020
The NSSE survey closed on May 22, a week later than usual.
Despite FSSE’s launch in late March, more than 90 institutions went ahead with their administration to collect information from their faculty. Many institutions included additional messaging for their faculty to encourage participation and set expectations for responses.
The best way to know what faculty are thinking during your FSSE administration is to give them some guidance. We strongly encouraged all institutions to be in contact with their faculty before and during their FSSE administration to provide guidance as they will likely have similar questions about how to respond. Institutions may want their faculty to think about their "typical" teaching situation, their teaching behaviors from a course in the fall semester, or about their new online approach, and we feel it would be best for institutions to relay such instructions to their faculty.
Many institutions have moved to online-only teaching situations, and FSSE questions are written with online learning in mind. Faculty can still validly respond to these questions, but it’s possible your results may change. In a time of crisis, it may be more important than ever to have information on how your students are learning both for future improvements in online learning or crisis management but also for understanding the context of student engagement during this outbreak.
Although FSSE staff are working remotely at this time, they are still available to help! Contact us at fsse@indiana.edu with any questions or to schedule a phone call.
The closing date of FSSE was extended to allow faculty to be able to respond through May 14, 2020.
Many campuses are facing Fall 2020 enrollment uncertainties. Because BCSSE’s web-version pricing is based on entering first-year enrollment, we will delay invoicing until after July 15, 2020 to allow for adjustments to enrollment estimates for web administration schools. This delay allows schools more time to update fall enrollment estimates that may place them in a different pricing tier. See BCSSE pricing for more information.
The BCSSE registration form prefills enrollment information from the most recent IPEDS data. However, schools that register prior to July 15 may request an adjustment to their fall enrollment estimates to be reviewed and applied to invoicing after July 15. Schools that register after July 15 are asked to provide fall enrollment adjustments within 48 hours of registration. Please send adjusted enrollment estimates to bcsse@indiana.edu, or contact us at the same email with questions regarding this policy.
BCSSE 2020 registration is open and will stay open through the summer. There is no registration close date. Institutions can register for BCSSE up to two weeks before survey administration. Keep in mind that the survey is intended to be administered prior to the start of fall term, or within two weeks after the start of fall courses. Visit Administering BCSSE to learn more.
Please email bcsse@indiana.edu if you need additional information or have any questions.
BCSSE added questions to the web version of the survey about the impact of the pandemic on the high school experiences of first-year students who graduated in 2020. There is no additional fee for these additional COVID questions. Responses to these items will be merged with core BCSSE data for participating institutions. Visit the BCSSE Survey Instruments page to learn more about the questionnaire.
Colleges and universities not participating in BCSSE but interested in asking questions of their entering students are welcome to use the "Coronavirus Disruption and 2020 High School Graduates" item set free of charge. As a courtesy and when possible, we request that institutions share with us how they used their data from these items and aggregate results. For more information, visit the BCSSE website or email bcsse@indiana.edu.
Analysis of preliminary 2020 data collected as of May 1 were conducted in response to concerns about the impact of the COVID-19 pandemic that disrupted operations at nearly all participating institutions. While the COVID-19 pandemic disrupted operations at nearly every institution, as of March 18, 96% of NSSE 2020 institutions had already launched the survey and 92% had sent all but one or two of their scheduled reminder messages. Analyses to explore the effect of the disruption on results, show generally trivial effects on response rates and responses to engagement items.
We addressed three broad questions: (a) Did the pandemic change response rates or the types of students who responded to the survey, (b) did the trend of responses change after disruptions to campus operations, and (c) do Engagement Indicators differ pre- and post-disruption?
Effects on Response Rates
Given the small number of institutions that administered NSSE after late March, analysis of the effect on response rates is inconclusive. If there was an effect, COVID-19 most likely increased response rates. A difference-in-difference causal analysis—leveraging data from matched institutions that participated in recent years—showed no significant differences due to COVID for any of the five recruitment messages. In addition, we were pleased to see that overall response rates for the 2020 administration were 2 to 3 percentage points higher on average over past years!
Descriptive Response Patterns by Date
In much of the NSSE, students are asked to respond about their experiences during the course of the entire academic year. So, to understand how responses may have been influenced by disruptions related to the pandemic, we examined trends throughout the administration (February through April) to identify potentially altered responses after, say, mid-March. After reviewing all core items, there were hardly any changes that were likely due to the disruption. Nearly all changes that were present among later respondents could be explained by institutional type and student demographics (online course taking being the exception).
Multivariate Analysis of EIs
Multivariate analysis of Engagement Indicators included three distinct examinations: (a) How early and late responders compare overall and across schools, (b) whether COVID-19 disruptions changed schools’ EIs, and (c) how much average EIs changed over the course of the survey administration. The result? COVID-19’s effect on EIs appears minimal. Being a late responder or late administration school corresponds with slightly higher scores for Learning Strategies, Effective Teaching Practices, Quality of Interactions, and Supportive Campus Environment.
Recommendations
Most institutions completed their NSSE recruitment campaigns prior to disruptions, generating the large majority of respondents, and analyses show trivial impact on responses. This provides some assurance of data quality and continuity of results. For these reasons, NSSE advises against excluding post-COVID respondents from reports.
NSSE will post data files to the Institution Interface around mid-July. With your own data in hand, consider using the covid variable to compare results before and after the disruption and in relation to findings in NSSE’s analysis.
At any time you may also download your disposition file from the Interface (see “Download your disposition file” below the Administration Snapshot). The disposition file is an Excel spreadsheet of every student in your population file and their response status, including the date and time of response.
Again, assessment activities like NSSE and FSSE matter even more in a year of turbulence. Our preliminary analyses should provide additional assurances that the data collected will contribute to your ongoing assessment plans. They will also help you assess your students’ experience in these unprecedented circumstances and study the impact of disruption to inform future planning.