Search for Publications and Presentations
SEARCH RESULTS
Keyword(s): "Response Rate"
17 results found
|
|
|
|
|
Publications
How important are high response rates for college surveys?
Fosnacht, K., Sarraf, S., Howe, E., & Peck, L. K.
The Review of Higher Education, 40
(2), 2017
hide abstract
Surveys play an important role in understanding the higher education landscape. However, declining survey participation rates threaten this source of vital information and its perceived utility. Although survey researchers have long assumed that the best way to obtain unbiased estimates is to achieve a high response rate, many survey researchers have begun to question the widely held assumption that low response rates provide biased results. Due to the prevalence of survey data in higher education research and assessment efforts, it is imperative to better understand the relationship between response rates and data quality. This study investigates this assumption with college student assessment data. It utilizes data from hundreds of samples of first-year and senior students with relatively high response rates using a common assessment instrument with a standardized administration protocol. It investigates how population estimates would have changed if researchers put forth less effort when collecting data and achieved lower response rates and respondent counts.
An alternative approach: Using panels to survey college students
Sarraf, S. A., Hurtado, S., Houlemarde, M., & Wang, X.
AIR Professional File,
(Fall), 2016
hide abstract
Eight short surveys based on select items from the National Survey of Student Engagement (NSSE) were administered to approximately five hundred students over a nine-week period at five diverse colleges and universities. The goal of the experiment was to investigate what impact a survey panel data collection approach would have on recruitment, survey data quality indicators, and scale properties. Results indicated higher response rates, shorter survey duration, and minimal impact on scale factor structures. However, both cost of incentives and panel member attrition make this alternative survey method less attractive than it would be otherwise.
Living with smartphones: Does completion device affect survey responses?
Lambert, A. D., & Miller, A. L.
Research in Higher Education, 56
(2, Special Forum Issue), 2015
hide abstract
With the growing reliance on tablets and smartphones for internet access, understanding the effects of completion device on online survey responses becomes increasing important. This study uses data from the Strategic National Arts Alumni Project, a multi-institution online alumni survey designed to obtain knowledge of arts education, to explore the effects of what type of device (PC, Mac, tablet, or smartphone) a respondent uses has on his/her responses. Differences by device type in the characteristics of survey respondents, survey completion, time spent responding, willingness to answer complex and open-ended questions, and lengths of open-ended responses are discussed.
back to top
Scholarly Papers
Survey lottery incentives and institutional response rates: An exploratory analysis
Sarraf, S., & Cole, J.S.
Association for Institutional Research Annual Forum, Orlando, FL, 2014, May
Project Slides
hide abstract
Many institutional and educational researchers are well aware that response rates for assessment surveys have been declining over the past few decades (Dey, 1997; Laguilles, Williams, & Saunders, 2011). As a result, many researchers have noted that our ability to adequately assess student academic experiences, satisfaction, engagement, use of campus resources, and other
important topics in higher education are at risk (Pike, 2008). Consequently, use of incentives are one tool that many institutional researchers have come to rely on to boost or hold steady their response rates for various campus student surveys. Though research regarding the efficacy of incentives to boost survey response rates in higher education is scant, the research that does exist suggests that incentives are an effective way to boost institutional response rates (Heerwegh, 2006; Laguilles, Williams, & Saunders, 2011). The purpose of this study is to investigate the efficacy of lottery incentives (the mostfrequently used incentive approach) to boost responses rates for institutions using the National Survey of Student Engagement (NSSE).
How important are high response rates for college surveys?
Fosnacht, K., Sarraf, S., Howe, E., & Peck, L.
Association for Institutional Research Annual Forum, Long Beach, CA, 2013, May
Presentation Slides
hide abstract
How important are high survey response rates for estimating population statistics related to the college experience? Given a general decline in survey participation rates among college students, the answer to this question has broad implications for institutional researchers who often use surveys. Survey methodologists have found that low response rates do not necessarily bias results. This study tests this proposition using results from about 250 colleges and universities that administered NSSE. Findings indicate that survey population estimates based on simulated low response rates are very similar to those based on actual high response rates.
How much effort is needed? The importance of response rates for estimating undergraduate behaviors
Fosnacht, K., Sarraf, S., Howe, E., & Peck, L.
American Educational Research Association Annual Meeting, San Francisco, CA, 2013, April
hide abstract
Survey methodologists have found that low response rates do not necessarily bias results. This study tests this proposition using results from several hundred colleges that administered a student survey. Findings indicate that survey population estimates based on low response rates are very similar to those based on high response rates.
back to top
Presentations
How are survey response rates changing? Findings from NSSE
Sarraf, S.
Association for Institutional Research Annual Forum, Denver, CO,
2019, May
hide abstract
How much have survey response rates changed at colleges and universities over the last decade? Using a National Survey of Student Engagement longitudinal data set (2010 to 2018) based on approximately 1,000 institutions, this study investigates how much response rates have changed over the years, the degree of variability in response rates within any given year, and what factors influence these outcomes, such as school undergraduate enrollment and the use of survey incentives. While reviewing this poster presentation, attendees will also gain an appreciation for one statistical method well suited for understanding change over time-latent growth curve modeling.
Assessing small populations: Recognizing everyone counts in your counts
BrckaLorenz, A., & Hurtado, S.
Student Affairs Assessment and Research Conference, Columbus, OH,
2018, June
hide abstract
Quantitative and survey research depends heavily on large sample sizes, but a focus on the “average student” in quantitative analyses often hides diverse voices. Participants in this session will discuss common issues and solutions associated with giving voice to small populations of college students (e.g., gender variant, multiracial, LGBQ+). Participants will discuss administration issues related to small populations such as increasing response rates, identifying special subpopulations, and writing more inclusive survey questions. Tips for disaggregating, responsibly aggregating, and choosing inclusive comparative information will be provided. Additionally, participants will discuss strategies for analyzing and communicating about the results from small populations as well as approaches for communicating about the validity and data quality from small sample sizes.
Assessing small populations: Recognizing everyone counts in your counts
BrckaLorenz, A., Fassett, K., & Hurtado, S.
Association for Institutional Research Annual Forum, Orlando, FL,
2018, May
hide abstract
Quantitative and survey research depends heavily on large sample sizes, but there are a variety of reasons why larger sample sizes may not be possible. Participants in this presentation will discuss common issues and solutions associated with assessing small populations of college students and instructors. Examples will focus on the experiences of gender variant and LGBQ+ students and faculty. Participants will also learn about and discuss administration issues related to small populations such as increasing response rates and identifying special subpopulations. Next, participants will learn about and discuss strategies for analyzing and communicating the results from small populations. Finally, participants will learn about and discuss approaches for communicating the validity and data quality from small sample sizes.
Improving survey data quality through experimentation
Sarraf, S., & Cole, J.
Association for Institutional Research Annual Forum, Washington, DC,
2017, May
hide abstract
With help from a large and diverse group of colleges and universities over the past decade, the National Survey of Student Engagement has conducted various randomized experiments aimed at improving survey response rates and minimizing missing data. This session provides an overview of this effort and a summary of major findings. Attendees will gain a better understanding of various issues that could help with developing their own surveys, including smartphone optimization, email subject lines, progress indicators, survey page length, and using learning management systems for recruitment.
Improving survey data quality through experimentation
Sarraf, S., & Cole, J.
North East Association for Institutional Research Annual Conference, Baltimore, MD,
2016, November
hide abstract
With help from a large and diverse group of colleges and universities over the past decade, the National Survey of Student Engagement has conducted various randomized experiments aimed at improving survey response rates and minimizing missing data. This session provides an overview of this effort and a summary of major findings. Attendees will gain a better understanding of various issues that could help with developing their own surveys, including smartphone optimization, email subject lines, progress indicators, survey page length, and using learning management systems for recruitment.
Collecting, analyzing, and reporting on data from small populations
BrckaLorenz, A., Hurtado, S., & Nelson Laird, T.
Association for Institutional Research Annual Forum, New Orleans, LA,
2016, June
hide abstract
Quantitative and survey research depends heavily on large sample sizes, but there are a variety of reasons why larger sample sizes may not be possible. Participants in this session will discuss common issues and solutions associated with assessing small populations of college students and instructors, with considerations for special subpopulations (gender variant, multiracial, etc.) as well as considerations for small institutions. Participants will also learn about and discuss administration issues related to small populations such as increasing response rates and identifying special subpopulations. Next, participants will learn about and discuss strategies for analyzing and communicating about the results from small populations. Finally, participants will learn about and discuss approaches for communicating the validity and data quality from small sample sizes.
Analysis of student-level NSSE response rates based on pre-college engagement
Gonyea, R. M., & Korkmaz, A.
Association for Institutional Research Annual Forum, Seattle, WA,
2008, May
back to top
Webinars
Survey Recruitment Using Student Portals and Learning Management Systems
Shimon Sarraf, Assitant Director for Survey Operations and Project Services
September 29, 2016
hide description
During the 2016 NSSE administration, thirty-seven institutions used their
student portal or learning management system (SP/LMS) to supplement their student recruitment efforts. Please join Shimon for a free webinar to learn more about this recruitment approach, results suggesting it can increase response rates, and the steps to take in order to do something similar for your next NSSE administration.
Improving student participation rates : What we’ve learned about incentives and promotion
Shimon Sarraf, Assistant Director
October 2, 2014
hide description
Join Shimon Sarraf, NSSE Assistant Director for Survey Operations, to learn more about the relationship of incentives and campus promotions to response rates. Based on recent research presented at the AIR Annual Forum in spring 2014, this webinar will focus on answering the following questions:
- What kinds of incentives do participating NSSE institutions typically use?
- Which ones appear to be most effective at increasing student participation?
- What impact do campus promotional campaigns have on response rates?
- For those that invest in promotional campaigns, how do they implement them and who is involved?
Encouraging student participation in NSSE
Brian McGowan, NSSE Institute Project Associate
February 28, 2012
hide description
In this pre-recorded webinar, we provide strategies to help institutional users encourage student participation and improve student response rates during the NSSE administration process. This session highlights effective approaches to engage students, faculty and staff members, and parents. Institution-specific examples are offered to show how schools have disseminated their NSSE results across campus and in their recruitment materials.
Improving student response rates
Tony Ribera and Brian McGowan
January 25, 2011
hide description
Ideas for promoting student participation in NSSE administration. The focus of the webinar will be on advertising campaigns and other creative approaches, including a number of specific examples, that can generate buzz on campus about the NSSE survey. Presenters discuss modes of capturing student attention and share approaches to include students in assessment conversations.
back to top
