How many participating NSSE institutions use incentives?

Incentives are quite popular among NSSE institutions and campuses have been using them more frequently than ever before. The proportion of NSSE schools using incentives has increased from 35% in 2010 to 61% in 2020.

Should my institution offer students an incentive to participate in NSSE?

We generally recommend offering an incentive, especially if an institution has struggled with response rates in the past. Some of the highest NSSE response rates have been at institutions that have included an incentive as part of a campus-wide promotional campaign. Each institution, however, must weigh the likely impact of the incentive against its financial cost.

It is worth noting that the use of incentives is one part of a larger promotional campaign. If offering an incentive is not possible, institutions should remember to coordinate promotional efforts with other campus offices and student organizations and should emphasize in other ways that NSSE participation is important to campus decision makers.

Before offering an incentive, we advise NSSE campus project managers to consult Institutional Review Board (IRB) and business office staff to ensure the incentive plan is allowable and feasible (see the last question).

What types of incentives are typically offered? Do institutions offer multiple incentive types?

Incentives are most commonly awarded by lottery, but guaranteed prizes are also used. Among NSSE institutions that offered an incentive in 2013, 92% entered respondents in a lottery; the remainder gave a prize to all respondents, and some offered both. The vast majority of institutions (87%) offered only one type of incentive, the most common being a gift card for a local establishment such as a campus bookstore or ice cream shop (42%). Electronic devices (e.g., tablets) were the next most common (27%), with cash and general-purpose gift cards (e.g., Amazon.com) trailing at 15% each.

How effective are incentives? Do certain types generate more response than others?

NSSE analyses suggest that lottery incentives generally boost response rates by 3 to 6 percentage points, depending on the incentive type used (Sarraf & Cole, 2014). As prizes, electronic devices appear most effective with general-purpose gift cards and cash close behind. Specific gift cards appear to boost response but not as much as these other incentive types; results suggest a 2% lower response rate compared to electronic devices for example.

A 2011 study (Laguilles, Williams, & Saunders) found a 5- to 9-point increase in college student survey response rates using lotteries. Other studies found that a prepaid incentive of $2 to be even more helpful, with response-rate increases of 11 to 17 points (Millar & Dillman, 2011; Parsons & Manierre, 2014). The total amount spent on incentives also made a difference in the NSSE study. Each additional dollar spent per sample member increased response rates by about 4 percentage points.

Does incentive use diminish data quality (item nonresponse, straightlining, etc.)?

Published research suggests that incentives do not affect data quality indicators (Singer & Ye, 2013; Toepoel, 2012), including item nonresponse and response distributions. Preliminary findings from a recent internal NSSE analysis suggest minimal impact as well.

Are there other considerations for using incentives?

It is important to consider your campus culture when determining whether to use an incentive. On your campus, is it common practice to offer an incentive for survey participation? If so, it may be helpful to offer one. However, if this is not typical on your campus, consider the possible effect of students' expectations of an incentive on response rates with future surveys that do not offer one.

Are there other ways to boost response rates at levels comparable to those from incentives?

Effective promotional campaigns appear to increase response rates by about 4 to 5 percentage points. This is particularly true for first-year students, where promotional campaigns, especially those that use multiple strategies (e.g., posters, advertisements, YouTube videos) and coordinate among multiple campus offices and constituencies, appear to increase response rates above and beyond lottery incentives alone (Sarraf & Cole, 2014).

How do institutions typically promote NSSE participation to students? What are successful models?

Campus posters are most frequently used to promote student participation in NSSE. A number of institutions also promote via social media (e.g., Facebook, Twitter, Instagram) or through learning management systems (e.g., Blackboard). Many institutions also enlist faculty to encourage survey participation by their students. Anecdotal and empirical evidence suggest a multi-faceted approach will be most effective, in which institutional leaders, multiple campus offices, academic departments, and student leaders promote NSSE.

What special considerations are there for campuses in implementing incentive and promotional campaigns?

All incentives must be approved by NSSE. (Institutions submit text describing the incentive on the NSSE Institution Interface during Phase II of survey preparations, beginning in October and concluding in November.) Under NSSE's IRB protocols, institutional recruitment messages must inform students of their chances of winning any prize by stating the number of prizes offered along with (a) the number of eligible students, (b) the number of respondents from a recent prior administration, or (c) the approximate odds of winning. Incentives may not be so valuable as to potentially coerce participation, and promotional materials may not call excessive attention to incentives.

We also recommend that you consult with your local IRB about your incentive plan as well as your business office for tax reporting considerations.

References

Laguilles, J. S., Williams, E. A., & Saunders, D. B. (2011). Can lottery incentives boost web survey response rates? Findings from four experiments. Research in Higher Education, 52, 537-553.

Millar, M. M., & Dillman, D. A. (2011). Improving response to web and mixed-mode surveys. Public Opinion Quarterly, 75, 249-269.

Parsons, N. L., & Manierre, M. J. (2014). Investigating the relationship among prepaid token incentives, response rates, and nonresponse bias in a web survey. Field Methods, 26, 191-204.

Sarraf, S., & Cole, J. S. (2014, May). Survey lottery incentives and institutional response rates: An exploratory analysis. Paper presented at the annual forum of the Association for Institutional Research, Orlando, FL.

Singer, E., & Ye, C. (2013). The use and effects of incentives in surveys. Annals of the American Academy of Political and Social Science, 645, 112-141.

Toepoel, V. (2012). Effects of incentives in surveys. In L. Gideon (Ed.), Handbook of survey methodology for the social sciences (pp. 209-223). New York, NY: Springer.