Digging Deeper Into HIP Quality

Digging Deeper Into the Quality of High-Impact Practices

Story Features

Internships: Challenging, Valuable, Inclusive
- Black Students Voice Internship Experiences
Service-Learning Has Benefits for Online Learners
Undergrad Research Enriches Learning Across Major & Race/Ethnicity
FY Students: Not Knowing About HIPs is Biggest Obstacle
How Important are HIPs to Faculty?
Reference and Resources

HIPs Must Be “Done Well” to Achieve Benefits

Students involved in high-impact practices (HIPs—transformative learning experiences such as service-learning, undergraduate research, internships, and capstones) generally enjoy higher levels of learning and success. The positive benefits to students, especially those from identity groups historically and contemporarily underserved by higher education, strong interest by faculty, and endorsement from employers has helped advance HIP adoption. Yet, in the rush to expand implementation and involve more students, we must heed NSSE founder George Kuh’s (2008) reminder to expect the benefits from HIPs only “when done well.” Defining “done well” means addressing continuing questions of access, quality, assessment, and equity.

We find that exposure to the eight quality elements varies across HIPs and mostly reflects patterns represented in the literature.

A smiling student with a camera with a superimposed graphic of sample results from the HIP module for decorative purposes only.

HIP Quality Topical Module

NSSE’s HIP Quality module measures the extent to which high-impact experiences entail features that define a quality practice, such as demanding considerable time and effort, helping students engage across difference and in diverse settings, making space for reflection and integration, conveying high expectations, and providing substantive feedback.

Review the module

Exposure to the eight essential HIP elements—high expectations, significant investment of time and effort, substantive interaction with faculty and peers, experiences with diversity and in unfamiliar situations, frequent feedback, opportunities for reflection and integration of learning, real-world application, and public demonstrations of competence—varies by HIP (Kinzie et al., 2020). In an ideal world, every HIP would manifest all eight elements of quality, but this is not demonstrated in practice.

Notably, the foundational literature specific to each HIP emphasizes some elements over others, and some not at all. For example, the literature about undergraduate research touts the benefits of interaction with faculty, while the benefits of engaging with difference are largely absent. In contrast, service-learning has a strong emphasis on diversity, while interaction with faculty has minimal emphasis. Given this, we find that exposure to the eight quality elements varies across HIPs and mostly reflects patterns represented in the literature.

In this second installment of Engagement Insights: Survey Findings on the Quality of Undergraduate Education—Annual Results 2022, we use data from the NSSE HIP Quality Topical Module, launched in 2022, to gain a more nuanced understanding of three selected HIPs: internships, service-learning, and research with faculty. Parallel FSSE data also inform some of these findings.

Note: We collected data from 27 institutions that adopted the HIP Quality module in 2022, with over 30,000 total respondents. Over 19,000 provided information about a high-impact practice they had done, while those who had not done any of the HIPs answered questions about their plans to participate (N>5,600) or barriers to participation (N>1,200).

Internships and Field Experiences: Challenging, Valuable for Careers, and Fostering Inclusion

While connected to early career development and acquiring real-world, on-the-job experience, internships and other field experiences (e.g., co-ops, student teaching, and clinical placements) can also expose students to different situations and interactions. They can also challenge students to draw connections between the practical and the disciplinary, and better understand social issues.

Seniors who participated in an internship or field experience benefited in a variety of ways. About four in five said it substantially (“quite a bit” or “very much”) contributed to their growth in understanding concepts in their courses or major, working effectively with people from different backgrounds, and solving real-world, complex problems. Nearly all felt their experience helped them acquire post-graduation job- or work-related skills (92%) and planning (86%).

Helping individuals in need in a community that I didn’t grow up in has significantly affected my life since starting college.
—SENIOR, MORGAN STATE UNIVERSITY, MAJORING IN PHYSICAL EDUCATION AND SPORTS ADMINISTRATION

Table 1. Activities Where Seniors Frequently Engaged in Their Internships
How often have you done the following?	Percentage
Connect what you were learning to your major field or career goals	86
Interact with people from a different background or identity	72
Find yourself in settings or circumstances that were new or unfamiliar to you	69
Track your experience with informal writing (notes, journal, blog, etc.)	58
Connect what you were learning to societal problems or issues	51
Work with other students	48
Discuss your experience with other students in an organized setting	48

Note: Frequently is defined as "often" or "very often."

During their internships, many seniors frequently (“often” or “very often”) connected what they were learning to their major field or career goals (86%), interacted with people from diverse (racial/ethnic, economic, political, religious, etc.) backgrounds (72%), or found themselves in new or unfamiliar settings or circumstances (69%). About half frequently connected their learning to broader societal problems (Table 1).

Black Students Voice Their Internship Experiences

Black or African American students were largely satisfied with their internships and felt they were challenged to do their best work, but we were interested to hear their responses to the two open prompts at the end of the module:

WHAT WAS MOST SATISFYING ABOUT THIS EXPERIENCE?

Comments provided reflect appreciation for the opportunities to develop real-world, work-related experiences as well as the importance of addressing social inequities.

“I am able to observe professionals doing the job that I aspire to do myself. I am given tasks to develop my abilities and I am given constructive criticism to mark areas that need improvement. I appreciate the level of intellectual challenge and emotional satisfaction I get from being involved.”

“Helping individuals in need in a community that I didn’t grow up in but that has significantly affected my life since college.”

WHAT WAS LEAST SATISFYING ABOUT THIS EXPERIENCE?

On the other hand, some black students expressed concerns with the time commitments these experiences often require:

“Just feeling overwhelmed with the workload”

“Balancing work, school, internships, and in class times”

“Balancing it with my job & school has been a little difficult.”

Though Less Common, Service-Learning Benefits Online Learners More

**Figure 1. Participation in High-Impact Practices by Class** Go to an accessible version of figure 1 (Participation in HIPs by class)

Note: Percentages are of those responding “done or in progress” or (where noted) “plan to do,” except for service-learning where percentages are of those responding at least “some” of their courses included a community-based project.

By offering high-quality service-learning courses, colleges and universities can address academic relevance, develop workplace skills, foster appreciation for cultural diversity, promote critical thinking, and integrate learning from multiple disciplines. In 2022, about half of first-year students and three in five seniors had taken at least one service-learning course (Figure 1)—percentages that have remained steady for over a decade.

However, with the shift toward remote learning, institutions are adapting online courses to include service-learning components (Faulconer, 2020). Some online service-learning programs arrange projects in-person and on location in the student’s community, some arrange online-only projects, and the remainder may be a hybrid of in-person and online activities. Among students taking mostly remote courses in 2022, 44% of first-year students and 56% of seniors had taken at least one service-learning course—about 5 to 7 percentage points lower than students overall.

**Figure 2. Seniors’ Perceptions of Service-Learning Quality by Course Modality**Go to an accessible version of this figure: Seniors’ Perceptions of Service-Learning

Note: Percentages for the first item above represent the highest rating (at least ‘6’) on the scale ranging from 1=poor to 7=excellent. Percentages for the how often items are the combination of “very often” and “often,” and for the perceived gains items are the sum of “very much” and “quite a bit.”

In general, the perceived quality of service-learning varies considerably, with less than half (38% of first-year students and 46% of seniors) evaluating their service-learning program with the highest ratings (at least ‘6’ on the scale from 1=poor to 7=excellent). Nonetheless, online instructors and program administrators may be pleased to see that mostly remote seniors rated the overall quality of their service-learning experiences more highly than their in-person counterparts (Figure 2). Also notable for online learners in service-learning, seniors perceived they had gained more in understanding course concepts, understanding societal problems, solving complex real-world problems, and preparing for plans after graduation, compared to their peers in mostly in-person courses. At the same time, service-learning programs for online learners were on par with their in-person counterparts in terms of interactions with diverse others and examining issues of equity or privilege.

A notable exception to this positive news is that students taking mostly remote courses were much less likely to be exposed to settings or circumstances that were new or unfamiliar to them. This may be because many remote learners are arranging their own service site visits, calling upon familiar contacts within their communities, or completing their service in a remote manner.

Note: The sample included 3,235 first-year and 1,894 senior students enrolled at 25 institutions that administered the HIP Quality Topical Module in 2022, and who had completed at least three weeks of a service-learning experience. In this sample, 5% of first-year students and 26% of seniors were taking mostly remote course, as compared to 74% of first-year students and 41% of seniors who were taking mostly in-person courses.

Undergraduate Research Enriches Learning Across Majors and Racial and Ethnic Identities

Participation in research projects with faculty broadens and deepens undergraduate learning and supports a range of skill developments. Our data show that seniors who participated in research with faculty were challenged at high levels and perceived substantial (“very much” or “quite a bit”) gains in applying theory to practice (86%), job- or work-related skills (83%), solving complex real-world problems (79%), and preparing for plans after graduation (78%). What is more—with mentoring identified as a vital role of faculty in undergraduate research—4 in 5 students frequently (“very often” or “often”) received helpful feedback from faculty during their experience.

**Figure 3. Emphasis on Equity and Diverse Ideas in Undergraduate Research by Race/Ethnic Identity for Seniors** Go to an accessible version of this figure: Emphasis on Equity and Diverse Ideas

Note. Percentages are the sum of “very much” and “quite a bit.”

Undergraduate Research—Exploring Engagement Across Difference

Aggregate results suggest that the quality of undergraduate research is strong. But while the perceived quality of undergraduate research is generally high across students of all racial and ethnic identities (72% on average), results reveal differences in student experiences related to diversity and equity. For example, the extent to which undergraduate research experiences substantially (“very much” or “quite a bit”) emphasized respecting the expression of diverse ideas ranged from 40% to 100% among seniors of different racial or ethnic identities. Similarly, substantial emphasis on examining issues of equity or privilege results also ranged from 20% to 82% across groups (Figure 3). Granted, these results are based on small populations and the differences deserve further inquiry.

In addition, closer consideration of results for these two items (examining equity and respecting diversity) shows that doing research with faculty had the lowest scores in comparison to the five other HIPs. Although there are existing programs such as McNair Scholars and Undergraduate Research Opportunity Programs (UROP) that intentionally engage students from racially marginalized groups, there is little undergraduate research literature in general that documents an emphasis on experiences with difference. Given the range of results by race and ethnic identity found here, these results could point to an opportunity to improve quality in undergraduate research by intentionally embedding opportunities to examine issues of equity and to encourage the expression of diverse ideas in research experiences.

**Figure 4. Perceived High Expectations, Gains in Learning Course Concepts in Undergraduate Research by Major Category for Seniors**Go to an accessible version of this figure: Perceived High Expectations, etc.

Note: Percentages for the perceived gains item are the sum of “very much” and “quite a bit,” and percentages for the challenge item are of those giving the highest rating (at least ‘6’) on the scale ranging from 1=not at all to 7=very much.

The quality of HIPs can also vary across major fields (Figure 4). For example, there is sizeable variation in perceptions that the research experience challenged students to do their best work. The percentage of students within each major perceiving the highest challenge (at least ‘6’ on a scale from 1=not at all to 7=very much) ranged from 92% within communications, media, and public relations majors to 61% among business majors. On the other hand, across all major categories, students expressed an elevated level of connection between their research experience and the courses in their majors. For example, 95% of students in arts and humanities, 90% of students in social sciences, and 87% in biological sciences, agriculture, and natural resources perceived substantial (“very much” or “quite a bit”) gains in understanding concepts in their courses or major.

Understanding this variation should help academic program leaders emphasize the strong connection between undergraduate research and the learning of course concepts and should encourage opportunities for students to do their best work in their research projects with faculty.

Note: The research with faculty sample included 358 first-year and 1,566 senior students enrolled at 26 institutions that administered the HIP Quality Topical Module in 2022.

First-Year Students are Clear: Lack of Information Would Keep Them From Participating in HIPs

**Figure 5. Anticipated Barriers to HIP Participation for First-Year Students** Go to an accessible version of these results: Anticipated Barriers to HIP Participation for First-Year Students

Note: The service-learning question does not include a "plan to do" option so was not eligible for these questions.

First-year students who planned to participate in HIPs (N=4,457) were asked to identify potential barriers to their participation. The top reason by far was not knowing enough about the opportunity (61%), followed by not being able to fit it in their schedule, cost barriers, or feeling unprepared—all around 30%. Least among their concerns was feeling discouraged by people in their life, that people like them were unlikely to participate, or that they were studying remotely. This finding suggests that HIP recruitment and promotions may be doing a better job across different social identities.

The anticipated barriers varied by HIP in predictable ways (Figure 5). For instance, while the most common reason for not participating was lack of knowledge, affordability was the highest concern for study abroad (56%). On the other hand, relative to other HIPs, a far greater share of students interested in learning communities or senior capstones did not know enough about the experience (78% and 75%, respectively).

Institutions should be encouraged that for the many first-year students who intend to participate in HIPs, the greatest potential barrier can be mitigated through better promotion, information sessions, curricular integration, and the like.

From FSSE

Faculty Emphasize Importance of HIPs

**Figure 6. Faculty Perceptions of the Importance of Selected High-Impact Practices** Go to an accessible version of these results (Faculty Perceptions of the Importance of Selected High-Impact Practices)

Note: Data are from more than 7,100 faculty members at 78 colleges and universities that were surveyed in 2022.

The importance faculty place on high-impact practices is a valuable aspect of institutional culture, as the collective practices and values of faculty members can potentially affect the student experience. At institutions where faculty more strongly believe HIPs are important, students are more likely to participate in them (Nelson Laird et al., 2014). In the 2022 Faculty Survey of Student Engagement (FSSE), we find that a majority of faculty believe that internships or field experiences (84%), service-learning (62%), and undergraduate research (54%) are “important” or “very important” (Figure 6).

Disciplinary differences do exist, however. For example, faculty from education, health professions, and social service professions place more importance on service-learning participation, whereas faculty from the physical and biological sciences place more importance on undergraduate research participation.

In Closing

HIPs rely on quality, and consistency—being “done well”—to achieve their learning outcomes. Evidence about quality can be useful for faculty and administrators interested in maximizing HIP effectiveness by suggesting where instructional practices, expectations, and assignments should be redesigned to ensure greater exposure to all eight elements.

References and Resources

Faulconer, E. (2021). eService-learning: A decade of research in undergraduate online service–learning. American Journal of Distance Education, 35(2), pp. 100-117.

Kinzie, J., McCormick, A. C., Gonyea, R. M., Dugan, B., & Silberstein, S. (2020, July). Assessing quality and equity in high-impact practices: Comprehensive report. Indiana University Center for Postsecondary Research.

Kinzie, J., Silberstein, S., McCormick, A. C., Gonyea, R. M. & Dugan, B. (2021). Centering Racially Minoritized Student Voices in High-Impact Practices. Change, 53(4), 6-14.

Kuh, G. (2008). High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter. Washington, DC: Association of American Colleges and Universities.

Nelson Laird, T., BrckaLorenz, A., Zilvinskis, J., & Lambert, A. (2014).
Exploring the effects of a HIP culture on campus: Measuring the relationship between the importance faculty place on high-impact practices and student participation in those practices. Paper presented at the Association for the Study of Higher Education Annual Conference, Washington, DC.

Zilvinskis, J., Kinzie, J., Daday, J., O’Donnell, K & Vande Zande, C. (2022). Delivering on the Promise of High-Impact Practices: Research and Models for Achieving Equity, Fidelity, Impact, and Scale. Sterling VA: Stylus.

Zilvinskis, J., Kinzie, J., Daday, J., O’Donnell, K & Vande Zande, C. (2022). Delivering on the Promise of High-Impact Practices: A New Resource for Assessment. Assessment Update, 34(4), 1-2, 16.