In 2019, researchers at the Center for Postsecondary Research that administer the National Survey of Student Engagement (NSSE) developed a new itemset asking students about the quality of their participation in high-impact practices (e.g., capstone, internship, undergraduate research experiences, etc.) through a grant supported by the Lumina Foundation. I used the data collected to build and a test a model of a quality element for high-impact practices as a portion of my dissertation.
Using collated literature, I created six themes, or aspects of quality, that I believed quality educational experiences should encompass. These aspects entailed opportunities for students to: apply their learning, receive frequent feedback, take part in active learning activities, understand learning expectations, engage in valuable interactions with others, and place time or effort toward the high-impact practice. I mapped multiple survey items from the new itemset to the six aspects to create a latent construct, the "Quality Element," through a confirmatory factor analysis. Figure 1 shows a visual representation of the six themes and the associated survey items. To provide more rigor to the analysis, I fit the quality element to the data twice by splitting the total cases (N=12,870) into random halves; the factor loading and omega (interpreted similar to Cronbach's Alpha) values indicted a fairly good fit. And, the CFI, TLI, and RMSEA indices showed an excellent model fit for both halves (CFI>.90, TLI>.90, RMSEA<.06).
Looking at the coefficients, all the survey items positively and significantly loaded, but with varying sizes. I found the survey item regarding learning expectations (HIP1902b) loaded the strongest while the quantity of active learning activities (HIP1915) loaded the weakest. These items respectively asked students about the extent in which the activities they would do as part of their high-impact practice were explained to them, and the frequency in which students gave a formal presentation for class, gave a formal presentation for a wider audience, or wrote a paper or report.
Next, I performed a multigroup confirmatory factor analysis (MGCFA) to test if the quality element applied to, or was invariant by, six high-impact practices (capstones, learning communities, internships, service-learning courses, study abroad experiences, and undergraduate research experiences). In short, yes, the quality element held (configural & metric invariance) for the six high-impact practices; it implies this model is appropriate to use to examine these high-impact practice experiences.
Some of the additional research questions examined in the dissertation included, how do perceptions of the quality element differ by student characteristics, and how does the quality element relate to student intrapersonal and practical perceived gains? For more details, see the dissertation located on ProQuest.