Time well spent: Flipped classrooms and effective teaching practices.
Fassett, K., BrckaLorenz, A., Strickland, J., & Ribera, A.
In E. Alqurashi Fostering Student Engagement with Instructional Technology in Higher Education ICI Global, 2019.
Good teaching practices are the crux of student education and require constant evaluation to meet current generations' learning needs. Flipped classrooms have sought a foothold in higher education to provide opportunities for deep learning through the delivery of content online prior to attending class while having activities related to processing and applying the information during class. Using a large-scale, multi-institution study of faculty teaching flipped courses, this study empirically links flipped procedures to other forms of effective educational practice and additionally focuses on the motivations and impacts on the faculty side of this pedagogical practice. Findings indicate numerous learning and development benefits for students with implications for supporting and motivating faculty across disciplines, faculty identities, and course types.
Full version

Independent colleges and student engagement: Descriptive analysis by institutional type
Gonyea, R. M., & Kinzie, J.
Washington, DC: Council of Independent Colleges, 2015.
Critics of traditional, residential, liberal arts colleges and universities contend that this form of higher education is outmoded, too costly, and no longer educationally relevant for 21st century students. Economies of scale, large classes taught by contingent faculty members and graduate students, and increasing reliance on technology and online learning, so the argument goes, are the only cost-effective means of meeting the educational challenges of the future. This report, prepared for the Council of Independent Colleges (CIC), draws on the most current NSSE data, from 2013 and 2014, that include more than 540,000 first-year and senior students enrolled at more than 900 four-year colleges and universities. Findings are presented with comparisons across four institutional types: (1) baccalaureate and master‘s level private institutions (CIC‘s predominant membership profile), (2) baccalaureate and master‘s level public institutions, (3) doctoral private institutions, and (4) doctoral public institutions. Included in the analysis are measures from the updated NSSE that includes ten new Engagement Indicators, six High-Impact Practices, the Perceived Gains scale, and a Satisfaction scale. Findings from this study affirm the effectiveness of independent colleges and universities for undergraduate student learning. Students at private institutions are more likely to be engaged in educationally effective experiences than their peers at public institutions. Areas of distinction in the private institution undergraduate experience include a more academically challenging education, better relations with faculty members, more substantial interactions with others on campus, and the consistent perception that students have learned and grown more, in comparison with public institutions.
Full version

Will these trees ever bear fruit? A response to the special issue on student engagement
McCormick, A. C., & McClenney, K.
The Review of Higher Education, 35(2), 307–333, 2012.
There is broad consensus that U.S. higher education needs to do better. Researchers, policymakers, and practitioners have called attention to a range of challenges: too many students enter college unprepared for college-level work, yet many developmental programs are little more than revolving doors; too many students who begin college never graduate, often accumulating considerable debt; the most rapid enrollment growth is among the groups that higher education has historically served least well?so institutions have to do more to ensure their students‘ success; students‘ development of generalized critical-thinking and problem-solving skills falls short of what we want and need; we are not producing enough graduates in science, technology, engineering, and math; cost escalation is unsustainable, with most of the growth occurring outside of core educational functions; and the United States is losing ground to other countries with regard to postsecondary degree attainment. And as we confront these challenges, the national understanding of college quality is dominated by beauty contests that privilege reputation and resources over teaching and learning. The higher education research community has the capacity to contribute to our understanding of and response to these challenges. Indeed, scholars have engaged with many of them. Any could justifiably serve as the organizing theme for a special issue of one of the field‘s leading scholarly journals. Given the range of important topics where systematic, focused scholarly treatment could advance both research and practice, we find it curious that student engagement trumps these subjects as meriting a special issue of the Association for the Study of Higher Education‘s signature scholarly journal. We might be flattered that our work is seen as deserving such attention, but we are instead dismayed that the ?special issue on student engagement? was in fact devoted to critiques focused exclusively on the two university-based research and service projects that we direct; that it included no contributions from scholars with a record of inquiry on student engagement; and that we had no opportunity to respond to the critique in the special issue itself so as to better advance scholarly discourse and professional practice. While our projects have always welcomed reasoned critique (continuous improvement based on feedback is a hallmark of both projects), we find these precedents worrisome. We are nevertheless grateful for the opportunity to submit this response after the fact. In the following pages, we situate our response relative to the long-decried disconnect between higher education research and practice, a gap that our respective projects attempt to bridge. We offer brief comments about the Olivas preface, mostly to correct factual errors and omissions, and then provide more detailed responses to the substantive critiques in the articles by Porter; Dowd, Sawatzky, and Korn; Campbell and Cabrera; and Nora, Crisp, and Matthews (all 2011).
Full version

STEM/non-STEM differences in engagement at U.S. institutions
Nelson Laird, T. F., McCormick, A. C., Sullivan, D. F., & Zimmerman, C. K.
Peer Review, 13(3), 23–26, 2011.
A recent paper by one of us (Nelson Laird) and some colleagues brought some sobering news of differences between STEM (science, technology, engineering, and mathematics) and non-STEM undergraduates with regard to approaches to learning that promote more complex, deeper understanding. Using data from the National Survey of Student Engagement (NSSE) and the Faculty Survey of Student Engagement (FSSE), Nelson Laird and colleagues examined disciplinary differences in the extent to which students are exposed to educational environments that promote deep approaches to learning. These approaches to learning are important because ?[s]tudents who use deep approaches to learning tend to perform better as well as retain, integrate, and transfer information at higher rates than students using surface approaches to student learning? (Nelson Laird, Shoup, Kuh, and Schwarz 2008, 470). Nelson Laird and colleagues found?using models with extensive statistical controls?that, nationally, STEM faculty generally use pedagogies that encourage higher-order, integrative, and reflective learning significantly less than faculty in non-STEM fields and, not coincidently, STEM seniors experience ?deep approaches to learning? less than seniors in non-STEM fields (for descriptions of the three measures, see Nelson Laird et al. 2008). The differences were small for Higher-Order Learning, the scale that is concerned with analysis, synthesis, and judgment regarding evidence?relatively good news?but quite large for the Integrative and Reflective Learning scales. The study by Nelson Laird and colleagues is a part of a larger body of work about students engaging in educationally purposeful activities?those educational practices known to positively influence valued educational outcomes, activities such as active and collaborative learning and those that involve much student?faculty interaction, as noted in many of the articles in this issue of Peer Review. We know of the positive impact of pedagogies of engagement not only on general student learning, but also on STEM learning, from years of research. It is discouraging that, nationally, faculty in STEM fields tend to have lower expectations for integrative and reflective learning relative to other faculty, and that results from seniors reflect those differences. The Integrative Learning scale assesses how often students use ideas from various sources and courses, include diverse perspectives in class discussions or writing assignments, and discuss ideas from readings or classes with faculty members and others outside of class. The Reflective Learning scale is a combination of responses to questions about trying out different perspectives and thinking about one‘s own beliefs. The kinds of intellectual self-reflection skills these questions are about are surely as important in the STEM disciplines as they are in other disciplines, but we see that STEM majors have far fewer opportunities to develop these skills than students in other majors. Indeed, one might argue that it is especially in STEM that students should acquire these skills, given the way empirical evidence tends to be seen as harder in science than in other disciplines. Discovering a bad premise or assumption and being open to other interpretations are just as important in STEM disciplines as elsewhere. These results caused us to want to look more closely at STEM/non-STEM differences and to determine whether there are circumstances where STEM seniors buck the general trends and are as engaged or more engaged than their non-STEM peers.

A comparison of student and faculty academic technology use across disciplines
Guidry, K. R., & BrckaLorenz, A.
EDUCAUSE Quarterly, 33(3), , 2010.
Our study extends the research into faculty and student use of contemporary academic technologies by asking five questions: How often do students report using academic technologies? How often do faculty report using academic technologies? Do students in different disciplines use these technologies more or less than their peers? Do faculty in different disciplines use these technologies more or less than their peers? Are there noticeable differences between how often students and faculty use these technologies? Our study examined responses to a pair of surveys ? the National Survey of Student Engagement (NSSE) and the Faculty Survey of Student Engagement (FSSE) ? administered in the spring of 2009.

Research and discovery across the curriculum
Elrod, S., Kinzie, J., & Husic, D.
Peer Review, 12(2), 4–8, 2010.
The national conversation on undergraduate research is gaining momentum, in part because of its identification as one of the ten high-impact educational practices identified in an analysis of data from the National Survey of Student Engagement (NSSE), published by the Association of American Colleges and Universities (Kuh 2008). However, the trend is not new; the Council on Undergraduate Research (CUR) has been a leading proponent of undergraduate research for the past thirty years. According to CUR, undergraduate research is ?an inquiry or investigation conducted by an undergraduate student that makes an original intellectual or creative contribution to the discipline.? Ramirez and Hoagland (2003) state that faculty and students should be encouraged to ??collaborate as partners in their explorations of uncharted intellectual terrain. The symbiosis established between the faculty member and undergraduate collaborator energizes and informs the faculty member‘s teaching and research while simultaneously introducing the student to the joys of discovery as well as to lessons in persistence, problem-solving and critical thinking.? Thus, it is as much a matter of effective teaching and learning as it is a matter of research and scholarship. In STEM (science, technology, engineering, and mathematics) disciplines, Project Kaleidoscope, working over the past twenty years to advance effective STEM education, has also played a role. The National Science Foundation‘s Research Experience for Undergraduates (REU) and Research in Undergraduate Institutions (RUI) programs and various scientific societies have also helped to promote, support, and highlight the work of undergraduate researchers. Many funding agencies, philanthropic foundations and other organizations have also touted and supported the educational benefits of undergraduate research across all disciplines for decades, including the National Humanities Alliance, which holds an annual Humanities Advocacy Day. Undergraduate research experiences are also held in high regard by faculty members: more than 50 percent of faculty members reported on the Faculty Survey of Student Engagement (FSSE) that participation in a research project with a faculty member is important for students (National Survey of Student Engagement 2008). Students in all disciplines are also increasingly calling for such experiences (Society of Physics Students 2008). In this article, we provide an analysis of the value of undergraduate research and suggest that its essence be used to infuse a pedagogy of research and discovery into courses across the curriculum for a more relevant, real-world, research-rich educational experience.

Promoting student success: What business leaders can do
Lovett, C. M.
Bloomington, IN: Indiana University Center for Postsecondary Research, 2005.
More than any other group of stakeholders, business leaders are aware that this country‘s ability to remain competitive in a global, technology-based economy is ever more closely tied to its ability to produce more and better prepared college graduates. Graduating more students from various backgrounds who are well prepared to meet the social, civic and economic challenges of the future is a national priority. Every college and university can improve its graduation rates and enhance the quality of its undergraduate programs by creating the conditions that matter to student success. Decades of research studies show that student engagement--the time and effort that students devote to their studies and related activities--is a key factor in student success. The Documenting Effective Educational Practice (DEEP) project gathered data from 20 very different institutions of higher learning whose student scores on the National Survey of Student Engagement (NSSE) were higher than what was predicted on the basis of their student and institutional profile, history, and other factors.
Full version

Student experiences with information technology and their relationship to other aspects of student engagement
Nelson Laird, T. F., & Kuh, G. D
Research in Higher Education, 46(2), 211–233, 2005.
Full version

Disciplinary differences and undergraduates' information-seeking behavior
Whitmire, E.
Journal of American Society for Information Science and Technology, 53 (8) 631-638, , 2002.

The relationships between computer and information technology use, student learning, and other college experiences
Kuh, G. D., & Hu, S.
Journal of College Student Development, 42(3), 217–232, 2001.

Time well spent: Flipped classrooms and effective teaching practices
Fassett, K. T., BrckaLorenz, A., Strickland, J., & Ribera, A.
American Educational Research Association Annual Meeting, Toronto, Canada, 2019, April.
Good teaching practices are the crux of student education and require constant evaluation to meet current generations‘ learning needs. Flipped classrooms have sought a foothold in higher education to provide opportunities for deep learning through the delivery of content online prior to attending class while having activities related to processing and applying the information during class. Using a large-scale, multi-institution study of faculty teaching flipped courses, this study empirically links flipped procedures to other forms of effective educational practice and additionally focuses on the motivations and impacts on the faculty side of this pedagogical practice. Findings indicate numerous learning and development benefits for students with implications for supporting and motivating faculty across disciplines, faculty identities, and course types.
Full version

Taking surveys with smartphones: A look at usage among college students
Sarraf, S., Brooks, J., & Cole, J.
American Association for Public Opinion Research Annual Conference, Anaheim, CA, 2014, May.
The widespread adoption of mobile technologies has dramatically impacted the landscape for survey researchers (Buskirk & Andrus, 2012), and those focusing on college student populations are no exception. The National Survey of Student Engagement (NSSE), one of the largest U.S. college survey assessment projects, annually surveys hundreds of thousands of undergraduate students at college and university campuses throughout the United States and Canada. Internal NSSE analyses show the number of smartphone respondents is increasing each year.1 This analysis showed that in 2011, only about 4% of NSSSE respondents used a smartphone, but by 2013 that figure had increased to 13%. Preliminary results from the 2014 administration suggest the percentage continues to increase, with roughly 18% of respondents using smartphones to complete the survey. Using 2013 NSSE data, the purpose of this study is to examine college student demographics and engagement results by smartphone respondent status. The results of this study will provide insights into the prevalence of college?aged survey respondents using smartphones, and the impact this technology has on survey responses.
Full version

Student perspectives on the importance and use of technology in learning
BrckaLorenz, A., Haeger, H., Nailos, J., & Rabourn, K.
Association for Institutional Research Annual Forum, Long Beach, CA, 2013, May.
Under constant demands to improve the quality of higher education within an increasingly digital world, technology is often seen as a way to increase learning and collaboration on college campuses. The current generation of college students has grown up with technology, and these students are among the earliest adopters of new advances in technology (Jones, 2002; Haney, 2011). Allowing students to connect to their campus community, collaborate with peers, acquire new information, and demonstrate their learning through technology is essential for college campuses seeking to meet the needs of today‘s college students. Though access and use of technology is highest in traditional age college students, disparity still exists in who uses technology and at what age they are exposed to it (Jones et al., 2009; Pew Research Center, 2012; Wilson, Wallin, & Reiser, 2003). Race, gender, age, socioeconomic status, among other factors, can impact the level of technical proficiency students come to college with and should be considered in how technology is implemented on college campuses (Goode, 2010; Hargittai, 2010; Huang, Hood, & Yoo, 2012). In order to effectively use technology to improve education, we must investigate how students are currently using technology, what they want from their colleges in the use of technology, how technology impacts educational outcomes, and how these factors differ for different student populations. How do these perceptions of importance vary by different types of students and students in different institutional settings? To what extent has students‘ technology use enabled them to understand, demonstrate their understanding, or study on their own or with others? How does such uses of technology relate to other important forms of educationally effective engagement? This study uses a large-scale dataset to explore these issues and provide information to institutions of higher education to effectively implement greater technology use among students.
Full version

A comparison of student and faculty academic technology use across disciplines
Guidry, K. R., & BrckaLorenz, A.
Association for Institutional Research Annual Forum, Chicago, IL, 2010, May.

Full version

Interactive technology and effective educational practices
BrckaLorenz, A., & Garver, A. K.
Association for Institutional Research Annual Forum, Chicago, IL, 2010, May.

Full version

Gender & racial-ethnic gaps among entering science, technology, engineering and mathematics (STEM) majors
Cole, J. S., & Kinzie, J.
Association for Institutional Research Annual Forum, Seattle, WA, 2008, May.

Full version

Student experiences with information technology and their relationship to other aspects of student engagement
Nelson Laird, T. F., & Kuh, G. D.
Association for Institutional Research Annual Forum, Boston, MA, 2004, June.

Faculty Members are Not the Problem: Improving Faculty Teaching Environments to Foster Teaching Excellence
BrckaLorenz, Allison; Nelson Laird, Tom
Assessment Institute, Indianapolis, IN, 2023, October.
Given challenges with technology, the pressures of the academy, political meddling in higher education, inequitable conditions, and students whose needs are complex and changing rapidly, faculty members find themselves struggling with workload, their own health issues, competing priorities, and how to be effective teachers in a challenging time. Using data from two large-scale multi-institution assessment projects, we invite you to examine with us aspects of faculty teaching environments that contribute to faculty members' success as educators. By using measures of, for example, institutional policies and processes, access to instructional resources, and institutional climates for diversity, we will illustrate how a better understanding of the teaching environment can improve faculty development efforts. Join us for an exchange of ideas about ways to foster environments that motivate teaching excellence and support faculty in both their work and personal lives.
Full version

Examining Black STEM Students' Experiences with Collaborative Learning
Priddie, Christen
American Educational Research Association Annual Meeting, San Diego, CA, 2022, April.
Educators continue to examine how to increase Black student representation in Science, Technology, Engineering, and Mathematics (STEM) fields in higher education. There is an increased call to implement collaborative learning in STEM classrooms, but current implementations do not articulate how collaborative learning can be culturally relevant for Black students. Using a critical approach, the purpose of the current study was to establish knowledge of Black STEM studentsâ?? collaborative learning experiences. Results showed that Black STEM students valued the maintenance of their contributions in collaborative learning while placing little value in being recognized for their collaborative contributions. The current study highlighted how STEM educators should center Black STEM studentsâ?? values and experiences while transforming an active learning practices already being implemented.
Full version

Reforming STEM education: Examining change by discipline, 2004&ndash2012
Kinzie, J., Nelson Laird, T. F., Mack, K., & Shute, C.
AAC&U STEM Conference, Seattle, WA, 2015, November.
Participants will gain awareness of trends in effective educational practices in STEM fields; explore how different STEM fields have or have not been changing; and consider trends as a basis for assessing effectiveness of STEM education reform efforts and what may be needed for ongoing reform. Much of the effort in reforming undergraduate STEM education over the last several decades has focused on improving undergraduate teaching and learning and demonstrating the effectiveness of certain educational practices. This session will highlight results from a current study using time series data from the National Survey of Student Engagement between 2004 and 2012 for a diverse cohort of institutions to assess trends in the use of effective educational practices in particular STEM fields. Discussion will focus on interpreting findings by comparing engagement between STEM and non-STEM and across STEM fields, and investigating trends in relation to reform efforts, including Project Kaleidoscope (PKAL). Results will be presented to encourage audience interaction and interpretation of findings with special attention to differences by field and what this suggests for institution, department, and STEM field transformation in teaching and learning.
Full version

Using the updated NSSE to support evidence-informed improvement and accreditation
Kinzie, J.
Higher Learning Commission Annual Conference, Chicago, IL, 2015, March.
The National Survey of Student Engagement (NSSE) and its affiliated surveys, FSSE and BCSSE, provide a fresh look at engagement, including insights about learning with technology, quantitative reasoning, and learning strategies. This presentation highlights findings, including those from the survey's new Topical Modules, and illustrates effective uses of NSSE results in accreditation as well as approaches to supporting evidence-informed improvement.
Full version

A fresh look at student engagement for accreditation and improvement
Kinzie, J.
Higher Learning Commission Annual Conference, Chicago, IL, 2014, April.
The updated National Survey of Student Engagement (NSSE), and its affiliate surveys, FSSE and BCSSE, provide a fresh look on engagement, including insights about learning with technology, quantitative reasoning, and learning strategies. This session will highlight findings, and demonstrate effective uses of NSSE results in accreditation self-studies and quality improvement.
Full version

Faculty technology use patterns: Comparing HBCUs and PWIs
Shaw, M. D., Cole, E. R., Harris, C. J., & Nelson Laird, T. F.
Association for the Study of Higher Education Annual Conference, Las Vegas, NV, 2012, November.

Full version

National Survey of Student Engagement technology initiatives community update
Arroway, P., BrckaLorenz, A., & Guidry, K. R.
EDUCAUSE Annual Conference, Philadelphia, PA, 2011, October.

Full version

Undergraduate research experience: Intention and doing for science, technology, engineering, and mathematics (STEM) majors
Kormaz, A., Cole, J., & Buckley, J.
American Educational Research Association Annual Meeting, New Orleans, LA, 2011, April.

Full version

Using NSSE and FSSE to link technology to student learning and engagement
Guidry, K. R., Garver, A., & BrckaLorenz, A.
ELI Annual Meeting, Austin, TX, 2010, January.

Full version

Instructional technology: A welcome change?
BrckaLorenz, A., Norris, K., & Palmer, M.
Professional & Organizational Development Conference, Houston, TX, 2009, October.

Full version

Technology and student engagement: Lessons learned from the National Survey of Student Engagement
Nelson Laird, T. F.
IHETS/IPSE All Partners Conference, Indianapolis, IN, 2005, April.

Full version

Rose-Hulman Institute of Technology: Assessing the Effectiveness of Campus Programs and Services for First-Year Students and Seniors
In Engagement insights: Survey findings on the quality of undergraduate education—Annual results 2016, 3.
Full version

Oregon Institute of Technology: Exploring General Education and Learning Outcomes
In Engagement insights: Survey findings on the quality of undergraduate education—Annual results 2016, 4.
Full version

Carlow University: Assessing Learning with Technology to Enhance Instructional Practice
In Engagement insights: Survey findings on the quality of undergraduate education—Annual results 2016, 8.
Full version

Topical Modules: Learning with Technology
In A fresh look at student engagement—Annual results 2013, 22 - 23.
Full version

Online Learners
In Promoting engagement for all students: The imperative to look within—2008 results, 15 - 16.
Full version

Information Technology
In Converting data into action: Expanding the boundaries of institutional improvement—2003 annual report, 17.
Full version

Women in Science, Technology, Engineering and Math
In From promise to progress: How colleges and universities are using student engagement results to improve collegiate quality—2002 annual report, 22.
Full version

Promoting in a Crunch: Increasing NSSE Response Rates
Dajanae Palmer & Bridgette Holmes
February 3, 2020.

Recording

Last minute prep for your NSSE 2014 administration
Cindy Ahonen and Katherine Wheatle
February 21, 2014.

Recording

Last minute prep for your NSSE 2013 administration
Jillian Kinzie, Associate Director, NSSE Institute
January 28, 2013.

Recording