Our team of faculty and student researchers at California State University, Stanislaus are engaging in an on-going qualitative data analysis of Stanislaus State Student Success indicators using data collected from the National Survey for Student Engagement (NSSE). Stan State developed an applied learning model, where graduate assistants collaborate with faculty to directly apply research methods training and qualitative analytical techniques to an institutional data set. In our on-going research using NSSE data, we examine coded themes using both deductive and inductive analytical approaches for exploring factors that contribute to student success.
We describe this practical and unique approach to data analysis and outline what we have learned about our students.
Students at CSU Stanislaus
California State University, Stanislaus serves a diverse student body of more than 10,000 with two campuses in the Central Valley -- a beautiful 228- acre campus in Turlock and the Stockton Center located in Stockton's historic Magnolia District. At Stanislaus State, 62% of the University's students are Pell-eligible and 73% are first-generation college students. With 43 undergraduate majors, 41 minors, and more than 100 areas of concentration, along with 16 master's degree programs, 7 credential programs, and a doctorate in educational leadership. Stanislaus State is a vibrant place of diversity, with 54% of its students self-identifying as historically underrepresented minorities. As a Hispanic-Serving Institution (HSI) since 2003, nearly 52% of its student body identifies as Hispanic/Latino.
The Faculty Scholar/Student Research Model
To foster broader engagement from students and faculty in institutional research and assessment processes and better understand the needs of our students, the Office of the Provost, in coordination with the Office of Strategic Planning, Enrollment Management, and Innovation, launched a pilot program connecting two faculty scholars with graduate students to review and analyze open-ended institutional survey data using qualitative data analysis software (NViVo). The faculty scholars designed the research questions and methodology for analyzing the open-ended questions. The faculty scholars provided the graduate students (in History and Child Development) with qualitative research methods training. After an overview in research methodology, the students were tasked with mining the qualitative data of student responses to discover factors for student success.
Methods and Analysis
The data for this research study is secondary data from a convenience sample of Stanislaus State students (First-Year Students and Seniors) who completed the open-ended items included in the National Survey for Student Engagement (NSSE) instrument in 2011 (N=122), 2014 (N=81), and 2017 (N=130).
At the end of the survey, students were invited to openly share comments about their learning experiences. Stan State students were given the following prompt: If you have any additional comments or feedback that you’d like to share on the quality of your educational experience, please type them below.
Our coded themes were used in both deductive and inductive analytical approaches for exploring factors that contribute to student success. Graduate students developed an initial coding framework for open-ended responses using free codes. We then re-coded the data using the following engagement indicators (Table 1) to develop a hierarchical coding framework. Some responses fit the NSSE indicators, but others did not. For the latter, we created new codes.
Table 1: NSSE Engagement Indicators
Results from student open-ended responses tended to reflect codes/indicators such as Student/Faculty Interaction, Supportive Environment, and Quality of Campus Interactions. Codes/Indicators like Academic Challenge, and Learning with Peers were coded less often because students mentioned these topics less often. New codes that were mentioned more often were Career Statements, Student Life, Financial Cost, Lack of Available Classes, and Academic Advising statements. Additional codes were discovered that may relate to engagement and student success, but did not fit with the existing NSSE indicators.
We identified six indicators through our qualitative analysis and compared these indicators among cohorts:
(1) Student success and satisfaction from participation in educationally purposeful activities at Stanislaus State,
(2) Stanislaus State requirements and the enriching and rigorous nature of coursework,
(3) Student attitudes and perceptions of the college environment at Stanislaus State for facilitating graduation and retention rates,
(4) Thematic and coded estimates of educational and personal growth since starting college among Stanislaus State freshmen and graduating senior students,
(5) Descriptive statistics of background and demographic information among Stanislaus State students, and
(6) Attitudes towards High Impact Practices that promote equity, diversity, and quality of the learning among Stanislaus State students.
What Did We Learn?
The qualitative data from NSSE helped us build student focus groups around the six coded themes and then conduct more in-depth exploration of the strongest themes identified in the open-ended responses. Data from the qualitative analysis and follow-up results from the focus groups are being discussed by the University's Faculty Learning Communities and Committees that are tasked with improving student graduation rates. Through the Graduation Rate Excellence and Assessment Team (GREAT) the NSSE data will allow administrators and faculty to understand the challenges and obstacles that students are facing with graduating in four years.
GREAT is currently using the NSSE data to promote programming for students and faculty that focuses on building relationships and creating an environment that facilitates equity, inclusiveness, and social justice practices. For example, the open-ended data is being used to inform faculty and administrators about the degree of student engagement in college activities that correlate to student learning and personal development. Also, the open-ended responses are helping to inform our Assessment Team on class participation, academic rigor, amount of effort being put into class work and overall experience. All of this data is being collected and analyzed to aid in the development of evidence based High-Impact Practices that will better serve our student body.
Findings will continue to be reviewed, as more cohorts are surveyed, to understand which practices, programming, and opportunities may be essential elements contributing to student success over time. We successfully presented our findings to the University during a meeting with the Stan State GREAT Team and Stan State Provost Office and we received considerable feedback and encouragement to continue examining Student Open-Ended Responses using qualitative analysis. Finally, we hope that the Faculty Scholar/student research model may be adopted by other campuses or divisions to directly engage both students and faculty in institutional research and assessment processes.
For more information on NSSE’s Student Comments, click here!