When it comes to updating NSSE items, sometimes it is necessary to consult with outside experts to make sure we are "getting things right" when it comes to the most recent research and terminology. However, we still want to make sure that new items will fit seamlessly within the existing NSSE survey. This invited guest blog describes the process for updating the NSSE disability item that is part of the demographic section of the core survey.
Forming Our Research Team
As part of this project, our team came together as colleagues presenting at conferences and networks within higher education survey research. At the Fall 2019 Annual Association for the Study of Higher Education Conference, Dr. Kirsten Brown and I presented on a panel Conceptualizing disability, discussing the tradeoffs of measuring disability along with the panel organizer Dr. Ellen Broido and Dr. Ryan Wells. During this conference, I ran into colleague Dr. Allison BrckaLorenz and mentioned the panel, and during this conversation Dr. BrckaLorenz invited the group to propose an updated version of the disability item for the National Survey of Student Engagement (NSSE). She also introduced me to Ms. Bridgette Holmes, a current Graduate Research Assistant for NSSE, who is studying Black students with disabilities for her dissertation.
The three of us, Dr. Brown, Ms. Holmes, and myself, began meeting weekly in spring 2020 to further explore how disability is measured via survey research. We followed the work of Dr. Jason Garvey, who studies the way sexual orientation is measured in higher education. Part of our agenda included a proposal to update the current NSSE items measuring disability. In a special issue for the Journal of Postsecondary Education and Disability, I had written about the tradeoffs of these questions (along with the survey items on the Community College Survey of Student Engagement) and had used them in my own research. Meanwhile, Dr. Brown contributed her expertise on the scholarship of disability research in higher education, notably serving as a co-author for Disability in higher education: A social justice approach along with several other works. Ms. Holmes added her familiarity with the survey, and also provided insight into the ways our research can be used for advocacy for students with disabilities.
The National Survey of Student Engagement Disability Items Spring 2020
Proposing Revisions
Our proposal included two major changes. First, our research team wanted to update the stem of the first question. The existing language came from the medical model of disability, including the word "diagnosed." Our criticism of this approach is that it removes the efficacy of the student and puts diagnosis in the hands of a third-party medical professional. Additionally, it is not inclusive for students who may have been diagnosed as a child but have not been evaluated recently, or students who become disabled later in life. Furthermore, our team wanted to expand the options for the follow-up question, which would provide researchers and educators with a better option for disaggregation. Improvements to this item included differentiating between the deaf and blind communities, adding options not previously represented in the survey such as chronic health issues, and differentiating between mental health issues such as depression and anxiety.
Our research team consulted over 20 versions of surveys measuring disability to create a draft of the new items. The draft was forwarded to 7 experts on disability in higher education for feedback before being sent to the NSSE Item Revision Team. While discussing the new item, the team worked through issues such as considering the functionality of one vs. two survey questions related to disability, aligning the grammar in question stem and response options, and consistency with language that is used by disabled communities and scholars on this topic.
Using Data for Policy and Practice
Below are the final survey items that will be included in the spring 2021 NSSE administration. Of course, any updates to a survey item will have tradeoffs such as increasing length or inability to combine multiple years of data, but our hope is that the new version will be more welcoming and inclusive for respondents with disabilities. Our research team also believes this version will provide more nuanced and detailed data for researchers and educators. For example, in the current era, being able to identify students with depression and/or anxiety is critically important.
Furthermore, connecting this aspect of identity to NSSE's measures of student engagement will provide more information on the pathways to success for students. On a campus level, institutional researchers can explore engagement within broad categories of disability through the aggregate NSSE data, and also use their institution's student-level data to learn more about this aspect of identity. On a national level, we believe these items could serve as an example for other survey researchers to better understand this group.
