Inevitably, if you work with survey data, you have noticed some very small subgroups in your data. These small subpopulations might exist for a number of reasons at your institution, but many times they are historically marginalized groups, such as students of color or LGBTQ students. The way we analyze and interpret the results of small populations is important, because we want to ensure we are not further contributing to the marginalization of these groups at our institutions.
Over the last few years, we have received questions from a number of institutions wanting to better manage their small population data. Some of the questions included:
- How do we analyze subgroups with very few responses?
- How do we better identify the needs and experience of students from underrepresented backgrounds?
- How do we better share these data and results with others on campus?
In an effort to better support your ability to be more inclusive in data sharing and analysis, we have developed a resource with a handful of tips to get you started. We hope whether you are presenting data and reports internally or sharing research at conferences that these tips will be useful to you.
The full resource can be found on our website, but here's a quick overview of the tips:
- Disaggregate your data. Aggregated data can mask variation, and give a perception of students' common experiences that is based on the institution's majority populations.
- Pay attention to small populations. Avoid the tendency to disregard small groups of students. They need special consideration at every step of assessment: before, during, and after data collection.
- Consider your framework. Make sure to take time to select a framework to help answer your research questions and ensure it does not perpetuate a deficit perspective about marginalized groups.
- Rethink comparisons and reference groups. Be careful to not reinforce that certain groups (White, straight, or cisgender students, etc.) and their experiences are the standard to which all other groups must be compared. Remember, too, that percentage differences, effect sizes, and descriptive statistics are legitimate forms of analysis.
- Aggregate responsibly. There are some instances when disaggregating data might actually cause more harm, because the data becomes identifiable. If you need to aggregate to keep students' confidentiality, make sure you are transparent with how and why you made this choice.
This resource was recently shared at the Association for Institutional Research Annual Forum and the Student Affairs Assessment and Research Conference. In these sessions, the presenters reviewed each of the tips in more detail and offered examples of application.
If you have questions about applying these tips, do not hesitate to let us know! Or, if you have an example of how you have analyzed and interpreted data from small populations, we would love to hear it. Feel free to contact us at nsse@indiana.edu.