Involving Youth in Participatory Instrument Review: Successes and Challenges
Background: Every two years, communities across the United States collect data on health behaviors among youth using the CDC's Youth Risk Behavior Survey (YRBS). This lengthy survey collects, among other things, information on demographics, sexual activity and violence, mental health, substance use, and protective factors like relationships with trusted adults. School and community organizations use this up-to-date view of what challenges their students are facing to develop new programming, update policies, and strengthen support networks. Since 2017, MXM Research Group has supported YRBS implementation and analysis for Waltham, MA, in collaboration with the local public school district and a community-based organization, Waltham Partnership for Youth (WPY; see our blog post about the 2019 report). In 2021, MXM and WPY facilitated a deep community engagement process and made major updates to YRBS language.
The Question: Our goal was to help the Waltham community see and understand trends over long stretches of time. As we approached YRBS implementation in 2023, we wanted to respect the changes made for 2021 while acknowledging that language and social constructs are constantly evolving. Whose expertise should we prioritize? Expertise from the deep community engagement of 2021 (including school staff, youth representatives, and community leaders)? Or expertise from the current moment in 2023 (noting that we didn’t have the budget or time to repeat a deep community engagement process)?
“How are you defining expertise and who holds it?” asks one of our Data Equity Deck cards from the Scoping + Design phase.
Data + Soul Approach: When asking youth or community members for their input, it is important to heed their feedback with rigor. Ultimately, we balanced community expertise from 2021 with present-day expertise from 2023 by identifying only a subset of questions that were most likely out-of-date, and facilitating review sessions with two groups of student leaders. Leaning into one of WPY’s core values to “include youth as co-creators of our work, ” students helped us update language on bullying, substance use, and demographics, for example, remove low-priority questions, and add new ones. Importantly, we updated language about “sexual intercourse” to be more generally about “sex,” thus avoiding a heteronormative and narrow assumption about what counts and what doesn’t when responding to YRBS questions. Final decisions about language and questions were made jointly by WPY and MXM because we held a longer view of how data would be useful to address youth health challenges. We communicated decisions back to the student leaders and shared our deep appreciation for their expertise in helping update the 2023 survey.
Result: We believe that our collective efforts fostered more realistic and authentic responses from students such that students could truly see themselves reflected in the YRBS questions and results. However, the updates we made to select questions and the potential changes in student responses related to pandemic stresses made us more cautious in how we interpreted and communicated trends between 2021 and 2023. For example, we were careful to add a caveat when comparing responses about “sexual intercourse” in 2021 to responses about “sex” in 2023. While we believe that survey updates were necessary to honor student expertise, we are reminded that using participatory methods also necessitates more time for analysis and sensemaking. Unfortunately, given our project constraints, we were unable to return to student leaders to help interpret the findings. Regardless, our team and WPY felt confident with the improved precision and fidelity of the survey questions, such that the Waltham community could trust and act on YRBS findings to support the health of the youth they serve.