Research and analysis

Uniformed Youth Fund evaluation reports: impact evaluation

Published 11 September 2025

Applies to England

Acknowledgments

This evaluation work relating to the Uniformed Youth Fund would not have been possible without the support and contributions of many individuals and organisations. We would like to extend our sincere gratitude to all who have assisted in this important work.

Department for Culture, Media and Sport (DCMS)

We are grateful for the support from the DCMS for funding and guiding this evaluation and for their commitment to understanding how activities such as the uniformed youth groups impact on young people’s lives and well-being. Specific thanks to: Sam Burthem, Natasha Fuyane, Hedvig FribergJonsson, Jake McBride, Kirby Swales, Anya Valli, and Michaela Wragg.

Participating uniformed youth organisations

We extend our deepest thanks to the eight uniformed youth organisations that participated in this evaluation. Their willingness to facilitate fieldwork was crucial to the success of this project. We are especially grateful to the staff and volunteers who took the time to participate in interviews and focus groups, providing invaluable insights into the delivery of the fund and its impact on the ground.

Youth Advisory Board

A special thank you goes to the members of the Youth Advisory Board. Those who wished to be named here included: Jerusalem Edward Amaeshi, Katie Ellis, May Hussain, Sharlize Munro-Labuda, Maisie Pollard, Kaivan Shah, Jason Tao. This group of dedicated young people, drawn from a range of uniformed youth groups, provided essential feedback on our data collection tools and materials, ensuring they were accessible and relevant for young people. Their contribution has been instrumental in shaping a youth-informed evaluation.

Young people

We would also like to thank the young people who shared their experiences and personal journeys with us. We are grateful for their trust in us to represent their views accurately.

Dartington Service Design Lab

We would like to acknowledge our partners at the Dartington Service Design Lab: Rachel Bromley, Leanne Freeman, Finlay Green, Julie Harris, Sean Manzi, Cristina Preece, Zhenni Qin, Katie Upsdale, and Hannah Wilson. Their expertise in youth-centred research and their work in establishing and collaborating with the Youth Advisory Board have been invaluable to this evaluation.

Ipsos UK colleagues

Finally, we would like to thank our colleagues at Ipsos UK who were instrumental in the successful delivery of this project. We are grateful for the hard work and dedication of the data collection and analysis teams: Grace Atkins, Marzieh Azarbadegan, Sally Barber, Kim Bohling, Raynette Bierman, Akshay Choudhary, Evie Cogley, Catherine Crick, Catherine Fenton, Facundo Herrera, Joel Hooper, Jessica Ozan, Alex Pangalos, Kristen Taylor and Jack Watson.

Executive summary

About the evaluation

Ipsos UK and Dartington Service Design Lab were appointed by the Department for Culture, Media and Sport (DCMS) to deliver an evaluation to understand the impact of Uniformed Youth Organisation (UYO) participation on young people. UYOs provide opportunities for young people to take part in structured activities to build skills and relationships, often while wearing a uniform.

The evaluation was conducted as part of the Uniformed Youth Fund (the Fund), which provided funding to eight UYOs to increase capacity and opportunities for young people, especially in areas with unmet demand and disadvantaged areas. A separate report has been published on how the Fund was delivered and the extent to which it achieved its objectives to expand opportunities.[1]

Evaluation aims and methods

The evaluation focused on the following research questions, exploring impacts on young people:

1. What was the impact of participating in uniformed youth groups on social and emotional skills, practical skills and wellbeing?

2. Did these impacts vary by the types of activities delivered?

3. What was the impact of participating in uniformed youth groups on educational attainment?

The impact evaluation used a mixed-methods approach, using both a quasi-experimental design (QED) and a theory-based contribution analysis, to assess the impact of UYO participation on young people. The QED compared young people who are members of UYOs with a weighted comparison group of young people who are not. Data for the QED was collected through the National Pupil Database (NPD) and two rounds of surveys of young people. A total of 1,179 young people completed the surveys at both time points, between eight and 11 months apart. The contribution analysis drew upon the survey data, interviews with young people (n=31 at two time points) and eight in-depth case studies. A Youth Advisory Board (YAB) was created to enhance the research design and incorporate young people’s voices in the evaluation design and delivery.

Findings

The findings from the contribution analysis and QED are discussed under four overarching themes of: wellbeing, socio-emotional development, skills and attainment, and community connectedness. In our QED analysis, we tested for differences in the outcomes across UYO members and the weighted comparison group at both time points. Where we observed that outcomes were consistent across both time points, we only reported outcomes from wave 2 to streamline findings. Where meaningful differences were observed across the two survey waves, these have been included in the reporting.

Wellbeing

Because it’s something I do regularly, it’s something I can always look forward to. And it always makes me happy. I love being around these people.

(St John Ambulance member)

There was strong evidence to support the claim that UYOs bolster members’ wellbeing through supportive and inclusive environments where young people feel safe, respected, and accepted. Among survey respondents, 97% agreed they felt safe at their group, and 89% agreed they felt supported. Within these supportive environments, young people often developed trusted relationships with both peers and adult leaders.

Members’ happiness and wellbeing are also supported through participating in a range of activities like sports, outdoors adventures, and arts and crafts. These activities offer new experiences, promote physical health, and provide a relaxed space for creativity, displacing more passive activities. Most survey respondents (93%) agreed that they like the activities that their group does. Some participants are involved in other positive activities, such as organised sports, choir and other youth groups, but UYO membership offers a beneficial complement and/or alternative to these. The QED analysis suggests there is a ‘protective effect’ of UYO membership on wellbeing. That is, the survey analysis shows higher wellbeing outcomes among UYO members, which were also stable across both survey time points. There were lower levels of wellbeing and a decline between the two time points among the comparison group.

Social and emotional development

Because you learn a lot of teamwork skills [doing UYO activities]. Everyone eventually will be able to lead a team, but also you have to learn how to listen to someone.

(Boys’ Brigade, boy, 13)

The evidence supporting the positive impact of UYO membership on social and emotional development in young people is strong. UYOs contribute towards fostering this development through structured activities and supportive environments. Spending time with peers in team-building exercises enhances cooperation skills, communication, and confidence. QED results from both survey time points revealed that UYO members self-reported higher levels of empathy, responsibility, teamwork, problem-solving and initiative compared to non-members. Notably, UYO members scored 10-percentage points higher in teamwork (83% vs 72%, p<0.001) and 8-percentage points higher in responsibility (78% vs 70%, p=0.010) at the second survey time point. The impact of UYO membership on socio-emotional development interestingly varies by socio-economic status. Young people from more deprived areas show particularly strong differences in responsibility (81% among members vs 70% in the comparison group, 11-percentage points, p=0.02) compared to those from less deprived areas (75% vs 70%, 5-percentage points). A similar pattern appears for empathy, with greater benefits for members from deprived areas (75% vs 66%, 10-percentage points, p=0.05) compared to less deprived areas (82% vs 78%, 4-percentage points).

Skills and attainment

It’s teamwork, teambuilding, leadership. Skills you don’t really get in school. That’s what cadets is – skills beyond school, learning life skills.

(Volunteer, Volunteer Police Cadets)

UYOs enhance education and career opportunities for young people by providing valuable life and work skills through diverse activities. These activities help participants develop essential skills such as first aid, communication, leadership, teamwork, problem-solving, time management, and community engagement. While formal qualifications are not predominantly gained through these organisations, the informal achievements, like badges, provide a sense of validation and achievement, boosting further skill acquisition and personal development.

Some young people reported enhanced motivation and engagement with their current studies due to the skills learned through UYO activities. The QED analysis found that UYO members averaged 69.6 points compared to 64.2 points for the comparison group – a difference of 5.4 points (p=0.00). However. these findings are from a small subset of the analytical sample (n=247), so these findings should be interpreted with caution.

The contribution of UYOs on young people’s readiness for the labour market was evident in groups with direct career links (such as Volunteer Police Cadets) and among older participants. These groups provide skills that are increasingly recognised and valued as compared to skills typically developed in conventional educational settings. This makes UYOs a valuable component in youth development, especially in preparing them for future challenges both educationally and professionally.

Community connectedness

I feel like the uniform for any of the uniformed groups empowers you…you feel like you belong in the uniform, and you have a sense of responsibility.

(St John Ambulance, girl, 15)

There was strong evidence to support the claim that UYOs foster a sense of community connectedness among young people through uniform and shared experiences. The participants clearly stated a range of positive emotions linked to their sense of belonging and collective identity associated with the wearing of the uniform. The uniform itself became a representation of this identity within which there were shared values and a social structure present. Some participants did allude to also having an identity and sense of belonging associated with non-UYOs such as sports groups and religious communities in which they participated.

The QED analysis from both survey time points found significant differences between UYO members and the comparison group in terms of their role in their community. At the second survey time point, we found that 61% of UYO members agreed with feeling like a valued member of their community, 16-percentage points higher than the comparison group (p<0.01). In more deprived communities, this difference widens, suggesting a larger effect for these members. Although a slightly smaller percentage of young people (58%) felt valued, this was 20-percentage points higher than the comparison group (p<0.01).

There was evidence of a moderate strength for the role of UYOs encouraging and reinforcing social action and civic participation, especially in terms of helping behaviours and engagement in volunteering. For many members, their civic participation was limited to the activities within their group. Among survey respondents, 24% (n= 98) indicated they did voluntary work outside of their UYO, which was the same as the comparison group of non-members (24%, n=187).

Conclusion and recommendations

The evaluation findings align with a growing body of research demonstrating the positive effects of uniformed youth group participation, particularly on non-cognitive outcomes such as self-confidence, teamwork, and empathy. To maximise the positive impact of UYOs, several recommendations are proposed:

1. Targeted support should be provided to UYOs to prioritise outreach and activities for disadvantaged youths, addressing financial and logistical barriers to participation. This is supported by evidence indicating that UYO membership can be especially beneficial for young people from disadvantaged backgrounds, supporting their socio-emotional development. The UYF had a particular focus on increasing opportunities in areas of socioeconomic deprivation, creating more than 13,000 new spaces in these communities.

2. UYOs should focus on promoting long-term engagement through leadership opportunities and advanced training programmes to foster skills development. The evaluation found that skills like responsibility and teamwork appeared stronger among young people who had been members for longer lengths of time.

3. The focus for UYOs should remain distinct from the remit of schools with regards to raising attainment. Young people frequently spoke of appreciating that UYOs felt very different from their school experience in that they had different activities, developed different skills and relationships, and viewed it as a space to unwind from school. The evidence suggests that this separation may be very important for realising the non-attainment outcomes, such as communication skills and wellbeing, which may in turn contribute to improved attainment in the long-term.

4. The overarching theory of change and contribution story indicates that a few key mechanisms are particularly important for improving outcomes for young people: creating supportive and inclusive environments, delivering varied and enjoyable activities, providing opportunities to build trusted relationships with adults and peers, and enabling members to take on leadership responsibilities. This evaluation suggests that UYOs are largely effectively delivering there mechanisms, but more detailed and programme-specific reviews may be beneficial.

5. Investment in volunteer training and support is crucial to ensure high-quality activities and positive relationships with young people. The evaluation found clear linkages between the UYO environment and activities and members’ wellbeing and development of social and emotional skills. Well-trained staff and volunteers are crucial to fostering these positive spaces and opportunities.

6. Further research, including longitudinal studies, is needed to better understand the long-term impact of UYO participation on educational attainment, career pathways, and resilience. This would help address some of the limitations of the current evaluation.

1. Introduction

Ipsos UK and Dartington Service Design Lab were appointed by the Department for Culture, Media and Sport (DCMS) to deliver evaluation work related to the Uniformed Youth Fund (the Fund). The evaluation findings are divided into two reports as there were two distinct lines of inquiry. This report explores the impact of participating in uniformed youth groups on young people. A separate report provides a process evaluation of the Fund, including more detail on the context and rationale of the Fund.

1.1 Overview of the Uniformed Youth Fund

The Uniformed Youth Fund is an £18.3 million investment to expand access to uniformed youth groups. As part of the National Youth Guarantee[2], the Fund aimed to improve the capacity to UYOs to increase the number of opportunities young people have to engage in regular clubs and activities, adventures away from home, and opportunities to volunteer. The ultimate aim of this funding was to improve the wellbeing of young people and support their development of life and work skills by expanding the reach and range of activities for young people that drive these outcomes.

Eight Uniformed Youth Organisations (UYOs) received funding: Boys’ Brigade, Girls’ Brigade, Girlguiding, Jewish Lads’ and Girls’ Brigade (JLGB), Scouts, Sea Cadets, St John Ambulance, and Volunteer Police Cadets. Uniformed youth groups are run by thousands of adult volunteers, and activities can include in-person sessions, camps, international trips, and community-based social projects. Each UYO has its own programme which usually includes skills development, accreditations (such as the Duke of Edinburgh’s Award, first aid certificate, and BTEC (Business and Technology Education Council) qualifications)) and social action (e.g. ‘when people give their time and other resources for the common good, such as volunteering, community-owned services, community organising, and neighbourly acts’)[3]. There are also accreditations and programmes that are unique to the UYOs’ background and remit – for example, Marine Society and Sea Cadets offer accreditations in British Canoeing Union and Royal Yachting Association.

Previous research has shown that participation in UYOs leads to a variety of positive outcomes for young people. A quasi-experimental study (using difference-in-difference) found that participation in UYOs led to improvements in life skills and character traits, such as communication skills, empathy, and resilience[4]. Research by the Youth United Foundation in 2018 found that UYOs encourage social mixing among young people[5]. The report, based on an online survey of uniformed and non-uniformed young people (n=2015, non-experimental) and 52 qualitative interviews with young people, their parents/guardians, and volunteer group leaders, found that UYO members are at least 10-percentage points more likely than non-UYO members to say that they spent time with people of a different ethnicity, socio-economic status, or religion.

This is useful evidence in support of young people engaging with UYOs, but the evidence base is still quite limited and has notable gaps, including sufficient detail on the key activities and mechanisms that bring about change for young people, as well as impacts on academic attainment. This evaluation aims to further bolster the evidence base and begin to address some of these gaps.

1.2 Impact evaluation aims and questions

The impact evaluation was not focused on the impact of the Fund but had a broader remit to create good-quality evidence about the benefits young people get from being part of a uniformed youth group – albeit, primarily focusing on only those UYOs that were supported by the Fund.

The key questions were:

1. What was the impact of participating in uniformed youth groups on social and emotional skills, practical skills and wellbeing?

2. Did these impacts vary by the types of activities delivered?

3. What was the impact of participating in uniformed youth groups on educational attainment?

These questions were addressed through a mixed-methods evaluation approach described in more detail in the next section.

2. Methods

This section contains a brief description of the impact evaluation methodology and its limitations. It also provides an overview of the evidence sources used. More detail is available in the Annexes.

2.1 Evaluation methodology

The evaluation had three main elements used to identify UYO participation impacts on young people: development of an overarching Theory of Change (ToC), a quasi-experimental counterfactual analysis[6], and a theory-based contribution analysis. Alongside these activities, a Youth Advisory Board (YAB) was created to enhance the design of the research and include young people’s voice.

2.1.1 Youth Advisory Board

Twelve young people aged 16-18 from across the funded UYOs were recruited to take part in the YAB. They took part in seven workshops throughout the evaluation to provide feedback on key aspects of the research including survey design, interview approaches, and accessibility of final reporting. More detail on their contribution to the research is detailed in Annex 5.3.

2.1.2 Overarching Theory of Change

The Fund supported eight UYOs, and while all had certain elements in common, there was also variation in programme aims and activities across the organisations. In order to answer the identified research questions, an initial shared ToC was needed to cover the range of activities, mechanisms of change, and potential outcomes across the funded UYOs. The overarching ToC was created through a series of workshops with staff members from each of the UYOs. It was then developed and refined by the evaluation team to capture the key causal pathways of interest to the impact evaluation. Therefore, it does not represent the full range of diverse outcomes for young people across the UYOs. The YAB provided final input, including identifying priority outcomes and their hypothesised causal pathway. The initial draft is presented in section 3.1.

2.1.3 Quasi-experimental analysis

The evaluation employed a quasi-experimental design (QED) to estimate whether there was a causal effect between UYO membership and the outcomes of interest and the magnitude of this effect. Several other approaches were considered (e.g. waitlist control, regression discontinuity and difference-in-difference) but proved unfeasible given the study’s constraints. More detail on the other approaches considered and reasons for rejection are provided in Annex 5.3.

Our approach compared young people who were members of UYOs with a carefully selected comparison group of young people who were not. To make this comparison fair, we used statistical techniques to rebalance our comparison group so it closely resembled UYO members in key characteristics like age, ethnicity, gender, and socio-economic background. This rebalancing process helps us isolate the true impact of UYO membership from other factors that might influence outcomes. We applied a robust analytical method that accounts for potential biases in two ways: by weighting the comparison group to better resemble UYO members, and by modelling outcomes separately for each group. Data came from two survey waves covering both members and non-members, plus administrative records from the Department for Education’s National Pupil Database (NPD). Crucially, we did not have baseline data from before young people joined UYOs, so our two survey waves capture outcomes at different points after membership began rather than measuring change from before to after joining. This approach allows us to test whether our findings remain consistent across different measurement points, strengthening confidence in the results. The chosen method maximises our use of available data whilst maintaining analytical rigour. Further technical details appear in Annex 6.

2.1.4 Theory-based contribution analysis

To complement the QED and allow for exploration of outcomes that could not be easily quantitatively measured, the evaluation also used contribution analysis, which brought together quantitative and qualitative data. Contribution analysis is a theory-based impact evaluation method that provides a robust framework for systematically assessing whether an intervention (UYO membership) contributed to a set of expected outcomes (identified in the overarching ToC). It does so by articulating and testing a set of ‘contribution claims’ about how UYO membership is anticipated to contribute to the intended outcomes relative to ‘alternative explanations’ e.g. other interventions or activities contributing to similar outcomes. More detail on the contribution claims and alternative explanations are provided in section 3.1 and Annex 5.3.

Together, the contribution of UYO membership and other factors form a causal package that lead to an outcome. This recognises that outcomes are often the result of a combination of causal factors. It was therefore important to determine how necessary UYO membership was for the observed outcomes. This process included assessing whether the evidence supported or refuted the contribution claim, and whether evidence of changes in outcomes could be directly or indirectly linked to UYO membership or an alternative explanation.

To assess the evidence gathered for each contribution claim, the analysis accounted for several factors that informed judgments about the strength of UYO membership’s contribution to outcomes (contribution strength rating) and the level of confidence in the evidence gathered (evidence confidence rating). These were primarily based on whether evidence was:

• reported directly or indirectly – more weight was placed on evidence collected directly from young people about their outcomes.
• consistent across multiple sources – more confidence was placed on findings where evidence converged across sources, for example, both interviews and surveys with young people.
• weakened by methodological limitations – ratings were tempered where there were potential weaknesses in the data (see section 2.3).

Both contribution strength and confidence were assessed on a 1-3 scale indicating weak, moderate, or strong support.

2.2 Evidence Sources

The quasi-experimental analysis and contribution analysis drew on four sources of data, which are shown in Table 2.1 and detailed further below.

Table 2.1: Summary of evaluation methods

	Survey	National Pupil Database	Young people interviews	Case studies
Quasi-experimental analysis	Method used	Method used	Method not used	Method not used
Contribution analysis	Method used	Method used	Method used	Method used

2.2.2 Survey of young people

Two rounds of surveys of young people were conducted, which was the primary data collection for the QED. The survey group included both young people in UYOs (the ‘treated’ group) and those who were not (the ‘untreated’ comparison group).

Content

The survey included questions to assess differences in outcomes between the two groups. The outcomes assessed included:

• wellbeing, assessed using the Office for National Statistics’ (ONS) scale for measuring personal wellbeing, known as the ONS4[7]
• resilience, assessed using the Brief Resilience Scale (BRS)[8], which measures an individual’s ability to recover from stress and adversity
• socio-emotional skills, assessed using selected statements from the Young People’s Survey (YPS)[9]
• community engagement and belonging, assessed using three distinct self-report items focused on collective efficacy, self-efficacy for community action, and community connectedness.

It also included questions about other groups and activities the young people take part in. For the ‘treated’ group (explained in next section), there were questions about length of time in their UYO, frequency of attendance, and activities undertaken in their UYO. Annex 6.3 provides more details on the outcome measures.

Recruitment and administration

The ‘treated’ respondents were recruited via central staff teams of UYOs who directed parents to an online consent form. Parents provided their consent and facilitated their child(ren)’s access to the survey invitation. The comparison group was recruited via a random sample of young people in the NPD. Letters were posted to parents, which asked them to pass the survey link onto their child if they were happy for them to take part.

The survey included a question for all participants about whether they were a member of a UYO and which one[10]. If a respondent indicated they were part of any UYO (regardless of whether it was receiving funding),[11] they were included in the ‘treatment’ group[12]. This means that impact findings relate to the UYO sector more broadly, not just those that were part of the Fund. Survey respondents were aged 10-18 at the point of recruitment in the 2022/23 school year in line with the Fund aims.

Surveys were administered at two points - wave 1 ran February to April 2024 and wave 2 took place November 2024 to January 2025. This means young people may have had between eight and 11 months between completing the surveys, as well as a change in their year at school. This was deemed sufficient to both capture meaningful changes in experience and outcomes, as well as assess which outcomes were stable over time. For our analysis in this report, we only included those respondents who completed both surveys, which gave us a total sample of 1,179 young people - 34% (n=406) in the treated group and 66% (n=773) in the comparison group. Some demographic data and outcomes have missing data, so reported sample sizes for some analyses in this report may not total 1,179.

In our analysis, we tested the outcomes across both time points to see if they were sensitive to time. Where meaningful differences were observed between time points, these have been included in the reporting. Where we observed that outcomes were consistent across both survey time points, the report focuses only on outcomes from wave 2 to streamline the findings. Wave 2 data was chosen, as it represented the maximum amount of treatment time.

Respondent demographics

Tables 2.2 and 2.3 show the breakdown of demographics across the analysed sample, UYO members and the comparison group. The sample’s average age was approximately 15, ranging from 10 to 19. Girls comprised about 60% of the sample. Individuals of White ethnicity constituted approximately 75% of the sample. The sample included 43% of participants who lived in an area of deprivation (based on the English Index of Multiple Deprivation (IMD), deciles 1-5 refer to areas with higher levels of deprivation). This study was designed for internal validity (i.e., to understand impact using a comparison group). It was not designed for external validity (i.e., to be representative of the general population of UYO members and/or young people), so we have intentionally not made any comparisons with national averages, as we did not recruit a sample that reflected the national population.

Table 2.2: Survey sample characteristics - gender, age, ethnicity and IMD Decile

		Total	UYO Member	Non-UYO Member
Gender (n=1143)	Girl or woman	60%	57%	62%
	Boy or man	39%	42%	38%
	Non-binary	1%	1%	0%
Age (n=1118)	10-14	40%	49%	36%
	15-19	60%	51%	64%
Ethnicity (n=1179)	White	75%	86%	70%
	Asian or Asian British	11%	6%	14%
	Black, Black British, African or Caribbean	5%	3%	7%
	Mixed or multiple ethnic groups	5%	3%	6%
	Other ethnic group	2%	1%	3%
IMD Decile (n=1107)	1-5	43%	41%	44%
	6-10	57%	59%	56%

Table 2.3: Survey sample characteristics - region[13]

	Total	UYO Member	Non-UYO Member
Region (n=1178)	East England	10%	10%	10%
	East Midlands	9%	10%	8%
	Greater London	11%	9%	12%
	North East	8%	9%	8%
	North West	11%	12%	10%
	South East	11%	9%	11%
	South West	10%	6%	14%
	West Midlands	9%	10%	8%
	Scotland	0%	1%	0%
	Wales	1%	1%	1%
	Prefer not to say	20%	17%	22%

2.2.3 National Pupil Database

Data was also collected through the NPD to estimate impact on attainment outcomes. The NPD is a comprehensive administrative dataset maintained by the Department for Education (DfE) that contains educational records for all pupils in state schools in England, including exam results, attendance, and demographic information. We used this dataset to access participants’ Attainment 8 scores, which is one of the UK’s standard measures of secondary school achievement.

All survey respondents were given the opportunity to consent to having their survey data matched to the NPD. Overall, 56.9% (n=231) of treated respondents and 66.7% (n=515) of comparison respondents consented to data matching and were successfully matched for a total sample of n=746.

2.2.4 Interviews with young people

In-depth interviews were conducted with young people in UYOs that received funding. A longitudinal interview design was employed with 36 participants to capture changes over time. The two rounds of interviews were conducted in May/June 2024 and November/December 2024, resulting in 67 data interactions. Participants were sampled from the survey respondents who consented to being contacted about the interviews. The primary sampling criterion was UYO, aiming for four members from each. Recruitment also aimed to achieve balance in region, age, and gender. Details on the interviewed sample are presented in Table 2.4 and Annex 5.3.

The interviews used two age-appropriate semi-structured topic guides, one for 10-14-year-olds and another for 15-18-year-olds. The topic guides were informed by the YAB and findings from the case studies. Questions focused on how being a member of their respective UYO had influenced their confidence levels, wellbeing, mental health, social skills, practical skills, and opportunities. The guides also explored how and in what ways these changes had taken place (i.e. the key causal mechanisms), and the extent to which these changes were a direct result of UYO membership.

Table 2.4: Interview sample characteristics - UYO, gender, and age

		Interview 1	Interview 2
UYO	Boy’s Brigade	3	2
	Girl’s Brigade	4	4
	Girlguiding	6	6
	Jewish Lads’ and Girls’ Brigade (JLGB)	3	3
	Marine Society and Sea Cadets	5	5
	Scouts	6	5
	St John Ambulance	4	3
	Volunteer Police Cadets	5	3
Gender	Girl or woman	19	17
	Boy or man	15	12
	Non-binary or gender questioning	2	2
Age	10-12	14	13
	13-15	10	8
	16 -18	12	10
Total		36	31

2.2.5 Case Studies

Eight in-depth case studies were conducted – one with each UYO. Case study locations were purposively selected with support from the central team at each UYO with an aim to conduct the case studies across a range of geographical areas, unit sizes, and lengths of time in operation. The case studies were not intended to provide a representative sample of local UYO units. Rather, the case studies were used to develop the contextual framework for the contribution analysis by providing rich insight into how activities are delivered on the ground and how young people are engaged with and impacted by these activities within the context of each UYO.

Each case study involved an in-person visit to a local unit of the UYO and resulted in a total of 88 interviews with staff, volunteers, members, and wider stakeholders (see Annex 5.3 for detail) and an observation of young people participating in an UYO activity.

Semi-structured topic guides were used as the base for the interviews, though these on-site interviews (especially those with young people) were more informal than the interviews conducted outside the case studies. Interviews with adults explored the type of young people that the UYO reaches in their area, the activities they deliver and why, and perceived outcomes and impacts for young people. Topic guides for young people included questions on why they joined, the types of activities members take part in at the UYO, what young people enjoy the most about UYOs, what could be improved, and how the UYO involvement impacts their social and emotional skills and longer-term outcomes.

2.3 Evaluation limitations

The evaluation had a number of limitations, detailed below.

• Lack of pre-treatment baseline data limits our ability to account fully for unobservable characteristics, which are personal traits or circumstances that we cannot measure but might influence both who joins UYOs and the outcomes from attending and taking part in a UYO. This means we might be comparing young people who differ in ways we have not measured, such as motivation levels or family support, which could affect our conclusions about the impact of being part of a UYO. This limitation stems from the practical challenges of studying youth organisations: young people often have irregular attendance patterns, moving in and out of membership, making it difficult to establish clear starting points. Additionally, whilst we collected self-reported start dates and membership duration, these may face recollection issues and do not provide the precise timing needed for robust baseline (pre-treatment) comparison. While our weighting approach addresses many observable selection factors, true baseline measurements would strengthen causal claims by allowing us to track changes from before young people joined.

• UYO membership is not static, and participants moved from being UYO members to not and vice versa, even within the relatively short time frame between survey waves. This means that defining the treatment status of participants was more complicated, and we know from the survey data that a large proportion of the comparison group (467 out of 744) had previously been a member of a UYO. Due to limited knowledge about when this occurred or the length/intensity of participation, these individuals remained part of the comparison group. However, this raised concerns that recent treatment exposure could dilute observed effects. That is, if former UYO members in the comparison group had benefited from participation and maintained those benefits, then the overall comparison group would have higher average outcomes, and we would be underestimating the true difference between treated and untreated young people. To address this limitation, we conducted robustness checks comparing outcomes between comparison group participants with and without previous UYO membership. These analyses found no statistically significant differences across all measured outcomes, suggesting that prior membership does not materially affect our treatment effect estimates. Whilst this provides reassurance about our findings, the possibility of some dilution effect cannot be entirely ruled out.

• Efforts were made to shorten the survey length as far as possible to support engagement, for example, one statement per sub-domain was selected from the YPS. This means that some outcomes measure tools were potentially less rigorous. Furthermore, when it comes to measuring resilience, there is no ‘gold standard’ tool. The BRS was selected, as it was highly rated in terms of psychometric properties and was short with just six items (other scales had 25+ items)[14]. However, it was not explicitly developed for a pre-teen/teenage population, so it may not have accurately measured resilience in this age group.

• The findings from case studies are likely prone to positivity bias. The groups identified for case studies were selected by senior leaders within each UYO, and it is therefore possible that groups with more positive experiences were selected. Equally, participants may have felt less comfortable speaking about the disadvantages or ways to improve the UYO with staff/volunteers present. To mitigate this, case study data has been triangulated with interview and survey data.

• While our weighted regression-adjusted approach creates a stronger counterfactual than simple regression, we cannot control for unobservable factors affecting both programme participation and outcomes. Put simply, we are working to minimise bias - systematic errors that could skew our results - rather than eliminate it entirely. Think of bias as contamination in our findings: our statistical techniques clean away much of this contamination by accounting for measurable differences between groups, giving us a much purer estimate of the programme’s true impact. However, some bias may remain because we cannot measure everything that influences who joins UYOs and their outcomes, such as personal motivation or family attitudes. This limitation means our findings should be interpreted as suggesting causal effects rather than conclusively establishing them.

3. Findings

3.1 Introductions

In this section, we present the findings from the impact evaluation. The findings from the contribution analysis, survey and case studies are discussed in relation to each other under the four overarching themes of: wellbeing, social emotional development, skills and attainment, and community connectedness. These themes represent the foci of the contribution claims, which are themselves derived from the overarching ToC. Below, each thematic findings section presents the contribution story as originally conceived, then provides an updated claim that accounts for the evaluation findings. The findings are then described and a summary of the evidence for each claim is given alongside consideration of the alternative explanations.

3.1.1 Theory of Change

The ToC is presented in Figure 3.1, and a condensed narrative of the theory is presented below. More detail on the ToC and the process for its development is presented in Annex 5.2.1.

• Inputs

Across UYOs, inputs included funding, volunteer time and skills, facilities or hubs located in the community, uniforms, skilled and knowledgeable staff, and partnerships with other organisations, stakeholders, and community members.

• Activities

UYOs offer a wide range of activities for their members. These include leadership programmes and award structures, training for volunteers and young people, and residentials or stays away from home. Many UYOs provide opportunities for young people to engage in youth social action (e.g. volunteering), sports and outdoor activities, arts and crafts, and other activities that aim to promote positive mental and physical health. They also offer specific activities tailored to their organisation’s mission, such as first aid for St John Ambulance or a faith focus for JLGB.

• Mechanisms of change

For the uniformed youth group activities to be effective and achieve the intended outcomes, the following mechanisms of change were identified as having to be in place:

• The UYOs offer a wide variety of fun and accessible activities that provide young people with opportunities to learn and grow in a supportive environment.
• The UYOs encourage young people to take self-directed action, which empowers them to make positive changes in their communities, develop leadership skills, and make informed choices.
• Within UYOs, young people have a consistent emotionally and physically safe, yet challenging, space where they can experience failure and develop resilience.
• UYOs provide a nurturing, affirming, and inclusive community for young people to build trusted relationships with their peers, staff, and communities.

• UYOs support a sense of collective identity among their members through the use of a uniform and shared experiences. This collective identity promotes a sense of belonging and unity within the group.

• Overarching outcomes

Based on the development of the ToC, UYO membership was expected to contribute to three main outcome domains, though it is important to recognise that outcomes are heavily interwoven with one another.

• Wellbeing

The ToC outlines that participating in UYOs is expected to lead to increased happiness and physical and mental health for young people as they engage in fun activities with peers. Ultimately, these outcomes are anticipated to improve their long-term wellbeing.

• Skills, education and employment

Participating in UYOs is expected to provide young people with valuable social, emotional, life and work skills, such as empathy and teamwork. Young people are also expected to experience increased confidence and a sense of empowerment as they are given opportunities to take on responsibilities, which is expected to increase their autonomy and contribute to a longer-term effect on their resilience and self-esteem. Potential life and work skill development includes leadership, communication, interpersonal and problem-solving skills, as well as recognised qualifications and technical skills relevant to their UYO. This skill development is expected to contribute to young people’s attitude towards and engagement within education. For the older age groups, it is anticipated that learning skills that are valued by employers and increased networking may support young people into the labour market.

• Community cohesion

The ToC also outlines that UYO membership can support the development of new relationships and networks. As young people build trust with their peers and the UYO community, it is expected that they will develop a sense of collective identity and belonging to a movement or community. In the longer-term, this was seen to encourage active citizenship, prosocial behaviours (such as actively talking to new people and helping others), a habit of volunteering and community engagement. Ultimately, this was expected to contribute towards broader community cohesion.

3.1.2 Contribution claims

As previously described, the theory-based impact evaluation relies upon a Contribution Analysis (CA) approach. This approach uses the overarching ToC to create a set of contribution claims (CC) about how the intervention is thought to produce its anticipated outcomes. Each claim identifies specific expected outcomes, as well as mechanisms of change (the specific steps through which change is achieved). The eight claims developed comprise the contribution ‘story’ – how the overall intervention is expected to lead to the identified outcomes. They have been grouped by the overarching outcome themes set out above, and each theme and CC are explored in detail in the following sections drawing from evidence gathered through interviews, surveys, and case studies (the full list can also be found in Annex 5.3). With quotes, we have provided characteristics of the interviewee where possible (UYO, gender, and age). In some instances, not all demographic data is provided – either to protect anonymity of the participant, because the participant did not provide it, or because the quote is from a focus group and is difficult to attribute correctly. In each section, the CC as drafted early on during the evaluation is presented alongside an updated version that reflects the findings.

Figure 3.1: Overarching theory of change

Use zoom on your browser to view.

_{NB: This is the ToC as drafted at the beginning of the evaluation. The outcomes that are outlined in bold in the ToC diagram refer to the outcomes that young people from the YAB thought were most impacted by UYO membership and therefore the highest priority}

See Section 3.1.1 for a text narrative of the diagram.

3.1.3 Alternative explanations

CA has an explicit focus on identifying possible alternative explanations of impact, or other factors (aside from UYO membership) that might have contributed to change. CA involves collecting data specifically to test these alternative explanations. This is especially valuable in complex settings, such as in the case of the eight UYOs, where multiple other factors (such as other activities young people take part in) may contribute to the same long-term outcomes. The evidence for the alternative explanations is addressed in each section, and the full list of these explanations can be found in Annex 5.3.

3.2 Wellbeing

This section explores the contribution of participating in UYOs on the wellbeing of young people. It examines the extent to which organisations offer supportive and inclusive environments and how this contributes to feelings of safety, the development of trusted relationships, and overall happiness and life satisfaction. The section also looks at how engaging in enjoyable activities offered by UYOs such as sports, outdoor pursuits, and arts and crafts, affects the mental wellbeing and behaviour of participants.

3.2.1 Happiness and wellbeing through safe environment and trusted relationships

The evidence confirmed that the majority of young people experienced supportive and inclusive environments at uniformed youth groups, which had an impact on their sense of safety, feeling respected, and self-acceptance. These organisations fostered atmospheres that were consistently described as welcoming, non-judgmental, and accepting of all individuals, regardless of their backgrounds or personal circumstances. Among survey respondents, 97% (n=390) agreed they felt safe at their group, and 89% (n=360) agreed they felt supported.

It’s a safe space for us. Because if at school you’d get bullied, you wouldn’t get bullied at Guides. And it’s a happy place for everyone.

(Girlguiding, girl)

As a transgender person I can go and not be treated any different for how I am.

(Girl, 15)

Within these supportive environments, young people often developed trusted relationships with both peers and adult leaders. They felt the adults genuinely cared about their wellbeing, would listen to their concerns, and help them if needed. This was also reflected in the survey with 95% (n=383) agreeing they knew who to talk to if they needed help at their group. Leaders were seen as approachable, attentive to individuals’ needs, and willing to provide both emotional and practical support.

And if they feel or notice that we’ve got problems, they just drop everything and listen, which is something I’m really, really happy with… They’re more than willing to give you space, talk things out at your own pace which for me being autistic is a very big one… They’re really willing to listen – if you’re upset and just want to rant, they’ll let us within reason.

(Boys’ Brigade, boy)

This dependable support from caring adults stood out to many as different from what they experienced in other settings. While some had trusted relationships with teachers or family members, others portrayed school as a less supportive environment with teachers often coming across as overwhelmed and peers that were sometimes hurtful. Furthermore, the personalised support from uniformed group leaders who took the time to get to know them as individuals made them feel valued.

There was strong evidence that the supportive environments and positive relationships in groups contribute to young people’s overall happiness, life satisfaction, and wellbeing. Many young people rated their happiness when attending their groups as being very high, and almost all survey respondents (97%, n=391) agreed they enjoy being at their group.

It just makes me overall happier. At school on Thursdays, I have horrible lessons, but I know that cadets is afterwards.

(Marine Society and Sea Cadets)

If you’ve had like a bad day at school, you can just go there and know that you’re just gonna do something that you enjoy and spend time with some, like, different people.

(Girlguiding, girl, 12)

Attending uniformed youth groups appeared to boost young people’s mental health in multiple ways. For some, having a protective space that uplifts their mood and promotes positive behaviours helps buffer against mental health difficulties. For example, one young person credited participating in their group with stabilising their mental health.

But also, it’s made my mental health so much better. Because I used to have quite unstable mental health. And now, through St John, because it’s something I do regularly, it’s something I can always look forward to. And it always makes me happy. I love being around these people.

(St John Ambulance)

Some young people also indicated the groups provide a beneficial coping outlet when facing challenges in other areas of life.

For me, it’s been a massive part of my life – [had a lot of problems with other young people at school] – coming here is letting off a weight – it’s somewhere I can be free and be me.

(Boys’ Brigade, boy)

For some young people, they also had access to strong support systems, caring relationships and other emotionally supportive environments, such as at school or with family, that also contributed to their overall wellbeing. In these cases, while still beneficial, the unique contribution of uniformed youth group membership towards wellbeing may be more limited, as those needs were being met through multiple avenues. Nonetheless, the cumulative impact of having an additional supportive environment and relationships appeared to extend the benefits.

The quantitative findings from the survey provide further support for uniformed youth groups providing a ‘protective’ effect for wellbeing. As shown in Figure 3.2, mean scores for both groups on the ONS4 (the survey measure of wellbeing) were moderately high – around 7 points on a 0–10-point scale. At wave one, there was a 3.7% difference (0.25 points, p=0.034) between young people in UYOs and the comparison group. This difference increased to 5.3% (0.36 points, p=0.002) at wave two, largely driven by a decline in wellbeing in the comparison group.

Figure 3.2: Wellbeing scores (ONS4) from quasi-experimental analysis, UYO members vs comparison

ONS4 Score Wave	UYG members’ mean outcome	Weighted comparison group mean outcome
ONS4 Score (Wave 1)	7.15	6.89
ONS4 Score (Wave 2)	7.05	6.69

_{Source: Survey data Wave 1 (n=1,028) and Wave 2 (n=1,028), balanced panel. Results show estimated treatment effects using inverse probability weighting with regression adjustment (IPWRA).}

The consistency of higher wellbeing in the uniformed youth group across time points strengthens confidence that UYO membership supports better wellbeing outcomes among young people. The difference in wellbeing was larger among those from less deprived areas at 6.6% (0.44 points, p<0.001). Health status appeared to moderate differences significantly, with young people in UYOs reporting fair/poor health experiencing a much larger benefit of 14.4% (0.84 points, p=0.01) compared to just 1.2% (0.07 points, p=0.76) for those reporting very good health.

However, it is important to acknowledge that there were also some instances of negative experiences with peers within the groups. There were some reported occasions of bullying with 14% (n=56) of survey respondents saying they had been bullied or seen others bullied, and some interviewed young people had also observed or experienced bullying. Some young people felt excluded, particularly by older members, who they felt would ‘gang up on them’ and were not perceived as friendly. Interviewees explained that the leaders spoke to the parents of the bullies, who were penalised, and then staff disseminated wider anti-bullying messages amongst the group. However, experience of exclusion made one young person ultimately leave their group.

Contribution story assessment

Contribution claim 1

Experiencing a supportive and inclusive environment helps young people feel safe and develop trusted relationships with peers and adults, which contributes to their happiness, life satisfaction, and mental health and wellbeing.

Evidence summary

Overall, the evidence strongly supports this contribution claim. Groups offer an emotionally safe space where young people feel accepted, respected, and free to express themselves authentically. Caring adult leaders who are invested in their welfare and attentive to their individual needs foster a sense of belonging and allow young people to develop trusted relationships. Participating in UYOs gives young people consistent access to an uplifting experience they look forward to, a valuable coping outlet to offset life stressors, and a space that both improves and protects wellbeing. The qualitative evidence is complemented by the quantitative analysis that showed better wellbeing outcomes among uniformed youth group members. However, it is important to note that the analysis may not have been able to fully account for pre-existing differences between members and non-members i.e. members may already have higher levels of wellbeing. However, if this is the case, there is good evidence to support that taking part in uniformed youth groups can help maintain positive wellbeing.

Alternative explanations

Some young people had access to strong support systems, caring relationships and other emotionally supportive environments, such as at school or with family, that also contributed to their overall wellbeing. However, most evidence pointed to significant added value resulting from uniformed youth group membership.

Contribution strength

Strong

Evidence confidence

Strong

Updated claim

No changes required

3.2.2 Happiness and wellbeing through activities

Participating in the various activities offered by UYOs, including sports, outdoor pursuits, arts and crafts, appeared to contribute to increased happiness, life satisfaction, and wellbeing among young people. Most young people expressed enthusiasm for the range of activities they experienced, and 93% (n=376) of survey respondents agreed they like the activities their group does. Many reported feeling excited to try new things and develop new skills and feeling happy and proud afterwards. Some felt that the safe and encouraging environment of their UYO made them feel more positive about trying new activities, even if they were initially unsure.

It’s helped me realise that I can do stuff even with my disability because it’s made me do stuff I never thought I can do, which is actually quite amazing.

(Girl, 18)

They’ve got the on-site activities, which includes crate stacking. So obviously that encourages them to, like, work as a team, to enable everyone to get as high as they can, and obviously encouraging them if they’re a little bit scared, kind of, pushing them outside the boundaries, because a lot of people don’t do that day-to-day. Same with archery, like, just giving them a fun activity, practice makes perfect. It’s nice to see them get excited when they finally get one on the board, if they haven’t done it before.

(St John Ambulance, staff)

Nearly half of the survey respondents (46%, n=184) reported that they had participated in reflection, wellbeing or mental health-focused activities, and 52% (n=209) participated in arts and crafts activities through their uniformed youth group in the last six months. UYOs also provided ample opportunities for physical activity. Among survey respondents, 47% (n=190) said they had played sports in their UYO and 54% (n=218) had taken part in outdoor activities, such as climbing, hiking or sailing in the last six months (see Annex 5.4 for details on activities in groups). For many young people, involvement in these activities allowed for them to be active, which enhanced their physical and mental health.

There are lots of opportunities to do like camping, hiking, and mountain biking… I really enjoyed them… They made me want to be more active.

(Scouts, boy, 12)

Furthermore, participants frequently framed UYO’s activities as a welcome alternative to the pressures of school. Many felt school was demanding and rule-bound whereas groups provided a relaxed space to destress and be themselves.

[School] feels like you have to follow the rules and listen to teachers but in Boys’ Brigade the leaders feel more human and are more understanding than teachers… also more flexible than school.

(Boys’ Brigade, boy, 13)

Some young people contrasted the active, varied experiences at groups with more passive behaviour, like staying home and being on their phones, which they admitted they tended to do more prior to joining their UYOs. Engagement in the groups has therefore helped to displace less productive activities and promote more prosocial attitudes and behaviours.

If Sea Cadets didn’t exist, I would probably just stay at home all day and do nothing[…] At Sea Cadets, I realise I need to help people more.

(Marine Society and Sea Cadets, girl, 10)

It gives me a reason to be good, because I know if I misbehave, I don’t get to come to Cadets.

(Marine Society and Sea Cadets)

Most UYO members reported they were engaged in enjoyable, productive activities in other areas of their lives, such as school clubs and sports, that likely also contributed to their wellbeing. In total, 97% (n=392) of survey respondents took part in other activities outside of school. For example, 53% (n=214) of survey respondents played sports outside of their UYO, 37% (n=152) took part in music lessons, 25% (n=103) participated in performing arts, such as dance, drama or choir, and 13% (n=52) attended a youth club. Therefore, uniformed groups were often not the sole source of engaging, healthy activities for participants. However, being a member of a uniformed youth group appeared to bring significant value for young people and sometimes had a greater contribution due to the breadth of opportunities.

Scouts is more adventurous than my youth club as I have a wider range of activities to choose from… I am able to try more new things.

(Scouts, boy, 11)

Looking also at the comparison group of survey respondents, a similar proportion of young people who were not uniformed youth group members took part in activities outside of school (95%, n=731). This is interesting given that the survey findings showed uniformed youth group members had better wellbeing outcomes, despite similar levels of activity outside of school (excluding UYO activities). However, this may also relate to differences in activity types – some of which may promote wellbeing more than others. While groups were similar in terms of playing sports, it was less common for comparison group members to have music lessons, take part in performing arts, or attend youth clubs (see Annex 5.4 for details on participation in out-of-school activities). Young people in the comparison group were more likely to specify a job as an out-of-school activity.

Contribution story assessment

Contribution claim 2

Participating in activities young people enjoy, including various sports and outdoor activities that promote physical activity and healthy lifestyles and/or other activities such as arts and crafts, contribute to increased happiness, life satisfaction, and mental health and wellbeing. Taking part in positive activities can also prevent/reduce negative behaviours e.g. anti-social behaviour. Improvements in overall health and behaviour can reduce societal costs in the long term.

Evidence summary

The evidence for this contribution claim shows young people are exposed to new, exciting experiences that promote physical health, offer a relaxed space to be active and creative, and displace passive, solitary activities with engaging group ones. While limited direct evidence was provided for prevention of anti-social behaviour, the prosocial norms and behaviours promoted may have this effect over time.

Alternative explanations

Some participants reported they were engaged in enjoyable, productive activities in other areas of their lives, such as school clubs and sports, that also contributed to their wellbeing – UYO members had slightly higher rates of participation in out-of-school activities than the comparison group. Therefore, uniformed groups’ unique impact on activity participation and wellbeing may be more modest for young people with pre-existing involvement in other positive activities like school clubs. Overall, UYOs provide a beneficial complement and/or alternative to other youth activities that fosters wellbeing through fun, social engagement and personal development.

Contribution strength

Strong

Evidence confidence

Strong

Updated claim

Participating in activities young people enjoy, including various sports and outdoor activities that promote physical activity and healthy lifestyles and/or other activities such as arts and crafts, contribute to increased happiness, life satisfaction, and mental health and wellbeing. Taking part in positive activities can also prevent/reduce negative behaviours.

3.3 Social and emotional development

This section explores the impact of UYO membership on the development of social and emotional skills. It explores the influence of team-building and structured activities on the development of cooperation skills and deeper social networks. This section also examines the role of experiencing a safe yet challenging space, where young people can develop new skills and learn from their experiences, in the development of their confidence, self-esteem and resilience.

3.3.1 Cooperation skills and deeper connections through structured activities

Young people reported that spending time with others through team-building activities encouraged them to work towards a common goal in a fun, sociable, and communal environment. Young people often noted that they did not have this type of experience in other settings outside the UYO. In fact, many stated that team-building activities provided them with cooperation skills that they then applied in different settings and helped them to socialise more at school.

It’s teamwork, teambuilding, leadership. Skills you don’t really get in school. That’s what cadets is - skills beyond school, learning life skills.

(Volunteer, Volunteer Police Cadets)

Participants also felt UYO activities improved their cooperation skills by improving their confidence in social settings and ability to communicate well with others. Members said the activities gave them opportunities to speak with new and diverse groups of people and provided a chance to lead teams. Improvements in confidence were often noticed by volunteers and staff too.

The confidence that the girls have gained from being part of Scouts has allowed them to speak out more in school.

(Volunteer/staff, Scouts)

[These activities] have offered a huge, huge benefit because before Girlguiding and Scouts, I was very shy.

(Girlguiding and Scouts, girl)

Because you learn a lot of teamwork skills [doing UYO activities]. Everyone eventually will be able to lead a team, but also you have to learn how to listen to someone.

(Boys’ Brigade, boy, 13)

When asked about socio-emotional skills in the survey, young people in uniformed youth groups showed higher levels of agreement than the comparison group of non-members across five statements, each associated with a different socio-emotional domain:

• Empathy

  I feel bad when someone gets their feelings hurt.

• Problem-solving

  I start a new task by thinking about different options for doing it.

• Initiative

  I work as long and hard as necessary to get a job done.

• Responsibility

  I take responsibility for my actions, even if I make a mistake.

• Teamwork

  I respect other points of view, even if I disagree.

Figure 3.3 shows the percentage of young people who agreed or strongly agreed with the statement corresponding to each domain. The largest difference appeared in statements on Teamwork, where members scored 10-percentage points higher (83% vs 72%, p<0.001), and Responsibility, with an 8-percentage point difference (78% vs 70%, p=0.010). There were also meaningful differences for the statement relating to Empathy (79% vs 73%, p=0.023) and a positive difference approaching significance for the Initiative statement (67% vs 61%, p=0.061). The difference between groups for the Problem-solving statement was smaller and non-significant (51% vs 47%, p=0.197).

Figure 3.3: Socio-emotional skills (YPS) from quasi-experimental analysis, UYO members vs comparison

Socio-emotional skills (YPS)	UYO members’ mean outcome	Weighted comparison group mean outcome
Teamwork	82.5%	72.3%
Empathy	79.2%	72.7%
Responsibility	77.7%	70.1%
Initiative	66.9%	60.8%
Problem solving	51.0%	46.5%

_{Source: Survey data Wave 2 (sample sizes vary by outcome ranging from 1,029 to 1,061). Results show estimated treatment effects using IPWRA.}

When examining socio-emotional skills by socio-economic status, Figure 3.4 reveals important patterns in how benefits vary across different levels of deprivation. Young people from more deprived areas (IMD 1-5) show particularly strong differences in the statement relating to Responsibility (81% vs 70%, 11-percentage points, p=0.02) compared to those from less deprived areas (IMD 6-10, 75% vs 70%, 5-percentage points). A similar pattern appears for the statement on Empathy, with greater benefits for young people from deprived areas (75% vs 66%, 10-percentage points, p=0.05) compared to less deprived areas (82% vs 78%, 4-percentage points). For the Teamwork statement, the difference remains substantial across both groups but is slightly larger in more deprived areas (83% vs 71%, 12-percentage points) than in less deprived areas (82% vs 72%, 10-percentage points). These findings suggest UYO membership may have a particularly valuable impact on socio-emotional development for young people from more economically disadvantaged backgrounds.

Figure 3.4: Socio-emotional skills (YPS from quasi-experimental analysis, UYO members vs comparison by socio-economic group (IMD)

Teamwork - more deprived	83.1%	70.9%
Teamwork - less deprived	82.1%	71.8%
Responsibility - more deprived	81.0%	70.3%
Responsibility - less deprived	75.3%	70.5%
Empathy - more deprived	75.2%	65.5%
Empathy - less deprived	82.0%	78.1%
Initiative - more deprived	66.9%	61.2%
Initiative - less deprived	67.0%	61.2%
Problem solving - more deprived	46.4%	44.1%
Problem solving- less deprived	54.3%	49.3%

_{Source: Survey data wave 2 (sample sizes varies by outcome and estimation). Results show estimated treatment effects using IPWRA.}

In addition to developing social and emotional skills, taking part in UYO activities also helped members make new connections. Among survey respondents, 97% (n=391) said they had made friends in their group. By taking part in activities requiring collaboration, problem-solving and a level of personal understanding not offered in other settings, young people described a deeper sense of connection in their friendships at groups.

You wouldn’t normally be put in a scenario with your best friend from school in a tent in the freezing cold or, like, trying to cook outside, but in cadets, you are. And I think you kind of, like, just get to know them on a different level, I guess, [than] school friends.

(Marine Society and Sea Cadets, girl, 16)

The combination of these elements made the friendships developed in uniformed youth groups feel distinct. Most participants stated that these friendships would not have happened in another setting, and many viewed them as highly valuable to their social life, sometimes even more so than those developed at school. This was partly because participants saw friendships formed at groups to be based more on shared interests. Furthermore, young people felt that they chose these friendships more actively than those at school, where they are less able to choose who they engage with.

Relationships are easier to build. You’re on common ground. You all believe in the same stuff.

(St John Ambulance & Scouts, boy, 16)

However, this was not universal for all young people. A minority of participants stated that their friendships at school were stronger than those at their group, and some participants did not feel that their friendships within the group were particularly distinct from those at school or other settings. This was often because there was a significant degree of overlap in their social network across school, other activities, and their group. A minority of participants explained that a disparity between ages could also be a barrier to building friendships at their group, because they felt less likely to share similar interests. For example, this was observed within St John Ambulance, which does not separate groups within a unit by age.

Contribution story assessment

Contribution claim 3

Spending time with other young people, including team-building and structured activities, contributes to young people’s development of cooperation skills (communication, interpersonal, teamwork) and more friendships/social networks.

Evidence summary

Overall, these findings suggest strong evidence that UYO membership contributed to participants’ development of cooperation skills, more friendships, and deeper connections.

Alternative explanations

Participants also spoke about developing skills and friendships through school or other groups. In general, evidence for these alternative explanations was weaker than the contribution of being a UYO member. Importantly, participants distinguished their UYO experiences and friendships from others and often placed more value on these.

Contribution strength

Strong

Evidence confidence

Moderate

Updated claim

Spending time with other young people, including team-building and structured activities, contributes to young people’s development of cooperation skills (teamwork, communication, confidence, openness) and more valued friendships and deeper connections with other young people.

3.3.2 Confidence, self-esteem, and resilience through challenge and support

Most young people described their group as a safe space to explore new activities and go outside of their comfort zone. Interview participants said they felt supported by leaders and peers to challenge themselves and make mistakes in order to learn. Young people described their group as offering a supportive environment distinct from school, which allowed them to build their confidence in valuable skills, such as public speaking. Participants who were part of UYOs with badge systems also felt that achieving badges, and gaining more responsibility as a consequence, improved their confidence significantly.

At school, if you have an idea and it’s wrong, they make you feel bad about it. Whereas at Guides, if you have an idea, everyone will try your idea. If it completely fails, it doesn’t matter. It’s a learning experience. And it just builds your confidence a lot.

(Girlguiding, girl, 14)

Many young people also felt that their self-esteem improved because of their involvement in group activities. Participants described how leaders positively encouraged members to try a new activity, which then allowed them to develop a new skill, and feel more confident in themselves. For instance, one Scouts member developed a passion for the game Warhammer through his Scouts group, which made him realise he was more artistic than he previously thought.

I just find it really good. It’s fun…I found that, like I’m actually an alright painter and I found that I really enjoy those kinds of things, like the miniature paintings and the building of it.

(Scouts, boy, 11)

Sea Cadets is different to school where teachers intimidate you to follow rules… I feel like I do believe in myself more. And I feel like I can achieve things I couldn’t before I joined.

(Marine Society and Sea Cadets, girl, 14)

Although gaining resilience was not often specifically spoken about amongst participants, some described how group activities developed their ability to not ‘give up’ when facing difficulty and discomfort. For instance, one JLGB member spoke about doing an assault course at a JLGB summer camp, where he had to crawl through mud to get to the finish line.

I’m proud of making it through that bog thing on the second try because…I fell in once…I got out and went at it again.

(JLGB, boy, 11)

[Sea Cadets] helps me keep trying and I feel like if I had not started going to Sea Cadets, then I’d probably give up [on many things] quite easily.

(Marine Society and Sea Cadets, girl, 14)

Most participants did not report that group activities developed their resilience. However, that may be because they perceived these activities, whilst sometimes challenging, as lower-risk and fun. As such, they did not perceive failure within activities as something to ‘bounce back from’, but instead saw failure as just part of the learning process. They associated higher-risk activities that left them open to negative feelings, such as failure, with school. Moreover, young people discussed the influence of other groups, their home life, and school life on their ability to handle difficult situations or push through challenges.

The lack of conclusive evidence on resilience was corroborated by the quantitative analysis. As part of the survey, young people were asked statements that explored their resilience, using the Brief Resilience Scale (see Figure 3.5). Overall, UYO members and the comparison group reported similar levels of agreement with the statements and negligible change across timepoints. At Wave 2, UYO members on average scored 3.17 compared to 3.09 for the comparison group. This represents a 2.6% higher score that does not reach statistical significance (p=0.16). Both scores fall within the range of 3.00-4.30, indicating normal resilience levels. As such, the quantitative and qualitative evidence are both fairly weak for supporting the claim that UYO membership leads to greater resilience.

Figure 3.5: Resilience (BRS) from quasi-experimental analysis, UYO members vs comparison

Figure 3.5: Resilience (BRS)	UYO members’ mean outcome	Weighted comparison group mean outcome
Resilience Score (Wave 1)	3.20	3.11
Resilience Score (Wave 2)	3.17	3.09

_{Source: Survey data Wave 1 and Wave 2 (sample size =1,009). Results show estimated treatment effects using IPWRA.}

Contribution story assessment

Contribution claim 4

Experiencing an emotionally and physically safe yet challenging space where young people can try new activities and experience both achievements and failure allows them to build confidence and self-esteem and learn from their mistakes, which contributes to greater resilience.

Evidence summary

There is strong evidence that UYO membership contributed to participants’ development of confidence and self-esteem. This is in line with the findings from a large-scale randomised controlled trial evaluating the impact of uniformed group activities, which found that participation in UYOs had positive effects on a range of social and behavioural outcomes – including self-confidence and teamwork (See et al., 2017). This same trial found a positive impact on resilience, while this evaluation found weaker evidence to support the claim that improvements in confidence lead to greater resilience in young people.

Alternative explanations

The evidence to support alternative explanations for the development of confidence and self-esteem, in school and other groups, was generally weaker than the contribution of UYO membership. For resilience, alternative explanations also included other groups, school or home life, and evidence for these alternative explanations appeared equal to the contribution of being a UYO member.

Contribution strength

Moderate

Evidence confidence

Moderate

Updated claim

Experiencing an emotionally and physically safe yet challenging space where young people can try new activities, develop new skills, and experience both achievements and failure allows them to build confidence and self-esteem and learn from their mistakes.

3.4 Skills and attainment

This section explores the impact of UYO membership on the development of practical skills. It explores how the development of a range of soft skills and obtaining qualifications or technical skills contribute to a young person’s engagement with education and access to the labour market. It then covers how leadership opportunities at UYOs contribute to confidence among young people, impacting their autonomy and self-esteem over time.

3.4.1 Enhancing education and career opportunities through activities and skill-building

Young people reported engaging in a variety of activities through their group including camping weekends, sports, and problem-solving games. These activities helped participants develop valuable life and work skills including first aid, communication and social skills, leadership and teamwork, problem solving, time management and organisational skills and community engagement.

Camping weekends played a significant role in developing essential soft skills and were a common experience in the survey group with 48% (n=192) saying they had taken part in an overnight camping trip with their group in the last six months. Participants reported meeting other young individuals for the first time during these events, facilitating the development of communication skills. Additionally, collaborating with their group to prepare meals helped enhance organisational and time management abilities, while the challenge of building tents or shelters fostered problem-solving skills.

You learn but you’re having fun at the same time. There is a shelter building thing at this camp and [my friend] didn’t think she was going to enjoy it, but she really did and she learnt new skills about how to make it sturdier and everything.

(Girlguiding)

There were fewer young people reporting access to formal recognised qualifications or technical skills through their group. If participants mentioned formal training, it was mostly either first aid or leadership. However, obtaining informal qualifications like activity-specific badges were reported by interviewed members of all UYOs, excluding Volunteer Police Cadets who have a rank structure where young people gain another stripe on their uniform. Among survey respondents, 76% (n=307) indicated they had worked towards an award or badge in the last six months. Young people were positive about gaining badges because they felt their hard work was recognised and celebrated, which increased their motivation to obtain more skills.

You can show your accomplishments off with Girls’ Brigade because you get a badge and there’s a ceremony, whereas at school it’s not as much of an accomplishment.

(Girls’ Brigade, girl, 10)

Generally, young people mentioned that group activities motivated them with their current education, rather than future pathways. Young people reported that attending their group gave them the opportunity to decompress outside of school, which in turn meant they could be more focused with their current studies. There were some examples of young people who reported that attending their group motivated them to work harder at school, either because the UYO taught them about the importance of studies or because the group positively influenced their motivation to complete school tasks.

Scouts makes me work harder…it’s giving me momentum to get things done before they’re due, like homework prep.

(Scouts, boy, 12)

Beyond the skills young people report gaining through their UYO activities, we also examined whether membership translated into measurable educational outcomes. Using data from the NPD, we analysed ‘Attainment 8’ scores - the UK’s standard measure of GCSE achievement across eight subjects. See the Technical Annex for further details on how Attainment 8 is calculated.

Our quasi-experimental analysis reveals that UYO members achieved significantly higher attainment scores than comparable non-members (Figure 3.6). UYO members averaged 69.6 points compared to 64.2 points for the comparison group - a difference of 5.4 points (p=0.00).

These findings are from a small subset of the analytical sample (n=247)[15], so these findings should be interpreted with caution. It is also worth noting that the average Attainment 8 score for the 2023/24 school year was 45.9, which means our overall sample has a much higher attainment profile than the population.[16]

Figure 3.6: Attainment 8 from quasi-experimental analysis, UYO members versus comparison

Attainment 8 score	UYO members’ mean outcome	Weighted comparison group mean outcome
Attainment 8 score (Wave 2)	69.6	64.2

_{Source: National Pupil Database (NPD), Department for Education, England and variables from Survey Wave 1 and Wave 2 (sample size =247). Results show estimated treatment effects using IPWRA.}

There was less evidence of the link between skills gained at their UYO and young people’s progression into further education. There was one St John Ambulance member who reported that attending the group looked good on their personal statement when applying for a paramedic course at university.

Participants who were members of UYOs with direct links to career paths, such as Marine Society and Sea Cadets, Volunteer Police Cadets and St John Ambulance, were more likely to believe that the skills gained from their group would lead to better access to the labour market. Two young people who were Sea Cadet members for a long period of time felt that the skills they have gained at the group, such as leadership skills and navigation, were more useful in their naval career aspirations than an English GCSE.

Leadership courses, cooking, first aid, engineering - there are a lot of courses you can do and take advantage of - they also look very good on your CV.

(Marine Society and Sea Cadets)

Volunteer Police Cadet members, particularly those interested in joining the police or pursuing a career in law, reported that attending their UYO enabled them to obtain the relevant skills for that role and consequently have stronger job applications. One member reported that they received mentoring and support from a leader with their application to join the police.

I would say that I’ve definitely got more skills than I think that [peers not attending Volunteer Police Cadets] do. And I have had that experience of having to be out and doing activities that they haven’t done or probably will never do.

(Volunteer Police Cadets, boy, 16)

St John Ambulance members reported that the skills gained through their UYO such as first aid, teamwork, and handling emergency situations were invaluable skills for life and future medical roles.

I know how to stop bleeding now, and I know what to do if someone’s hurt. So, I guess that’s a skill I’ve unlocked. And I also know how to tie stuff, like bandages, I know how to do that.

(St John Ambulance)

Definitely communication and also, like the presenting skills, because that’s something quite crucial. Teamwork, I think, as well. That’s quite like a crucial skill for medicine.

(St John Ambulance, girl, 16)

The evidence was weaker among young people who attended UYOs that are not as strongly linked to a specific career area. However, these participants did note that attending their UYO meant they obtained transferrable skills such as taking initiative and responsibility, or had easier access to the Duke of Edinburgh via their group. Members felt that these opportunities would be seen positively on their CV and improve their access to the labour market.

Contribution story assessment

Contribution claim 5

Engaging in fun and diverse activities outside of school that teach young people valuable life and work skills (leadership, emotion management, empathy, responsibility, initiative, and problem solving) and having access to recognised qualifications/technical skills contributes to better attitudes towards, engagement and progress with education and, among older young people, access to the labour market.

Evidence summary

Overall, the evidence supports the contribution of UYOs in facilitating the acquisition of valuable life and work skills such as leadership, responsibility, initiative, and problem-solving. These skills cultivate better attitudes towards education, with some young people reporting enhanced motivation and engagement with their current studies due to the skills learned through UYO activities. There is also evidence that members have higher attainment outcomes (GCSE Attainment 8) than non-members – albeit from a small sample. While formal qualifications are not predominantly gained through these organisations, the informal achievements, like badges, instil a sense of validation and accomplishment, boosting further skill acquisition and personal development. The impact of UYOs on young people’s readiness for the labour market is more evident in groups with direct career links, providing skills that are increasingly recognised and valued as compared to those developed in conventional educational settings. The impact of UYOs is particularly profound for older participants concerning labour market access. This makes UYOs a valuable component in youth development, especially in preparing them for future challenges both educationally and professionally.

Alternative explanations

The evidence to support alternative explanations for the development of confidence and self-esteem, in school and other groups, was generally weaker than the contribution of UYO membership. For resilience, alternative explanations also included other groups, school or home life, and evidence for these alternative explanations appeared equal to the contribution of being a UYO member.

Alternative explanations

Activities within more formal education are also critical for the development of life and work skills and preparing young people for continued/further engagement in educations and/or the labour market.

Contribution strength

Moderate

Evidence confidence

Moderate

Updated claim

Engaging in fun and diverse activities outside of school that teach young people valuable life and work skills (leadership, communication, responsibility, initiative, and problem solving) contributes to better attitudes towards and engagement with their current education, improved attainment, and, among young people attending sector-specific UYOs, improved access to the labour market.

3.4.2 Building autonomy and self-esteem through leadership

Unsurprisingly, older members reported that they had received more opportunities than younger participants to take on responsibilities or leadership roles at their UYO because they were more experienced or transitioning to become a young leader. Among survey respondents, around a third (33%, n=133) identified as a leader who helps run the sessions, organise activities and support younger members in the group, and another 46% (n=184) want a leadership role in their group in the future. Leadership activities included volunteering as a young leader or attending leadership training courses, which fostered skills such as public speaking and managing groups. For example, one member reported taking on leadership training so that she could get promoted within the group:

I did [basic first aid and leadership] training as I wanted to do leadership roles and thought it would look good on a CV and be good for any job. I want to be responsible and work up the ranks and get to a higher position to make an impact in the job I do…I want to show that people who are trans like myself can do things.

(Girl, 15)

Older participants also mentioned additional responsibilities such as taking initiative in organising events or activities and helping staff members with daily operations and activities which could include setting up, tidying up, or supporting others during group sessions.

Some days we design sessions for the younger Scouts, so we plan activities like making food. That really develops our leadership skills.

(Scouts)

Among older members, there was strong evidence that taking on responsibilities and leadership roles led to increased confidence and initiative. Participants reported that their group gave them more freedom and responsibility than at school.

The survey findings provide further support for UYO membership contributing to the development of the self-belief and skills to be a leader (see Figure 3.7). UYO members were nearly 10-percentage points more likely to agree or strongly agree that young people can make a difference in the world (p=0.03) and 18-percentage points more likely to say they have the skills to make positive change in their community (p<0.01).

Figure 3.7: Youth empowerment beliefs and community change from quasi-experimental analysis, UYO members vs comparison

Belief or Skills	UYO members’ mean outcome	Weighted comparison group mean outcome
Youth empowerment belief	81.1%	71.5%
Community change skills	74.6%	56.5%

_{Source: Survey data Wave 2 (sample size varies by estimation). Results show estimated treatment effects using IPWRA.}

Although the qualitative data indicated that older members were more likely to report changes regarding leadership opportunities and skills, there was no notable difference between younger (10 to 13-year-olds) and older (14 to 18-year-olds) members in these survey responses. Figure 3.8 shows that among 10 to 13-year-olds and 14 to 18-year-olds in UYOs, 81% agreed or strongly agreed that young people can make a difference in the world. Older members were slightly more likely to say they had the skills to make positive change (75%), compared to 73% of younger members.

Figure 3.8: Youth empowerment beliefs and community change skills from quasi-experimental analysis, UYO members by age group.

Beliefs or Skills	UYO members’ mean outcome
Youth empowerment belief - Age 10-13	80.6%
Youth empowerment belief - Age 14+	81.5%
Community change skills - Age 10-13	73.2%
Community change skills - Age 14+	75.4%

_{Source: Survey data wave 2 (sample size varies by estimation).}

Contribution story assessment

Contribution claim 6

Taking on responsibilities and leadership roles develops young people’s leadership skills and ability to make informed choices, which contributes to their confidence and sense of empowerment. Over time, this improves their autonomy and self-esteem.

Evidence summary

The evidence indicates that older UYO members gained significant opportunities to develop leadership skills. Activities such as volunteering as young leaders and attending leadership training courses enriched their skills in areas like public speaking and group management. This sense of empowerment fostered by UYOs helps cultivate independence, initiative and self-esteem, critical for personal and professional growth. The evidence linked to autonomy was less clear, particularly among younger participants who had fewer opportunities to lead at their UYO. However, we can infer that the development of initiative (i.e. actively identifying and taking on new things) links to autonomy.

Alternative explanations

Young people involved in organising UYO activities and supporting others displayed increased initiative and autonomy compared to their experiences at school, which they found less engaging.

Contribution strength

Strong

Evidence confidence

Moderate

Updated claim

Taking on responsibilities and, among older members, leadership roles develop young people’s leadership skills and ability to make informed choices, which contributes to their confidence and sense of empowerment. Over time, this improves their initiative and self-esteem.

3.5 Community connectedness

This section examines how young people in uniformed youth groups have a sense of connectedness to their communities within their UYO and externally through their UYO. This is explored through the uniform and shared experiences, which were expected to support the formation of collective identity and belonging. Furthermore, this section explores how UYO involvement might encourage young people to engage in helping and volunteering activity within their communities.

3.5.1 Growing belonging and identity through uniforms and shared experiences

The concept of what constitutes uniform and when uniforms or different types of uniforms are worn varied across the UYOs. There are organisations, such as JLGB, for whom the uniform is often a polo shirt and, in some instances, they are not expected to wear a uniform at all. For the Marine Society and Sea Cadets, the uniform is more formal with more components, and they have different uniforms for different activities. Even with these differences in what constitutes a uniform, there are more similarities in young people’s associations to uniforms than there are differences.

Young people articulated a wide range of feelings that came from wearing the uniform. For some, this meant the feeling of being included, being part of a group, and feeling connected to others. In multiple instances, this feeling was likened to being part of a family. For others, the uniform represented a sense of equality, a feeling that no-one was better than anyone else.

There’s nothing really that could separate us because we’re all in the same group and it just seems that we’re all in it together.

(Marine Society and Sea Cadets, girl, 11)

We get people from really disadvantaged backgrounds and people who are really well-off, and that’s really interesting watching them make friends and things like that. Usually, it’s fine because once you’re here, you’re all in a uniform and it doesn’t really matter.

(Boys’ Brigade, volunteer)

This shared sense of belonging even extended to young people from other units of their UYO and other UYOs. The participants felt that seeing someone else in a uniform gave them a positive perception of that person and an instant connection to them. They felt that they could engage in conversation with that person, that they already had a common ground, and that they wanted to connect with them.

I’d know that we have something in common. Like, we might get along a bit better than somebody who don’t do Scouts because you have the same interests in that and you know that they do.

(Scouts, boy, 11)

In some instances, young people shared that their group did not have sufficient uniforms for all new recruits. The participants recounted feeling disconnected from the group and a sense of embarrassment at not everyone in their unit having the full uniform.

[…]I feel like it’d be quite a bad impact… Yeah, it looked like we’re under-funded and they can’t deliver the uniform.

(Girl, 15)

As a point of comparison, young people gave mixed responses regarding whether UYO uniforms were perceived as different from school uniforms. The comfort of the uniform was considered important; some preferred their school uniform and others their UYO uniform. However, members often spoke of feeling proud in their UYO uniform – a feeling that was not associated with their school uniform.

I feel like the uniform for any of the uniformed groups empowers you… you feel like you belong in the uniform, and you have a sense of responsibility.

(St John Ambulance, girl, 15)

Beyond the UYO community, young people felt that the uniform was useful for conveying their purpose to their broader community and enabled them to build trust.

It gives you a sense of purpose and rightfulness. And when you are wearing it, you represent the police. And everything you do which is good, you give the police a good name, which helps to build trust.

(Volunteer Police Cadets, boy, 18)

Some of the public just see us as we’re all paramedics or something, but they recognise it as a healthcare uniform, which means there’s an immediate amount of trust and an immediate relationship built with your patients.

(St John Ambulance, 16)

As described in previous sections, young people enjoy and benefit from participation in a broad range of activities provided through groups. Alongside the uniform, these shared experiences enhanced the feeling of belonging and having a shared purpose.

It’s like on camp I’m a different person and it’s much easier to interact with people and everyone is, kind of, linked there because we’re all there for a shared purpose… So, it is very different than with friends at home and yes, I feel like I can be much closer with people on camp than here.

JLGB, 16

Some activities were also seen as beneficial in creating stronger ties to communities beyond their group. One Volunteer Police Cadets member described her door-knocking work as opening her eyes to the diversity of her community. Through these face-to-face interactions she gained a deeper perspective of those living around her from different backgrounds and gained confidence in her communication with others. JLGB members also spoke of how their participation helped them to connect with other Jewish young people and learn more about their own culture and customs.

The survey findings support the claim that UYO participation can lead to an increased sense of belonging. As shown in Figure 3.9, 61% of UYO members agreed with feeling like a valued member of their community, 16-percentage points higher than the comparison group (p<0.01). In more deprived communities, this difference widens, suggesting a larger effect for these members. Although a slightly smaller percentage of young people (58%) felt valued, this was 20-percentage points higher than the comparison group (p<0.01). The difference in less deprived areas was 13-percentage points (p<0.01).

Figure 3.9: Sense of belonging from quasi-experimental analysis, UYO members vs comparison

Sense of belonging	UYO members’ mean outcome	Weighted comparison group mean outcome
Community belonging	61.0%	45.0%
Community belonging - High IMD	62.8%	50.0%
Community belonging - Low IMD	58.4%	38.4%

_{Source: Survey data wave 2 (sample size varies by estimation). Results show estimated treatment effects using IPWRA.}

The overall sense of identity and belonging was considered by the interviewed participants to grow over time. The more that they engaged in group activities and accrued shared experiences with other young people, the stronger this bond became.

It’s a community and group within yourself. Like, you begin to know each other and there becomes a lot of friendships. Like, even with GCSEs, I took a bit of time off so I could be studying, and I had people still message me asking if I was okay and when I’d be coming back.

(Volunteer Police Cadets, boy, 16)

However, survey findings indicated that community connection is not as pronounced among those with longer involvement in UYOs. Of those members who had been involved with their organisation less than three years, 64% felt like valued members of their community compared to 58% who have been involved more than three years (p=0.01).

Contribution story assessment

Contribution claim 7

Wearing a uniform and sharing experiences with other young people contributes to their sense of (collective) identity and belonging within their UYO and/or wider community (e.g. faith, movement, national, international).

Evidence summary

Young people clearly stated a range of positive emotions linked to their sense of belonging and collective identity associated with the wearing of the uniform. The uniform itself became a representation of this identity within which there were shared values and a social structure present. Shared experiences were central to the positive emotions associated with the identity, and the sense of identity and belonging grows.

Alternative explanations

Some participants did allude to also having an identity and sense of belonging associated with non-UYOs such as sports groups and religious communities in which they participated. The YAB saw community as something that was more akin to a Venn with overlapping aspects. There was something that was thought to be different about the UYOs in the scale of the national/global community that it represented and the association with it feeling like a family. Overall, this was considered to be distinct to other settings, as the respondents did not speak with the same strength and positivity about their school or the other non-UYOs they are involved with. For example, how young people associated positive sentiments such as pride, empowerment and responsibility was different to school uniforms.

Contribution strength

Strong

Evidence confidence

Moderate

Updated claim

Wearing a uniform and sharing experiences with other young people contributes to their sense of collective identity and belonging within their UYO. Their sense of collective identity and belonging increases with sustained participation in the UYO over time.

3.5.2 Empowering young people to social action and civic participation

In this study, we define social action as engaging in general helping behaviours such as helping at home with chores or helping a neighbour bring in their shopping. Civic participation is conceptualised by active participation in more formal volunteering roles and democratic activity, which might include volunteering as a leader with a UYO or at the local library.

Increased helpfulness and volunteering

Participants often reported an increased recognition of the need for and willingness to help at home, in school and to a certain extent within their neighbourhoods. This ranged from helping a teacher hand out books at school to tidying up at home and helping neighbours with gardening. Younger members, in particular, reported a change in these helping behaviours.

I feel like I stepped forward a lot more to help people.

(JLGB, boy, 11)

Many of the young people reported engaging in volunteering activities with their groups. These activities centred around fundraising, helping out at local organisations such as food banks, and litter picking. Some members also indicated that they volunteered to take on leadership roles within their group, which was more commonly reported among older participants. Of those volunteering in their group, some found it challenging to maintain the commitment due to competing pressures, such as the need to focus on their education. This is supported by the survey findings. Of the 60 respondents who indicated they had stopped attending their UYO, 40% (n=24) reported the need to focus on their school or college work as a reason why.

Yeah, to be honest, that’s the reason why I’ve just, you know, stopped doing those things [volunteering activities] because I just want to focus on more of my school stuff.

(Girlguiding, girl, 17)

For many members, their civic participation was limited to the activities within their group. Among survey respondents, 24% (n=98) indicated they did voluntary work outside of their UYO, which was the same as the comparison group of non-members (24%, n=187). There were similarly low levels of civic engagement (e.g. going to rallies or demonstrations) among UYO members and the comparison group – 3% (n=14) and 4% (n=28), respectively. Based on interviews with those members who engaged in other outside volunteering, they were often older, and volunteering was done as a family activity. These activities included helping out at local hospitals and food banks, mentoring young people, activities in their religious community, and volunteering to deliver music workshops. Family involvement also appeared to have a bigger contribution on civic engagement than UYO activities as well.

Well, the first time, my dad just asked me if I’d like to go to it with him, because he was going to go and I had the time. So, I said, yeah, of course. And then he just told me last time that it was in the calendar for last Sunday and I said, yeah, sure, I’ll happily go to it.

(JLGB, 16)

UYO values and role modelling

There was a set of values that participants considered to underlie their want to engage in social action and civic participation. The most common of these values were respect, kindness, and helping others being the right thing to do. Members indicated that these same values were aligned with their UYO.

[St John’s Ambulance] are quite good because they encourage to you help others and stuff…. I think everyone’s going to need help sooner or later in life. You get back what you give out.

(St John Ambulance and Girlguiding, 14, Girl)

They teach us to be respectful and kind to everyone and to help people when they need it.

(Marine Society and Sea Cadets, girl, 10)

The leaders and staff of the UYOs were seen to be role models for the young people. Participants expressed respect for volunteers and had established trusting relationships with them.

[The Scout leader] is always helping everybody out and if somebody gets stuck, then he helps them instead of criticising what they’ve done wrong.

(Scouts, boy, 11)

Many young people expressed the desire to eventually volunteer for their UYO. Some cited their current leaders and staff as inspiring this aim, as they wanted to provide a similar positive experience to other young people that they received.

Contribution story assessment

Contribution claim 8

Encouraging young people to take self-directed action in their community (e.g. mentoring and volunteering, addressing social/environmental issues) leads to increased social action and civic participation and empowers young people to make positive changes in their communities, which contributes to their development as active citizens e.g. creating a habit of volunteering. Taking part in social action also supports community cohesion.

Evidence summary

There was evidence of a moderate strength for the role of UYOs encouraging and reinforcing social action and civic participation, especially in terms of helping behaviours and engagement in volunteering. This was through the strong values portrayed by the organisations and the activities they undertook such as volunteering. There did seem to be a difference in civic participation by age. Younger participants and those transitioning to further education were less likely to sustain their engagement in civic participation. For those who were younger this might result from a lack of autonomy and those who are older it was a lack of free-time due to the pressures of school, work and employment. It was not possible to substantiate the claim that increased social action or civic participation contributed to a young person’s development as an active citizen. Similarly, there was no evidence for the claim that taking part in social action supports community cohesion, but there was also no evidence against this.

Alternative explanations

There was evidence for alternative explanations for young people’s engagement in social action and civic participation or at least factors playing a strong mediating role in influencing the actions of participants. Young people who were regularly engaged in volunteering activities outside of their UYOs were also commonly participating in other groups and activities linked to volunteering such as the Duke of Edinburgh award. There also appeared to be an influence of the family circumstances of the young person. If their parents were active in volunteering and other forms of civic participation, they themselves would also be more likely to be involved in such activities outside of their UYO.

Contribution strength

Moderate

Evidence confidence

Moderate

Updated claim

Giving young people opportunities to take part in community activities through their UYO leads to increased social action and civic participation and empowers young people to engage with their communities in a positive way. Modelling social action and civic participation by UYO leaders encourages young people to engage in these behaviours.

4. Conclusion

4.1 Contribution to the evidence base

• Wellbeing

The evaluation found strong support for the claim that UYOs foster wellbeing through supportive and inclusive environments. Young people consistently reported feeling safe, respected, and accepted within their groups. These environments facilitated the development of trusted relationships with peers and adult leaders, contributing to overall happiness and life satisfaction. Quantitative analysis further supported these findings, revealing better and stable wellbeing outcomes over time among UYO members compared to a weighted comparison group, which showed a decline over time, suggesting a ‘protective effect’ of UYO membership.

• Social and emotional development

UYO membership demonstrated a strong positive impact on social and emotional development. Structured activities and supportive environments within UYOs fostered cooperation skills, communication, and confidence. QED results indicated that UYO members exhibited higher levels of empathy, responsibility, and teamwork compared to non-members. Notably, young people from more deprived areas showed particularly strong differences in responsibility and empathy, highlighting the valuable impact of UYO membership on socio-emotional development for those from disadvantaged backgrounds.

• Skills and attainment

UYOs enhance education and career opportunities by providing valuable life and work skills. Participants developed essential skills such as first aid, communication, leadership, teamwork, problem-solving, time management, and community engagement. While formal qualifications were not predominantly gained through these organisations, informal achievements, like badges, instilled a sense of validation and accomplishment. The impact of UYOs on young people’s readiness for the labour market was more evident in groups with direct career links (such as Volunteer Police Cadets). The evaluation also found evidence of positive educational outcomes, with UYO members achieving significantly higher GCSE Attainment 8 scores than matched peers, suggesting that the skills and confidence developed through UYO participation may translate into measurable academic benefits.

• Community connectedness

UYOs foster a sense of community connectedness through uniforms and shared experiences. Participants expressed positive emotions linked to their sense of belonging and collective identity associated with wearing the uniform. The QED analysis found that UYO members felt like valued members of their community significantly more often than the comparison group, with this difference widening in more deprived communities. UYOs also played a role in encouraging and reinforcing social action and civic engagement, especially in terms of helping behaviours and volunteering.

Based on the survey findings, UYO members had slightly higher rates of participation in out-of-school activities than the comparison group. However, young people identified their UYO participation as contributing to the outcomes above in distinct ways compared to other settings, such as school, family life, and other out-of-school activities. UYO members therefore appeared to benefit from complementary experiences that reinforce positive outcomes.

The findings of this impact evaluation align with a growing body of literature highlighting the positive effects of uniformed youth group participation on young people’s outcomes. While earlier research often relied on correlational data or evaluations conducted by the organisations themselves, some studies employing rigorous methodologies have provided stronger evidence for the causal impact of UYO participation.

There is further evidence of the potential long-term ‘protective’ effect of UYOs on wellbeing. Dibben, Playford and Mitchell (2017)[17] conducted a birth-cohort study (n=9603), which found a lower risk of a mood or anxiety disorder at age 50 for previous Scouts-Guides members, compared to those who had never participated. The authors hypothesise that several elements of the Scout-Guide approach may provide the foundation for this long-term impact including physical activity, time spent outdoors, and opportunities to build peer relationships.

See et al (2017)[18] conducted a randomised controlled trial in 71 UK schools with 7,781 students aged 13 to 14 to investigate the impact of uniformed group activities on non-cognitive outcomes. It similarly found that participation in these activities had positive effects on self-confidence, teamwork, resilience, career aspirations, empathy, and charitable activities. Cheng and Humphrey’s (2025)[19] recent study evaluated the efficacy of Football Beyond Borders (FBB), a targeted social-emotional learning intervention that includes engaging activities like classroom sessions, football, therapy, and rewards trips. While not a uniformed youth group, the intervention offered similar out-of-school activities. The study found FBB had a significant positive effect, increasing participants’ scores on a wellbeing scale, relative to a matched control group.

Our finding that UYO membership may have a particularly valuable impact on socio-emotional development for young people from more economically disadvantaged backgrounds aligns with research suggesting that open access youth activities are often particularly beneficial for young people from disadvantaged and marginalised backgrounds. For example, another randomised controlled trial evaluating the impact of the Children’s University programme – which combines outdoor learning, after-school clubs, and community social action – found that disadvantaged pupils eligible for free school meals showed better gains in teamwork and social responsibility.[20]

The same study, which involved 1,840 year 5 pupils in 68 primary schools, also found slight progress in pupils’ reading and maths performance. Another study with a more specific focus on uniformed youth groups, involving 71 schools and over 7,700 students, found no significant academic benefits for Year 9 students’ academic development.[21] However, the study also faced implementation challenges due to lack of resources and support. This study found a positive, significant impact on Attainment 8 scores (derived from GCSEs taken in Year 11), but with a small sample. UYO impact on academic outcomes appears to be an area that would benefit from further research.

Overall, we found limited evidence on how UYO membership contributed to young people’s resilience. However, there is other research, such as the multi-generational impact study of Marine Society and Sea Cadets membership,[22] which found that Marine Society and Sea Cadets fostered confidence that enabled members to navigate life with greater resilience. This study involved 3,000 former members who were older at the time of the study and whose ability to understand resilience and the impact of UYO membership differs from the young people surveyed here. Also, as noted in the Methodology section, the BRS scale was not developed for young people so may not accurately capture their experience. This suggests that further research is required to properly understand the contribution UYOs make to young people’s resilience.

While our evaluation primarily focused on short-term outcomes, the multi-generational study of Marine Society and Sea Cadets found that 95% of participants said the organisation had a positive impact on their life, long after they left. This highlights the potential for UYOs to provide young people with a “positive start in life”, particularly those who struggle to engage in formal education.

4.2 Recommendations

Based on the findings of this impact evaluation and the wider literature, the following recommendations are made to further enhance the positive impact of UYOs:

• 1. Targeted support for disadvantaged youth

Given the particularly strong benefits observed for young people from more deprived areas, UYOs could further prioritise creating new spaces and outreach activities to increase participation among this population. The Government and other funders could also focus their support to UYOs in alignment with these aims The Fund had an explicit focus on increasing capacity for delivery in areas of deprivation and this is detailed further in the process evaluation report. Further interventions to improve access to socioeconomically disadvantaged young people could include reducing financial barriers to membership, providing transportation assistance, and tailoring activities to meet the specific needs and interests of disadvantaged youth.

• 2. Promote long-term engagement

While positive effects were observed across membership durations, UYOs should strive to promote long-term engagement to maximise the development of skills like responsibility and teamwork, where we observed stronger differences among young people who had been members for longer. This could involve creating leadership opportunities for older members or offering advanced training programmes.

• 3. Support the added value of UYOs relative to school

Given some positive evidence that UYO membership can contribute to academic attainment, it may be tempting to consider adapting UYO activities to further enhance this outcome. However, young people frequently spoke of appreciating that UYOs felt very different from their school experience in that they had different activities, developed different skills and relationships, and viewed it as a space to unwind from school. The evidence suggests that this separation may be critical for realising the non-attainment outcomes, such as communication skills and wellbeing, which may in turn contribute to improved attainment in the long-term.

• 4.Review programming and curriculum to ensure strong focus on key mechanisms

The overarching ToC and contribution story indicates that a few key mechanisms are particularly important for improving outcomes for young people: creating supportive and inclusive environments, delivering varied and enjoyable activities, providing opportunities to build trusted relationships with adults and peers, and enabling members to take on leadership responsibilities. This evaluation suggests that UYOs are largely effectively delivering these mechanisms, but more detailed and programme-specific reviews may be beneficial. Interviews with young people indicated that leadership responsibilities were more common among older members, so creating more age-appropriate opportunities for younger members may be beneficial to ensure access across all ages.

• 5. Invest in volunteer training and support

The success of UYOs relies heavily on the dedication and skills of adult volunteers (see the Process Evaluation Report). Therefore, it is crucial to invest in volunteer training and support to ensure they are equipped to provide high-quality activities and create positive relationships with young people. This training should focus on creating inclusive environments, promoting social and emotional development, and fostering community engagement.

• 6. Further research

Additional research is needed to better understand the long-term impact of UYO participation, particularly on educational attainment and career pathways.

Longitudinal studies that track young people over time would provide valuable insights into the sustained benefits of UYO membership. Further research is also needed to overcome some of the limitations of this evaluation, for example, the lack of baseline data.

By implementing these recommendations, UYOs can further enhance their positive impact on young people’s lives, contributing to their wellbeing, social and emotional development, skills and attainment, and community connectedness.

5. Annex

5.1 Evaluation framework

This table presents our evaluation questions and the data sources that will be used to answer them. Two ticks indicate that this data source will be key to answering the evaluation question, whereas one tick indicates that it will provide supporting evidence.

Table 5.1: Evaluation questions

	Delivery team and UYO staff interviews	Volunteer interviews	Interviews with young people	Case study visits to units/events	Learning events hosted by Youth United Foundation (where relevant)	Data collected by Groundwork	Data collected from local groups evaluation team	Survey of young people	Analysis of National Pupil Database data
Process evaluation
Was the funding successful in reducing the waiting lists for uniformed youth groups and creating more places (both at new and existing groups)? This should include a quantitative assessment of whether the waiting lists were reduced and overall capacity increased.	Supporting Evidence	Supporting Evidence	Evidence not provided	Evidence not provided	Evidence not provided	Key evidence source	Key evidence source	Evidence not provided	Evidence not provided
To what extent were created places concentrated in deprived areas?	Supporting Evidence	Supporting Evidence	Evidence not provided	✘Evidence not provided	Supporting Evidence	Key evidence source	Key evidence source	Evidence not provided	Evidence not provided
What types of young people took up the opportunities created by the fund? (how young people are recruited; whether there is targeted recruitment of under-represent young people; whether nature of engagement varies by characteristics).	Supporting Evidence	Key evidence source	Evidence not provided	Evidence not provided	Supporting Evidence	Supporting Evidence	Key evidence source	Supporting Evidence	Evidence not provided
How did the different uniformed youth groups go about eliminating their waiting lists? What worked well and what worked less well? How do groups tend to use waiting lists?	Key evidence source	Key evidence source	Evidence not provided	Evidence not provided	Key evidence source	Evidence not provided	Evidence not provided	Evidence not provided	Evidence not provided
Did the funding improve the long-term sustainability of uniformed youth group places?	Supporting Evidence	Key evidence source	Evidence not provided	Evidence not provided	Supporting Evidence	Evidence not provided	Evidence not provided	Evidence not provided	Evidence not provided
Did the fund improve the recruitment and retention of adult volunteers to uniform youth groups?	Evidence not provided	Key evidence source	Evidence not provided	Evidence not provided	Evidence not provided	Key evidence source	Key evidence source	Evidence not provided	Evidence not provided
How was the fund delivered on the ground in terms of setting up new groups?	Supporting Evidence	Key evidence source	Evidence not provided	Evidence not provided	Supporting Evidence	Supporting Evidence	Evidence not provided	Evidence not provided	Evidence not provided
How was the fund delivered on the ground in terms of expanding capacity at existing groups?	Supporting Evidence	Key evidence source	Evidence not provided	Evidence not provided	Supporting Evidence	Supporting Evidence	Evidence not provided	Evidence not provided	Evidence not provided
What could be improved with the delivery?	Key evidence source	Key evidence source	Supporting Evidence	Evidence not provided	Key evidence source	Evidence not provided	Evidence not provided	Evidence not provided	Evidence not provided
Impact Evaluation
What was the impact of participating in uniformed youth groups on social and emotional skills, practical skills and wellbeing?	Evidence not provided	Evidence not provided	Key evidence source	Key evidence source	Evidence not provided	Evidence not provided	Evidence not provided	Key evidence source	Evidence not provided
Did these impacts vary by the types of activities delivered?	Evidence not provided	Evidence not provided	Key evidence source	Key evidence source	Evidence not provided	Evidence not provided	Evidence not provided	Supporting Evidence	Evidence not provided
What was the impact of participating in uniformed youth groups on educational attainment?	Evidence not provided	Evidence not provided	Supporting Evidence	Evidence not provided	Evidence not provided	Evidence not provided	Evidence not provided	Supporting Evidence	Key evidence source

5.2 Initial overarching theory of change

The initial overarching ToC was created through a series of workshops with staff members from each of the UYOs in the spring of 2023. It was then developed and refined by the evaluation team to capture the key causal pathways of interest to the impact evaluation. The YAB provided final input, including identifying priority outcomes and their hypothesised causal pathway.

Inputs

Across UYOs, inputs included funding to carry out activities, the support of volunteers who offer their time and skills, facilities or hubs located in the community to facilitate accessibility and convenience for young people, uniforms that aim to create a sense of unity and belonging, skilled and knowledgeable staff to deliver high-quality services, and existing partnerships and relationships with other organisations, stakeholders, and community members to provide support and resources, and support trust and credibility.

Activities

UYOs offer a wide range of activities for their members. These include structured leadership programmes and awards, which aim to help young people develop leadership skills. To help ensure that everyone has the knowledge and skills needed to participate in various activities and programmes, UYOs offer training for both volunteers and young people. They also organise local or international residentials providing young people with opportunities to experience outdoor education, develop teamwork skills, and form lasting friendships.

Many UYOs provide opportunities for young people to engage in youth social action to encourage young people to engage in social and environmental issues and make a positive impact in their communities. They also offer mentoring and volunteering opportunities within the community.

UYOs offer opportunities for young people to participate in various sports and outdoor activities, promoting physical activity and healthy lifestyles, arts and crafts and other activities that promote positive mental health. They also offer specific activities tailored to their organisation’s mission and objectives, such as skill development in first aid or faith development.

Mechanisms of change

For the intervention to be effective and achieve the intended outcomes, the following mechanisms of change were identified as having to be in place:

• UYOs provide a nurturing, affirming, and inclusive community for young people to build trusted relationships with their peers, staff and their communities.
• The UYOs will offer a wide variety of fun and accessible activities that provide young people with opportunities to learn and grow in a supportive environment.
• Within UYOs, young people have a consistent emotionally and physically safe yet challenging space where they can experience failure and develop resilience.
• The UYOs encourage young people to take self-directed action, which empowers them to make positive changes in their communities and develop leadership skills. Giving young people this agency means they feel trusted and are encouraged to make informed choices.
• UYOs support a sense of collective identity among their members through the use of a uniform and shared experiences. This collective identity promotes a sense of belonging and unity within the group.

Overarching objective and outcomes

The overarching intended outcomes for young people participating in UYOs included improvement in their wellbeing, to increase community cohesion, and contribute to the development of skills that young people will benefit from over their life course.

In the short-term, the ToC outlines that participating in UYOs is expected to lead to increased happiness for young people as they engage in a range of fun and accessible activities with their peers. This in turn is expected to increase their life satisfaction, improve their mental health and contribute to a long-term effect on their overall wellbeing. In addition to increased happiness, young people may also experience increased confidence and a sense of empowerment as they are given opportunities to take on responsibilities and leadership roles and are encouraged to learn from their mistakes. This is expected to increase their autonomy and contribute to a longer-term effect on their resilience and self-esteem. Ultimately this is also anticipated to improve their long-term wellbeing.

The ToC also outlines that UYOs can foster a greater sense of safety and trust between peers and with the wider community, providing a consistent and nurturing environment where young people can build strong and trusted relationships with those they would not have otherwise (encouraging mixing of different communities). These safe and trusted relationships allow young people to build their confidence learning from mistakes. Another anticipated outcome was a sense of collective identity that membership can create for young people. This supports a sense of belonging to a movement, national and/or international community. In the longer term this was seen to encourage active citizenship, prosocial behaviours, a habit of volunteering and community engagement. Ultimately this is seen as vital for supporting community cohesion and reducing negative behaviours.

The ToC illustrates that participating in UYOs is expected to provide young people with valuable life and work skills, including leadership, communication, interpersonal skills, and problem-solving, as well as recognised qualifications and technical skills relevant to their UYO. There are also opportunities for young people to develop key emotional and social skills such as emotion management, empathy, teamwork, responsibility, and initiative, which may be beneficial to young people’s attitude, engagement and progress within education. For the older age groups, learning skills that are valued by employers and increased networking to support young people into the labour market were outlined as key outcomes of being a UYO member.

Assumptions

1. The key assumptions that underpin the ToC include: The programmes offered are relevant and appealing to young people. The structure and variety of activities attract a diverse group of young people.

2. Delivery is high-quality and those running programmes are trained to create safe spaces for young people.

3. Young people remain engaged over time to ensure the outcomes are realised.

4. The sense of belonging to a local group transfers to a wider sense of belonging to a larger community over time.

5.3 Methodology

5.3.1 Ethical considerations

Ipsos draws on relevant ethical codes such as the Economic and Social Research Council (ESRC) Research Ethics Framework, the Social Research Association (SRA) ethical guidelines and the Market Research Society (MRS) Code of Conduct. As the evaluation involved young people taking part in surveys, remote in-depth interviews, focus groups, and case study research, Ipsos ensured that the appropriate measures were put in place so that the research followed ethical guidance. Researchers were all Disclosure and Barring Service (DBS) checked and remained acutely aware of their safeguarding duties throughout the evaluation.

All participants were asked to provide informed consent at the recruitment stage, after being sent age-appropriate information sheets and privacy notices. We made sure that participants being asked for consent to take part, were able to make an informed decision by developing an information sheet explaining the research, what and who it is for, its voluntary nature, details on confidentiality and anonymity, our disclosure policy, and contact details for the research team to help answer any further questions.

For under-16s, we secured parental/carer consent to approach young people before asking participants for their own consent. To ensure that participants did not feel pressure to take part in the research, we minimised the risk by making it clear at all stages of the research that participation is entirely voluntary and that a young person’s decision about whether they take part will not impact them. At the start of all interviews and focus groups with young people, researchers emphasised that participation is voluntary and that they can stop the interview at any time. Survey participants were asked to provide permission to be recontacted for qualitative research activities within the project. We were aware that the sample may also include people with learning disabilities, cognitive impairment, mental/physical health issues, accessibility needs, and those who speak limited English. Researchers have remained proactive and responsive in accommodating the needs of all participants and ensuring that tools and approaches are accessible and inclusive.

There are appropriate channels in place for team members to action safeguarding duties if they are concerned for the wellbeing of any participants. Young people who have taken part in the qualitative research activities were provided with information about how to access support if they feel distressed or feel that they are at risk of harm or concerned for their wellbeing.

5.3.2 Youth Advisory Board (YAB) detail

The YAB was engaged across seven workshops to provide feedback on multiple aspects of the evaluation including the overarching ToC, survey questions, best practice for interviewing young people and disseminating findings. The objectives and findings from each of the workshops are presented in Table 5.1.

Table 5.2: Youth Advisory Board workshop objectives and findings

Workshop number	Format	Number of Attendees	Timing	Objectives	Findings
Workshop One	In-Person	8 young people	June 2023	The first workshop was held in person in London, aiming to familiarise everyone with each other and the evaluation. In this workshop, board members were given information about the evaluation and their role, shared their intentions and hopes, as well as an opportunity to get to know one another. In July, Dartington held an online catch-up session with four board members who were unable to attend the in-person session.	The board shared their aims and hopes for joining the YAB, which could be grouped into three categories: To improve research accessibility and give young people a non-critical platform; to share feedback and offer positive contributions. To co-create a safe space for everyone to feel comfortable to contribute. To acknowledge the importance of and include diverse lived experiences as evidence in decision-making.
Workshop two	Online	8 young people	July 2023	The second workshop invited the board to review the revised ToC for the evaluation in terms of language, structure, and relevance to young peoples’ experiences. The YAB members also delivered presentations about their UYO to the group.	The board identified the common goals and/or outcomes across UYOs based on members’ presentations and these largely overlapped with outcomes in the ToC: Having fun and building networks with other people. Building a sense of belonging in safe environments. Young people have the opportunity for new experiences and explore personal development through trial and error. Structured activities and learning prepare young people for the real world and workplace by developing their life and employment skills. The board gave some minor comments about some of the more technical language in the ToC which led to removing or amending this language. Other elements of the ToC were split, and some of the causal logic revised. Finally, the board chose the three most relevant outcome pathways (highlighted by thicker outlines on Figure 1.1).
Workshop three	Online	9 young people	July 2023	The workshop was set to review quantitative outcomes measurement tools to collect data for the impact evaluation. To align with the survey development, the board contributed some survey design principles, tested the YPS (Young People’s Survey created by the Centre for Youth Impact) questions, and suggested areas for measurement in the survey that they thought would reveal the impact UYOs have on members.	The board discussed a number of factors that will make young people more likely to complete a survey including language, layout, length, question forms and deployment methods. In summary, the YAB thought the survey would need to look professional with good visual designs and straightforward access; the questions asked are succinct and relevant to UYOs; multiple choices and open questions are more inviting to young people; appreciation needs to be shown for participants’ time and contribution with incentives (e.g. voucher draw) as optional rewards if the survey is longer than 10 minutes; and research and survey intentions must be clearly explained in all sections. These comments were applied to revising the first draft of the survey, which was later shared with the YAB for further feedback. Through a voting and prioritising activity, the board selected the five most relevant questions for the survey to enquire about social and emotional skills that young people may have developed at UYOs. In discussions about what to measure to best manifest UYOs’ impacts on members, common themes emerged, which can be seen in the finalised survey questionnaire. The common themes were: support and trust, fun, sense of safety, sense of belonging and community, sense of accomplishment and confidence, and friendships and peer learning.
Workshop four	Online	10 young people	September 2023	The workshop was planned to take place in August 2023 but was delayed to September. It focused on interview deployment plans and suggested interview questions.	Through breakout groups, the YAB discussed approaches to consider for the young people interviews and other qualitative work. A clear introduction to the research and interviews. Flexibility of time and forms, including options of both in-person and online interviews to suit individual situations, options for interviewees to bring along 1-2 trusted adults/friends and setting a time range of 20-30 minutes for interviews to accommodate individual needs. Interviewers take on a more casual or formal style to match that of the interviewee. Multiple data recording methods to cater to interviewees’ preferences. A safe and familiar environment where interviewees can access trusted adults. Interviewers show understanding of what’s said (e.g. providing timely feedback and/or encouragement). Accommodate UYO-specific requirements (e.g. female interviewers preferred for Girl Guiding members). In the same workshop, the YAB were also invited to a role-play game where they interviewed each other in pairs or threes. In this space, they came up with potential interview questions that would assess the effect that UYO membership has as related to the mechanisms and short-term outcomes set in the ToC, in a way they thought young people would understand and respond to. These have informed the development of the topic guides for young people interviews.
Workshop five	Online	7 Young people	April 2024	This workshop was originally planned for February 2024 to focus on interpretations of data collected from surveys and interviews, which would feed into the interim report. However, because of a change in the method for the survey and an extension to the survey window, early insights from the survey were included and the workshop focused on reviewing and testing the topic guides for the next round of interviews with young people.	The YAB were presented with the contribution claims to be tested in the interviews and related questions from the topic guide. They were asked for their thoughts related to: Timing given to each section and questions; The accessibility and sensitivity of the language; The relevance of questions to their uniformed youth group; The relevance of the question to the contribution claim. In general, the YAB thought the interview questions were well connected to the contribution claims and could elicit relevant answers. Many of their comments centred on improvements to make questions more individual- and/or UYO-specific for interviewees to feel included and give more relevant answers. They also suggested where parts of the interview could be shorter, including questions that might not be relevant to all young people or UYOs, and which might need more time; where questions could be clearer; and where prompt questions might be leading. YAB members also shared their perspectives on some of the early survey findings presented.
Workshop six	Online	7 young people	November 2024	The timing of this workshop was adjusted to fit in with the completion of the analysis of round 1 of the interviews with the young people for the impact evaluation. The workshop sought to sense check the findings of the analysis with the YAB focusing on understanding; what resonated with them, what they found surprising, what did not seem correct and what they thought might be missing. These questions were explored for each of the contribution claims and the alternative explanations.	The YAB provided a depth of insight and clarity to the contribution claim analysis for the round one interviews and the alternative explanations. A short summary of some of the key topics is provided below: Claim 1 – uniform as a symbol of community, UYO uniforms have an inherent sense of meaning and history, promotion of equality and unity, respect and recognition. Claim 2 – forming social connections, close-knit friendships, confidence, teambuilding through a UYO activities which are unique and different to what is offered elsewhere. Claims 3 and 4 – UYOs providing consistency, socialisation, alternative learning spaces, motivating tasks and have an increasing awareness and support of mental health. Claim 5 – providing a supporting and caring environment, varied levels of support from UYO staff, providing leadership opportunities. Claims 6 and 7 – practical skills building which is accessible for the young people. Highlighted the diverse motivations for participation in UYOs. Claim 8 – volunteering beneficial for both personal growth and supporting their community. Being visible in the community.
Workshop seven	Online	7 young people	April 2025	This workshop was timed to coincide with the completion of the contribution analysis of the young people interviews following the second round of interviews. There were a number of questions and points of clarification arising from the contribution analysis so the workshop was designed to focus on conducting sense-making of the contribution analysis findings and exploring those areas where further information was required to more fully interpret and contextualise the findings. Time was also given to discussion of how young people might most usefully engage with the evaluation to inform the design of young person focused outputs. Finally, the planning process was initiated for a final celebration event for the YAB.	The design of this workshop was informed by the questions raised and gaps in understanding identified by the research team. Some of the key constructs explored with the YAB and their resulting insight used to inform the interpretation of the analysis are summarised below: Feeling safe and supported enabling young people to try new things, to not fear failure and being confident to engage with new experiences; Resilience understood as being able to handle things when feeling overwhelmed, weathering the storm – bending but not breaking, knowing there are other people there to support you; UYOs support young people’s resilience by providing a support network of both peers and leaders. Having trusted people there to listen and help come up with ideas when needed; The concept of ‘fun’ as a multi-faceted construct including - activities, learning, the environment, flexibility, banter, self-expression; UYOs helping to maintain young people’s wellbeing as they feel part of something bigger than themselves, they have something to look forward to, they get opportunities they would not get elsewhere, there is both development and reward from participation, helps the young people to understand their strengths and weaknesses in a supportive environment; Participation in UYOs helped young people to develop both career specific skills and transferable skills. It supported their engagement with school and social situations outside; UYOs helping young people to understand the value of community, providing the opportunity to feel valued and trusted, being part of something; Young people understand themselves as part of many different communities that intersect and overlap.

5.3.3 Quasi-experimental design selection detail

Several study designs were considered but proved unfeasible given the study’s constraints.

Although most organisations had a waitlist, a waitlist control design was not feasible due to organisational differences. Each participating organisation used different selection criteria and mechanisms, making their waitlists incomparable. Unlike programmes with standardised scoring systems, these organisations employed varied assessment approaches. Being waitlisted at one organisation could reflect capacity constraints, whilst at another it might indicate specific skill gaps or geographic factors. This variation would create significant bias, as waitlisted individuals would not represent equivalent comparison groups. The resulting heterogeneity would undermine valid causal inference, leading us to adopt alternative quasi-experimental methods that better account for organisational diversity.

Regression discontinuity design was not possible as participating organisations lacked standardised scoring systems or clear cut-off points for selection. Their varied selection mechanisms meant no sharp threshold existed around which to compare similar applicants. Difference-in-differences analysis was also ruled out due to the absence of pre-treatment baseline data—a consequence of the irregular participation patterns typical in this context.

These constraints, combined with relatively modest sample sizes, led to the selection of inverse probability weighting over traditional matching approaches. Matching methods are more data-intensive and risk losing participants who fall outside the common support region, potentially reducing statistical power. Whilst the two survey waves do not provide before-and-after comparisons, they allow us to test whether estimated effects remain consistent across different time measurements, strengthening confidence in our results. The chosen approach maximises sample utilisation whilst maintaining analytical rigour.

5.3.3 Quasi-experimental design selection detail

We used contribution analysis (CA) to assess whether and how UYO membership contributes to observed outcomes for young people. CA is a common method of theory-based impact evaluation that provides a robust framework for systematically assessing whether an intervention contributed to a set of expected outcomes. In this context, the ‘intervention’ is UYO membership, and the ‘expected outcomes’ include the outcomes identified in the overarching ToC – such as improved confidence, better skills for life and work, and improved wellbeing. CA also supports an understanding of how and why UYO membership contributes to these outcomes, by exploring the changes that take place at the individual level to produce each outcome. These changes can be described as ‘mechanisms of change’.

The CA approach is well-suited to the scope and nature of the impact evaluation research questions for several reasons:

• CA has an explicit focus on identifying possible alternative explanations of impact. In the context of this evaluation, this involves considering other factors (aside from UYO membership) that might have contributed to change. CA involves collecting data specifically to test these alternative explanations. This is especially valuable in complex settings, such as in the case of the eight UYOs, where multiple other factors (such as other activities young people take part in) may contribute to the same long-term outcomes.
• CA is underpinned by a ToC approach. This involves identifying the key causal pathways in the ToC (see section 3.1.1 and Figure 3.1 of this report). A causal pathway is a sequence of steps that explains how a specific long-term outcome is expected to be achieved. Once the key causal pathways have been identified, CA involves developing these pathways into contribution claims. A contribution claim is a statement that explains how part of an intervention is expected to contribute to a particular outcome. Collectively, the contribution claims form a contribution story. The contribution story explains how the overall intervention is expected to lead to the outcomes of interest.

Contribution claims were developed from the overarching ToC, which captured intended outcomes for young people across all eight UYOs. This ToC has been developed with input from representatives across all eight UYOs.

Our analysis plans for the theory-based impact evaluation adheres to the four key principles of CA:

1. Drawing on expert knowledge to define the causal pathways to be tested. In the context of this evaluation, expert knowledge primarily includes input from the UYO team members who have been directly involved with the co-production of the overarching ToC. The experience and knowledge of these team members has been used to contextualise and add nuance to the key causal pathways. The overarching ToC workshop also involved a discussion focusing on mechanisms of change, to ensure that the contribution claims that will be tested by CA are relevant, appropriate, and important. Expert knowledge also included input from young people themselves, via the YAB.
2. Using iterative data collection and analysis to test the causal hypotheses of the intervention. An iterative approach allows data collection methods and tools to evolve throughout an evaluation in response to emerging findings. Learnings from the initial case study fieldwork were used to refine and contextualise the contribution claims, deepening our understanding of how each contribution claim is expected to operate in the context of each UYO. The topic guides for the first round of qualitative data collection were then tailored to ensure that the data collected could be used to robustly assess the contribution story of UYO membership. The topic guides were then updated again after the first round of fieldwork based on emerging findings, as well as any gaps in the data.
3. Recognising that other factors affect the change being measured and accepting that the intervention might not be the most valid explanation. The CA framework, as well as the data collection tools themselves, were designed to identify and explore alternative explanations for any observed outcomes. This allowed us to provide a holistic, comprehensive, and well-reasoned assessment of the UYOs’ contribution to observed outcomes.
4. Focusing on the credibility and plausibility of the story, and transparency about the logic underpinning it. By focusing on understanding and testing the causal mechanisms within each contribution claim, we were able to make a strong assessment of the overarching contribution story.

Developing the contribution claims

Contribution claims are statements that describe how an intervention is expected to contribute to an outcome of interest. Contribution claims relate closely to the causal pathways within an intervention’s ToC. Using the Overarching Theory of Change, we developed an initial set of eight core contribution claims (collectively forming a contribution story). Each contribution claim was considered applicable to some or all the eight UYOs, but with slight variations in the way it was expected to operate for each UYO, depending on its context.

These contribution claims were then validated and contextualised by the case study fieldwork, before being tested in more detail by the longitudinal interviews with young people. Due to the iterative nature of CA, the contribution claims were revised throughout the evaluation process, including at the final analysis and reporting stage.

Case study analysis – setting the landscape for CA

We used the case study fieldwork to contextualise the contribution claims. We developed an analysis framework for the analysis of case study data, which contained one worksheet for each of the eight case studies, within which evidence was mapped out against key themes, including: how activities were delivered; how young people were engaged; types of young people engaged; relationships between young people, volunteers, and staff; context of local area; how activities have changed over last 6 to 12 months; relationships between UYO and local community; impact of UYO membership on young people; impact of UYO on local community. These fields were populated with qualitative data collected throughout case study visits, including data from observational analysis and interviews with young people, volunteers, delivery staff, and wider stakeholders.

Following the analysis of case study data, we produced a concise, two-page summary of findings for each case study. These summaries provided a snapshot of each UYO’s activity at the local level, including verbatim quotations where relevant. The summaries set out the contextual landscape within which the contribution claims are expected to operate for each UYO.

The case study analysis and findings were used to validate and refine the initial set of contribution claims, which involved tweaking the wording of contribution claims as necessary to reflect the findings from case study analysis. The topic guides for interviews with young people were then refined on the basis of the updated contribution claims, to ensure that the remaining qualitative data collection activities tested the contribution story robustly, systematically, and consistently.

Contribution analysis framework

Following the case study fieldwork and the finalisation of the hypothesised contribution story, we developed a contribution analysis framework that was structured around the core contribution claims. The framework included: a breakdown of the contribution claims (including key mechanisms of change); key data and evidence that supports the contribution claim; key data and evidence that identifies or supports alternative explanations; and considerations of strength of evidence. Strength of evidence was assessed by considering factors such as diversity of voice, volume of voice, authority of voice, and considerations of bias. This included assessing whether evidence claims came from multiple sources (diversity of voice) and the credibility and expertise of the source (authority of voice) to determine how much weight to give to the evidence. For example, more weight was placed on outcomes reported by young people directly relative to outcomes about young people indirectly reported by staff or volunteers.

The framework was broadly divided into two sections – ‘UYOs’ Contribution Story’ (containing evidence of how UYO membership contributes to key outcomes), and ‘Alternative Performance Stories’ (containing evidence of alternative explanations for key outcomes alongside each contribution claim). This contribution analysis framework was populated with qualitative data collected through in-depth interviews with young people. After each round of interviews, we identified key evidence gaps and the topic guides for the remaining interviews were further refined to focus on collecting data that could address these gaps, ensuring that we were in a strong position to robustly assess the full contribution story once we had completed all qualitative fieldwork. The topic guides were also updated to include questions about any emerging alternative explanations.

Once the contribution analysis framework was populated, we analysed each of the core contribution claims. Analysis focused on identifying and assessing the strength of evidence to support each claim, and the strength of evidence to support any alternative performance stories (i.e. alternative explanations). We explored changes in perceived outcomes (i.e. changes in how the contribution claims are operating) throughout the course of the evaluation at each of the data-collection time points. Analysis considered whether and how the impact of UYO membership on young people differs according to personal characteristics (including age), nature/consistency of engagement, and type of UYO. The analysis of the individual contribution claims was then pieced together to provide a comprehensive assessment of UYOs’ contribution story as a whole.

5.3.5 Contribution claims and alternative explanations

Below are the initial contribution claims developed for the evaluation.

Wellbeing

• 1. Experiencing a supportive and inclusive environment helps young people feel safe and develop trusted relationships with peers and adults, which contributes to their happiness, life satisfaction, and mental health and wellbeing.

• 2. Participating in activities young people enjoy, including various sports and outdoor activities that promote physical activity and healthy lifestyles and/or other activities such as arts and crafts, contribute to increased happiness, life satisfaction, and mental health and wellbeing. Taking part in positive activities can also prevent/reduce negative behaviours e.g. anti-social behaviour. Improvements in overall health and behaviour can reduce societal costs in the long-term.

Social emotional development

• 3. Spending time with other young people, including team-building and structured activities, contributes to young people’s development of cooperation skills (communication, interpersonal, teamwork) and more friendships/social networks.

• 4. Experiencing an emotionally and physically safe yet challenging space where young people can try new activities and experience both achievements and failure allows them to build confidence and self-esteem and learn from their mistakes, which contributes to greater resilience.

Skills and attainment

• 5. Engaging in fun and diverse activities outside of school that teach them valuable life and work skills (leadership, emotion management, empathy, responsibility, initiative, and problem solving) and having access to recognised qualifications/technical skills contributes to better attitudes towards, engagement and progress with education and, among older young people, access to the labour market.

• 6. Taking on responsibilities and leadership roles develops young people’s leadership skills and ability to make informed choices, which contributes to their confidence and sense of empowerment. Over time, this improves their autonomy and self-esteem.

Community connectedness

• 7. Wearing a uniform and sharing experiences with other young people contributes to their sense of (collective) identity and belonging within their UYO and/or wider community (e.g. faith, movement, national, international).

• 8. Encouraging young people to take self-directed action in their community (e.g. mentoring and volunteering, addressing social/environmental issues) leads to increased social action and civic participation and empowers young people to make positive changes in their communities, which contributes to their development as active citizens e.g. creating a habit of volunteering. Taking part in social action also supports community cohesion.

Alternative explanations

• AE1 Other activities: Young people are engaging in other out-of-school activities (e.g. youth clubs, volunteering) following similar pathways and leading to similar outcomes.
• AE2 School activities: Young people are engaging in school-run activities (e.g. sports, music, theatre) following similar pathways and leading to similar outcomes.
• AE3 School qualifications: Young people are engaged at school and complete exams and gain qualifications, which allows them to progress into further education and in the labour market.
• AE4 Other adult relationships: Young people are building trusted adult relationships in other settings (e.g. school, mentoring programmes).
• AE5 Other peer friendships: Young people are developing valued friendships in other settings (e.g. school, sports, youth clubs).
• AE6 Other networks: Young people have strong personal and community networks providing them with a sense of belonging and identity (e.g. faith groups, sports clubs).
• AE7 Social capital: Young people joining UYOs have the social capital (i.e. don’t come from deprived backgrounds) that will naturally build their confidence and leadership skills with time.

5.3.6 Further detail on young people interview sample

Table 5.3: Young people interview sample by region

Region	Interview 1	Interview 2
Midlands	8	6
East England	4	4
Greater London	5	5
South East	4	4
South West	4	4
Wales	2	1
North West	7	5
North East	2	2

5.3.7 Further detail on case study sample

Table 5.4: Case study participants

	Young People	Staff	Volunteers	Wide stakeholders	Total number of interviews
Boy’s Brigade	7	0	3	0	10
Girl’s Brigade	5	0	2	1	8
Girlguiding	8	0	3	0	11
JLGB	9	0	2	0	11
Marine Society and Sea Cadets	10	0	2	0	12
Scouts	12	1	2	0	15
St John Ambulance	9	1	1	0	11
Volunteer Police Cadets	8	0	2	0	9
Total	68	2	17	1	88

5.4 Findings

Table 5.5 provides an explanation for contribution analysis ratings provided in the main body of the report.

Table 5.5: Contribution analysis ratings

Updated contribution claim 1	Contribution strength	Evidence confidence	Explanation
Contribution claim 1: Experiencing a supportive and inclusive environment helps young people feel safe and develop trusted relationships with peers and adults, which contributes to their happiness, life satisfaction, and mental health and wellbeing	Strong	Strong	Clear linkages between UYO environment and relationships and wellbeing. Evidence collected directly from young people. Validated ONS wellbeing scale. Consistent findings across interviews and surveys with young people.
Contribution claim 2: Participating in activities young people enjoy, including various sports and outdoor activities that promote physical activity and healthy lifestyles and/or other activities such as arts and crafts, contribute to increased happiness, life satisfaction, and mental health and wellbeing. Taking part in positive activities can also prevent/reduce negative behaviours.	Strong	Strong	Clear linkages between UYO activities and wellbeing, often alongside other positive activities. Evidence collected directly from young people. Validated ONS wellbeing scale. Consistent findings across interviews and surveys with young people.
Contribution claim 3: Spending time with other young people, including team-building and structured activities, contributes to young people’s development of cooperation skills (teamwork, communication, confidence, openness) and more valued friendships and deeper connections with other young people.	Strong	Moderate	Clear linkages between UYO opportunities and development of socio-emotional skills. Evidence collected directly from young people. YPS scale tested with young people but truncated for this study with only one statement per sub-domain. Consistent findings across interviews and surveys with young people.
Contribution claim 4: Experiencing an emotionally and physically safe yet challenging space where young people can try new activities, develop new skills, and experience both achievements and failure allows them to build confidence and self-esteem and learn from their mistakes.	Moderate	Moderate	Clear linkages between UYO opportunities and development of confidence/self-esteem, but weak links to resilience. Evidence collected directly from young people. BRS scale not developed for pre-teen/teenage population.
Contribution claim 5: Engaging in fun and diverse activities outside of school that teach young people valuable life and work skills (leadership, communication, responsibility, initiative, and problem solving) contributes to better attitudes towards and engagement with their current education, improved attainment, and, among young people attending sector-specific UYOs, improved access to the labour market.	Moderate	Moderate	Clear linkages between UYO activities and development of skills, but weaker links for education and labour market. Evidence collected directly from young people. Objective NPD attainment data but small sample size.
Contribution claim 6: Taking on responsibilities and, among older members, leadership roles develop young people’s leadership skills and ability to make informed choices, which contributes to their confidence and sense of empowerment. Over time, this improves their initiative and self-esteem.	Strong	Moderate	Clear linkages between UYO activities and development of skills and confidence. Evidence collected directly from young people. Survey statements not validated. Consistent findings across interviews and surveys with young people.
Contribution claim 7: Wearing a uniform and sharing experiences with other young people contributes to their sense of collective identity and belonging within their UYO. Their sense of collective identity and belonging increases with sustained participation in the UYO over time.	Strong	Moderate	Clear linkages between UYO membership and uniforms with sense of belonging and collective identity. Evidence collected directly from young people. Standard survey statements from Community Life but not tested on young people. Consistent findings across interviews and surveys with young people.
Contribution claim 8: Giving young people opportunities to take part in community activities through their UYO leads to increased social action and civic participation and empowers young people to engage with their communities in a positive way. Modelling social action and civic participation by UYO leaders encourages young people to engage in these behaviours.	Moderate	Moderate	Some linkages between UYO membership and social action and civic participation. Alternative explanations appeared to have a stronger contribution. Evidence collected directly from young people. Reliant on interview data only.

Table 5.6 presents the responses from UYO members about their participation in various activities as part of their UYO. Table 5.7 presents the responses from the full sample about whether they had participated in particular activities that might have similar characteristics and benefits to attending a UYO.

Table 5.6: UYO activities over the past six months

	Total
Base: All Wave 2 treatment	403
Weekly or fortnightly meetings in-person	343
	85%
Online meetings	53
	13%
Music or drama activities	77
	19%
Playing sports	190
	47%
Life skills such as cooking or money management	192
	48%
Reflection, wellbeing or mental health-focused activities	184
	46%
Arts and crafts activities	209
	52%
Outdoor activities such as climbing, hiking or sailing	218
	54%
Stayed overnight - camping	192
	48%
Stayed overnight – indoors	150
	37%
Attended an event in your community	272
	67%
Visited another country outside the UK	25
	6%
Worked towards an award or badge specific to [UYO]	307
	76%
Worked towards another award or qualification	158
	39%
Volunteered or helped other outside [UYO]	172
	43%
Worked with younger age groups, or worked towards becoming a young leader or group leader	175
	43%
None of these	6
	1%
Prefer not to say	4
	1%

Table 5.7: Out-of-school activities, UYO members vs comparison

	Total	UYO Members	Non-UYO Members
Base: All Wave 2 respondents	1179	406	773
Going to a youth club	114	57	57
	10%	14%	7%
Playing sports or sports training, including martial arts	615	222	393
	52%	55%	51%
Religious services, groups or classes	188	77	111
	16%	19%	14%
Extra tuition in school subjects	198	68	130
	17%	17%	17%
Reading books for pleasure, or a book discussion group	524	208	316
	44%	51%	41%
Music lessons, such as singing or playing an instrument	323	156	167
	27%	38%	22%
Performing arts, such as dance, drama or choir	236	107	129
	20%	26%	17%
Another type of artistic activity, such as craft, painting, or creating animations	267	103	164
	23%	25%	21%
Visiting Museums, galleries or historic places	301	123	178
	26%	30%	23%
Video gaming	585	219	366
	50%	54%	47%
Board games or tabletop games	365	144	221
	31%	35%	29%
Voluntary or community work, helping others	285	104	181
	24%	26%	23%
Going to political meetings, rallies or demonstrations	42	16	26
	4%	4%	3%
Other (please specify)	109	36	73
9%	9%	10%
I don’t take part in any activities outside school	56	14	42
	5%	3%	5%
Prefer not to say	6	2	4
	1%	0%	1%

6. Technical Annex: QED study

6.1 Introduction

This annex outlines our quasi-experimental approach to assessing the impact of UYO membership on young people’s development. Whilst randomised controlled trials represent the methodological gold standard, practical constraints necessitated an alternative design that still enabled robust causal inference.

Our evaluation framework examined both immediate and longer-term outcomes. The primary focus included wellbeing and socio-emotional competencies, measured through validated instruments: the ONS4 wellbeing questions, the Brief Resilience Scale (BRS), and an adapted version of the Young People’s Survey (YPS). We also assessed community engagement through three key civic efficacy and belonging indicators. Data collection occurred at two distinct time points via participant surveys.

For longer-term outcomes, we examined educational attainment through NPD linkage, using participants’ personal identifiers including name, date of birth and postcode to track academic progression.

Whilst our weighted, regression-adjusted approach creates a stronger counterfactual than simple regression, we cannot control for unobservable factors affecting both programme participation and outcomes. This limitation means our findings should be interpreted as suggesting causal effects rather than conclusively establishing them. The following sections provide a detailed account of our methodology, analytical strategies, findings, and limitations.

6.2 Aims and research questions

6.2.1 Aims

This study aimed to determine the causal impact of UYO membership on young people’s development. Given the inability to randomise membership, we designed a quasi-experimental approach to identify these effects whilst accounting for selection bias.

6.2.2 Research Questions

The evaluation addressed the following research questions:

• Short-term impacts: To what extent did UYO membership affect young people’s wellbeing as measured by the ONS4 indicator?
• Social-emotional development: Did participation in UYOs lead to measurable improvements in young people’s social and emotional skills as assessed through the Brief Resilience Scale (BRS) and adapted domains from the Young People’s Survey (YPS)?
• Educational outcomes: What impact did UYO membership have on participants’ educational attainment, as measured by GCSEs, compared to non-members with similar characteristics?
• Differential effects: Did the impacts of UYO membership vary across different demographic groups?
• Dosage effects: Was there evidence of a relationship between duration of UYO involvement and the magnitude of observed outcomes?

6.3 Methodological Approach

Our study evaluates the causal impact of UYO membership using a robust statistical approach. We selected a doubly robust method.

We faced a fundamental challenge: young people choose to join UYOs rather than being randomly assigned. This self-selection creates potential bias since UYO members might differ systematically from non-members in ways that affect our outcomes of interest.

We implemented the inverse probability weighted regression adjustment (IPWRA) method to address this challenge. This approach offers significant advantages over more common techniques like propensity score matching (PSM) or coarsened exact matching (CEM). Both PSM and CEM discard unmatched observations, wasting valuable data, and rely entirely on a single model being correctly specified.

In contrast, our IPWRA method keeps all observations in the analysis by creating statistical weights rather than physically matching individuals. Instead of finding pairs of similar people and throwing away those who cannot be matched, we assign weights to each person in our comparison group to make the overall group statistically similar to UYO members. This means a comparison group member who looks very similar to typical UYO members gets a higher weight, whilst those who are quite different get lower weights. Crucially, IPWRA also provides a safety net: it combines this weighting approach with separate outcome models for each group. If our weighting model is imperfect, the outcome models can compensate, and if the outcome models are wrong, the weighting can still provide reliable estimates. This “doubly-robust” feature means we get consistent results as long as at least one of our two models is correctly specified, rather than requiring both to be perfect (Słoczyński et al., 2022).This provides greater confidence in our findings, which is particularly important when evaluating complex interventions.

It is important to note that while our approach controls for observable characteristics, some unobservable factors may remain unaccounted for. Although we aim to estimate causal effects, our treatment estimates may still contain some self-selection bias. Young people who join UYOs might differ in unobservable ways (such as motivation or personality traits) that also influence their outcomes.

The following sections detail our data sources, treatment definition, outcome measures, covariates, and the specific implementation of our estimation strategy.

6.3.1 Data

The evaluation combined survey data for short-term outcomes with administrative data for long-term outcomes through a carefully designed data collection strategy. Figure 6.1 displays the participant flow for the study.

For the treatment group, UYOs distributed our survey link to their members. The survey collected wellbeing and social-emotional skills measures using validated tools: ONS4 wellbeing questions, the BRS, and selected items from the YPS. We conducted two survey waves—Wave 1 in March 2024 and Wave 2 in December 2024. Having two waves allowed us to test whether the measurements are sensitive to seasonality or change in time.

The survey also requested consent for NPD linkage. For consenting respondents, we securely transferred identifying information (name, date of birth, and postcode) to the Department for Education (DfE). The DfE then matched these identifiers to NPD records, enabling analysis of educational outcomes. We drew a random sample of 30,000 pupils from the NPD for the comparison group. We distributed the same survey to this sample, with responses linked directly to their NPD records. Only respondents who provided explicit consent were included in the final analysis.

The resulting matched dataset contained survey responses measuring wellbeing outcomes, NPD data providing educational outcomes, and demographic information, and a treatment/comparison identifier.

Our analytical approach addressed the challenge of multiple memberships by collecting information on participation in other youth organisations or extracurricular activities, which we incorporated as control variables in our models.

Figure 6.1: Participant flow diagram

Use zoom on your browser to view.

Figure 6.1 shows the participant flow for the study which was described in section 6.3.1. The study began with a random selection of 30,000 pupils from the NPD and additional recruitment by the UYOs. After two waves of surveys, the final sample was 1,179 young people who responded to both waves of surveys. For the young people who consented to having their data matched to education outcomes, this was done using a pupil reference number or their names, date of birth and postcodes.

6.3.2 Treatment variable

We derived the treatment variable from self-reported responses in the second wave (27 November to 20 December 2024). Individuals who reported being part of a UYO are classified as treated. Moreover, these individuals could be part of a UYO for less than a month or several years.

6.3.3 Outcomes

This evaluation examines four core outcome domains identified through ToC workshops with UYO stakeholders. These discussions established wellbeing, resilience, socio-emotional skills, and community engagement as priority areas that align with UYO programme objectives. The selected outcomes capture both personal development and wider social impact, reflecting the multi-faceted nature of youth development.

Understanding the measurement scales is essential for interpreting the impact findings in their proper context.

Office of National Statistics Personal Wellbeing Domain (ONS4)

The ONS4 provides a robust framework for measuring personal wellbeing among children and young people. This tool comprises four distinct questions that assess different aspects of subjective wellbeing and are combined into a composite wellbeing indicator (ONS, 2025).

The four questions cover life satisfaction, sense of purpose, happiness, and anxiety. Respondents rate each aspect on a scale from 0 to 10, with higher scores generally indicating greater wellbeing (except for the question regarding anxiety, which is reverse-scored, marked with [R]). The ONS4 has been widely used in national surveys since 2011, including the Annual Population Survey and the Millennium Cohort Study. This extensive application supports its validity as a measure.

For our evaluation, the ONS4 offers several advantages. It is suitable for children aged 10 and above, making it appropriate for our target population. The questions are straightforward and easily understood, reducing potential confusion for participants. The tool’s brevity also minimises respondent burden whilst capturing key wellbeing dimensions. This efficiency makes it particularly valuable for educational research, where time constraints often apply.

Changes in ONS4 responses will help us assess whether the intervention influences participants’ subjective wellbeing across multiple domains. This information will complement our resilience measures to provide a more complete picture of the effects of the intervention.

Table 6.1: ONS4 definition of items from the questionnaire

Items	Question	Concept
ONS Item 1	Overall, how satisfied are you with your life nowadays, where 0 is “not at all satisfied” and 10 is “completely satisfied”?	Life satisfaction
ONS Item 2	Overall, to what extent do you feel that the things you do in your life are worthwhile, where 0 is “not at all worthwhile” and 10 is “completely worthwhile”?	Worthwhile
ONS Item 3	Overall, how happy did you feel yesterday, where 0 is “not at all happy” and 10 is “completely happy”?	Happiness
ONS Item 4	On a scale where 0 is “not at all anxious” and 10 is “completely anxious”, overall, how anxious did you feel yesterday?	Anxiety [R]
Overall ONS4 score	Composite measure of all questions	Overall wellbeing

Brief Resilience Scale (BRS)

The BRS measures an individual’s ability to recover from stress and adversity. This six-item self-report instrument assesses how quickly people bounce back from difficult experiences (Smith et al., 2008).

Participants respond to statements about recovery from challenges on a five-point scale ranging from “Strongly Disagree” (1) to “Strongly Agree” (5). Table 6.2 describes each questionnaire item and its intended concept.

The scale includes three positively worded statements and three negatively worded statements that are reverse-scored (marked with [R]). We preserved the original 1-5 scale for analysis, with the overall resilience score calculated as the average across all six items.

BRS scores should be interpreted according to established thresholds: 1.00-2.99 indicates low resilience, 3.00-4.30 represents normal resilience, and 4.31-5.00 shows high resilience. Higher scores consistently reflect greater resilience capacity. However, the scale’s developers acknowledge that these thresholds may require adjustment based on age, gender, ethnicity, and health factors (Smith et al., 2023), and the scale’s validation has focused primarily on adult populations (Kunzler et al., 2018).

Table 6.2: BRS definition of items from the questionnaire

Items	Question	Concept
BRS Item 1	I tend to bounce back quickly after hard times.	Quick recovery
BRS Item 2	I have a hard time making it through stressful events.	Stress Management [R]
BRS Item 3	It doesn’t take me long to recover from a stressful event.	Rapid stress recovery
BRS Item 4	It is hard for me to snap back when something bad happens.	Adversity response [R]
BRS Item 5	I usually come through difficult times with little trouble.	Ease through challenges
BRS Item 6	I tend to take a long time to get over setbacks in my life.	Recovery Time [R]
Overall BRS score	Composite measure of all questions	Overall resilience

_{Note: those items with [R] means they have been reversed.}

The BRS provides a focused measure of psychological resilience rather than related factors like social support or coping strategies. This distinction makes it particularly valuable for evaluating interventions designed to strengthen resilience in educational settings.

Prior research has established strong psychometric properties for the BRS, with good internal consistency and test-retest reliability. The scale correlates appropriately with related constructs like optimism and social support (Fung, 2020; Konaszewski et al., 2020).

For our evaluation, changes in BRS scores will help determine whether the intervention enhances participants’ ability to recover from setbacks in educational contexts.

Young People’s Survey (YPS)

The study measures socio-emotional development using the Young People’s Survey (YPS), a validated self-report tool that assesses transferable functional traits. The full survey contains 24 items. Due to concerns about survey length and resulting survey fatigue for the young people, we adapted the survey to include one statement from the five of the six socio-emotional skill domains measured in the survey: empathy, problem solving, initiative, responsibility and teamwork. We selected statements that most closely aligned with the theory of change.

Each item uses a five-point scale from “Never” (1) to “Always” (5), where higher scores indicate greater competency (see Table 6.3). To ease interpretation, we converted responses to binary outcomes (0-1). Responses in the highest categories (4-5) were coded as 1, while lower values (1-3) were coded as 0.

Table 6.3: YPS definitions of items from the questionnaire

Item	Question	Socio-emotional skill domain
YPS Item 1	I feel bad when someone gets their feelings hurt	Empathy
YPS Item 2	I start a new task by thinking about different options for doing it	Problem solving
YPS Item 3	I work as long and hard as necessary to get a job done	Initiative
YPS Item 4	I take responsibility for my actions, even if I make a mistake	Responsibility
YPS Item 5	I respect other points of view, even if I disagree	Teamwork

Community and Belonging Outcomes

The evaluation examines community engagement through three distinct self-report items described in Table 6.4. Participants rate their agreement with each statement on a Likert scale from strongly disagree to strongly agree.

These measures capture how UYO participation might influence young people’s perception of their role and value within communities. For analytical clarity, we converted each item to a binary indicator, coding ‘agree’ or ‘strongly agree’ responses as 1, and all other responses as 0. Table 6.5 below summarises the scale range for each outcome.

Table 6.4: Community and belonging definitions of items from the questionnaire

Items	Question	Concept
Belonging Scale Item 1	I believe that young people have the power to make a difference in the world	Youth empowerment belief
Belonging Scale Item 2	I have the skills to make a positive change in my community	Community change skills
Belonging Scale Item 3	I feel like I am a valued member of my community	Community belonging

Table 6.5: Outcome measures scales

Outcome measure	Scale Range
ONS Wellbeing	0-10
BRS Resilience items	1-5
BRS overall	1-5
YPS items	0-1
Community Belonging items	0-1

Education attainment (NPS-Attainment 8)

Attainment 8 is the UK Government’s primary measure of secondary school pupil achievement at Key Stage 4 (GCSE level). Attainment 8 data is collected through the NPD, which contains comprehensive information about all pupils in state-funded schools in England, including their attainment, attendance, and demographic characteristics. The Attainment 8 measure evaluates pupils across eight qualifications, including mathematics (double-weighted) and English (double-weighted if sitting both English literature and English language), three qualifications that count towards the English Baccalaureate (EBacc), and three additional GCSE or approved non-GCSE qualifications. Each GCSE grade receives points on a 9-1 scale, where 9 represents the highest achievement and 1 the lowest (GOV.UK, 2024).

For example, a pupil achieving the maximum grade 9 in all subjects would score 90 points (8 × 9 = 72, plus double-weighting for English and mathematics adds 18 points) (DfE, 2025). In the 2023-24 academic year, the national average Attainment 8 score was 45.9 points out of a maximum 90 (GOV.UK, 2025).

6.3.4 Covariates

Our analysis includes several key demographic covariates. Age is calculated using date of birth and Wave 2 survey date. The Index of Multiple Deprivation (IMD) decile uses 2019 data to represent area-level deprivation based on postcode. We group self-reported ethnicity into six categories (White, Asian, Black, Mixed, Other, or Prefer not to say) to ensure sufficient sample sizes within groups. Region indicates the participant’s UK location. Gender includes six self-reported categories: Girl or Woman, Boy or Man, Non-binary, gender not listed, questioning, or prefer not to say. Health status captures whether participants experienced physical or mental illness within the past 12 months (Yes, No, Don’t know, or Prefer not to say).

Table 6.6 and Table 6.7 present descriptive statistics for the analysed sample for the primary outcome (ONS4). The total sample size for the primary outcome is 1,129; however, some characteristics have missing data for certain individuals, as shown by the varying sample size in the last column.

The sample’s average age os approximately 15, ranging from 10 to 19.[23] Girls comprise about 57% of the sample. Individuals of White ethnicity constitute approximately 75% of the sample. Approximately 24% of the sample reported experiencing a physical or mental health issue within the past 12 months. The IMD decile, ranging from 1 to 10 (with lower deciles indicating higher levels of deprivation), represents the decile in which an individual resides.

The statistics are broken down by UYO members and non-members to illustrate differences between these groups at the time of survey completion. It is important to note that the non-member group shown here represents the unweighted comparison sample used for descriptive purposes. This is not the same as the weighted comparison group used in the quasi-experimental analysis, which adjusts for observable differences between members and non-members. The substantial differences visible in these raw statistics highlight why weighting adjustments are necessary to create a balanced comparison group for causal analysis. These descriptive statistics therefore provide context for understanding both the characteristics of our sample and the rationale for our weighting approach.

Table 6.6: Descriptive statistics of the sample (unmatched)

Variable	Population	Mean	SD	Min	Max	N
Age	Total	14.96	1.96	10	19	1074
	UYO Member	14.38	2.02	10	19	366
	Non-UYO Member	15.26	1.86	11	19	708
Girl	Total	0.57	0.49	0	1	1129
	UYO Member	0.53	0.5	0	1	385
	Non-UYO Member	0.59	0.49	0	1	744
Health issue	Total	0.24	0.43	0	1	1129
	UYO Member	0.27	0.44	0	1	385
	Non-UYO Member	0.23	0.42	0	1	744
Low IMD decile (1-5)	Total	0.43	0.49	0	1	1066
	UYO Member	0.42	0.49	0	1	372
	Non-UYO Member	0.44	0.49	0	1	694

Table 6.7: Ethnicity distribution of the sample (unmatched)

Ethnicity	% Total (N=1129)	% UYO (N=385)	% Non-UYO (N=744)
White	75.29	86.23	69.62
Asian	11.34	6.23	13.98
Black	5.31	2.86	6.59
Mixed	4.96	3.12	5.91
Others	2.13	0.78	2.82
Prefer not to say	0.97	0.78	1.08

6.3.5 Estimation strategy

Our approach uses a statistical method called IPWRA. This technique helps us estimate the causal impact of UYO membership whilst accounting for potential selection bias.

The IPWRA method works in two complementary ways. First, it creates a model that predicts which young people are likely to join UYOs based on their characteristics. Second, it builds separate models for our outcomes of interest for both members and non-members. This dual approach gives our method a valuable property: even if one model is imperfect, our results remain reliable as long as the other model is correct.

Our analysis follows three clear steps. First, we create a statistical model to predict each young person’s likelihood of UYO membership based on their characteristics. This creates a probability score for each participant that reflects how closely they match the typical profile of UYO members. Young people with similar profiles to typical members receive higher probability scores, while those with vastly different characteristics receive scores closer to zero. This approach allows us to account for selection factors without assuming all treated participants have identical joining probabilities.

Second, we use these probabilities to create a balanced comparison through statistical weighting. Participants with very low probabilities of treatment receive lower weights in our analysis. This approach prevents our results from being skewed by including comparison participants who are fundamentally different from UYO members.

Finally, we calculate the average differences in predicted outcomes between UYO members and non-members. These differences represent our estimated effects of UYO membership.

The strength of this approach is its ability to create fair comparisons. By accounting for factors that influence both UYO membership and outcomes, we can better isolate the true impact of participation. This helps us answer whether UYO membership makes a difference and how much difference it makes for various outcomes.

6.4 Main findings

This section presents the causal impact of UYO membership on young people’s development across multiple domains. We examine wellbeing, resilience, socio-emotional competencies, and community belonging outcomes. The findings represent average treatment effects based on our quasi-experimental methodology. Full statistical results, including effects for different demographic subgroups and duration of participation, appear in detailed tables in the annex. Here, we focus on interpreting the key patterns and their practical significance for understanding UYO impact.

6.4.1 UYO membership effects on wellbeing and social-emotional development

This section presents the main effects of UYO membership across key outcome measures. The results show how participation influences young people’s wellbeing, resilience, social-emotional skills and community engagement compared to similar non-members.

Wellbeing (ONS4)

The ONS4 wellbeing measure shows a positive and statistically significant impact of UYO membership. Young people in UYOs reported higher wellbeing scores than comparable non-members. At Wave 2, UYO members scored 7.05 on average compared to 6.69 for the comparison group. This represents a treatment effect of 0.36 points (p=0.002). In percentage terms, UYO members scored 5.3% higher than their peers.

This pattern was consistent across both measurement periods. At Wave 1, UYO members scored 7.15 compared to 6.89 for the comparison group, with a treatment effect of 0.25 points (p=0.034). This represents a 3.7% higher score for UYO members.

The consistent positive difference across time points strengthens confidence that UYO membership supports better wellbeing outcomes among young people. Although our method has tried to minimise selection bias, some potential self-selection bias may remain in these findings. Despite this caveat, the consistent positive difference across time points suggests UYO membership supports better wellbeing outcomes among young people.

Resilience (BRS)

The resilience measures show mixed results between UYO members and the comparison group. Two specific aspects reveal notable differences. At Wave 2, UYO members scored significantly higher on stress management (2.86 versus 2.67), representing a 7.1% higher score (p=0.012) – UYO members had also scored marginally higher at Wave 1 (2.82 versus 2.68; p=0.05). They also scored marginally higher on Recovery Time at Wave 1 (3.32 versus 3.18), a 4.4% higher score (p=0.054).

The overall Resilience Score showed no statistically significant differences at either wave, though the direction remained consistently positive. These findings suggest UYO membership may specifically enhance young people’s stress management skills, whilst having a more limited impact on general resilience.

While our analysis accounts for socio-demographic factors, other unobservable characteristics might influence these outcomes.

Socio-emotional development (YPS)

The YPS results show varied differences between UYO members and the comparison group across socio-emotional competencies. UYO members score higher in empathy (79% vs 73% at wave 2), a 6-percentage point difference (p=0.023). They show the strongest advantage in teamwork (83% vs 72% at wave 2), a 10-percentage point difference (p<0.001).

Responsibility scores also differ meaningfully (78% vs 70% at wave 2), an 8-percentage point difference (p=0.010). Initiative shows a positive difference (67% vs 61% at wave 2), a 6-percentage point difference approaching significance (p=0.061). Problem solving shows a smaller, non-significant difference (51% vs 47% at wave 2, p=0.197).

UYO members scored significantly higher on teamwork, responsibility, initiative, and problem-solving at wave 1, suggesting good consistency in findings over time.

While our analysis controls for observable characteristics, some self-selection factors may remain unaccounted for. The consistent pattern suggests UYO participation may enhance interpersonal awareness and social responsibility, though some differences might reflect pre-existing traits that attracted young people to UYOs initially.

Community and belonging

The community outcomes show the strongest overall differences among all measured domains. In all three community and belonging measures, UYO members score notably higher than the comparison group.

Youth empowerment belief shows higher scores among UYO members (81% vs 71% at wave 2), a 10-percentage point difference (p=0.001). Community change skills demonstrate a substantial difference (75% vs 57% at wave 2), an 18-percentage point advantage (p<0.001). Community belonging reveals the largest gap (61% vs 45% at wave 2), a 16-percentage point difference (p<0.001). Findings were consistent at wave 1.

These consistent patterns suggest a meaningful relationship between UYO participation and young people’s perceptions about community engagement and belonging. While our analysis addresses many potential sources of bias, young people with pre-existing interest in community involvement might be more drawn to UYOs initially, which could partly explain these differences.

6.4.2 Educational attainment outcomes of UYO participation

The analysis suggests that UYO participation has a statistically significant positive impact on Attainment 8 scores. UYO members achieved an average score of 69.5, compared to 64.1 for the non-UYO comparison group. This represents a treatment effect of 5.4 points (p = 0.004).

These results suggest that UYO members demonstrate higher attainment scores compared to their non-member counterparts. While we control for additional factors such as Free School Meal (FSM) eligibility and Special Educational Needs provision in our analysis, it is important to acknowledge that educational outcomes can be influenced by numerous unobservable variables that our study cannot capture. This analysis also uses a substantially smaller sample given that only 746 young people consented to having their survey data linked to NPD data. Further, we are only able to observe Attainment 8 outcomes for those participants who are old enough to have sat their GCSEs (completed Year 11), which resulted in a sample of n=247.

6.4.3 How does the impact change by sub-groups?

Our subgroup analysis examines how UYO benefits vary across different populations. Rather than testing all possible groups, we focus on those highlighted in our qualitative research. Interviews and focus groups revealed specific experiences among certain populations that warranted deeper investigation.

We include ethnicity as a standard subgroup variable to ensure our analysis captures equity considerations, even though this was not specifically prominent in our qualitative findings. Additionally, we analyse outcomes by socio-economic status using both the IMD and FSM eligibility, reflecting the programme’s focus on addressing socio-economic deprivation.

This targeted approach helps identify which young people gain the most from UYO membership and whether the programme delivers equitable benefits across different demographic and socio-economic groups.

ONS4

The wellbeing benefits of UYO membership differ notably across demographic groups. Black young people show the largest difference with a 1.52-point higher score (7.97 vs 6.45, p=0.01), representing a 23.5% difference compared to the comparison group.

Boys score 0.43 points higher (6.84 vs 6.42, p=0.01) than their comparison group counterparts. Girls show a smaller difference of 0.20 points (7.41 vs 7.22, p=0.22) that is not statistically significant.

Health status appears to play an important role. Young people reporting fair/poor health score 0.84 points higher when participating in UYOs (6.69 vs 5.84, p=0.01). Those reporting good health show a moderate difference of 0.39 points (7.56 vs 7.17, p<0.001), while those with very good health show minimal difference (6.20 vs 6.13, p=0.76).

Both age groups show similar positive differences (0.44 points for both), though the difference is only significant for younger participants aged 10-13 (p=0.05). The difference appears stronger among those from less deprived areas (0.44 points, p<0.001) compared to more deprived areas (0.07 points, p=0.68). Non-career-oriented groups show a larger difference (0.46 points, p=0.02) than the comparison group (0.23 points, p=0.12).

These patterns should be interpreted cautiously, particularly for ethnicity findings where sample sizes are small (n=50-114).

BRS

The data shows specific differences in resilience between UYO members and the comparison group. At Wave 2, UYO members score notably higher on stress management (2.86 vs 2.67), a 7.1% higher score that reaches statistical significance (p=0.01).

Looking at overall resilience scores, UYO members consistently score slightly higher than the comparison group (3.17 vs 3.09 at Wave 2), a 2.6% higher score that does not reach statistical significance (p=0.16).

The pattern holds across socio-economic groups. Young people from both less deprived and more deprived areas show similar positive differences in their resilience scores (0.08 points, or 2.6% higher, for both groups). Neither reaches statistical significance, suggesting the relationship between UYO participation and resilience remains modest across different backgrounds.

While our analysis accounts for socio-demographic factors, other unobservable characteristics might influence these outcomes. These findings indicate UYO activities may support specific aspects of resilience like stress management, while having a less pronounced relationship with overall resilience.

YPS

Socio-emotional outcomes differ across demographic groups for UYO members. Young people from more deprived areas show stronger differences in responsibility (11-percentage points, p=0.02) compared to peers from less deprived areas (5-percentage points). A similar pattern appears for empathy, with greater benefits for young people from deprived areas (10-percentage points, p=0.05).

Non-career-oriented UYOs show more substantial differences than career-oriented UYOs across all domains. This is particularly notable in teamwork (16-percentage points for non-career-oriented vs 8-percentage points for career-oriented) and responsibility (11-percentage points vs 1-percentage point). These findings suggest activities in non-career-oriented UYOs may connect more strongly with socio-emotional development outcomes. Different UYO formats appear to relate differently with young people’s development across various backgrounds.

While our analysis accounts for many observable factors, we cannot rule out that unobserved characteristics might influence both UYO selection and these outcomes.

Community and belonging

The community and belonging benefits of UYO membership vary across demographic groups. Young people from more deprived areas show notably stronger differences in community change skills (74.7% vs 51.9%) with a 23-percentage point difference. Their community belonging scores (58.4% vs 38.4%) show a 20-percentage point difference compared to just 13-percentage points for less deprived areas.

Age also influences outcomes. Older participants (14+) show significantly stronger differences in community belonging (61.0% vs 41.5%, p<0.001) with a 20-percentage point gap. Younger members show a smaller 9-percentage point difference (60.9% vs 52.4%, non-significant). Youth empowerment belief shows more consistent benefits across age groups.

Duration of membership shows mixed patterns. Both shorter and longer duration members show similar positive differences in community change skills (around 18-19-percentage points).

These patterns suggest UYO organisations may provide particularly valuable community connection for older teens and those from more deprived backgrounds. However, unobserved factors might influence both UYO participation and these outcomes.

6.4.4 How does impact change by length of exposure to membership

This section explores whether the benefits of UYO membership differ with longer participation. We compare outcomes between members with shorter duration (one month to three years of membership) and longer duration (more than three years of membership). The sample comprised 382 individuals, with 56.5% having a longer membership duration and 43.5% a shorter duration. This comparison helps identify observed differences in outcomes among members who have been involved in their UYO for shorter or longer time periods.

While we observe differences between newer and longer-term members, we must interpret these with caution. Our analysis controls for age, socio-economic factors and other observable characteristics that might predict both membership duration and outcomes. However, some self-selection factors may remain unaccounted for. For example, young people with naturally higher wellbeing might choose to stay in UYOs longer, which could contribute to some of the differences we observe.

Despite these limitations, comparing outcomes across membership duration provides useful insights into how the UYO experience might evolve over time.

ONS4

The duration of UYO membership shows minimal difference in wellbeing outcomes. Both shorter-duration and longer-duration members show similar positive differences compared to the comparison group. Shorter-duration members score 7.14 compared to 7.11 for longer-duration members.

BRS

Longer-duration of UYO membership appears to slightly enhance resilience outcomes compared to shorter participation, though the difference is modest. Members with longer participation show scores of 3.18 compared to 3.08 for their comparison counterparts, a difference of 0.10 points or 3.2% higher. Shorter-duration members score 3.16 compared to 3.08, a difference of 0.08 points or 2.6% higher. However, neither effect reaches statistical significance (p=0.16 and p=0.24, respectively).

This suggests resilience development through UYO participation may follow a gradual pattern rather than producing immediate benefits, though the evidence for duration effects remains tentative.

YPS

The duration of UYO membership reveals distinct patterns across different socio-emotional domains. Shorter-duration members show stronger differences for empathy (81% vs 72%, a 10-percentage point difference, p=0.01) and initiative (70% vs 61%, a 9-percentage point difference, p=0.04).

Conversely, responsibility (80% vs 69%, an 11-percentage point difference, p=0.01) and teamwork (86% vs 72%, a 13-percentage point difference, p<0.001) suggest stronger differences among longer-duration members.

These findings should be interpreted with caution. Our cross-sectional design means we cannot determine whether these patterns reflect the development of skills over time or if different types of young people remain in UYOs for longer periods. The differences might reflect self-selection, with those naturally stronger in certain competencies more likely to continue their membership.

Community and belonging

The duration of UYO membership reveals an interesting pattern for community outcomes. Shorter-duration members show substantially stronger differences on youth empowerment belief (88% vs 72%, a 16-percentage point difference, p<0.001) compared to longer-duration members (73% vs 72%, only a 1-percentage point difference, non-significant).

A similar pattern appears for community belonging, with shorter-duration members scoring 64% vs 45% (19-percentage point difference) compared to longer-duration members at 58% vs 45% (13-percentage point difference).

Community change skills show nearly identical differences regardless of duration (75% vs 56% for shorter-duration, 74% vs 56% for longer-duration).

6.4.5 Robustness checks and covariate balance

We conducted multiple robustness checks to test whether our main findings remain stable across different model specifications and sample restrictions. These checks focus on our primary outcome measure, the ONS wellbeing score.

Our main analysis uses a balanced panel of respondents who completed both Wave 1 and Wave 2 surveys. To test sensitivity, we ran additional models with varying specifications:

• All available data

Using all responses from Wave 1 (n=1,043) rather than just the balanced panel. This model shows consistent results (treatment effect of 0.24, p=0.05).

• Models without age adjustment

Testing whether age controls influence our results. The treatment effects remain significant and actually strengthen slightly to 0.49 (p<0.01) for Wave 2 and 0.35 (p<0.01) for Wave 1.

• Minimal controls model

Using only essential covariates.[24] This shows a smaller effect (0.16) that doesn’t reach statistical significance (p=0.20), suggesting our full set of controls helps identify the treatment effect more precisely.

• NPD data models

Using the subset of cases with NPD linkage. These models show similar patterns though with reduced statistical power due to smaller sample sizes (n=787).

• Models without regression adjustment

Using augmented inverse probability weighting (AIPW) without the regression adjustment component of our main IPWRA approach. Results remain relatively consistent, confirming our findings are not dependent on the specific estimation method.

• Comparison group homogeneity

Testing whether prior uniform youth organisation membership within the comparison group affects our results. Using the same regression specification as our main analysis, we found no statistically significant differences between comparison group participants with and without previous uniform group experience across all outcomes (p > 0.05), confirming that prior membership does not contaminate our treatment effect estimates.

The treatment effect remains positive across all specifications, ranging from 0.12 to 0.49. Most importantly, our main findings (treatment effects of 0.36 and 0.25 for Waves 2 and 1) fall within this range, suggesting they represent a reliable estimate of UYO impact.

The consistent direction of effects across these varied specifications strengthens confidence in our main conclusions. While effect sizes vary somewhat with different modelling approaches, the overall pattern confirms a positive relationship between UYO membership and wellbeing outcomes.

Balance tests

Balance tests assess whether covariates are distributed similarly between treatment and comparison groups before and after applying weights. This is crucial for causal inference in quasi-experimental designs, as imbalanced covariates can bias treatment effect estimates.

The standardised difference measures the distance between group means, standardised by their pooled standard deviation. Values closer to zero indicate better balance. The weighted column shows values after applying inverse probability weights from our treatment model.

Before weighting, we observe notable imbalances in ethnicity (-0.339) and age (-0.457). After weighting, all standardised differences fall below ±0.05, indicating excellent balance across all covariates. This substantial improvement suggests our weighting approach effectively mitigates selection bias.

Table 6.8: Standardised differences

Covariate	Standardised differences (Unweighted)	Standardised differences (Weighted)
Ethnicity	-0.33924	-0.01542
Region	-0.05173	0.02276
Gender	0.10804	-0.00520
Age	-0.45701	-0.04384
Health status	-0.09787	0.00475
IMD decile	0.01112	-0.00915

The variance ratio compares the variance of covariates between treatment and control groups. Ideal values are close to 1.0, indicating similar spread in both groups.

Before weighting, ethnicity shows the most problematic ratio (0.521), suggesting considerably lower variance in the treatment group. After weighting, all ratios fall between 0.92 and 1.27, within the acceptable range (typically 0.5 to 2.0). The weighted variance ratios confirm our balancing approach has successfully addressed differences in distribution shape, not just central tendency.

Overall, these results demonstrate that our inverse probability weighting procedure effectively balances observed covariates between groups, strengthening the credibility of our treatment effect estimates.

Table 6.9 Variance ratios

Covariate	Variance ratio (Unweighted)	Variance ratio (Weighted)
Ethnicity	0.52093	1.23364
Region	0.96686	1.02030
Gender	1.09435	0.92189
Age	1.21057	1.26173
Health status	1.00076	1.01567
IMD decile	1.02155	1.00745

Comparison group homogeneity

A potential concern arose regarding the homogeneity of our comparison group. Within the comparison group, some participants reported previous membership in uniform youth organisations whilst others had no such experience.

This raised questions about whether prior uniform group exposure might influence outcomes, potentially contaminating our comparison group with participants who had similar experiences to the treatment group. To address this concern, we conducted a robustness check examining whether these two sub-groups within the comparison group differed significantly on key outcome measures.

Using the same regression model employed in our main analysis (including all covariates), we found no statistically significant differences between participants with and without previous uniform youth organisation experience across all measured outcomes (p > 0.05 for all comparisons). This finding supports the validity of treating our comparison group as homogeneous and confirms that prior uniform group membership does not confound our treatment effect estimates.

6.5 Conclusion and limitations

This section summarises key findings, examines differential impacts, discusses methodological limitations, and explores implications for practice.

6.5.1 Main treatment effects

Based on our quasi-experimental methodology, UYO membership shows significant positive impacts across multiple domains:

• Wellbeing

UYO members report consistently higher ONS wellbeing scores (5.3% higher) than comparable non-members, with statistically significant differences across both measurement waves.

• Resilience

Members show modest improvements, particularly in stress management (7.1% higher scores). The overall resilience effect is positive but not statistically significant.

• Socio-emotional development

UYO participation correlates with stronger social competencies, especially in teamwork (10-percentage points higher) and responsibility (8-percentage points higher).

• Community engagement

The strongest effects appear in community domains, with substantial differences in community change skills (18-percentage points higher) and community belonging (16-percentage points higher).

• Educational attainment

UYO members achieved significantly higher Attainment 8 scores than matched peers (69.5 versus 64.1 points), representing a 5.4-point treatment effect that is statistically significant and equivalent to approximately four months of additional academic progress.

• Effect across waves

All treatment effects remain consistent across both measurement waves, suggesting reliable and stable impacts rather than temporary changes.

While our methodology addresses selection bias based on observable characteristics, some self-selection factors likely remain unaccounted for. Young people with certain unmeasurable traits may be more drawn to UYOs initially. This potential remaining bias warrants caution when interpreting the strength of causal relationships.

6.5.2 Effects by sub-groups

UYO benefits vary meaningfully across different populations:

• Deprivation context

Young people from more deprived areas show particularly strong gains in responsibility (11-percentage points), empathy (10-percentage points), and community change skills (23-percentage points).

• Ethnicity

Black young people experience exceptionally large wellbeing benefits (1.52 points higher, representing a 23.5% difference).

• Health status

Young people reporting fair/poor health gain substantially more from UYO membership (0.84 points higher on wellbeing scales).

• Age

Older participants (14+) benefit more from community engagement aspects, with stronger effects on community belonging (20-percentage points difference).

• Programme focus

Non-career-oriented UYOs show stronger socio-emotional outcomes across all measured domains compared to career-oriented programmes.

Some subgroup findings require particular caution in interpretation due to small sample sizes. This is especially true for ethnic minority groups, where sample sizes range from 50 to 114 participants.

6.5.3 Effect of duration

Our analysis of membership duration reveals distinct patterns:

• Wellbeing shows similar positive differences for both shorter-duration (0.38 points) and longer-duration members (0.35 points).
• Resilience shows slight increases with longer participation, though differences remain modest.
• Socio-emotional competencies show varying temporal patterns: empathy and initiative show stronger differences among newer members, while responsibility and teamwork appear stronger with sustained membership.
• Youth empowerment beliefs show stronger differences among newer members (16-percentage points) compared to longer-term participants (1-percentage point), suggesting potential early enthusiasm effects.
• Community change skills show consistent benefits regardless of membership duration (approximately 18-19-percentage points).

6.5.4 Limitations

Our study employs robust statistical methods while acknowledging certain methodological constraints. Our doubly robust approach effectively minimises selection bias, though like all observational studies, it cannot fully account for unobservable factors that might influence both UYO membership and outcomes.

The available data provides valuable insights despite some limitations. While our analysis includes postcode-level deprivation measures, we also have FSM data from the NPD. However, this variable contains few valid responses, limiting its analytical value for this study. Similarly, our two measurement waves offer useful temporal perspective, though baseline data before UYO membership began would provide additional analytical advantages.

Our weighting approach successfully creates comparable treatment and comparison groups based on observable characteristics. This provides a strong foundation for causal inference, even as we recognise that some unmeasured factors may differ between young people who join UYOs and those who do not. These limitations are common in real-world educational evaluations and do not undermine our core findings. Instead, they contextualise our results and highlight opportunities for future research designs that could build upon this important evidence base.

6.6 QED analytical tables

Each table presents our analysis of UYO’s impact on specific outcomes.

The ‘UYO members’ mean outcome’ shows what young people in UYO actually achieved. The “Weighted comparison group mean outcome” shows what these same young people would likely have achieved without UYO participation, based on similar non-participants.

The ‘Treatment effect’ is the difference between these values. This represents our best estimate of UYO’s impact. P-values below 0.05 indicate statistically significant differences. The confidence intervals show the range where the true effect likely falls.

Our analysis uses IPWRA methodology (Inverse Probability Weighted Regression Adjustment). This approach creates a balanced comparison by weighting similar non-participants and adjusting for background factors. This helps address potential selection bias in our non-randomised design.

Sample sizes vary slightly due to survey completion patterns. All estimates account for demographic factors and other relevant variables as outlined in our methodology section.

6.6.1 ONS4

Table 6.10 ONS4 outcome means and treatment effects (wave 1 and 2)

Outcome measure	UYO members’ mean outcome	Weighted comparison group mean outcome	Treatment effect coefficient	Standard error	P- value	Lower Bound (95%CI)	Upper Bound (95%CI)	Sample size
Wave 1
ONS4 Score (W1)	7.15	6.89	0.25	0.12	0.03	0.02	0.49	1,028
Wave 2
ONS4 Score (W2)	7.05	6.69	0.36	0.12	0.00	0.13	0.58	1,028

Table 6.11: ONS4 outcome means and treatment effects by sub-groups (wave 2)

Outcome measure	UYO members’ mean outcome	Weighted comparison group mean outcome	Treatment effect coefficient	Standard error	P-value	Lower Bound (95%CI)	Upper Bound (95%CI)	Sample size
Ethnicity
ONS4 Score (W2) - White British	7.04	6.72	0.32	0.12	0.01	0.08	0.57	785
ONS4 Score (W2) - White Other	6.64	6.15	0.49	0.55	0.37	-0.59	1.56	114
ONS4 Score (W2) - Asian	6.95	6.99	-0.04	0.56	0.95	-1.13	1.06	52
ONS4 Score (W2) - Black	7.97	6.45	1.52	0.57	0.01	0.40	2.64	50
Gender
ONS4 Score (W2) - Male	6.84	6.42	0.43	0.16	0.01	0.11	0.75	584
ONS4 Score (W2) - Female	7.41	7.22	0.20	0.16	0.22	-0.11	0.50	412
Health status
ONS4 Score (W2) - Health: Very good	6.20	6.13	0.07	0.23	0.76	-0.38	0.53	242
ONS4 Score (W2) - Health: Good	7.56	7.17	0.39	0.14	0.00	0.12	0.66	615
ONS4 Score (W2) - Health: Fair/Poor	6.69	5.84	0.84	0.31	0.01	0.24	1.45	152
Age
ONS4 Score (W2) - Age 10-13	7.47	7.03	0.44	0.22	0.05	0.00	0.87	279
ONS4 Score (W2) - Age 14+	6.78	6.33	0.44	0.13	0.00	0.18	0.71	749
IMD status
ONS4 Score (W2) - IMD 1-5 (More Deprived)	6.91	6.84	0.07	0.18	0.68	-0.28	0.42	435
ONS4 Score (W2) - IMD 6-10 (Less Deprived)	7.11	6.67	0.44	0.15	0.00	0.15	0.73	593
UYO type
ONS4 Score (W2) - Career-oriented vs Comparison	7.03	6.79	0.23	0.15	0.12	-0.06	0.53	978
ONS4 Score (W2) - Non-career-oriented vs Comparison	7.25	6.79	0.46	0.19	0.02	0.09	0.83	978
Length of membership
ONS4 Score (W2) - Shorter Duration vs Comparison	7.14	6.76	0.38	0.14	0.01	0.09	0.66	1,025
ONS4 Score (W2) - Longer Duration vs Comparison	7.11	6.76	0.35	0.16	0.03	0.03	0.67	1,025

6.6.2 BRS

Table 6.12: BRS outcome means and treatment effects (wave 1 and 2)

Outcome measure	UYO members’ mean outcome	Weighted comparison group mean outcome	Treatment effect coefficient	Standard error	P-value	Lower Bound (95%CI)	Upper Bound (95%CI)	Sample size
Wave 1
Total BRS Score (W1)	3.20	3.11	0.09	0.05	0.11	-0.02	0.19	1,009
BRS1 - Quick recovery (W1)	3.47	3.48	0.00	0.07	0.95	-0.14	0.13	1,045
BRS3 - Rapid stress recovery (W1)	3.28	3.21	0.08	0.07	0.27	-0.06	0.22	1,050
BRS5 - Ease through challenges (W1)	3.15	3.08	0.06	0.07	0.35	-0.07	0.20	1,041
BRS2 - Stress Management [R] (W1)	2.82	2.68	0.14	0.07	0.05	0.00	0.28	1,050
BRS4 - Adversity response [R] (W1)	3.10	2.99	0.10	0.07	0.16	-0.04	0.25	1,047
BRS6 - Recovery time [R] (W1)	3.32	3.18	0.14	0.07	0.05	0.00	0.28	1,036
Wave 2
Total BRS Score (W2)	3.17	3.09	0.08	0.05	0.16	-0.03	0.18	1,009
BRS1 - Quick recovery (W2)	3.44	3.35	0.10	0.07	0.15	-0.04	0.23	1,045
BRS3 - Rapid stress recovery (W2)	3.20	3.14	0.06	0.07	0.44	-0.08	0.20	1,050
BRS5 - Ease through challenges (W2)	3.17	3.06	0.11	0.07	0.11	-0.03	0.24	1,041
BRS2 - Stress Management [R] (W2)	2.86	2.67	0.18	0.07	0.01	0.04	0.32	1,050
BRS4 - Adversity response [R] (W2)	3.09	3.07	0.02	0.07	0.82	-0.13	0.16	1,047
BRS6 - Recovery time [R] (W2)	3.22	3.22	0.01	0.07	0.92	-0.13	0.15	1,036

Table 6.13: BRS outcome means and treatment effects by sub-groups (wave 2)

Outcome measure	UYO members’ mean outcome	Weighted comparison group mean outcome	Treatment effect coefficient	Standard error	P-value	Lower Bound (95%CI)	Upper Bound (95%CI)	Sample size
IMD status
Total BRS Score (W2) - IMD 1-5 (More Deprived)	3.15	3.07	0.08	0.09	0.38	-0.10	0.25	429
Total BRS Score (W2) - IMD 6-10 (Less deprived)	3.18	3.10	0.08	0.07	0.25	-0.06	0.22	580
Length of membership
Total BRS Score (W2) – Shorter-Duration vs Comparison	3.16	3.08	0.08	0.07	0.24	-0.05	0.21	1,006
Total BRS Score (W2) – Longer-Duration vs Comparison	3.18	3.08	0.10	0.07	0.16	-0.04	0.24	1,006

6.6.3 YPS

Table 6.14: YPS outcome means and treatment effects (wave 1 and 2)

Outcome measure	UYO members’ mean outcome	Weighted comparison group mean outcome	Treatment effect coefficient	Standard error	P-value	Lower Bound (95%CI)	Upper Bound (95%CI)	Sample size
Wave 1
YPS1 - Empathy (W1)	0.76	0.71	0.05	0.03	0.09	-0.01	0.11	1,054
YPS2 - Problem solving (W1)	0.51	0.40	0.11	0.03	0.00	0.05	0.18	1,029
YPS3 - Initiative (W1)	0.70	0.59	0.11	0.03	0.00	0.05	0.17	1,061
YPS4 - Responsibility (W1)	0.79	0.69	0.09	0.03	0.00	0.04	0.15	1,056
YPS5 - Teamwork (W1)	0.84	0.76	0.08	0.03	0.00	0.03	0.13	1,059
Wave 2
YPS1 - Empathy (W2)	0.79	0.73	0.07	0.03	0.02	0.01	0.12	1,054
YPS2 - Problem solving (W2)	0.51	0.47	0.04	0.03	0.20	-0.02	0.11	1,029
YPS3 - Initiative (W2)	0.67	0.61	0.06	0.03	0.06	0.00	0.13	1,061
YPS4 - Responsibility (W2)	0.78	0.70	0.08	0.03	0.01	0.02	0.13	1,056
YPS5 - Teamwork (W2)	0.83	0.72	0.10	0.03	0.00	0.05	0.16	1,059

Table 6.15: YPS outcome means and treatment effects by sub-groups (wave 2)

Outcome measure	UYO members’ mean outcome	Weighted comparison group mean outcome	Treatment effect coefficient	Standard error	P-value	Lower Bound (95%CI)	Upper Bound (95%CI)	Sample size
IMD status
YPS1 - Empathy (W2) - IMD 1-5 (More Deprived)	0.75	0.66	0.10	0.05	0.05	0.00	0.19	449
YPS1 - Empathy (W2) - IMD 6-10 (Less Deprived)	0.82	0.78	0.04	0.04	0.27	-0.03	0.11	605
YPS2 - Problem solving (W2) - IMD 1-5 (More Deprived)	0.46	0.44	0.02	0.05	0.67	-0.08	0.13	439
YPS2 - Problem solving (W2) - IMD 6-10 (Less Deprived)	0.54	0.49	0.05	0.04	0.27	-0.04	0.14	590
YPS3 - Initiative (W2) - IMD 1-5 (More Deprived)	0.67	0.61	0.06	0.05	0.27	-0.04	0.16	455
YPS3 - Initiative (W2) - IMD 6-10 (Less Deprived)	0.67	0.61	0.06	0.04	0.18	-0.03	0.14	606
YPS4 - Responsibility (W2) - IMD 1-5 (More Deprived)	0.81	0.70	0.11	0.05	0.02	0.02	0.20	453
YPS4 - Responsibility (W2) - IMD 6-10 (Less Deprived)	0.75	0.70	0.05	0.04	0.21	-0.03	0.13	603
YPS5 - Teamwork (W2) - IMD 1-5 (More Deprived)	0.83	0.71	0.12	0.05	0.01	0.03	0.21	454
YPS5 - Teamwork (W2) - IMD 6-10 (Less Deprived)	0.82	0.72	0.10	0.04	0.01	0.03	0.18	605
Length of membership
YPS1 - Empathy (W2) - Shorter Duration vs Comparison	0.81	0.72	0.10	0.04	0.01	0.02	0.17	1,051
YPS1 - Empathy (W2) - Longer Duration vs Comparison	0.77	0.72	0.05	0.04	0.22	-0.03	0.13	1,051
YPS2 - Problem solving (W2) - Shorter Duration vs Comparison	0.49	0.45	0.04	0.05	0.38	-0.05	0.13	1,026
YPS2 - Problem solving (W2) - Longer Duration vs Comparison	0.49	0.45	0.03	0.05	0.47	-0.06	0.12	1,026
YPS3 - Initiative (W2) - Shorter Duration vs Comparison	0.70	0.61	0.09	0.04	0.04	0.00	0.17	1,059
YPS3 - Initiative (W2) - Longer Duration vs Comparison	0.62	0.61	0.01	0.05	0.82	-0.08	0.10	1,059
YPS4 - Responsibility (W2) - Shorter Duration vs Comparison	0.75	0.69	0.06	0.04	0.13	-0.02	0.14	1,053
YPS4 - Responsibility (W2) - Longer Duration vs Comparison	0.80	0.69	0.10	0.04	0.01	0.03	0.18	1,053
YPS5 - Teamwork (W2) - Shorter Duration vs Comparison	0.80	0.72	0.07	0.04	0.07	-0.01	0.15	1,056
YPS5 - Teamwork (W2) - Longer Duration vs Comparison	0.86	0.72	0.13	0.03	0.00	0.07	0.20	1,056
UYO type
YPS1 - Empathy (W2) - Career-oriented vs Control	0.80	0.71	0.09	0.04	0.02	0.02	0.16	1,002
YPS1 - Empathy (W2) - Non-career-oriented vs Comparison	0.82	0.71	0.10	0.04	0.01	0.02	0.18	1,002
YPS2 - Problem solving (W2) - Career-oriented vs Comparison	0.45	0.46	-0.01	0.05	0.81	-0.10	0.08	978
YPS2 - Problem solving (W2) - Non-career-oriented vs Comparison	0.55	0.46	0.09	0.05	0.11	-0.02	0.20	978
YPS3 - Initiative (W2) - Career-oriented vs Comparison	0.59	0.62	-0.03	0.05	0.52	-0.12	0.06	1,010
YPS3 - Initiative (W2) - Non-career-oriented vs Comparison	0.69	0.62	0.07	0.05	0.17	-0.03	0.17	1,010
YPS4 - Responsibility (W2) - Career-oriented vs Comparison	0.70	0.69	0.01	0.04	0.81	-0.07	0.09	1,004
YPS4 - Responsibility (W2) - Non-career-oriented vs Comparison	0.80	0.69	0.11	0.05	0.02	0.02	0.20	1,004
YPS5 - Teamwork (W2) - Career-oriented vs Comparison	0.80	0.72	0.08	0.04	0.04	0.01	0.15	1,007
YPS5 - Teamwork (W2) - Non-career-oriented vs Comparison	0.88	0.72	0.16	0.04	0.00	0.08	0.25	1,007

6.6.4 Community and belonging

Table 6.16: Community and belonging outcome means and treatment effects (wave 1 and 2)

Outcome measure	UYO members’ mean outcome	Weighted comparison group mean outcome	Treatment effect coefficient	Standard error	P-value	Lower Bound (95%CI)	Upper Bound (95%CI)	Sample size
Wave 1
Belonging 1 - Youth empowerment belief (W1)	0.81	0.73	0.08	0.03	0.01	0.02	0.13	1,065
Belonging 2 - Community change skills (W1)	0.72	0.58	0.14	0.03	0.00	0.08	0.21	1,059
Belonging 3 - Community belonging (W1)	0.59	0.47	0.12	0.03	0.00	0.06	0.19	1,058
Wave 2
Belonging 1 - Youth empowerment belief (W2)	0.81	0.71	0.10	0.03	0.00	0.04	0.15	1,062
Belonging 2 - Community change skills (W2)	0.75	0.56	0.18	0.03	0.00	0.12	0.24	1,054
Belonging 3 - Community belonging (W2)	0.61	0.45	0.16	0.03	0.00	0.09	0.23	1,056

Table 6.17: Community and belonging outcome means and treatment effects by sub-groups (wave 2)

Outcome measure	UYO members’ mean outcome	Weighted comparison group mean outcome	Treatment effect coefficient	Standard error	P-value	Lower Bound (95%CI)	Upper Bound (95%CI)	Sample size
IMD status
Belonging 1 - Youth empowerment belief (W2) - IMD 1-5 (More Deprived)	0.81	0.71	0.09	0.05	0.04	0.00	0.18	455
Belonging 1 - Youth empowerment belief (W2) - IMD 6-10 (Less Deprived)	0.82	0.70	0.11	0.04	0.00	0.04	0.19	607
Belonging 2 - Community change skills (W2) - IMD 1-5 (More Deprived)	0.75	0.52	0.23	0.05	0.00	0.13	0.33	453
Belonging 2 - Community change skills (W2) - IMD 6-10 (Less Deprived)	0.75	0.62	0.13	0.04	0.00	0.05	0.21	601
Belonging 3 - Community belonging (W2) - IMD 1-5 (More Deprived)	0.58	0.38	0.20	0.05	0.00	0.10	0.30	453
Belonging 3 - Community belonging (W2) - IMD 6-10 (Less Deprived)	0.63	0.50	0.13	0.04	0.00	0.04	0.22	603
Age
Belonging 1 - Youth empowerment belief (W2) - Age 10-13	0.81	0.74	0.07	0.05	0.19	-0.03	0.16	288
Belonging 1 - Youth empowerment belief (W2) - Age 14+	0.81	0.70	0.11	0.03	0.00	0.05	0.17	774
Belonging 2 - Community change skills (W2) - Age 10-13	0.73	0.56	0.17	0.06	0.00	0.06	0.28	285
Belonging 2 - Community change skills (W2) - Age 14+	0.75	0.57	0.19	0.04	0.00	0.11	0.26	769
Belonging 3 - Community belonging (W2) - Age 10-13	0.61	0.52	0.08	0.06	0.16	-0.03	0.20	286
Belonging 3 - Community belonging (W2) - Age 14+	0.61	0.42	0.20	0.04	0.00	0.12	0.27	770
Length of membership
Belonging 1 - Youth empowerment belief (W2) - Shorter Duration vs Comparison	0.88	0.72	0.16	0.03	0.00	0.09	0.23	1,059
Belonging 1 - Youth empowerment belief (W2) - Longer Duration vs Comparison	0.73	0.72	0.01	0.04	0.79	-0.07	0.09	1,059
Belonging 2 - Community change skills (W2) - Shorter Duration vs Comparison	0.75	0.55	0.19	0.04	0.00	0.11	0.27	1,052
Belonging 2 - Community change skills (W2) - Longer Duration vs Comparison	0.74	0.55	0.18	0.04	0.00	0.10	0.27	1,052
Belonging 3 - Community belonging (W2) - Shorter Duration vs Comparison	0.64	0.45	0.19	0.05	0.00	0.10	0.28	1,054
Belonging 3 - Community belonging (W2) - Longer Duration vs Comparison	0.58	0.45	0.13	0.05	0.01	0.04	0.22	1,054

6.6.5 Educational attainment

Table 6.18: Estimation results for different model specifications of education attainment (wave 1 and 2)

Estimation	UYO members’ mean outcome	Weighted comparison group mean outcome	Treatment effect coefficient	Standard error	P-value	Lower Bound (95%CI)	Upper Bound (95%CI)	Sample size
Wave 1 KS4 attainment score (Overall)	66.17	60.04	6.13	1.57	0.000	3.04	9.21	711
Wave 1 KS4 attainment score (Complete Cases)	72.22	63.99	8.22	2.07	0.000	4.16	12.28	247
Wave 2 KS4 attainment score (Complete Cases)	69.55	64.17	5.38	1.89	0.000	1.67	9.08	247

6.6.6 Robustness checks for ONS4

Table 6.19: Estimation results for different model specifications of ONS4 (wave 1 and 2)

Estimation	UYO members’ mean outcome	Weighted comparison group mean outcome	Treatment effect coefficient	Standard error	P-value	Lower Bound (95%CI)	Upper Bound (95%CI)	Sample size
Wave 2 ONS - Main Analysis	7.05	6.69	0.36	0.12	0.00	0.13	0.58	1,028
Wave 1 ONS - Main Analysis	7.15	6.89	0.25	0.12	0.03	0.02	0.49	1,028
Wave 1 ONS - All Available Data	7.10	6.86	0.24	0.12	0.05	0.00	0.47	1,043
Wave 2 ONS - Without Age (AIPW)	7.04	6.55	0.49	0.11	0.00	0.27	0.71	1,067
Wave 1 ONS - Without Age (AIPW)	7.08	6.73	0.35	0.12	0.00	0.12	0.58	1,067
Wave 1 ONS - Minimal Controls (AIPW)	6.91	6.76	0.16	0.12	0.20	-0.08	0.40	1,106
Wave 1 ONS - With NPD Data (All)	6.81	6.69	0.12	0.14	0.39	-0.16	0.40	807
Wave 1 ONS - With NPD Data (Complete Cases)	6.84	6.71	0.13	0.14	0.36	-0.15	0.41	787
Wave 2 ONS - With NPD Data (Complete Cases)	6.85	6.52	0.32	0.13	0.01	0.07	0.58	787

References

Fung, S. (2020). Validity of the Brief Resilience Scale and Brief Resilient Coping Scale in a Chinese Sample. International Journal of Environmental Research and Public Health, 17(4), Article 4.

Department for Education. (February 2025). Secondary accountability measures: Guide for maintained secondary schools, academies and free schools.

GOV.UK. (2024, November 7). GCSE results (Attainment 8).

GOV.UK. (2025, February 27). Secondary accountability measures (including Progress 8 and Attainment 8). GOV.UK.

Konaszewski, K., Niesiobędzka, M., & Surzykiewicz, J. (2020). Validation of the Polish version of the Brief Resilience Scale (BRS). PLOS ONE, 15(8), e0237038.

Kunzler, A. M., Chmitorz, A., Bagusat, C., Kaluza, A. J., Hoffmann, I., Schäfer, M., Quiring, O., Rigotti, T., Kalisch, R., Tüscher, O., Franke, A. G., Dick, R. van, & Lieb, K. (2018). Construct Validity and Population-Based Norms of the German Brief Resilience Scale (BRS). European Journal of Health Psychology.

ONS. (2025). Personal well-being user guidance—Office for National Statistics.

Słoczyński, T., Uysal, S. D., & Wooldridge, J. M. (2022). Doubly Robust Estimation of Local Average Treatment Effects Using Inverse Probability Weighted Regression Adjustment (No. arXiv:2208.01300). arXiv.

Smith, B. W., Dalen, J., Wiggins, K., Tooley, E., Christopher, P., & Bernard, J. (2008). The brief resilience scale: Assessing the ability to bounce back. International Journal of Behavioral Medicine, 15(3), 194–200.

Smith, B. W., deCruz-Dixon, N., Schodt, K., & Torres, F. (2023). Brief Resilience Scale (BRS). In Handbook of Assessment in Mindfulness Research (pp. 1–19). Springer, Cham.

[1] Read the process report

[2] DCMS (2022). Government outlines ambitious plans to level up activities for young people

[3] DCMS and New Economics Foundation (2016). A description of social action

[4] Family, Kids and Youth. (2015): Evaluation of the Social Action Journey Fund

[5] Youth United Foundation. (2018). Social integration: The role of uniformed youth groups.

[6] Counterfactual impact evaluation can adopt either an experimental approach (such as a randomised controlled trial) or a quasi-experimental approach. In our case, randomisation was not feasible, so we used quasi-experimental methods to create a valid comparison group through statistical techniques such as inverse probability weighting.

[7] Personal well-being user guidance

[8] Smith, B. et al, (2008). The Brief Resilience Scale: Assessing the Ability to Bounce Back. International journal of behavioral medicine. 15. 194 to 200.

[9] Young People’s Survey(YPS)

[10] Participants were asked to select their UYO(s) from a list, but also given the option to select other and provide a free text response if they did not see their organisation. The research team cleaned this dataset to appropriately code any ‘other’ responses to either the treatment or comparison group.

[11] Given that the research aim was to understand the impact of UYOs on young people, members of all UYOs were included as part of the ‘treatment group’. This had the practical benefit of increasing the sample size, but also accounted for the fact that some young people were part of both a funded and an unfunded UYO.

[12] Participation in UYOs varied between the two survey waves, meaning some participants started or stopped being UYO members. The analysis in this report focuses on participants who were current UYO members at wave 2.

[13] The Fund was focused on England only, however, a small number of participants from Scotland and Wales were identified in the sample using postcode data. This is likely the result of groups near borders such that some young people cross borders to attend their group.

[14] Windle, G., Bennett, K.M. & Noyes, J. A methodological review of resilience measurement scales. Health Qual Life Outcomes 9, 8 (2011). Ballard, M., Richard Gill, P., Hand, T., & MacKenzie, D. A critical evaluation of adolescent resilience self-report scales: A scoping review. Children and Youth Services Review, 157 (2024).

[15] Only 63% of survey participants consented to having their data matched to the NPD. And then we are only able to observe GCSE outcomes for those members who had completed Year 11 by the end of the 2023/24 school year.

[16] Department for Education. (27 February 2025). Key stage 4 performance.

[17] Dibben, C., Playford, C., & Mitchell, R., (2017). Be(ing) prepared: Guide and Scout participation, childhood social position and mental health at age 50-a prospective birth cohort study. Journal of Epidemiol Community Health, 71, 255-281.

[18] See B.H., Gorard S., & Siddiqui N. (2017). Does participation in uniformed group activities in school improve young people’s non-cognitive outcomes? International Journal of Educational Research, 85, 109 to 120.

[19] Preliminary Evaluation of a Targeted, School-Based Social and Emotional Learning Intervention for at Risk Youth: Football Beyond Borders

[20] Siddiqui, N., Gorard, S., & See, B. H. (2019). Can learning beyond the classroom impact on social responsibility and academic attainment? An evaluation of the Children’s University youth social action programme. Studies in Educational Evaluation, 61, 74 to 82.

[21] Gorard, S., See, B. H., Siddiqui, N., Smith, E., & White, P. (2016). Youth Social Action Trials: Youth United Evaluation report and executive summary. Education Endowment Foundation.

[22] My LegaSea: A multi-generational impact study of Sea Cadets

[23] Age reflects participants’ age at the time of the second survey; some participants may have turned 19 since taking the first survey.

[24] This model excludes IMD and health.