Research and analysis

Student Mentoring Programme – Birmingham City Council

Published 10 April 2024

Applies to England

Partnership for People and Place

The PfPP programme has piloted a new approach to cross-government working to improve local outcomes and efficiency of policy and programmes designed and delivered in place. PfPP funded 13 local government partners to deliver pilot projects focusing on hyper-local issues that could be tackled through better central or local government coordination. The overall objective of the PfPP programme was to test whether closer working between different central government departments and local places can bring measurable benefits to people who live there.

Local background – East Birmingham

Birmingham City Council undertook research in 2021 which found that youth unemployment had increased significantly since before the pandemic, exposing pre-existing inequalities within the city. The research highlighted that some young people were struggling to find the right opportunities to build their CVs and demonstrate their value to employers. This identified a gap to improve communication and information channels between employers and young people.

Birmingham City Council piloted a mentoring programme for a cohort of students at risk of facing unemployment. While the team’s initial intention was to focus on sixth form students, local consultations with schools highlighted that the Year 11 cohort had less provision and required more support. Through 5 locally representative mentors, the pilot aimed to connect young people into work and other opportunities to improve transitions from school to employment. They also held student peer sessions, a reverse mentoring event and wider inspirational visits and workshops with local employers.

Central government involvement

The Student Mentoring Programme intended to bring central government partners into the project primarily at the dissemination stage, seeking to share learnings and good practice and influence change. The programme team initially identified DWP and DfE as central government departments involved in supporting student transitions from education into employment. However, due to the change in focus from sixth form to Year 11 students, engagement with DWP was no longer aligned to the focus of the pilot. This made it more challenging to engage central government partners.

The project delayed interaction with DfE to enable time for greater clarification of collaboration requests. This led to the project connecting with DfE partners towards the close of the pilot. Engagement took the form of a meeting between the project team and DfE staff responsible for data sharing. The information shared during the meeting was helpful for the project by providing more detailed knowledge of data processes and linkage capabilities between central government and local government.

Delivery plans	Meetings	Visits	Involvement in delivery
DWP, DfE, DESNZ were listed in Birmingham’s initial delivery plan as departments with an interest in youth unemployment.	DfE, DWP	N/A	N/A

What was delivered?

Overall, the pilot engaged 215 students across 6 schools. Engagement varied between students, but generally included coaching/mentoring sessions, visits to local employers of interest to young people (such as the clean tech sector), attendance at the reverse mentoring event and participation in peer group sessions.

A reverse mentoring event on the 20th January 2023 brought students together with employers in a neutral and safe space. The event created space for students and employers to talk about what young people were looking for in work experience opportunities, future employment and how employers might engage younger prospective employees. This event also featured an inspirational speaker that young people could identify with, which was regarded as impactful by the project lead.

A member of the core pilot team was seconded for 6 months from the University of Birmingham to work in the City Council and explore opportunities around improving data sharing, working in the newly formed City Observatory. The project sought to understand if the student cohort included in the pilot could be tracked using DfE data to understand whether they went on to future pathways in education or employment. The team wished to compare this to a counterfactual of students outside of the cohort. This was not a possibility during the lifetime of the project due to difficulties sharing anonymised individual-level data at sufficient detail for the analysis required, and because there is always a time lag on collection of student destinations data.  However, this will now be undertaken post-project based on sharing of anonymised student identifier numbers. This will extend the post-project impact evaluation to March 2024, and has led to an intention to develop a formal research project to focus on young people’s progression in the East Birmingham area.

The project has built evidence of what works to support young people through transitions related to leaving school e.g. further education, apprenticeships, employment. The findings will be disseminated during an event in East Birmingham in March 2024.

Local communities

Students in a focus group with the evaluation team reflected that the programme provided a safe space to talk about life, their experiences and the pathways that interest them. Mentors were highlighted to have effectively supported students to develop skills and tools (such as CVs) to better position them for post-school transition.

I found the one-on-one’s really useful because when I’m around a bigger group I tend to keep to myself and don’t really say much but when I had the one-on-ones I spoke about my strengths and my weaknesses and what I’m scared about, what route I’m going with.

[Student – project participant]

Pilot leads and mentors felt that the programme helped improve student confidence about their future choices and broaden their understanding of career (and wider) options available to them. Mentors encouraged student to reflect on their ideal pathway, connect with local employers to access work experience and other training or development opportunities.

I made a CV, I was speaking to people who are successful in their area of work. I’m in a very positive place. I’ve applied for my sixth forms with the help of [my mentor].

[Student – project participant]

---

Delivery partners & local government

The pilot provided a service that increased collaboration and partnership working between local partners. The set up of PfPP enabled Birmingham City Council and its partners to work autonomously, focusing on their areas of expertise without the need for continuous monitoring. This built trust across the delivery team.

The pilot was co-designed with local schools. This worked well to increase trust between the pilot team and schools, building ongoing relationships across those involved. The pilot team identified data held by DfE which could relate to this project, such as data around apprenticeships, further education and educational performance. Through engagement with DfE, the team established that Longitudinal Educational Outcomes (LEO) data would not be applicable to the local PfPP project. However, alternative requests related to data held by DfE are being developed by the team for use in future projects.

Snapshot: Breakeven analysis

Breakeven analysis provides an estimate of the level of change in outcomes within each of the pilot areas that would be necessary for the pilot’s benefits to meet costs. Where there is absence of medium- or long-term person-level outcomes data, a breakeven analysis can provide an indication of how many beneficiaries would need to achieve specific outcomes for the programme to achieve a net positive economic value. Breakeven analysis on outcomes identified in each pilot’s Theory of Change, for which monetisable benefits could be estimated through the Annual Population Survey.^{[footnote 1]}

Based on the evidence available to date, it is at this stage uncertain whether the pilot will have achieved breakeven in terms of person-level outcomes. This analysis however can be used for further tracking of student-level outcomes into the future. We note that this breakeven does not account for intended outcomes including benefits arising from system-level outcomes including sharing of evidence around ‘what works’, and other person-level outcomes that could not be quantified in this analysis (e.g. increased awareness of employment opportunities, improved skill level, improved confidence through transitions, increased employer engagement).

The pilot aimed to provide Year 11 students in East Birmingham with a mentoring programme to support them to transition from school. This was designed so that young people could explore career pathways including apprenticeships and further education, to develop a better understanding of the options for them. The extent to which these outcomes are associated with marginal changes in life satisfaction was explored through statistical analysis of the Understanding Society survey.  

In consultation with the Student Mentoring Programme team, the following outcomes from Understanding Society were utilised within the breakeven analysis.^{[footnote 2]} These align with specific person-level outcomes a specified within the Student Mentoring Programme Theory of Change, as highlighted below.

Outcomes specified in Student Mentoring Programme Theory of Change	Outcomes used in breakeven analysis
Young people prevented from entering claimant and DWP system.	Reducing the number of young people (ages 16 – 24 inclusive) not in education, employment or training (NEET).
Reduced claimant count among young people in Birmingham.	Reducing the number of young people (ages 16 – 24 inclusive) not in education, employment or training (NEET).
Increased educational attainment and skill levels among young people in East Birmingham wards.	Increasing the take up of apprenticeships/ other vocational activities.

The outcomes targeted by the Student Mentoring Programme were used in a breakeven calculator to estimate the number of beneficiaries that would need to be reached for the pilot areas to breakeven in cost terms (with a range presented between the highest and lowest number of people that would need to benefit to breakeven). The analysis accounts only for outcomes to beneficiaries. The analysis does not account for wider impacts.

The breakeven analysis provides an upper and lower bound range which accounts for some of the uncertainties inherent in predicting social value improvements for these outcomes, which can be calculated without direct primary data collection (out of scope of the evaluation). The table below displays the results of this breakeven analysis for the Student Mentoring Programme.

Outcome	Reducing number of NEET young people	Take up apprenticeships/other vocations	Range
Value Per Beneficiary	£4,920	£1,385	£1,385-£4,920
Number of Beneficiaries to Breakeven	84	297	84-297

The breakeven calculator estimates that between 84 and 297 beneficiaries need to benefit such that the social welfare benefits of the programme offset the cost of the programme.^{[footnote 3]} The estimated range accounts for the difference between highest and lowest values assigned to each outcome. For example, reducing the number of NEET young people has an associated value of £4,920. Should this outcome be realised for 84 participants, the total benefits would equal the cost of the programme. This presents the lowest number of beneficiaries required to achieve breakeven and the outcome with the highest assigned value. Likewise, participants taking up apprenticeships/other vocations has an associated value of £1,385. If 297 students take up apprenticeships/other vocations as a result of support provided by mentors, the total benefits would equal the cost of the programme. This presents the highest number of beneficiaries required to achieve breakeven and the outcome with the lowest assigned value.

Based on the evidence reviewed to date, we are unable to determine whether a breakeven outcome is feasible in the longer term. Overall, the pilot engaged 215 students across 6 schools. Engagement varied between students, but generally included coaching/mentoring sessions, attendance at the reverse mentoring event and participation in peer group sessions. In interviews with the research team, students suggested that mentors are having a highly positive influence on young people. However, we are yet to capture outcomes data. While the programme has reached a sufficiently high number of students to date, there is high uncertainty about how this support will translate into education or employment outcomes.

Note that the application of the breakeven calculator to a full business case would require data to be collected that evidences the number of beneficiaries who experienced these outcomes, either through primary surveys (e.g. by replicating the Understanding Society or Community Life survey question on which this analysis was based in a survey on the target population) or administrative data (e.g. administrative records including demographic data, school attendance data and programme activity data).

1. Following the methodology set out in HMTs supplementary Green Book guidance for wellbeing appraisal, changes in reported life satisfaction can be used to monetise the social welfare implications of a policy. ↩

2. Baseline analysis of Understanding Society wave 12 data shows that the mean reported life satisfaction score in Liverpool had a value of 7.42 (when converted to an 11-point scale, uprated from 4.72 on a 7-point scale). This is lower than the national average of 8.20 (5.22 on a 7-point scale). ↩

3. The total cost of the student mentoring PfPP pilot in Birmingham was £410,730, of which £335,00 was direct funding and £75,730 was administrative costs. ↩