Official Statistics

Employment Data Lab Analysis: Hertfordshire County Council Pathways to Success Programme

Published 9 July 2026

This Employment Data Lab report presents estimates of the impact of Hertfordshire County Council’s Pathways to Success (PtS) programme, on the education and employment outcomes of the programme participants. The PtS programme is aimed at supporting 16 to 25-year-olds who are not in employment, education or training (NEET).

The results in this report have been generated using quasi-experimental techniques which introduce some uncertainty. The results should be used with a degree of caution. Further information can be found in Section 7, and in an associated methodology report.

Headline results

Increase in employment at 12 months

Between 5 and 12 percentage points more programme participants were in employment a year after starting the programme than had they not participated.

This result was statistically significant.

Increase in education and training courses passed

The percentage of participants who passed an education or training course a year after start was between 1 and 8 percentage points higher than had they not participated.

This result was statistically significant.

• The main analysis focuses on a sub-group of 800 evaluated participants (out of 1,294) who started the programme between January 2020 and April 2023, and were between the ages of 16 and 25 (for more information about exclusions from the evaluation see Who was evaluated as part of the analysis?

• The headline results focus on one-year outcomes. These outcomes were chosen in consultation with the user organisation before starting the analysis as the primary outcome measures to assess the success of the programme. Alongside these one-year headline measures, the Employment Data Lab team used administrative data to analyse participants’ labour market outcomes for up to two years after starting the programme. These two-year outcomes are presented in the main body of the report.

• For the main analysis, the Employment Data Lab team also used administrative data to analyse participants’ education outcomes for up to a year after starting the programme.

• Participants were compared to a comparison group of similar individuals to evaluate the programme.

1. What you need to know

What is the Employment Data Lab?

The Employment Data Lab is a service provided by a team of analysts at the Department for Work and Pensions (DWP). The Data Lab provides group-level benefits and employment information to organisations who have worked with people to help them into employment. The purpose is to provide these organisations with information to help them understand the impact of their programmes. Find more information about the Employment Data Lab and its background.

What is the Pathways to Success programme?

The Pathways to Success programme is a partnership programme led by Hertfordshire County Council that supported young people aged 16 to 25, who were not in employment, education or training (NEET) to develop skills needed to return to education or training or enter work.

Programme interventions included tailored one-to-one meetings with an advisor at a young people’s centre to reduce barriers into employment and support progression into employment, education or training. Participants were offered assistance with CV writing, help searching and applying for jobs, interview skill practice, career and educational route planning advice, and employability skills training. The programme also facilitated work experience opportunities, referred participants to youth work projects that met their needs, and promoted engagement with Career Hubs and centre-based services.

Who was evaluated as part of this analysis?

Hertfordshire County Council shared data on 1,294 participants who started the programme between January 2020 and August 2023. The main impact analysis focuses on a subset of 800 participants aged 16 to 25 who could be matched to administrative data at the time of the analysis.

For more details see Appendix A.

Participant information

The administrative data available for the 800 participants included in this analysis indicated that:

• 44% were female and 56% were male

• the average age was 17 years

• 74% were white

• 44% had previously had Special Educational Needs (SEN) provisions

• 31% had previously been eligible for Free School Meals (FSM)

• the average number of GCSEs with grades C or 4 or higher was 2

• the average number of GCSEs with grades D or 3 or lower was 2

Further information on those who were and were not included as part of the analysis and missing markers in the administrative data can be found in Appendix A and Appendix B.

The analysis in this report

This report presents analysis on the impact of the PtS programme by comparing education, employment and benefit outcomes of participants to those of a matched comparison group who did not participate. The comparison group is used to estimate the outcomes of participants had they not participated in the programme and was created using a method called propensity score matching (PSM). Further information about how the analysis was conducted can be found in the associated methodology document.

The following primary outcome measures were selected for this evaluation before the analysis was undertaken.

Primary outcome measures

• The percentage of the group classed as employed a year after starting on the programme.
• The percentage of the group who passed an education or training course at any time during the year after starting the programme.

The main analysis in this report (Section 2 and Section 3) presents the impacts for the sub-group of 800 individuals aged 16 to 25 who participated in the PtS programme. Standard two-year labour market outcomes are presented in addition to one-year primary outcomes.

2. The labour market impacts of the programme over time

The results show that, at one year, the programme led to:

• more classed as employed
• more classed as looking for work
• no statistically significant impact on those classed as inactive
• fewer classed as ‘other’

An increase in employment, at one year

Between 5 and 12 percentage points more programme participants were classed as employed a year after starting the programme than had they not participated.

The result was statistically significant.

More classed as looking for work, at one year

Between 2 and 7 percentage points more programme participants were classed as looking for work a year after starting the programme than had they not participated.

The result was statistically significant.

No statistically significant impact on those classed as inactive, at one year

Between 2 percentage points less and 2 percentage points more programme participants were classed as being on inactivity benefits a year after starting the programme than had they not participated.

The result was not statistically significant.

Fewer classed as ‘other’, at one year

Between 7 and 14 percentage points fewer programme participants were classed as ‘other’ a year after start.

The result was statistically significant.

Employment Data Lab reports use four categories of labour market status (see Section 7 for more details).

The figures and tables in this section show the number of people in each labour market category over the two years before and after starting the intervention. The participants are compared to a comparison group used to estimate the outcomes they would have achieved had they not participated in the programme. The difference between the groups can be interpreted as the impact of the programme.

The results in Figure 1 and Table 1 show that the programme led to a statistically significant increase in both employment and looking-for-work benefit claims at one and two years after programme start. The results also show that the programme had no statistically significant impact on inactivity benefit claims at one and two years after programme start, but that the programme did lead to a statistically significant decrease in the number of participants in the ‘other’ category at one and two years after programme start.

Figure 1: Impact of the programme on the numbers in each labour market category over the two years after programme start

The plots on the left show the percentages of the participant and comparison groups in each category. The difference (or impact of the programme) is shown in the plots on the right. The solid line shows the central estimate, and the shaded area around the line indicates the 95% confidence interval.

1a and 1b – Employed: The impact plot (1b) shows the programme had a positive and statistically significant effect on the percentage of participants who were employed. The outcomes plot (1a) shows the percentage of both groups employed at start is similar, but the percentages increase at a faster rate for the participant group over the year after start. Plot (1a) then shows that between year one and year two after start, the percentage of the comparison group in employment increases at a faster rate, causing the impact of the programme on percentage of participants in employment to decrease but still remain statistically significant.

1c and 1d – Looking for work: The impact plot (1d) shows the programme resulted in a statistically significant increase in those on looking-for-work benefits. The outcomes plot (1c) shows the percentage of both groups in the looking for work category increasing up to programme start, with the participant group continuing to increase for a short period after programme start whilst the comparison group remains constant in the 2 years after start. Plot (1c) then shows that the percentage in the category remains constant for the participant group, resulting in a sustained impact.

1e and 1f – Inactive: The impact plot (1f) shows the programme had no statistically significant impact on the percentage of participants in receipt of ‘inactive’ benefits. The outcomes plot (1e) shows the percentage of both groups on inactive benefits.

1g and 1h – Other: The impact plot (1h) shows the programme led to a statistically significant and sustained reduction in the percentage of participants in the ‘other’ category. The outcomes plot (1g) shows the percentage categorised as ‘other’ fell at a faster rate for the participant group initially. In the 2 years after start, the impact decreased but remained statistically significant.

Additionally, the outcomes plot (1g) shows that two years before start approximately 90% of participants are in the ‘other’ category. This could include: 

• individuals in education and not in employment or in receipt of looking-for-work or inactive benefits  

• individuals who are NEET and not in receipt of looking-for-work or inactive benefits 

For brevity, the latter subset is referred to as ‘other NEET’ and examined in detail in Section 3 and Section 4 of this report.

Table 1: Percentage of each group in each category at one and two years after programme start, along with estimated impacts and statistical significance

The “upper” and “lower” values give the 95% confidence interval around the central estimate of the impact. Percentage points are denoted by ppt.

Percentage of group in category:	Participant group (%)	Comparison group (%)	Impact: Central (ppt)	Impact: Lower (ppt)	Impact: Upper (ppt)	Significant
Employed (1 year)	47	38	9	5	12	Yes
Looking for work (1 year)	17	12	4	2	7	Yes
Inactive (1 year)	8	7	0	-2	2	No
Other (1 year)	36	46	-10	-14	-7	Yes
Employed (2 years)	53	49	4	0	7	Yes
Looking for Work (2 years)	17	14	3	0	6	Yes
Inactive (2 years)	12	11	1	-1	3	No
Other (2 years)	25	31	-6	-9	-3	Yes

Note: Values are rounded to the nearest whole number, so statistically significant impacts may have a 95% confidence interval with a lower bound of zero.

Note: Categories are not mutually exclusive, so percentages in category do not sum to 100% and impacts do not sum to 0.

Table 2 shows the effect of the programme on the percentage of participants who had at least one sustained employment spell at any point during the two years after start. Sustained employment is defined as being employed for six months or more in a row. During the two years after start between 5 and 12 percentage points more participants had a sustained employment spell.

Table 2: Percentage of the participant and comparison groups who had a sustained employment spell at any point during the two years after start

Percentage of group with sustained employment at any point…	Participant group (%)	Comparison group (%)	Impact: Central (ppt)	Impact: Lower (ppt)	Impact: Upper (ppt)	Significant
…in the 2 years after start	58	50	8	5	12	Yes

To explore the labour market impacts further, Table 3 shows an additional labour market impact measure – the percentage of each group who were in work at any point during the one and two years after starting the programme. The results indicate the programme led to a statistically significant increase in the number of participants in work at any point during:

• the year after programme start, employment increased between 9 and 16 percentage points

• the two years after programme start, employment increased between 8 and 14 percentage points

Table 3: Percentage of the participant and comparison groups in the employed category at any point during the 12 and 24 months after start

Percentage of group employed at any point…	Participant group (%)	Comparison group (%)	Impact: Central (ppt)	Impact: Lower (ppt)	Impact: Upper (ppt)	Significant
…in the year after start	59	47	12	9	16	Yes
…in the 2 years after start	76	65	11	8	14	Yes

Note: Specific limitations mean that the figures in this section should be treated with a greater degree of caution. See Section 7 for more details.

3. Impact on education and training

The results show that the programme led to:

Increase in education and training courses passed

The percentage of participants who passed an education or training course at some point during the year after start, was between 1 and 8 percentage points higher than had they not participated.

The result was statistically significant.

Table 4 shows the impact of the PtS programme on the percentage of participants who passed an education or training course at any time during the year after starting the programme.

Table 4: Percentage of the participant and comparison groups classed as passing a course during the year after programme start 

Percentage who passed a course at any point…	Participant group (%)	Comparison group (%)	Impact: Central (ppt)	Impact: Lower (ppt)	Impact:  Upper (ppt)	Significant
…in the year after start	33	29	4	1	8	yes

As mentioned in the previous section, approximately 90% of participants start in the ‘other’ category. To investigate this further, enrolment in education was combined with the ‘other’ category to derive the ‘other NEET’ category. This allows those not enrolled in education to be classified as ‘other NEET’ and consists of individuals who are NEET and not in receipt of either looking-for-work or inactive benefits.

The ‘education’ category is defined irrespective of labour market status and includes all individuals who are enrolled in education or training across the four labour market categories: employed, looking for work, inactive, and other.

Figure 2 shows the percentage of participants and comparison groups enrolled in education or training and the ‘other NEET’ category over the two years prior to starting the intervention through to one year after programme start. These outcomes are reported only up to one year after programme start because Department for Education (DfE) data is only available to the DWP Employment Data Lab up to July 2024.

Figure 2: Impact of the programme on enrolment in education or training and the ‘Other NEET’ category over the year after programme start 

2a and 2b – Education: The impact plot (2b) shows the programme led to a statistically significant increase in the percentage of participants enrolled in education or training. The outcomes plot (2a) shows the percentage of both groups enrolled in education or training.

2c and 2d – Other NEET: The impact plot (2d) shows the programme led to a statistically significant reduction in the percentage of participants in the ‘other NEET’ category. The outcomes plot (2c) shows the percentage categorised as ‘other NEET’ fell at a faster rate for the participant group in the year after start. This indicates that the decrease in the percentage of participants in the ‘other’ category, illustrated in Figure 1(g), may be driven by the decrease in percentage of ‘other NEET’.

4. Longer term impacts on education and training 

The Department for Education provides education and training data to the DWP Employment Data Lab, with data currently available up to July 2024. Therefore, the two years of education outcomes can only be estimated for a subset of 715 programme participants who started the programme before 1 July 2022. The figures and tables in this section represent this subgroup of the main cohort.  

Table 5 in this section shows the impact of the programme on people who passed an education or training course during the two years after start. The results suggest the impact of the PtS programme was a statistically significant increase of between 3 and 10 percentage points in courses passed during the two years after start.

Table 5: Percentage of the participant and comparison groups classed as passing a course during the two years after programme start for a subset of participants with two years of education outcomes

Percentage who passed a course at any point…	Participant group (%)	Comparison group (%)	Impact: Central (ppt)	Impact: Lower (ppt)	Impact:  Upper (ppt)	Significant
…in the 2 years after start	48	41	7	3	10	yes

Table 6 and Figure 3 provide a breakdown of the level^{[footnote 1]} of qualifications obtained by participants and the comparison group at any point during the two years after start. The results suggest the PtS programme led to a large positive impact in the number of people passing entry level and level one qualifications, as well as a more modest increase in those passing level two courses. The results also suggest that the programme led to a statistically significant reduction in the number of people obtaining a level three or higher qualification over the same period. 

Table 6: Percentage of each group who passed at least one course at the specified level at any point in the two years after programme start

Percentage obtaining a qualification, by level	Participant group (%)	Comparison group (%)	Impact: Central (ppt)	Impact: Lower (ppt)	Impact:  Upper (ppt)	Significant
Entry Level	16	5	12	9	14	yes
Level One	25	11	14	10	17	yes
Level Two	29	23	6	3	10	yes
Level Three	8	17	-9	-11	-7	yes

Figure 3: Percentage of the participant and comparison groups who passed at least one entry level, level one, level two, and level three or higher course at any point during the two years after programme start

Note: These levels are not mutually exclusive, so where an individual passed different courses at different levels, multiple levels would all be recorded. 

Note: These figures include qualifications obtained for education spells that were active prior to individuals starting the programme. 

Note: Specific data limitations mean that these figures should be treated with a greater degree of caution.  See  Section 7  for more details.

Figure 4 shows the percentage of participants and comparison groups enrolled in education or training and the ‘other NEET’ category over the two years before and after starting the intervention.

Figure 4: Impact of the programme on enrolment in education or training and the ‘Other NEET’ category over the two years after programme start 

4a and 4b – Education: The impact plot (4b) shows the programme led to a statistically significant increase in the percentage of participants enrolled in education or training during the year after start, but the impact decreased between one and two years after start. At two years after programme start, there was no statistically significant difference in the percentage of participants enrolled in education or training.

The decrease in enrolment between year one and two after start may be explained by participants passing courses and, subsequently, leaving education.

The outcomes plot (4a) shows the percentage of both groups enrolled in education or training.

4c and 4d – Other NEET: The impact plot (4d) shows the programme led to a statistically significant reduction in the percentage of participants in the ‘other NEET’ category. The outcomes plot (4c) shows the percentage categorised as ‘other NEET’ fell at a faster rate for the participant group initially, but remains constant one year after start, resulting in a sustained impact.

5. How to use the results of this report

Two primary outcome measures were chosen to assess the success of this programme. The results suggest that the programme had a positive statistically significant impact on both measures. This suggests the programme has been successful at:

• Increasing the percentage of the group classed as employed a year after starting.

• Increasing the percentage of the group that passing an education qualification at any time during the year after starting the programme.

A range of secondary outcome measures were also analysed in this report (see Appendix E) and can be used to learn more about the impacts of the programme. Results marked as statistically significant indicate an estimate that is unlikely to have occurred by chance (and more likely to be a causal impact of the programme). If a result is not statistically significant it does not mean that there was no impact, it just means there was insufficient evidence to verify this to the required threshold.

The estimates in this report were generated using quasi-experimental methods that can be less reliable than experimental methods such as a randomised control trial. The results should be used with a degree of caution.

The estimates relate to a programme working in a particular context. This report makes no assessment as to whether these impacts are generalisable to different contexts. The estimates were also made in a “business as usual” setting where participants and comparators were free to go on to access other support.

6. Hertfordshire County Council in their own words 

Hertfordshire County Council have provided a description of their programme in their own words and a response to the analysis. Those can be found in the Response to Employment Data Lab’s Analysis: Hertfordshire County Council in their own words.

7. About these statistics

This report presents estimates of the impact of a programme. This is achieved by comparing the outcomes of the programme participants to a credible estimate of their outcomes had they not participated in the programme. This is often referred to as the counterfactual. In this report the counterfactual was generated using a quasi-experimental technique called Propensity Score Matching (PSM). This involves constructing a comparison group of individuals who did not participate in the programme but who are matched on key characteristics that affect whether an individual takes part in the programme and the outcomes that they experience as a result of participation.

Once this comparison group has been constructed the outcomes of the two groups can be compared to generate the estimate of the impact of the programme. More information about this technique and how it is used in the Data Lab can be found in the methodology report.

Categorisation

The analysis in this report is based on the labour market outcomes of the participants (and a matched comparison group) in the two years before and after starting the intervention. This report uses four categories of labour market status for the analysis, detailed below.

• Employed: People who are either employed or self-employed.

• Looking for Work: People who are in receipt of Jobseeker’s Allowance (JSA), or in the Universal Credit (UC) “intensive work search”, “light touch out of work”, “light touch in work”, or “working enough” conditionality regimes.

• Inactive: People who are in receipt of inactive benefits such as Employment and Support Allowance (ESA) or in the UC “no work requirements” or “work focussed interview” conditionality regimes. Several other benefits also fall into this category, though the number of people on these benefits is small. See methodology report for details.

• Other: People who do not fall into the above three categories, this could include people who are in full-time education and not working or receiving benefits or those who are in custody.

These categories are not mutually exclusive, and it is possible to be in more than one category. For example, someone working fewer than 16 hours a week may also be in receipt of JSA and would be classed as “employed” and “looking for work”.

Statistical significance

The report highlights if the results are statistically significant or not. A statistically significant result is one that is unlikely to have occurred by chance because of sampling error. If a result is not statistically significant it does not mean that the intervention has no impact, it simply means that there is not enough evidence to verify this to a required threshold. In this report, unless otherwise stated, the threshold for significance is 95%.

This report sometimes presents the central estimate of a result along with the upper and lower confidence values. These upper and lower values create a range that you would expect the estimate to fall within if the test was to be redone, within a certain level of confidence. This level is set at 95 per cent unless otherwise stated. The confidence intervals will typically be stated in the tables of results and be presented on graphs and plots as either error bars or shaded regions.

Limitations

PSM is used to construct a comparison group of individuals that are matched on key characteristics that are linked to a person’s participation in the programme and the outcome variables of interest. The validity of the technique used in this report rests on the assumption that all the characteristics that are linked to a person’s participation in the programme and the outcome variables of interest have been sufficiently accounted for in the analysis, either explicitly or otherwise. This is a strong assumption that cannot be tested and depends on the data available and on the nature of each programme and its participants. This is reviewed on a case-by-case basis in the Data Lab and impact evaluations are only carried out where the validity of this assumption is plausible. That said, these are quasi-experimental techniques that tend to be less robust than true experimental methods, such as a randomised control trial, and the results must be treated with a degree of caution.

Particular caution should be applied when using and interpreting the education related results as they refer to enrolment on a course, whereas actual attendance and/or drop-outs are not always captured and accounted for. This has potential for someone to appear as though they are in education when in practice they are not (for example if they enrolled on a course and subsequently dropped out). Since these programmes are aimed at people who are NEET, i.e. Not in Employment, Education or Training, it is possible that this issue would be more likely to affect the participant group than the comparison group, therefore, introducing some bias into the results. Whilst there are indications in the data that this sort of bias may be present to a degree, sensitivity analyses have been conducted providing reassurance that the benefit and employment related impact estimates are robust.

Also, in PSM some participants will be excluded from the analysis because they have no matched comparator with a similar propensity score. This can potentially bias findings if that group has different outcomes. However, a sufficiently small percentage had no matched comparator, so this does not raise concerns about the representativeness of the results – for more information see Appendix C.

Where to find out more  

Read the Employment Data Lab analysis, information and guidance.

8. Statement of compliance with the Code of Practice for Statistics

The  Code of Practice for Statistics (the Code) is built around 3 main concepts, or pillars:  

• Trustworthiness – is about having confidence in the people and organisations that publish statistics 

• Quality – is about using data and methods that produce statistics 

• Value – is about publishing statistics that support society’s needs  

The following explains how we have applied the pillars of the Code in a proportionate way.

Trustworthiness  

Employment Data Lab reports, such as this, are published to provide User Organisations with an estimate of the impact of their programmes that support employment. Releasing them via an ad hoc publication will give equal access to all those with an interest in them.

Quality 

The methodology used to produce the information in this report has been developed by DWP analysts in conjunction with the Institute for Employment Studies. The information is based on data from the User Organisation and Government administrative data. The calculations have been quality assured by DWP analysts to ensure they are robust.

Value 

Producing and releasing these estimates provides User Organisations and the public with useful information about employment support provision that they may not have otherwise been able to generate or obtain.

Appendix A: Exclusions from the treatment group

Hertfordshire County Council shared data on 1,294 participants who took part in the programme between January 2020 and August 2023. The main impact analysis focussed on a subset of these participants who were aged 16 to 25 and lived in England when they started the programme. This was to ensure that participants had coverage in the education and training data available to DWP.

Figure 5 shows the distribution of programme start dates for all 916 programme participants who were matched to DWP administrative datasets, along with reasons for exclusion from the analysis.

Figure 5: Distribution of start dates of the programme participants

Figure 6 shows the stages that individuals were excluded from the analytical process. The final group of 800 participants, comprising the main analysis group used in the PSM, represent approximately 87% of the matched participants who took part during the relevant period.

Figure 6: Numbers of participants and the stages at which they were excluded from the analysis

Appendix B:  Participant group information 

The following table displays the participant group information for the full analysis sample who could be matched to administrative data. This is then broken down into the “evaluated”; those who were selected for the evaluation, the “non-evaluated”; those who were excluded from the evaluation, and “all” of the participants, for whom data was available. This table only includes participants who could be linked to the administrative data.

Table 7: Showing characteristics, benefits and employment information for the participant group

Variable	Evaluated	Non-Evaluated	All
Observations	800	116	916
Age (mean years)	17.1	16.2	17
18 years or under (%)	90	96	91
Over 18 years (%)	10	x	9
Male (%)	56	61	57
SEN marker set (%)	44	44	44
FSM marker set (%)	31	29	31
Care leaver/adopted marker (%)	6	x	6
Child in need marker (%)	13	x	13
Exclusion marker (%)	27	28	27
Permanent exclusion marker (%)	x	x	x
Employed marker (%)	27	x	25
Child on Child Benefit claim marker (%)	89	89	89
Other ethnicity (%)	x	x	x
Asian ethnicity (%)	x	x	3
Black ethnicity (%)	4	x	3
Chinese ethnicity (%)	x	x	x
Mixed ethnicity (%)	7	x	7
White ethnicity (%)	74	59	72
Missing ethnicity (%)	11	27	13
Entry level qualification (%)	72	x	65
Level 1 qualification (%)	70	x	64
Level 2 qualification (%)	54	x	49
Level 3 qualification (%)	9	x	8
Level 4 qualification (%)	x	x	x
Level 5 qualification (%)	x	x	x
Level 6 qualification (%)	x	x	x
Level 7 qualification (%)	x	x	x
Level 8 qualification (%)	x	x	x
Enrolled in ‘School’* at start (%)	54	58	54
Enrolled in Further Education* at start (%)	8	x	7
Enrolled in Higher Education at start (%)	x	x	x
Number of months ‘Employed’ in the previous two years	2	1	2
Number of months ‘Looking for work’ in the previous two years	1	0	1
Number of months ‘Inactive’ in the previous two years	1	0	1
Number of months ‘Other’ in the previous two years	10	16	10
Number of A*/A GCSE grades	0	0	0
Number of B GCSE grades	1	0	1
Number of C GCSE grades	1	0	1
Number of D GCSE grades	1	1	1
Number of E GCSE grades	1	1	1
Number of F/G GCSE grades	1	0	1
Total number of GCSE qualifications at grades A*to C	2	1	2
Total number of GCSE qualifications at grades D to G	2	2	2

Note: Some figures which have been suppressed for disclosure control purposes are denoted by an x.

*The ‘School’ category covers any education or training spells captured in the School Census, Pupil Referral Unit Census, Alternative Provision Census, Key Stage 4 or Key Stage 5 datasets

Appendix C: Matching the comparison group

PSM is used to construct a comparison group of individuals that are matched on key characteristics that are linked to a person’s participation in the programme and the outcome variables of interest. More information about this technique and how it is used in the Data Lab can be found in the methodology report.

Before proceeding with the analysis, the Data Lab team assessed the plausibility of constructing a comparison group that satisfies the conditional independence assumption that underlies PSM (see methodology report for more details). The programme was targeted at individuals with some characteristics that were well represented in the available data.

The comparison pool was selected from the Department for Education’s (DfE) administration data and was restricted to only include individuals who were in the same age range as the participants at the time of the programme start. This group was then assigned a pseudo-start date (in lieu of an actual start date) in a way that matched the distribution of participant start dates. This group was then reduced further in additional steps: 

• Firstly, individuals who were known to have participated in other European Social Fund (ESF) programmes were excluded. 

• The group was then restricted to geographic locations that were similar to, but excluded, the locations that the programme was delivered in. The selection of these locations was based on cluster analysis that used local labour market characteristics and indicators to group Local Authorities in Great Britain into 14 groups. See Appendix D for more info. 

• The comparison pool was restricted further by stratified sampling that matched the distributions of the comparison and participant groups on their age and labour market status at programme start/pseudo-start.

These steps resulted in a comparison pool of approximately 129,000 individuals who were then used in the matching process.

The matching estimator used to generate the impact estimates presented in this report was nearest neighbour matching using 100 nearest neighbours and a bandwidth of 0.01. Nearest neighbour matching involves running through each participant and matching them with the closest eligible individuals from the comparison pool, determined by closeness of the propensity scores. The sensitivity of the impact estimates to the choice of matching estimator was tested using a range of estimators and found to be insensitive. Further information about matching estimators can be found in the methodology and literature review documents.

Table 8 below shows a sample of the variables used in the matching process and the mean values of these variables both before and after matching. The table shows that before matching, the participant and comparison groups are not well matched, or balanced, as shown by sizeable differences in the mean values. After matching, the mean values of the participant and comparison groups are much closer. The percent bias and p-value columns provide information on how big the residual difference is and if this difference is statistically significant. Ideally one would like the percent biases to be small (below 5%) and there to be no statistically significant differences i.e, p-values above 0.05 (the 95 percent confidence level threshold).

Table 9 also presents summary statistics that relate to how well matched the participant and comparison groups are for the main run. It shows values for Rubin’s B, Rubin’s R and the maximum and median percent biases, all of which meet commonly accepted thresholds for the selected approach (see the methodology report for more details). The table also shows there were 2 participants (0.25 percent) who were off support (had no matched comparator). This is a sufficiently small percentage so as not to raise concerns about the representativeness of the result.

Table 8: Showing the mean value of each control variable for the main run, before and after matching

Variable	Unmatched Comparison Group	Unmatched participant group	Matched comparison group	Matched participant group	Percent bias after matching	P value
DFE_C_start (%)	51.8	53.6	50.8	53.5	5.4	0.28
CHAR_AGE	17	17.1	17.1	17.1	-4.9	0.33
SPELL_HIST_EMPLOYMENT (%)	22.8	26.7	28.7	26.6	-4.8	0.35
CHAR_AGE_SQ	291.7	293.4	295.7	293.3	-4.8	0.33
DFE_month_m_3 (%)	66.4	62.2	59.8	62.1	4.8	0.35
DFE_month_m_1 (%)	62.1	58.4	55.9	58.3	4.8	0.34
SPELL_OTHER_m6 (%)	83.5	80.5	78.8	80.6	4.8	0.36
SPELL_OTHER_m3 (%)	82.4	78.4	76.7	78.5	4.6	0.38
SPELL_OTHER_m9 (%)	85.2	83.8	82.2	83.9	4.6	0.38
SPELL_WORK_m9 (%)	12.6	10.8	12.3	10.9	-4.4	0.37
SPELL_HIST_UC (%)	5.6	12.2	13.4	12.1	-4.4	0.46
SPELL_OTHER_m1 (%)	82.1	77.6	75.9	77.6	4.3	0.41
SPELL_LFW_m1 (%)	3.4	8.9	9.8	8.9	-3.7	0.54
SPELL_HIST_C (%)	90.9	87.9	86.7	87.9	3.7	0.5
GEOG_CLUSTER_13 (%)	0.1	x	2	x	-3.6	0.6
SPELL_WORK_m6 (%)	13.6	12.3	13.6	12.4	-3.6	0.48
SPELL_WORK_m3 (%)	14	12.6	13.8	12.6	-3.5	0.48
GCSE_D	0.7	1	1	1	3.5	0.51
SPELL_OTHER_m2 (%)	82.2	77.9	76.6	78	3.5	0.51
SPELL_OTHER_m21 (%)	91.2	91.9	90.9	91.9	3.4	0.5
dfe_level_3_start (%)	14.2	8.5	9.5	8.5	-3.2	0.47
SPELL_OTHER_m18 (%)	89.7	90.4	89.4	90.4	3.2	0.53
SPELL_OTHER_m15 (%)	88.2	88.2	87.1	88.1	3.1	0.54
SPELL_OTHER_m12 (%)	86.8	86	85.1	86.1	3.1	0.55
SPELL_WORK_m21 (%)	7.9	5.7	6.5	5.8	-2.9	0.54
SPELL_WORK_m18 (%)	9.2	6.4	7.1	6.4	-2.8	0.55
SPELL_HIST_A (%)	27.1	46.8	48	46.6	-2.8	0.6
CHAR_INT_MONTH_6 (%)	12.1	13.8	14.8	13.9	-2.8	0.6
SPELL_WORK_m15 (%)	10.4	8	8.8	8	-2.7	0.57
SPELL_HIST_HB (%)	0.8	4.4	4.7	4.3	-2.7	0.68
GCSE_AA	1.8	0.2	0.2	0.2	-2.7	0.19
SPELL_OTHER_m24 (%)	92.5	94.8	94.2	94.9	2.6	0.57
SPELL_Inactive_m6 (%)	1.3	3.5	3.8	3.4	-2.6	0.67
CHAR_INT_MONTH_11 (%)	8.1	9	9.6	8.9	-2.5	0.63
SPELL_HIST_ESA (%)	0.3	x	1.5	x	-2.4	0.72
SPELL_WORK_m2 (%)	13.8	12.5	13.3	12.5	-2.4	0.63
DFE_month_m_6 (%)	71.4	68.7	67.6	68.6	2.3	0.65
SPELL_WORK_m12 (%)	11.5	8.9	9.6	8.9	-2.3	0.63
SPELL_Inactive_m12 (%)	0.9	x	3.1	x	-2.3	0.71
SPELL_HIST_PIP (%)	4.2	8.1	8.7	8.1	-2.2	0.7
SPELL_Inactive_m18 (%)	0.6	x	2.2	x	-2.2	0.73
DFE_exclusion (%)	10.1	26.6	27.3	26.5	-2.2	0.71
SPELL_WORK_m1 (%)	13.5	11.3	12.1	11.4	-2.2	0.66
SPELL_Inactive_m3 (%)	1.5	3.5	3.7	3.4	-2.1	0.72
SPELL_Inactive_m21 (%)	0.5	x	1.8	x	-2.1	0.74
GCSE_C	0.9	0.8	0.8	0.8	2.1	0.66
SPELL_Inactive_m1 (%)	1.7	3.5	3.7	3.4	-2	0.72
SPELL_WORK_m24 (%)	6.8	3.4	3.7	3.3	-2	0.64
SPELL_Inactive_m24 (%)	0.5	x	1.7	x	-1.9	0.76
SPELL_Inactive_m15 (%)	0.8	x	2.6	x	-1.9	0.76
CHAR_INT_MONTH_7 (%)	8.9	6	5.5	6	1.8	0.69
DFE_month_m_12 (%)	81.1	80.4	79.7	80.4	1.8	0.72
CHAR_INT_MONTH_8 (%)	7.4	5.5	5.9	5.5	-1.8	0.71
SPELL_Inactive_m2 (%)	1.6	3.5	3.7	3.4	-1.8	0.76
SPELL_Inactive_m9 (%)	1.1	3.2	3.4	x	-1.7	0.78
SPELL_LFW_m2 (%)	3.1	7.9	8.2	7.9	-1.6	0.79
CHAR_INT_YEAR_2020 (%)	46.1	45.8	45	45.8	1.6	0.75
DFE_ETHNICITY_MIXED (%)	4.1	7.2	7.6	7.3	-1.6	0.78
SPELL_HIST_CHB_CHILD (%)	82.7	89.3	88.7	89.3	1.5	0.73
CHAR_INT_MONTH_10 (%)	11	11.5	11	11.5	1.5	0.76
SPELL_LFW_m6 (%)	2.1	4.2	4.5	4.3	-1.5	0.8
dfe_level_0_start (%)	78.6	71.8	72.4	71.8	-1.5	0.78
DFE_ETHNICITY_BLACK (%)	2.7	3.5	3.7	3.5	-1.4	0.79
SPELL_LFW_m3 (%)	2.8	6.9	7.2	6.9	-1.4	0.82
PIT_SANC_HIST (%)	0.3	x	0.5	x	-1.4	0.8
DFE_month_m_9 (%)	77.4	78.3	77.7	78.3	1.3	0.79
CHAR_INT_MONTH_2 (%)	8.9	8.9	8.5	8.9	1.2	0.81
DFE_ETHNICITY_WHITE (%)	70.3	74.4	73.9	74.4	1.2	0.81
CHAR_INT_YEAR_2021 (%)	35.1	34.7	35.2	34.6	-1.1	0.82
CHAR_INT_MONTH_3 (%)	7.1	7	6.7	7	1.1	0.82
DFE_SEN (%)	20.2	44.4	43.8	44.3	1.1	0.85
DFE_month_m_24 (%)	59.3	66.7	67.1	66.6	-1	0.83
CHAR_CHILDREN (%)	0.7	x	0.8	x	-1	0.84
SPELL_HIST_H (%)	3.4	x	0.6	x	-1	0.72
DFE_FSM (%)	15.4	31.2	31.4	31	-1	0.86
DFE_CLA (%)	2.1	5.7	5.9	5.8	-0.9	0.88
GEOG_CLUSTER_14 (%)	26.2	14.1	14.5	14.1	-0.9	0.84
SPELL_HIST_CHB_PARENT (%)	0.8	x	1	x	-0.9	0.87
SPELL_HIST_JSA (%)	0.1	x	0.2	x	0.9	0.88
GEOG_CLUSTER_12 (%)	23.6	38.7	38.3	38.8	0.9	0.87
GEOG_CLUSTER_3 (%)	0.1	x	0.3	x	-0.9	0.89
SPELL_LFW_m18 (%)	0.7	x	1.4	x	0.8	0.89
CHAR_INT_MONTH_5 (%)	7.4	7.6	7.4	7.6	0.7	0.88
DFE_H_start (%)	3.5	x	0.2	x	-0.7	0.65
SPELL_HIST_IS (%)	x	x	0.2	x	0.7	0.91
DFE_ETHNICITY_ASIAN (%)	4.6	x	2.6	x	0.7	0.87
CHAR_INT_MONTH_12 (%)	5.2	5.5	5.3	5.5	0.7	0.89
CHAR_INT_MONTH_4 (%)	7.3	7.6	7.4	7.6	0.7	0.89
CHAR_INT_YEAR_2022 (%)	16.1	16.3	16.6	16.4	-0.7	0.9
PIT_INT_HIST (%)	0.6	x	0.8	x	-0.7	0.9
DFE_ETHNICITY_CHINESE (%)	0.3	x	0.3	x	-0.6	0.9
DFE_month_m_21 (%)	70.7	73.7	73.9	73.6	-0.6	0.9
SPELL_HIST_DLA (%)	6.5	11.2	11.4	11.3	-0.6	0.91
GEOG_CLUSTER_6 (%)	6.2	8.6	8.3	8.5	0.6	0.91
GEOG_CLUSTER_4 (%)	13.1	18.7	18.5	18.8	0.6	0.91
DFE_CIN (%)	6.7	13.3	13.6	13.4	-0.6	0.92
DFE_month_m_15 (%)	78.9	76.4	76.3	76.5	0.6	0.91
SPELL_HIST_ICA (%)	0.5	x	0.9	x	-0.6	0.92
dfe_level_1_start (%)	78.1	70.4	70.6	70.4	-0.6	0.91
GEOG_CLUSTER_2 (%)	0.1	x	0.1	x	-0.5	0.93
PIT_INT_START (%)	0.6	x	2.3	x	0.4	0.94
GEOG_CLUSTER_10 (%)	0.1	x	0.1	x	0.3	0.95
GEOG_CLUSTER_MISSING (%)	0.2	x	0.6	x	-0.3	0.96
SPELL_LFW_m12 (%)	1.2	x	2.4	x	-0.3	0.96
CHAR_INT_MONTH_9 (%)	4.8	4.4	4.4	4.4	-0.3	0.95
GEOG_CLUSTER_9 (%)	18.8	8.2	8.3	8.3	-0.3	0.94
GCSE_FG	0.2	0.6	0.6	0.6	-0.3	0.96
GCSE_B	1.8	0.7	0.7	0.7	-0.3	0.94
SPELL_LFW_m9 (%)	1.6	x	2.7	x	-0.3	0.96
GEOG_CLUSTER_8 (%)	11.5	8.7	8.7	8.8	0.3	0.95
DFE_A_start (%)	18.7	7.7	7.7	7.8	0.3	0.95
GCSE_E	0.4	0.8	0.8	0.8	-0.3	0.96
SPELL_LFW_m21 (%)	0.5	x	1.1	x	-0.2	0.97
dfe_level_4_start (%)	0.8	x	0.4	x	0.2	0.95
dfe_level_2_start (%)	71.7	54.4	54.5	54.4	-0.2	0.96
GEOG_CLUSTER_7 (%)	0.1	x	0.1	x	-0.2	0.97
SPELL_LFW_m24 (%)	0.4	x	0.6	x	-0.2	0.97
DFE_ETHNICITY_OTHER (%)	0.9	x	1.4	x	0.2	0.97
DFE_permanent (%)	0.6	x	1.7	x	0.2	0.97
SPELL_LFW_m15 (%)	0.9	x	1.6	x	-0.2	0.97
CHAR_SEX (%)	54.4	56.2	56.2	56.3	0.2	0.97
DFE_ETHNICITY_MISSING (%)	17.1	10.5	10.5	10.5	-0.1	0.98
CHAR_INT_YEAR_2023 (%)	2.7	3.2	3.3	3.3	-0.1	0.99
CHAR_INT_MONTH_1 (%)	11.8	13.3	13.2	13.3	0.1	0.99
dfe_level_6_start (%)	0.6	x	0.1	x	-0.1	0.98
DFE_month_m_18 (%)	75	76.7	76.6	76.6	-0.1	0.99
dfe_level_5_start (%)	0.7	x	0.2	x	0	1
SPELL_HIST_WTC (%)	x	x	x	x	0	1
SPELL_HIST_CTC (%)	x	x	x	x	0	1
SPELL_HIST_BB (%)	x	x	x	x	0	1
SPELL_HIST_BSP (%)	x	x	x	x	0	1
SPELL_HIST_IB (%)	x	x	x	x	0	1
SPELL_HIST_PIB (%)	x	x	x	x	0	1
SPELL_HIST_SDA (%)	x	x	x	x	0	1
SPELL_HIST_WB (%)	x	x	x	x	0	1
PIT_REF_START (%)	x	x	x	x	0	1
PIT_REF_START (%)	x	x	x	x	0	1
GEOG_CLUSTER_1 (%)	x	x	x	x	0	1
GEOG_CLUSTER_5 (%)	x	x	x	x	0	1
GEOG_CLUSTER_11 (%)	x	x	x	x	0	1
dfe_level_7_start (%)	x	x	x	x	0	1
dfe_level_8_start (%)	x	x	x	x	0	1

Note: Some figures which have been supressed for disclosure control purposes are denoted by an x

Note: The definition of the matching variables can be found in the methodology document

Table 9: PSM summary statistics used to assess the success of the matching for the main analytical run

	Summary Statistics
Matching estimator	100 Nearest Neighbours
bandwidth/calliper	0.01
Rubin’s B	16.23
Rubin’s R	0.67
Max % bias	5.36%
Median % bias	1.40%
Number on support	800
Number off support	2
Percent off support	0.25%

Appendix D: Regional cluster analysis

For the analysis presented in this report the comparison pool was selected from outside the regions where the programme was implemented. As discussed on the methodology report, care must be taken when doing this as local factors such as the availability of employment and public transport and levels of disadvantage, etc. all have an impact on the likelihood of someone finding and maintaining employment.

This analysis took advantage of cluster analysis carried out within DWP that groups the Local Authorities of Great Britain into 14 groups based on a range of variables about key features of the local labour market. These include local employment rates, unemployment-related benefit caseload, qualification levels, variables related to mental and physical health / disability characteristics of the local population.

This cluster information was used as a way of selecting the comparison pool from “similar” regions of Great Britain, and as a control variable in the propensity score matching.

Appendix E: Table of results

Table 10: Full list of results for the main run featuring participants aged 16 to 25 at programme start and started before 1 April 2023

See Table 10 in Tables: Pathways to Success programme.

Note: Some figures which have been suppressed for disclosure control purposes.

Table 11: Full list of results for a subset of participants with two years of education outcomes

See Table 11 in Tables: Pathways to Success programme.

Note: Some figures which have been suppressed for disclosure control purposes.

Appendix F: Glossary of Terms

Term	Meaning
Care experienced	Refers to individuals who have spent any amount of time in the care system at any point.
Common support/ On Support/ Off support	Once propensity scores have been assigned for each observation, the overlap of propensity scores between the participants and comparison group is called ‘common support’. Those who fall in the overlap are referred to as ‘on support’, those who do not fall into the overlap are ‘off support’.
Comparison group	Carefully selected subset of the comparison pool, selected to have outcomes as similar as possible, to act as a counterfactual.
CIN	Child in Need
DfE	Department for Education
DLA	Disability Living Allowance
DWP	The Department for Work and Pensions
ESA	Employment and Support Allowance
ESF	European Social Fund
FSM	Free School Meals
GCSE	General Certificate of Secondary Education
JSA	Jobseeker’s Allowance
NEET	Not in Employment, Education or Training
NUTS	Nomenclature of Territorial Units for Statistics
Participant group	The people who took part in the programme being evaluated.
PIP	Personal Independence Payment
Programme	The employment support provision under investigation.
Pseudo-start date	Dates assigned to the comparison pool in lieu of the real programme start dates of the participant group.
PSM	Propensity Score Matching
Quasi-Experimental	An experimental technique that looks to establish a cause and effect relationship between two variables, where the assignment to the participant or comparison group is not random.
Rubin’s B & R	A test used to evaluate the matching in PSM.
SEN	Special Educational Needs
Statistically significant	Describes a result where the likelihood of observing that result by chance, where there is no genuine underlying difference, is less than a set threshold. In the Data Lab reports, this is set at 5 per cent.
UC	Universal Credit
User Organisation	The organisation using the employment data lab service.

1. For more information see What qualification levels mean: England, Wales and Northern Ireland  ↩