Research and analysis

Job Finding Support programme: A Quantitative Impact Assessment

Published 6 February 2025

Crown copyright 2025.

You may re-use this information (not including logos) free of charge in any format or medium, under the terms of the Open Government Licence. To view this licence, visit

http://www.nationalarchives.gov.uk/doc/open-government-licence/ or write to the

Information Policy Team, The National Archives, Kew, London TW9 4DU, or email

psi@nationalarchives.gov.uk.

This document/publication is also available on our website at:

https://www.gov.uk/government/organisations/department-for-work-pensions/about/research#research-and-analysis-publications

First published February 2025

Views expressed in this report are not necessarily those of the Department for Work and Pensions or any other government department.

Statement of application of the Code of Practice for Statistics

The analysis in this report has been produced as far as possible in line with the Code of Practice for Statistics. The code is built around 3 main concepts, or pillars, trustworthiness, quality and value:

• trustworthiness – is about having confidence in the people and organisations that publish statistics

• quality – is about using data and methods that produce assured statistics

• value – is about publishing statistics that support society’s needs for information

The following explains how we have applied the pillars of the Code in a proportionate way.

Trustworthiness

DWP analysts work to a professional competency framework and Civil Service core values of integrity, honesty, objectivity, and impartiality. Analysts have produced these statistics and conducted rigorous quality assurance in line to the standards usually applied to ad hoc releases. Background and methodology information is also included in the release.

The analysis has been signed off by the expert lead analyst and the Department’s Head of Profession for Statistics was consulted on the production and publication process and the timing of the publication was pre-announced.

Care has been taken to ensure only those who needed to see the analysis prior to publication had access to it.

The detailed methodology, data sources and econometric approach taken in this research are set out in this report alongside the findings. The approach used builds on methodology used in the previous labour market programme evaluations

Quality

The process to produce the analysis in this report was conducted by professional analysts taking account of the latest administrative data and applying methods using their professional judgement. The analysis has been through a rigorous quality-assurance and sign-off process by other DWP analysts. The statistical methodology used in this report by the Employment Data Lab has been externally reviewed by the Institute for Employment Studies. This can be viewed in the report Literature review and methodological background to the Employment Data Lab.

Value

The publication of this release ensures the information are equally available to all users as well as providing transparency. This research provides important new evidence for Ministers, policy makers and external stakeholders on the impacts of the Job Finding Support programme.

Executive summary

Background

This report presents an impact assessment and accompanying cost benefit analysis of the first cohort of the Job Finding Support programme, who were referred to the programme between 1 January 2021 and 1 April 2021.

The Job Finding Support programme was implemented by the Department for Work and Pensions (DWP) in the wake of the COVID 19 pandemic as part of a wider “Plan for Jobs” initiative. It ran between January 2021 and January 2022, with the aim of providing “quick-fire” support and advice to jobseekers. The offer consisted of 4 one-to-one sessions involving mock interviews, help to identify transferable skills and advice on how to switch industries, as well as online group sessions to improve job search techniques. The support was intended to take place over a period of no more than 20 working days, with the majority of participants expected to complete within 10 working days from starting.

To be eligible, participants had to have the right to reside and work in United Kingdom and be of working age, 18 (16 in Scotland and Wales) to State Pension Age.  The programme was targeted at those who were not in employment or on a zero-hours contract and who had been claiming benefits for 13 weeks or less, however exceptions were possible.  Please see the Job Finding Support provider guidance for more details.

Methodology

The impact assessment, undertaken by DWP’s Employment Data Lab, looks at employment and benefit outcomes for individuals who were referred to the Job Finding Support programme from 1 January 2021 to 1 April 2021. Outcomes are tracked one year after being referred to the programme. Outcomes from this group are compared to a matched comparison group who were not referred to the Job Finding Support programme.

Individuals in the 2 groups are matched together to account for differences in characteristics between the 2 groups to ensure a fair comparison. The methodology is well established and is considered a plausible means of estimating the impact of interventions of this type.

Earnings are also tracked for 12 months from referral to the programme. These results are then used to produce a cost benefit analysis. This focuses on different perspectives where different groups value the costs and benefits of the Job Finding Support programme differently, including a society perspective that combines all perspectives together. This analysis follows the DWP Social Cost-Benefit Analysis Framework (Fujiwara 2010) methodology, in line with the methodology used in other departmental impact assessments.

Key Findings

• The Job Finding Support programme led to an increase in the number of people classed as employed one year after being referred to the programme.

• For those who started the Job Finding Support programme, the increase was between 7 and 9 percentage points higher than it would have been had they not participated.

• The impact for all who were referred to the programme (irrespective of if they started) was an increase of between 3 and 4 percentage points higher than had the programme not existed. This accounts for the fact that the majority of those who were referred did not go on to start on the programme.

• The Job Finding Support programme was designed and implemented within the context of the COVID 19 pandemic. This report makes no assessment of the generalisability of these results to other contexts.

• Job Finding Support participants were free to participate in other employment support programmes and there is evidence to suggest that individuals who did participate were more likely to take up other DWP funded programmes.

• For the participants, the Job Finding Support programme makes a return of £1.46 for every pound spent at 12 months.

• For the Exchequer, the Job Finding Support programme makes a return of £6.51 for every pound spent at 12 months.

• The Job Finding Support programme makes a return of £9.52 for every pound spent when combining all perspectives at 12 months.

• These results should be viewed in context of the relatively low unit cost of the programme, at £63 per individual. The programme only required a small increase in earnings to break even.

The Authors

The impact assessment was undertaken by James Crowe, Economist at the Department for Work and Pensions, Employment Data Lab. The cost benefit analysis was undertaken by Sophie Jarvis, Operational Research Analyst at the Department for Work and Pensions, Labour Market analysis.

Acknowledgements

We would like to thank DWP colleagues for their help towards producing this report including:

Adam Robinson, Joe Cann, Charlie Kitson, Mark Langdon, Jake MacDonald, Jess Maddison, James Lewis, the DWP Employment Data Lab and Craig Lindsay.

Glossary

Term	Definition
Average treatment effect on the treated	The average estimated impact of a policy intervention among the group who were affected by the intervention
Comparison group	The group of individuals who were not affected by the policy intervention
Conditionality	The conditions (for example work search activity) claimants must comply with in order to receive benefit payments
Inactive	People who are in receipt of inactive benefits such as Employment and Support Allowance (ESA) or in the UC “no work requirements” or “work focused interview” conditionality regimes. Several other benefits also fall into this category, though the numbers of people on these benefits is small.
Intention to treat	Cohorts of individuals based on the date they met the programme eligibility criteria, regardless of whether they went on to be referred to the programme
Other (labour market category)	People who do not fall into ‘employed’, ‘looking for work’ or ‘inactive’, this could include people who are in full-time education and not working or receiving benefits or those who are in custody.
Propensity score matching	A statistical technique in which individuals are identified as statistically similar to each other based on a set of characteristics
Regression	A statistical technique which estimates the extent to which changes in one or more variables are associated with changes in an outcome of interest
Looking For Work	The labour market regime in Universal Credit where claimants are expected to search for work and attend regular work search reviews with their work coach
Treatment group	The group of individuals affected by the policy intervention

Abbreviations

ATE	Average Treatment Effect
CI	Confidence Interval
CIA	Conditional Independence Assumption
CBA	Cost Benefit Analysis
CBR	Cost Benefit Ratio
DWP	Department for Work and Pensions
ESA	Employment and Support Allowance
HMRC	HM Revenue and Customs
ITT	Intention to Treat
JFAP	Job Finding Action Plan
JFS	Job Finding Support
JSA	Job Seekers Allowance
PAYE	Pay As You Earn
PSM	Propensity Score Matching
RAPID	Registration and Population Interaction Database
RTI	Real Time Information
SCBA	Social Cost Benefit Analysis
UC	Universal Credit

1. Introduction

1.1   Policy Background

The Job Finding Support programme was part of the Department for Work and Pension’s (DWP) “Plan for Jobs” package created in response to the Coronavirus pandemic.

The UK was facing unprecedented economic conditions, differing in its nature from past economic emergencies. Protecting public health meant closing many places of work, which significantly impacted the economy and labour market. In July 2020, the Government published the Plan for Jobs which contained measures aimed at getting people back into employment, keeping people in their jobs and creating new jobs. This included the introduction of the Job Finding Support programme to provide “quick-fire” support to jobseekers who had been claiming benefits for 13 weeks or less.

During the analysis period England entered the third national lockdown on 5 January 2021, and the lockdown was not eased until non-essential retail and outdoor hospitality was reopened on 12 April 2021 (similar arrangements were in place in Scotland and Wales).  This report makes no assessment as to whether the estimated programme impacts are generalisable to different contexts.

1.2 Aims and Scheme Design

The Job Finding Support programme aimed to provide “quick-fire” support to jobseekers who had been claiming benefits for 13 weeks or less, delivered remotely using digital communication channels. These jobseekers may not need significant help with their job search, but were thought to benefit from a short package of tailored support to help them understand current recruitment practices and sector specific approaches.

The offer consisted of a minimum of 4 hours flexible 1-2-1 digital support, and the opportunity to join at least one digital group session. Support included mock interviews, help to identify transferable skills and advice on how to switch industries, as well as group sessions to improve job search techniques. The support was intended to take place over a period of no more than 20 working days.

To be eligible, participants had to have the right to reside and work in the United Kingdom and be of working age, 18 (16 in Scotland and Wales) to State Pension Age.  The programme was targeted at those who were not in employment or on a zero-hours contract and who had been claiming benefits for 13 weeks or less, however exceptions were possible. 

Recruitment to the programme typically started with identification of eligible and suitable individuals by a work coach. The individual then had to declare a willingness to volunteer before being referred to a Job Finding Support provider, an external organisation who would deliver the support. An individual started on the programme by attending an initial meeting and completing necessary paperwork, such as a draft Job Finding Action Plan (JFAP). 

The programme ran between 1 January 2021 and 31 January 2022. 159,821 people were referred to the Job Finding Support programme, of which 49,981 (31%) went on to start. Reasons for not starting included the provider not being able to contact the individual during the initial contact period (up to 8 working days after referral), the individual not attending the initial meeting, or the individual dropping out after attending the initial meeting.

1.3 Purpose of the Analysis and Report Structure

The department has publicly committed to evaluating the impact of the scheme. The aims of evaluating the scheme are:

• measure the outcomes for the first cohort of participants – a sub-group of 9,857 individuals who were referred to the programme between 1 January 2021 and 1 April 2021, were in the “looking for work” category on their date of referral and went on to start the Job Finding Support programme

• quantify the impact of the scheme in terms of employment outcomes

• quantify the costs, benefits and value for money in a way that can be compared with other labour market schemes

• make an overall assessment of the effectiveness of the scheme in meeting its objectives

This analysis will use Propensity Score Matching (PSM) methodology to compare 2 similar groups of people, where the main difference is whether they participated in the Job Finding Support programme. Over time, PSM has been used for a range of DWP labour market interventions, from the Future Jobs fund to the Work Program and Kickstart. PSM is also the standard methodology used by DWP’s Employment Data Lab. This approach matches participants with non-participants based on their likelihood of participating in the Job Finding Support programme, where the likelihood is calculated based on a rich dataset of characteristics. This quasi-experimental methodology allows for matching people with similar characteristics and comparing average labour market outcomes to measure the impact of the scheme in isolation.

The plan for this report is as follows:

• section 2 describes the analytical approach covering potential sources of bias, limitations and cohort selection, as well group selection

• section 3 explains the impacts of the scheme and main sensitivity analyses

• section 4 presents the cost benefit analysis of the Job Finding Support programme

• section 5 concludes the findings of the impact evaluation, making an overall assessment of the impact of the scheme in meeting its objectives

2. Methodology and Sample Selection

2.1 Propensity Score Matching

The primary aim of this evaluation is to assess the impact of the Job Finding Support programme on the percentage of participants classed as employed one year after being referred. Propensity Score Matching (PSM) has been chosen as it best suits the features of this scheme.

The Job Finding Support programme is voluntary, which creates a self-selection issue. Participation in the programme may be influenced by personal factors that may affect overall outcomes. These demographics may be observable such as age or gender, but they could also be factors that are not easy to measure, such as motivation and enthusiasm, making it more difficult to isolate and control for this bias. Statistical methods such as PSM allow us to estimate the impact of the Job Finding Support programme in terms of the number of additional people who move into employment, while minimising this selection bias.

The starting point for PSM is to define an overall sample containing both participants and non-participants in the intervention under consideration. Once the sample has been defined, PSM is carried out as follows:

1. Data on the characteristics of individuals in the sample are used as the input to a logistic regression model, using a logit approach, to estimate the probability of each individual participating in the scheme. This probability is also known as an individual’s ‘propensity score’.

2. The propensity scores are then used to match participants to individuals in the comparison group with a similar likelihood of participating in the intervention. Here, the matching approach used was 100 nearest neighbours with replacement, meaning one non-participant could be matched to multiple participants. This involves running through each participant and matching them with the 100 closest eligible individuals from the comparison pool, determined by closeness of the propensity scores.

The logic of this is that by assigning reliable propensity scores, and then assessing people with similar scores across the treatment and comparison group, the groups should have similar characteristics overall, in terms of how likely they are to be treated. The only difference is the actual treatment effect itself. This methodology allows us to compare similar people across different groups and isolate the treatment effect to calculate it. PSM is commonly used for labour market programme evaluations, particularly voluntary ones such as the Job Finding Support programme.

2.2 Conditional Independence Assumption

For PSM to give an unbiased estimate of the treatment effect, there must be sufficiently rich data to ensure that the conditional independence assumption (CIA) is met. This states that the outcome must be independent of treatment assignment. This means that there are no differences between the matched treatment and matched comparison group which would affect outcomes. Therefore, if the Job Finding Support programme hadn’t existed, both the treatment and comparison groups would have had the same outcomes, as there would have been no treatment effect.

2.3 Data Sources

Data for Job Finding Support participants was collected throughout the programme, giving a detailed timeline of when somebody was referred, started, and completed the programme. A wider set of characteristics has been collected using DWP administrative datasets, particularly from UC datasets and the National Benefit Database for legacy benefits. These datasets are also used to collate benefit history variables used in the matching.

Data on earnings comes from the Real Time Information (RTI) data feed provided by HMRC. DWP receives a regular feed of RTI payslip data specifically for employment impact evaluations of UC claimants. Data on employment spells and outcomes come from the Registration and Population Interaction Database (RAPID), created by DWP using data held by DWP and HMRC. This is a longitudinal dataset holding data on the whole population.

2.4 Treatment Group Selection

The aim of the Job Finding Support programme impact evaluation is to compare the outcomes of those participating in the scheme, with what their outcomes would have been had they not participated – the “counterfactual” outcomes. The treatment group evaluated here is made up of the first cohort who participated in the programme – a sub-group of 9,857 individuals who were referred to the programme between 1 January 2021 and 1 April 2021, were in the “looking for work” category on their date of referral and went on to start the Job Finding Support programme. All participants were residents of England, Scotland or Wales. The definition of “inactive” may differ from other definitions found elsewhere.

Table 2.1: First cohort participant information

	Included in the primary analysis	JFS total
number of individuals	9,857	159,821
mean age (in years)	38.3	37.7
percentage who were male	53	58
percentage with dependent child flag	14	19
percentage with lone parent flag	7	11
percentage “employed” at start	17	26
percentage “looking for work” at start	100	94
percentage “inactive” at start	1	2
percentage “other” at start	0	2

For PSM to work effectively there must be no missing data that adds explanatory power to the propensity scores. To correct this, where information is missing, a dummy variable is derived and added to the list of matching variables.

To be eligible, participants had to have the right to reside and work in United Kingdom and be of working age, 18 (16 in Scotland and Wales) to State Pension Age.  The programme was targeted at those who were not in employment or on a zero-hours contract and who had been claiming benefits for 13 weeks or less, however exceptions were possible. 

Recruitment to the programme typically started with identification of eligible and suitable individuals by a work coach. The individual then had to declare a willingness to volunteer before being referred to a Job Finding Support provider, an external organisation who would deliver the support. An individual started on the programme by attending an initial meeting and completing necessary paperwork, such as a draft Job Finding Action Plan (JFAP). 

The programme ran between 1 January 2021 and 31 January 2022. 159,821 people were referred the Job Finding Support programme, of which 49,981 (31%) went on to start. Reasons for not starting included the provider not being able to contact the individual during the initial contact period (up to 8 working days after referral), the individual not attending the initial meeting, or the individual dropping out after attending the initial meeting.

2.5   Comparison Group Selection

2.5.1 Self-Selection Issue

The key question when constructing the comparison group is whether there is a self-selection issue for treatment. If participants who volunteered to be referred had non-observable characteristics that made them more likely to find work, this could bias the results upwards. The treatment group would have better characteristics (and therefore outcomes) than expected. Given these are non-observable traits such as motivation, this is challenging to control for in the methodology.

To control for this potential bias, Section 3 details a further analysis which explores the impact of the programme on everyone who volunteered to be referred, not just those who went on to start. This is arguably the most policy relevant analysis as it is the offering of treatment that is in the control of policy makers, more so than whether participants actively take up the treatment.

2.5.2 Comparison Group

The availability of data on all individuals who were referred to the Job Finding Support programme, as opposed to only those who went on to start, permits a fuller investigation of the programme. The schematic below, in Figure 2.1, shows 3 different groups of individuals associated with this programme.  Group A are the individuals who were referred and went on to start, Group B are those who were referred and who did not start, and group C represent those who were eligible but were not referred to Job Finding Support. 

Figure 2.1: Diagram showing the 3 different groups involved in the Job Finding Support analysis

Six different analyses comparing different combinations of these groups have been conducted, and the results, in terms of the impact on employment rates at 12 months, can be found in Section 3.

Run 1 shows the primary analysis in this report. This is generated by comparing individuals in group A to a matched comparison group selected from group C and shows the effect of the programme on those who started.

Run 2 combines groups A and B together and compares them to a different matched comparison group selected from group C.  This is the Intention to Treat (ITT) analysis which explores the impact of the programme on everyone who it was offered to, not just those who took it up.

Run 3 is a variation on run 1, but this time compares group A; those who started, to group B; those who were referred but did not start. This explicitly removes any selection bias associated with the referral process (as everyone included was referred), but it potentially introduces another form of bias associated with starting the programme having been referred. 

Run 4 compares those in group B (the referred non-starters) to a matched comparison group selected from group C. This looks at the impact (and/or bias) associated with being referred and then not starting the programme.

Runs 5 and 6 are variations of runs 3 and 4 respectively.  Instead of using all of group B (the referred non-starters), a subset of only those who did not start because the provider was unable to contact them was used.  This group is interesting as a) they did not have any interaction with the provider at all (e.g. no initial meeting) and b) they did not explicitly self-select themselves out of starting the programme in the way that the others in the referred non-starter group may have done.

2.6 Descriptive Statistics

2.6.1 Demographics

Table 2.2: Characteristics, benefit and employment information for the treatment group, comparison group, all starters and all referred non-starters. 

Variable	Cohort 1 Starters	Cohort 1 referred non-starters	All starters	All referred non- starters	JFS total
Observations	9,857	28,823	49,981	109,840	159,821
Age (mean years)	38.3	35.4	39.7	36.8	37.7
18 to 24 years (%)	19	25	13	17	16
25 to 34 years (%)	27	31	28	34	32
35 to 44 years (%)	18	18	21	21	21
45 to 54 years (%)	17	14	19	15	16
55 to 64 years (%)	16	11	18	12	14
65+ years (%)	0	0	0	0	0
Male (%)	53	61	53	60	58
RATW at start marker set (%)	1	1	2	3	3
Partner marker set (%)	7	7	8	9	8
Partner marker missing (%)	81	81	75	75	75
Dependent children marker (%)	14	14	19	19	19
Dependent – missing (%)	51	59	45	53	51
Lone parent marker set (%)	7	7	11	10	11
Lone parent marker missing (%)	53	61	47	56	53
DLA/PIP marker set (%)	1	2	2	2	2
DLA/PIP at start marker set (%)	1	1	2	2	2
Employed at start (%)	17	24	21	29	26
Looking for Work at start (%)	100	100	97	93	94
Inactive at start (%)	1	1	2	2	2
Other at start (%)	0	0	1	2	2
Number of weeks “Employed” in the previous 2 years	65	63	59	60	59
Number of weeks “Looking for Work” in the previous 2 years	16	19	20	24	23
Number of weeks “Inactive” in the previous 2 years	2	2	5	4	5
Number of weeks “Other” in the previous 2 years	26	27	26	25	25

2.7 Context and the impact of other interventions

The analysis in this report estimates the impact of the Job Finding Support programme in a real-life context where both the participants and the individuals in the comparison group are free to participate in other employment support programmes, either provided by DWP or external providers.  The analysis controls for participation in other DWP programmes in the run-up to referral to Job Finding Support but not after referral. The results in Table 2.3 show that those who started on Job Finding Support were 72% (12 percentage points) more likely than the comparison group (those who were not referred to Job Finding Support) to participate in another DWP programme in the 2 years following referral. It is possible that participation in these other programmes may account for some of the observed impacts presented below and should be considered when interpreting these results.

Table 2.3: the percentage of the participant and comparison groups who participated in another DWP employment support programme in the 2 years following referral

	Participant group (%)	Comparison group (%)	Impact central (ppt)	Impact lower (ppt)	Impact upper (ppt)	Significant yes/no
% who went on other DWP interventions within 2 years of referral	28.2	16.4	11.8	10.9	12.7	Yes

2.8 Matching Quality

As discussed in 2.1, the analysis in this report uses a technique called Propensity Score Matching (PSM) to construct a comparison group of individuals that are matched on key characteristics that are linked to a person’s participation in the Programme and the outcome variables of interest. 

The comparison pool was selected from DWP administration data and was restricted to only include individuals who were of working age and had an active UC, JSA or ESA spell during the period, between 1 January 2021 and 31 January 2022, when the Job Finding Support programme was accepting referrals. The next step was to assign a pseudo-referral date.  Since the programme was predominantly aimed at people who had only recently started on a benefit, the assignment of the pseudo-referral date was linked to the start date of the benefit spell as follows:

• Firstly, the referred group (restricted to only those on a “looking for work benefit”) were split into monthly cohorts based on the month in which the latest benefit spell started.  Spells which included transitions between “eligible” benefits without a gap of more than 7 days were considered to be single, coherent spells.

• The same was done for the comparison pool, based on spells that were active during the period of interest.  If there were multiple active spells one was selected at random.

• Stratified sampling was then used to ensure that the proportions of the comparison pool in each monthly cohort matched that of the referred group.

• Pseudo-referral months were then assigned to the comparison pool within each monthly cohort in a way that matched the distribution of start months for the referred group (within each cohort). A day within the month was then selected at random to complete the pseudo-referral date.

• Finally, the referred group was restricted to only individuals who were on a Looking for Work benefit at the time of their pseudo-referral date.

The pool was restricted to those who were referred between 1 January 2021 and 1 April 2021 to be consistent with the first cohort of the participant group.

The matching estimator used to generate the impact estimates presented in this report was nearest neighbour matching using 100 nearest neighbours and a bandwidth of 0.01. Nearest neighbour matching involves running through each participant and matching them with the closest eligible individuals from the comparison pool, determined by closeness of the propensity scores. Further information about matching estimators can be found in the Employment Data Lab’s methodology report and literature review documents.

Table A in Annex A shows a sample of the variables used in the matching process and the mean values of these variables both before and after matching. The table shows that before matching the participant and comparison groups are not well matched, or balanced, shown by sizeable differences in the mean values. After matching the mean values of the participant and comparison groups are much closer. The percent bias and p-value columns provide information on how big the residual difference is and if this difference is statistically significant. Ideally one would like the percent biases to be small (below 5%) and there to be no statistically significant differences i.e., p-values above 0.05 (the 95% confidence level threshold) – all percent biases are below 5%, and all p-values are greater than or equal to 0.05.

Several summary statistics were also used to assess the quality of the match. Rubin’s B and R summarise the covariate balance of the sample. B measures absolute difference in the mean propensity scores between the treatment and comparison group and should be under 25 to satisfy a balanced sample. R measures the ratio of treatment to comparison variances of the propensity scores and should be between 0.5 and 2 to satisfy a balanced sample. In the primary run, Rubin’s B and Rubin’s R were 6.45 and 0.92 respectively, the maximum percent bias of the control variables after matching was 2.89%. There were only 3 participants (0.03%) who were off support, a sufficiently small percentage so as not to raise concerns about the representativeness of the results.

3. Impacts

3.1   Outcome Measures

The primary outcome of the impact evaluation is the percentage of individuals classed as employed one year after being referred.

The aim of the Job Finding Support programme was to provide “quick-fire” support to short duration jobseekers to help them move into employment. This outcome is like those of other DWP employment schemes, given that most schemes have similar overall aims in line with departmental priorities to maximise employment.

We used RAPID data to find the percentage of participants who are employed one year after being referred to the programme (who went on to start the programme).

A range of secondary outcome measures were also analysed in this report, see Annex B for the complete table. These can be used to learn more about the impacts of the programme.

To estimate a cost benefit ratio, we need to know the Exchequer impact of lower benefit receipt because of the scheme. As with employment, we are tracking this at a monthly rate, as UC is paid monthly.

Outcomes are tracked up to 12 months from referral date. This is done by looking at mean impacts of the treatment and comparison group and calculating the difference.

3.2 Impact on labour market status

The results for those who started on Job Finding Support show that the programme led to:

More classed as Employed, at one year

• Between 7 and 9 percentage points more participants were classed as Employed one year after being referred to the programme, than had they not participated.

Fewer classed as Inactive, at one year

• Between 3 and 4 percentage points fewer participants were classed as Inactive one year after being referred to the programme than had they not participated.

Fewer classed as Other, at one year

• Between 2 and 4 percentage points fewer participants were classed as Other one and 2 years after being referred to the programme, than had they not participated.

The 3 above results are all statistically significant.

No significant change to those Looking for Work, at one year

• Between 0 and 2 percentage points more participants were classed as Looking for Work one year after being referred to the programme, than had they not participated. This result is not statistically significant

Table 2.4 shows the percentages of the participant and comparison groups in each labour market category at one year. The categories are defined as:

• Employed:  People who are either employed or self-employed

• Looking for Work:  People who are in receipt of Jobseekers Allowance (JSA), or in the Universal Credit (UC) “intensive work search”, “light touch out of work”, or “light touch in work” conditionality regimes. This also includes those in the “working enough” conditionality regime who are not in employment or self-employed.

• Inactive:  People who are in receipt of inactive benefits such as Employment and Support Allowance (ESA) or in the UC “no work requirements” or “work focused interview” conditionality regimes. Several other benefits also fall into this category, though the numbers of people on these benefits is small.

• Other:  People who do not fall into the above 3 categories, this could include people who are in full-time education and not working or receiving benefits or those who are in custody.

These categories are not mutually exclusive, and it is possible to be in more than one category. For example, someone working fewer than 16 hours a week may also be in receipt of JSA and would be classed as “employed” and “looking for work”. 

The difference between the matched participant and comparison group is estimated to be the impact of the programme. 

Table 2.4: Showing the percentage of each group in each category at one year after starting the programme.  The difference, or impact, is shown along with an indication of statistical significance.

Category	Participant group (%)	Comparison group (%)	Impact: Central (ppt)	Impact: Lower (ppt)	Impact: upper (ppt)	Significant yes/no
Employed	67	59	8	7	9	yes
Looking for Work	33	32	1	0	2	no
Inactive	4	7	-3	-4	-3	yes
Other	12	15	-3	-4	-2	yes

3.2.1 Labour market status over time

The plots in Figure 2.2 provide a graphical representation of how the programme impacted the percentage of participants in each labour market category over time.  Figure 2.2(b) shows that following a drop in the initial months, known as a “locking-in” period, the programme led to a statistically significant increase in the percentage of participants classed as employed one year after referral.  Figure 2.2(a) shows that one year after starting the programme the rates of employment are comparable to the rates during the period between one and 2 years prior to starting.  The comparison pool does not return to these rates during the one-year tracking period.

Figure 2.2: Plots showing the impact of the programme on the numbers in each labour market category over time. The ‘percentage in category’ plots (orange) show the percentages of the participant and comparison groups in each category. The ‘percentage point difference’ plots (blue) show the impact of the programme. The darker blue line shows the central estimate, and the shaded blue area is the 95% confidence interval.

2.2(a) Employed: One year after starting the programme, the rates of employment among participants are comparable to the rates one year prior to starting.

2.2(b) Employed: The impact plot shows that following a significant drop in employment in the initial months, known as a “locking-in” period, the programme led to statistically significant increase in the percentage of participants classed as employed at one year.

2.2(c) Looking for work: One year after starting the programme, the rates of looking for work are higher than the rates one year prior to starting for both the participant and comparison group.

2.2(d) Looking for work: The impact plot shows the programme resulted in a short sharp increase in the percentages classed as ‘looking for work’ that lasted for approximately 6 months before dropping to near zero for the second 6 months.

2.2(e) Inactive: The percentage inactive in the participant and comparison groups.

2.2(f) Inactive: The impact plot shows that the programme led to a statistically significant reduction in the percentage of participants classed as ‘inactive’, that was sustained over the one-year follow-up period.

2.2(g) Other: The percentage in the ‘other’ category in the participant and comparison groups.

2.2(h) Other: The impact plot shows that the programme led to a statistically significant and sustained reduction in the percentage of participants in the ‘other’ category.

3.3 Impact on weeks in employment

One year after referral, the average individual who started on Job Finding Support spent:   

• between 2 and 3 more weeks in the “Employed” category than if they had not been referred

• between one and 2 more weeks in the “Looking for Work” category than if they had not been referred

• one week less in the “Inactive” category than if they had not been referred

• one week less in the “Other” category than if they had not been referred

These results are all statistically significant.

Table 3.1 shows the impact of the programme on the average number of weeks spent in each labour market category one year after being referred.

The impact on durations in each category are modest. In the case of employment, the pronounced “locking in” period, shown in Figure 2.2(b) by an initial fall before the subsequent rise in employment impacts, reduces this measure.

Table 3.1: Shows the average number of weeks members of each group spent in each category one year after starting the programme.  The impact, or difference, is shown along with an indication of statistical significance.

Category	Participant group (weeks)	Comparison group (weeks)	Impact: Central (weeks)	Impact: Lower (weeks)	Impact: Upper (weeks)	Significant yes/no
Employed	28	26	2	2	3	yes
Looking for Work	31	30	2	1	2	yes
Inactive	1	2	-1	-1	-1	yes
Other	3	4	-1	-1	-1	yes

Note: The employed category includes those in low levels of work and receiving benefits such as Jobseekers Allowance (JSA) or Universal Credit (UC)

3.4 Impact on benefits and employment overlap

The “Employed” category includes those in low levels of work and receiving benefits such as JSA (Jobseeker’s Allowance) or UC (Universal Credit). The plot in Figure 3.1 shows the percentage of each group with an employment record alongside those who are employed and in receipt of “looking for work benefits”. It shows that a proportion of the increase in employment brought about by the programme includes people who are also in receipt of benefits. For more information on categories, see section 3.2.

Figure 3.1: Plot showing the percentages of the participant and comparison group in the Employed category and the Looking for Work and Employed

3.5 Intention to treat and sensitivity analysis

The availability of data on all individuals who were referred to the Job Finding Support programme, as opposed to only those who went on to start, permits a fuller investigation of the programme. The schematic below, in Figure 3.2, shows 3 different groups of individuals associated with this programme. Group A are the individuals who were referred and went on to start, Group B are those who were referred and who did not start, and Group C represent those who were eligible but were not referred to Job Finding Support. 

Figure 3.2: Diagram showing the 3 different groups involved in the Job Finding Support analysis

Six different analyses comparing different combinations of these groups have been conducted, and the results, in terms of the impact on employment rates at 12 months, can be found in Section 3.

Run 1 shows the primary analysis in this report. This is generated by comparing individuals in group A to a matched comparison group selected from group C and shows the effect of the programme on those who started.

Run 2 combines groups A and B together and compares them to a different matched comparison group selected from group C. This is the ITT analysis which explores the impact of the programme on everyone who it was offered to, not just those who took it up.

Run 3*is a variation on run 1, but this time compares group A; those who started, to group B; those who were referred but did not start. This explicitly removes any selection bias associated with the referral process (as everyone included was referred), but it potentially introduces another form of bias associated with starting the programme having been referred.

Run 4 compares those in group B (the referred non-starters) to a matched comparison group selected from group C. This looks at the impact (and/or bias) associated with being referred and then not starting the programme.

Runs 5 & 6 are variations of runs 3 and 4 respectively. Instead of using all of group B (the referred non-starters), a subset of only those who did not start because the provider was unable to contact them was used. This group is interesting as a) they did not have any interaction with the provider at all (e.g. no initial meeting) and b) they did not explicitly self-select themselves out of starting the programme in the way that the others in the referred non-starter group may have done.

Table 3.2: Showing the impact on employment rates at 12 months for 6 different scenarios.

Run	Run description	Impact central (ppt)	Impact lower (ppt)	Impact upper (ppt)
1	Group A vs C (primary analysis)	7.95	6.97	8.93
2	Group (A+B) vs C (ITT)	3.42	2.87	3.98
3	Group A vs B	5.50	4.37	6.62
4	Group B vs C	1.95	1.32	2.58
5	Group A vs “No Contact” subgroup of B	7.63	6.26	9.00
6	“No Contact” subgroup of B vs C	-0.01	-0.87	0.85

Run 2 shows the results of the ITT analysis which explores the impact of the programme on everyone who it was offered to, not just those who took it up. This is arguably the most policy relevant analysis as it is the offering of treatment that is in the control of policy makers, more so than whether participants actively take up the treatment.

The fact that the result of run 4 is not zero, and that the result of run 3 is different to run 1 may indicate that either: there is an impact or bias associated with being referred and not starting, or that there could be some residual bias that the PSM is not fully accounting for. Runs 5 and 6 are helpful for exploring this further. By using the “no-contact” subgroup of B, some of the sources of potential bias are removed.  For instance, one source of bias in the referred-non-starter group may have come from individuals self-selecting themselves off the programme due to finding employment between being referred and starting. This would positively bias the outcomes of this group as they would be more likely to find work in the period after referral. By focusing on the “no contact” subgroup of group B, the possibility of this self-selection bias is removed, and so this is arguably a more appropriate comparison pool to use. Reassuringly when this subgroup is used the results are supportive of the primary analysis. The result of run 5 is closer to run 1, falling within the confidence interval, and run 6 is very close to zero suggesting little bias or impact associated with being referred but otherwise having no contact with the programme.

Note: it is still appropriate to use all of Group B for the ITT approach (run 2) because this approach investigates the impact of the programme on everyone it was offered to, irrespective of the degree of participation.

As part of the sensitivity analysis, we also ran a variation of the analysis using an additional matching variable, number of Jobcentre Plus (JCP) appointments in the 30 days before referral, and created the matched comparison group using this variable. This was to ensure that the participant group wasn’t attending more appointments than the comparison group, as additional support from JCP could be a source of some of the observed impact on employment outcomes. The estimated impact of the programme on employment as a result of this additional matching variable was not statistically significantly different to the results in the main analysis (run 1).

3.6 Analysis using Pay As You Earn Real Time Information

Pay As You Earn (PAYE) Real Time Information (RTI) is earnings data that is collected by HMRC via their PAYE systems. Some of this data is shared with DWP to support the administration of benefits. The experimental use of this data was explored in this analysis and is presented below. The methodology for using this data is still in development, and its use should be considered experimental, and the results used with caution.

To be used in this analysis the RTI data was calendarized and then directly linked with the individual data. No steps were taken to remove or address outlier payments, whether positive or negative. As earnings data tends to be highly skewed and not normally distributed, the data were transformed using the inverse hyperbolic sine transformation prior to inclusion in the matching.

The inclusion of RTI earnings as a matching variable made a negligible difference to the impact estimates on the other measures, but it does allow for the impact of the programme on earnings to be explored. Figure 3.3(a) shows the mean monthly earnings for the participant and comparison groups over time before and after referral to the programme. The impact, Figure 3.3(b), is very similar to that observed for employment status in Figure 2.2(b), clearly showing a pronounced positive impact of the programme on mean monthly earnings. What is less clear is if this increase is simply the result of increased employment rates, or if there is an impact on the amounts people earn for those who are employed. Whilst it is possible to calculate the mean monthly earnings for only individuals with non-zero earnings, doing this involves post-treatment selection which undermines a comparison between the participant and comparison groups. 

Figure 3.3(a): Average monthly earnings of the participant and comparison groups over time.

Figure 3.3(b) Impact of the programme on average monthly earnings over time, with 95% confidence interval shown by shaded blue area.

3.7 Cohort sub analyses

The analysis in this report is based on the first 3 months of Job Finding Support programme data (those who were referred between 1 January and 1 April 2021). This cohort was chosen for reasons of data availability to ensure that a least one year of employment follow-up data was available at the time of performing the analysis. Six-month follow-up data was available for a further 2 cohorts. The results of these cohorts (April to June and July to September 2021) were explored and are displayed below in Figure 3.4.

The figures show that each of the 3 cohorts have broadly similar trends both before and in the 6 months after programme start, with a few notable differences. The plots shows that the employment rates in the run up to programme start were higher for the first cohort than the latter 2 (67% for cohort 1 at 12 months before, compared to 59% and 54% for cohorts 2 and 3 respectively). The plots also show that the tail of those on looking for work benefits in the run up to programme start are longer for those in the later cohorts than for the first. The rise in this tail seems to coincide with the starting of the COVID 19 pandemic, highlighted in each plot by the blue shaded box.

Figure 3.4 showing 6-month outcomes data for 3 cohorts of starters on Job Finding Support. 3.4(a) shows the January to March 2021 cohort. This is the cohort used for the main analysis presented elsewhere in this report. 3.4(b) shows the April to June 2021 cohort and 3.4(c) shows the Jul-Sept cohort.

Figure 3.4(a) Jan-Mar 2021 Cohort

Figure 3.4(b) Apr-Jun 2021 cohort

Figure 3.4(c) Jul-Sep 2021 cohort

4. Cost Benefit Analysis

The Job Finding Support Programme cost benefit analysis (CBA) is based on earnings impacts set out in section 3.6. This section explains how these impacts are derived and are used in the department’s Social Cost Benefit Analysis (SCBA) model to consider costs and benefits as a result. This methodology is consistent with previous departmental impact evaluations, based on guidance from The Green Book (2022), and using the department’s SCBA model.

These costs and benefits apply to the first participant cohort – a sub-group of 9,857 individuals who were referred to the programme between 1 January 2021 and 1 April 2021, were in the “looking for work” category on their date of referral and went on to start the Job Finding Support programme.

This group were compared with a matched comparison group of customers who were not referred to the programme. It is important to note that due to the time this intervention took place (the participant cohort were referred between January and April 2021), the results may not be generalisable to other groups of Universal Credit customers at other points in time. The Job Finding Support programme was designed and implemented within the context of the COVID 19 pandemic. This report makes no assessment of the generalisability of these results to other contexts. Programme participants were free to participate in other employment support programmes and there is evidence to suggest that individuals who did participate were more likely to take up other DWP funded programmes.

4.1 Methodology

4.1.1 Average monthly earnings

The average monthly earnings over a year following referral to the programme (months 0 to 12 inclusive) were calculated using Real Time Information (RTI) earnings data for the participant and comparison groups. The comparison group of non-referred customers was matched to the participant cohort (see section 2 ‘Methodology and sample selection’ for more details on matching). The average UC amount received by the participant cohort upon referral to the programme was used as a proxy for the characteristics of the participants in the SCBA model. The programme cost per individual was £63. These were used as inputs to the model, to calculate the cost-benefit ratio to the programme participants, the exchequer and society.

4.1.2 Perspectives Under Consideration

The CBA will consider 4 key perspectives when calculating the costs and benefits of the scheme, summarised in Table 4.1:

• Job Finding Support participants
• Employers
• The Exchequer, in other words, the government budget perspective
• Society

The participant perspective focusses on individual costs and benefits, in particular changes in wages and benefit entitlement. The employer perspective focuses on the costs of paying participants and the benefits of the additional output produced by participants. The Exchequer perspective covers the fiscal elements of the policy, such as income and indirect taxes, National Insurance, and reduced benefit spending, as well as the cost of departmental spending on the Job Finding Support programme. The society perspective sums up the net impact from the other perspectives, reflecting the participant, employer and Exchequer perspectives in combination.

For the purposes of this analysis, ‘society’ represents an aggregate of all British citizens. Therefore, a cost or benefit to participants, their employers or the Exchequer can also represent a cost or benefit to society.

Table 4.1: Monetised costs and benefits of Job Finding Support

Job Finding Support (JFS) programme impact	Participant perspective	Employer perspective	Exchequer perspective	Society perspective
Increase in output	0	+	0	+
Increase in wages post JFS	+	-	0	0
Reduction in operational costs	0	0	+	+
Reduction in benefits post JFS	-	0	+	0
Increase in taxes	-	-	+	0
Increase in travel costs	-	0	0	-
JFS programme costs	0	0	-	-

Key: ‘+’ denotes a net benefit; ‘-’ denotes a net cost; ‘0’ denotes neither cost nor a benefit.

Once the costs and benefits have been calculated, a Cost Benefit Ratio (CBR) is produced reflecting the balance of the two. If the total is greater than £1, then for every pound spent on the Job Finding Support programme, more than one pound has been earned back in benefits.

4.1.3 Calculating Benefits

Increase in output

This refers to the economic output produced by participants because of additional earnings. This output represents a benefit to employers (who sell it) and society (who consume it). The DWP does not have information on the value of this output, so it is necessary to make several simplifying assumptions. The labour market is assumed to be perfectly competitive. This implies that employers will hire workers up to the point where the value of an additional unit of output is equal to the associated marginal cost of production. The cost of production, and therefore the value of the output produced during employment, is assumed to equal the commensurate gross wage payments and employers’ National Insurance contributions.

Increase in wages post Job Finding Support

This refers to the average gross wages received by participants in the 12 months following the programme. Wages represent a benefit to participants but a cost to their employers. This means they do not represent a net cost or benefit to society.

Reduction in operational costs

Job Finding Support participants are less likely to receive support from Jobcentre Plus advisers following the programme because they are more likely to be earning more and therefore more likely to flow off UC. This translates into operational savings which represent a benefit to the Exchequer and society, as economic resources can be reallocated to alternative uses. However, the comparison group was also earning in the 12 months following the programme, and therefore may have moved from the intensive search regime into a lower support regime. In order not to overstate the benefits of the programme, we decided to exclude operational costs from the ratios. This only impacted the high impact scenario, the other 2 scenarios had no reduction in operational costs.

Reduction in benefits post Job Finding Support

This refers to the net reduction in benefit entitlement that occurs when participants increase their earnings because of participation in Job Finding Support. This represents a benefit to the Exchequer and a cost to participants, but no net cost or benefit to society. Changes in benefit entitlement and take up are estimated using the DWP Policy Simulation Model. The DWP Policy Simulation Model is a microsimulation model which combines data from the Family Resources Survey with information on the UK tax and benefit systems. This allows users to estimate the changes in benefit payments and tax revenue that occur when unemployed individuals with a given set of characteristics move into work.

Increase in taxes

This refers to the increase in income tax, National Insurance (employer and employee) and indirect tax revenue that occurs when participants increase their earnings because of participation in Job Finding Support. This represents a benefit to the Exchequer and a cost to both employers and participants, but no net cost or benefit to society. Increases in tax revenue are estimated using the DWP Policy Simulation Model. To estimate increases in indirect tax revenue, Office for National Statistics estimates of indirect tax burdens were applied to estimates of participants’ disposable income obtained from the DWP Policy Simulation Model.

Increase in travel costs

This refers to the additional travel costs that are incurred by participants during additional employment because of participation in Job Finding Support. This also represents a cost to society as the provision of additional travel services diverts economic resources from alternative uses.

4.1.4 Calculating Costs

The calculation for CBAs uses a flat unit cost, the cost to deliver the Job Finding Support programme per participant. This is a net negative to the Exchequer by default. Employers and participants are not affected directly by this. This therefore becomes a cost for society as well. The cost of the Job Finding Support programme was published by the National Audit Office in their Employment support briefing.

4.1.5 Limitations of this approach

The CBA estimates presented in section 4.2 are subject to 2 main caveats. First, the accuracy of the estimates depends on the robustness of the impact estimates from which they are derived and the validity of the assumptions upon which they are based (see section 4.1.3). Uncertainty regarding these inputs has been partially mitigated by undertaking sensitivity analysis (see section 4.3).

Second, the CBA estimates exclude several potentially significant costs and benefits due to a lack of robust evidence. For a thorough discussion of the non-monetised costs and benefits of employment programmes, see Fujiwara (2010). For example, it has not been possible to obtain robust estimates relating to:

• the additional leisure time which participants forego (this represents a potential cost to participants and therefore society)

• the non-pecuniary benefits associated with additional time in unsubsidised employment (these represent a potential benefit to participants and therefore society)

• the cost of hiring and training incurred by employers (this represents a potential cost to employers and therefore society)

• the reduction in crime^{[footnote 1]} which may result from the programme (this represents a potential benefit to society)

• the economic multiplier effect which may result from the programme (this represents a potential benefit to society).

These non-monetised costs and benefits should be borne in mind when interpreting the CBA estimates presented in section 4.2.

4.2 Estimates

Table 4.2: Estimated Costs and Benefits of the Job Finding Support programme at 12 months following referral to the program, per participant, £ (unrounded) 2020/21 prices.

JFS programme impact (12 months)	Participant perspective	Employer perspective	Exchequer perspective	Society perspective
Increase in output	-	600	-	600
Increase in wages post JFS	600	-600	-	-
Reduction in operational costs	-	-	-	-
Reduction in UC post JFS	-378	-	378	-
Increase in taxes	-32	-	32	-
Increase in travel costs	-	-	-	-
JFS programme costs	-	-	-63	-63
Total Benefits	600	600	410	600
Total Costs	-410	-600	-63	-63
Net Benefit	190	0	347	537
Cost Benefit Ratio (CBR)	1.46	1.00	6.51	9.52

4.3 Sensitivity Analysis

To test the robustness of these results, sensitivity tests have been conducted to test different assumptions used in the modelling.

High and low impact scenarios

Along with average monthly earnings over 12 months following referral to the programme, an upper and lower estimate was also calculated. This uses the standard error for the earnings for the control and treatment groups. To calculate this, we add and subtract the standard error x 1.96 from the monthly earnings over 12 months, and then take an average.

These can be used to produce higher and lower impact scenarios, in addition to the central scenario.

• Comparing the lower bound average earnings for the control group with the upper bound average earnings for the participant group gives the higher impact scenario (a larger difference between earnings in the control and participant group).

• Comparing the upper bound average earnings for the control group with the lower bound average earnings for the participant group gives the lower impact scenario (a smaller difference between earnings in the control and participant group).

These average earnings figures are shown in Table 4.3, and the lower, central and higher impact estimates are shown in Table 4.4.

Table 4.3: average earning figures

Control (lower, central, upper)	Average earnings	Participant (lower, central, upper)	Average earnings
Control (lower)	£633.37	Participant (lower)	£670.73
Control (central)	£640.36	Participant (central)	£690.86
Control (upper)	£647.34	Participant (upper)	£710.98

Table 4.4: Lower, central and higher earnings impacts and cost benefit ratios

	Lower impact	Central impact	Higher impact
Average earnings impact	£23.29	£50.50	£77.61
Participants ratio	1.46	1.46	1.59
Employer ratio	1	1	1
Exchequer ratio	3.25	6.51	8.98
Society ratio	4.76	9.52	14.29

Figure 4.1: Lower, central and higher impact cost benefit ratios

The lower impact scenario reduces the Exchequer and Society ratios and the higher impact scenario increases the Exchequer and Society ratios. This shows that the ratios are sensitive to the average monthly earnings, although in all 3 scenarios the ratios show good value for money. The ratios are particularly sensitive due to the low programme cost of £63 per person.

Persistence of impacts

The costs and benefits of the Job Finding Support programme for the first cohort have been estimated at 12 months post-referral. If the impacts persist beyond this then our earnings estimate will be an underestimate, therefore our benefits will also be an underestimate. Any benefits of the programme to participants beyond 12 months after referral are not reflected in the impact estimates or cost benefit analysis.

5. Conclusions

5.1 Impact Analysis

The analysis uses an average treatment effect (ATE) approach to compare outcomes between the first cohort of Job Finding Support programme participants and a matched comparison group of similar people who were not referred to the programme. This methodology allows us to compare similar groups of people directly against each other. The analysis uses many characteristic variables to control for different demographic traits that could influence labour market outcomes outside of the Job Finding Support programme. The aim of this analysis is to assess the direct impact of the programme on the percentage of individuals classed as employed one year after being referred.

Job Finding Support programme led to an increase in the percentage of people classed as employed one year after being referred to the programme of between 7 and 9 percentage points.

The intention to treat (ITT) analysis compared all referred individuals (whether they started or not) with a different matched comparison group who were not referred. This explores the impact of the programme on everyone who it was offered to, not just those who took it up. This found an increase in the percentage of people classed as employed one year after being referred to the programme of between 3 and 4 percentage points. This shows that there was a positive impact of offering the programme to people, even if they did not start.

We also compared participants with those who were referred but did not start. This found an increase in the percentage of people classed as employed one year after being referred to the programme of between 4 and 7 percentage points.

Several sensitivity tests were performed to check the validity of these results. The impacts appear to be consistently strong across different combinations of the groups who were referred or not referred, and who started or didn’t start. This gives confidence that the central results are robust.

The fact that we saw different result when comparing participants with not referred, vs participants with referred who did not start indicates that there could be bias associated with being referred and not starting. This was explored further by comparing participants with a subgroup of the ‘referred but didn’t start’ who not only did not start but had no contact with the programme following referral. This removes the possibility of selection bias from work coach referral to the Job Finding Support programme. This gives an increase of between 6 and 9 percentage points, which is supportive of the primary analysis.

5.2 Cost Benefit Analysis

Using the estimated impacts (average earnings), a Cost Benefit Analysis was performed to estimate the cost effectiveness of the scheme over the 12 months following the first cohort of participants’ referral to the Job Finding Support programme.

The programme was examined from the participant, employer and Exchequer perspectives, as each would value costs and benefits differently. These perspectives were then summed together to get an overall society perspective that reflects all 3. At the 12-month mark, for every pound invested:

• participants benefit by £1.46
• employers benefit by £1.00
• The Exchequer benefits by £6.51
• society benefits by £9.52

The Job Finding Support programme provides good value for money – this is largely due to the very low programme cost per person (a one-off £63), meaning the programme only needs to result in a small increase in earnings to outweigh the costs. There was a difference in average monthly earnings of £50 between the control and participant groups. The participant group would only have had to earn £6 more per month than the control to break-even from a societal perspective (giving a return of £1.14), and £8 more per month to break-even from the Exchequer perspective (giving a return of £1.04). However, such a low difference would likely not have been statistically significant.

Sensitivity analysis on the participant, Exchequer and society perspectives show that these results do varying across higher and lower impact average earnings scenarios. However, even with this variation, the programme provides good value for money from all perspectives, particularly Exchequer and society.

An alternative way to look at the cost benefit of the programme is the net benefit. As the unit cost per participant is small, the net benefit shows how much money per participant the programme is generating after costs (Table 4.2). If applied to the initial cohort who participated in the programme, around 10,000 individuals, this means at the 12-month mark, the net benefit is:

• participants benefit by £1.9million
• employers – zero net benefit
• The Exchequer benefits by £3.5million
• society benefits by £5.4million

It is important to note that due to the time this intervention took place (the participant cohort were referred between January and April 2021), the results may not be generalisable to other groups of UC customers at other points in time. The Job Finding Support programme was designed and implemented within the context of the COVID 19 pandemic. This Cost Benefit Analysis makes no assessment of the generalisability of these results to other contexts.

It is important to reiterate that the accuracy of the cost and benefit estimates is very much dependent on the robustness of the impact estimates from which they are derived and the validity of the assumptions upon which they are based. It should also be borne in mind that several potentially significant costs and benefits have been excluded from this analysis due to a lack of robust evidence. These include non-pecuniary benefits from the Job Finding Support programme such as improved motivation, or the costs of future training schemes. Job Finding Support participants were free to participate in other employment support programmes and there is evidence to suggest that individuals who did participate were more likely to take up other DWP funded programmes.

References

Fujiwara D. ‘The DWP Social Cost-Benefit Analysis framework (WP86)’ Department for Work and Pensions working paper 86, 2010.

HM Treasury, ‘The Green Book: appraisal and evaluation in central government’, 2022

Annex A: Mean values of matching variables before and after matching

Table A: Showing mean value of each control variable, for each group, before and after matching. The residual percent bias and p-value (after matching) are also shown.

Variable	Unmatched comparison group	Unmatched participant group	Matched comparison group	Matched participant group	percent bias	p value
l_week_m1	91.9	89.8	90.7	89.8	-2.89	0.05
o_week_m1	3.4	6.5	5.9	6.5	2.74	0.08
int_start	9.7	13.1	13.9	13.2	-2.26	0.14
binary_jsahist	18.5	33	33.9	32.9	-2.16	0.17
intervention_month	1.6	1.8	1.7	1.8	1.73	0.24
binary_uc_ltiwhist	31.7	19	18.4	19	1.41	0.27
l_month_m_1	61.6	59.7	60.4	59.7	-1.33	0.35
dur_m_o_q7	3.8	3.3	3.3	3.3	-1.31	0.34
o_month_m_15	29	25.3	25.8	25.3	-1.29	0.35
dur_m_o_q6	3.8	3.2	3.3	3.2	-1.27	0.36
dur_m_o_q4	3.6	3.2	3.3	3.2	-1.26	0.37
o_month_m_21	30.4	25.8	26.4	25.8	-1.26	0.37
w_month_m_3	55.5	51.7	51.1	51.7	1.25	0.38
dur_m_o_q8	4	3.4	3.5	3.4	-1.25	0.37
o_month_m_18	29.5	25.1	25.6	25.1	-1.24	0.37
dur_m_o_q5	3.7	3.2	3.3	3.2	-1.22	0.38
w_month_m_9	62.5	64.6	64.1	64.7	1.22	0.39
dur_m_w_q2	7.7	7.6	7.6	7.6	1.21	0.39
dur_m_w_q3	8.1	8.3	8.2	8.3	1.2	0.4
dur_m_l_q1	7.4	7.2	7.2	7.2	-1.2	0.4
o_month_m_12	29.4	26.1	26.6	26.1	-1.19	0.4
o_month_m_24	32.3	27.2	27.7	27.2	-1.19	0.39
dur_m_w_q4	8.3	8.6	8.6	8.6	1.16	0.41
w_month_m_15	65.4	68.7	68.2	68.7	1.15	0.41
f_children	22.4	14.3	13.9	14.3	1.14	0.38
dur_m_w_q7	8.4	8.9	8.8	8.9	1.12	0.42
dur_m_w_q6	8.5	9	8.9	9	1.11	0.43
dur_m_w_q8	8.2	8.7	8.6	8.7	1.11	0.43
dur_m_o_q2	3.4	3.5	3.5	3.5	-1.1	0.44
w_month_m_24	61.8	66.2	65.6	66.2	1.09	0.44
w_month_m_21	63.9	67.7	67.2	67.7	1.08	0.44
o_month_m_3	27.3	30.2	30.7	30.2	-1.08	0.46
w_month_m_6	60.9	61.6	61.1	61.6	1.07	0.45
w_month_m_18	64.8	68.6	68.1	68.6	1.04	0.46
missing_phone_flag	15.8	17.6	18	17.6	-1.04	0.48
f_miss_children	53.6	51.2	50.7	51.3	1.02	0.48
dur_m_w_q5	8.5	9	8.9	9	1.01	0.47
o_month_m_9	26.3	24.6	25	24.5	-0.99	0.48
f_age	37.4	38.3	38.2	38.3	0.99	0.49
f_loneparent	7.8	7.4	7.1	7.4	0.99	0.48
dur_m_o_q3	3.3	3.1	3.2	3.1	-0.98	0.49
f_miss_loneparent	56.6	52.8	52.4	52.9	0.96	0.5
o_month_m_6	25.9	25	25.4	25	-0.93	0.51
dur_m_w_q1	6.4	4.7	4.7	4.7	0.93	0.49
w_month_m_12	64.5	67.2	66.8	67.2	0.92	0.51
f_miss_partner	71.4	81.5	81.9	81.5	-0.92	0.48
f_sex	54.6	53.2	53.7	53.2	-0.9	0.53
w_month_m_2	51.2	40.6	40.2	40.6	0.86	0.54
f_age_sq	1585.8	1659.8	1651	1659.7	0.79	0.59
f_partner	14.7	6.8	6.6	6.8	0.7	0.55
uer_year_m_2	4	4.3	4.3	4.3	0.66	0.64
w_month_m_1	46.3	29.5	29.2	29.5	0.62	0.65
w_week_m1	42.3	18.9	18.7	18.9	0.6	0.63
uer_year_m_1	4.7	5	4.9	5	0.59	0.67
binary_uc_eehist	20.3	15.9	15.7	15.9	0.58	0.67
uer_year	4.5	4.8	4.8	4.8	0.57	0.68
w_start	41.7	17	16.8	17	0.55	0.65
i_start	1	0.6	0.7	0.6	-0.52	0.69
f_dwp_ethnicity_not_given	23.8	20.2	20	20.2	0.51	0.71
p_eo_year_m_2	9.9	10.3	10.3	10.3	0.51	0.7
l_month_m_12	6.6	6.8	6.7	6.8	0.5	0.72
binary_wtchist	10.4	8.6	8.4	8.6	0.49	0.72
binary_ctchist	10	8	7.9	8	0.47	0.73
f_dwp_ethnicity_white	4.4	5.3	5.2	5.3	0.46	0.76
binary_hbhist	10.1	7.8	7.7	7.8	0.45	0.74
binary_cbhist	15.3	13	12.8	13	0.45	0.75
binary_uc_wfihist	0.3	0.3	0.3	0.3	-0.43	0.76
i_week_m1	1.1	0.8	0.8	0.8	-0.41	0.75
l_month_m_2	39.4	38.5	38.7	38.5	-0.4	0.78
o_month_m_1	23.5	27.8	28	27.8	-0.4	0.78
o_month_m_2	23.5	27.8	28	27.8	-0.4	0.78
dur_m_l_q5	0.8	0.8	0.8	0.8	0.4	0.78
binary_icahist	2.3	1.9	1.9	1.9	-0.38	0.78
l_month_m_15	5.4	5.9	5.8	5.9	0.37	0.8
f_dwp_ethnicity_mixed	1	1.2	1.2	1.2	-0.36	0.81
dur_m_l_q4	1.6	1.3	1.3	1.3	0.35	0.8
binary_bbhist	x	x	x	x	0.34	0.81
f_dwp_ethnicity_missing	28.9	25.5	25.6	25.5	-0.33	0.81
binary_esahist	1.9	0.9	0.9	0.9	-0.32	0.78
binary_sahist	15.9	5.1	5	5.1	0.32	0.76
p_po_year_m_1	21.7	22.4	22.4	22.4	0.31	0.81
p_eo_year	9.2	9.5	9.5	9.5	0.31	0.82
l_month_m_18	5.4	6.1	6.1	6.1	0.29	0.84
dur_m_l_q6	0.7	0.8	0.8	0.8	0.27	0.85
f_dwp_ethnicity_other	1.6	1.7	1.7	1.7	-0.25	0.86
p_po_year	22.6	23.4	23.3	23.4	0.25	0.85
f_dwp_ethnicity_british	34.1	37.6	37.7	37.6	-0.24	0.87
i_month_m_24	3.1	2.7	2.7	2.7	0.23	0.86
dur_m_l_q7	0.7	0.8	0.8	0.8	0.23	0.88
dur_m_i_q8	0.4	0.4	0.3	0.4	0.22	0.87
missing_internetusers	1.8	0.8	0.8	0.8	0.21	0.85
missing_p_st_year_m_1	5.7	2.9	2.9	2.9	-0.21	0.86
binary_ishist	1.9	2.2	2.2	2.2	0.21	0.89
int_hist	2.6	2	2.1	2	-0.21	0.88
dur_m_o_q1	2.1	2.6	2.6	2.6	0.21	0.89
i_month_m_9	2.5	2.5	2.4	2.5	0.21	0.88
p_eo_year_m_1	9	9.4	9.4	9.4	0.2	0.88
f_dwp_ethnicity_chinese	0.1	0.1	0.1	0.1	-0.2	0.89
i_month_m_21	2.9	2.6	2.6	2.6	0.2	0.89
l_month_m_24	4.8	6.1	6	6.1	0.2	0.9
p_po_year_m_2	20.3	20.9	20.8	20.9	0.19	0.89
i_month_m_12	2.3	2.4	2.4	2.4	0.18	0.9
dur_m_i_q4	0.3	0.3	0.3	0.3	0.17	0.9
p_nvq3p_year_m_2	55.2	56.2	56.2	56.2	-0.17	0.89
p_nvq4p_year_m_2	38.3	38.8	38.9	38.8	-0.16	0.9
binary_emphist	72.7	81.9	81.8	81.9	0.16	0.91
dur_m_i_q6	0.3	0.3	0.3	0.3	0.16	0.91
missing_p_eo_year	5.8	2.7	2.7	2.7	-0.15	0.9
phone_flag	63.4	79.2	79.1	79.2	0.15	0.91
dur_m_l_q2	3	2.5	2.5	2.5	0.15	0.92
dur_m_i_q7	0.4	0.3	0.3	0.3	0.15	0.92
p_st_year	8	8.3	8.3	8.3	-0.15	0.91
p_nvq2p_year_m_2	71.1	72.7	72.7	72.7	-0.13	0.91
p_nvq1p_year_m_2	80.5	82.4	82.4	82.4	-0.13	0.91
dur_m_l_q8	0.7	0.8	0.8	0.8	0.13	0.93
dur_m_i_q5	0.3	0.3	0.3	0.3	0.12	0.93
p_st_year_m_2	9.3	9.6	9.6	9.6	-0.12	0.93
binary_dlapiphist	3.9	1.4	1.4	1.4	-0.12	0.91
f_dwp_ethnicity_black	2.8	4.1	4.1	4.1	0.12	0.94
l_month_m_6	21.1	16.9	16.8	16.9	0.11	0.93
internetusers	90.5	91.5	91.5	91.5	-0.11	0.92
l_month_m_3	26.2	22.6	22.5	22.6	0.11	0.94
binary_ibhist	0.1	x	x	x	x	x
i_month_m_15	2.5	2.5	2.5	2.5	0.1	0.95
missing_ear_year	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_po_year	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_nvq4p_year	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_nvq3p_year	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_nvq2p_year	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_nvq1p_year	5.4	2.5	2.5	2.5	-0.1	0.93
missing_ear_year_m_1	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_po_year_m_1	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_eo_year_m_1	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_nvq4p_year_m_1	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_nvq3p_year_m_1	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_nvq2p_year_m_1	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_nvq1p_year_m_1	5.4	2.5	2.5	2.5	-0.1	0.93
missing_ear_year_m_2	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_po_year_m_2	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_st_year_m_2	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_eo_year_m_2	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_nvq4p_year_m_2	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_nvq3p_year_m_2	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_nvq2p_year_m_2	5.4	2.5	2.5	2.5	-0.1	0.93
missing_p_nvq1p_year_m_2	5.4	2.5	2.5	2.5	-0.1	0.93
binary_uc_iwshist	64.6	73.6	73.6	73.6	0.09	0.94
binary_wbhist	x	x	x	x	x	x
missing_uer_year_m_1	5.4	2.5	2.5	2.5	-0.09	0.94
missing_uer_year_m_2	5.4	2.5	2.5	2.5	-0.09	0.94
i_month_m_6	2.2	2.3	2.2	2.3	0.09	0.95
dur_m_i_q1	0.2	0.2	0.2	0.2	-0.09	0.95
missing_uer_year	5.4	2.5	2.5	2.5	-0.09	0.94
missing_p_st_year	5.4	2.5	2.5	2.5	-0.09	0.94
binary_bsphist	0.3	0.2	0.2	0.2	-0.09	0.94
p_nvq2p_year	73.7	75.6	75.6	75.6	-0.08	0.94
p_nvq1p_year	82.4	84.7	84.7	84.7	-0.08	0.94
i_month_m_18	2.7	2.5	2.5	2.5	0.08	0.95
f_ratw_start	7.5	1.6	1.5	1.6	0.08	0.93
dur_m_i_q3	0.3	0.3	0.3	0.3	0.08	0.96
dur_m_i_q2	0.3	0.3	0.3	0.3	0.08	0.96
l_month_m_9	18.4	13.6	13.6	13.6	0.07	0.96
dlapip_start	3.3	1	1	1	-0.06	0.95
i_month_m_3	1.8	2	2	2	0.05	0.97
snc_hst_flg	x	x	x	x	x	x
ear_year_m_2	74.3	76.2	76.3	76.2	-0.05	0.97
i_month_m_1	1.5	1.3	1.3	1.3	-0.05	0.97
dur_m_l_q3	2.7	2	2	2	-0.04	0.97
i_month_m_2	1.7	1.8	1.8	1.8	0.04	0.98
binary_uc_wprephist	0.8	0.4	0.4	0.4	0.04	0.97
p_nvq4p_year_m_1	41.1	42	42	42	-0.04	0.98
ear_year_m_1	74.6	76.7	76.7	76.7	-0.03	0.98
p_nvq3p_year	58.2	59.7	59.7	59.7	-0.03	0.98
f_dwp_ethnicity_asian	3.2	4.4	4.3	4.4	0.03	0.99
binary_uc_ltoowhist	6.9	2.3	2.3	2.3	-0.02	0.98
l_month_m_21	5.2	6.1	6.1	6.1	0.01	0.99
p_st_year_m_1	8.4	8.7	8.7	8.7	-0.01	0.99
p_nvq2p_year_m_1	73.6	75.5	75.5	75.5	-0.01	1
ear_year	74	76.1	76.1	76.1	0.01	1
p_nvq3p_year_m_1	58	59.4	59.4	59.4	0.01	1
p_nvq1p_year_m_1	82.6	84.9	84.9	84.9	0.01	1
p_nvq4p_year	41.5	42.5	42.5	42.5	0.01	1
intervention_year	2021	2021	2021	2021	0.01	1
binary_pibhist	x	x	x	x	x	x
binary_sdahist	x	x	x	x	x	x
binary_uc_misshist	x	x	x	x	x	x
wp_hist	x	x	x	x	x	x
wp_start	x	x	x	x	x	x
prap_ref_hist	x	x	x	x	x	x
l_start	100	100	100	100	0.01	1
o_start	0.0	0.0	0.0	0.0	0.01	1

• The definition of the matching variables can be found in the Employment Data Lab methodology document.

• Some figures have been suppressed for disclosure control purposes.

Annex B: Full table of results

Table B Showing the full list of generated results

Outcome measure	Participant group	Comparison group	Impact central	Impact lower	Impact upper	p-value
no. weeks at 1 year - employed	27.8	25.6	2.1	1.7	2.5	0.00
no. weeks at 1 year – looking for work	31.3	29.6	1.7	1.4	2.0	0.00
no. weeks at 1 year - inactive	1.1	2.4	-1.3	-1.4	-1.1	0.00
no. weeks at 1 year - other	3.3	4.3	-1.0	-1.2	-0.8	0.00
% at 6 months - employed	59.5	53.3	6.2	5.2	7.2	0.00
% at 1 year - employed	66.8	58.9	8.0	7.0	8.9	0.00
% at 6 months - looking for work	51.0	50.1	0.9	-0.2	1.9	0.10
% at 1 year - looking for work	33.0	32.2	0.8	-0.2	1.7	0.12
% at 6 months - inactive	2.1	4.8	-2.6	-3.0	-2.3	0.00
% at 1 year - inactive	4.0	7.4	-3.3	-3.8	-2.9	0.00
% at 6 months - other	9.0	10.0	-1.0	-1.5	-0.4	0.00
% at 1 year - other	11.6	14.5	-2.9	-3.6	-2.3	0.00
% at 6 months in work only	37.9	35.3	2.6	1.6	3.6	0.00
% at 1 year in work only	51.5	46.0	5.5	4.4	6.5	0.00
% at 6 months in looking for work only	29.7	32.6	-3.0	-3.9	-2.0	0.00
% at 1 year in looking for work only	18.3	20.3	-2.0	-2.8	-1.2	0.00
% at 6 months in inactive only	1.7	3.9	-2.2	-2.5	-1.9	0.00
% at 1 year in inactive only	3.2	6.2	-3.0	-3.4	-2.7	0.00
% at 6 months in other only	9.0	10.0	-1.0	-1.5	-0.4	0.00
% at 1 year in other only	11.6	14.5	-2.9	-3.6	-2.3	0.00
% at 6 months in looking for work and work	21.2	17.3	3.9	3.1	4.8	0.00
% at 1 year in looking for work and work	14.6	11.8	2.8	2.1	3.5	0.00
% at 6 months in inactive and work	0.3	0.7	-0.4	-0.5	-0.3	0.00
% at 1 year in inactive and work	0.8	1.1	-0.3	-0.5	-0.1	0.00
% on other DWP intervention within 2 years of referral	28.2	16.4	11.8	10.9	12.7	0.00

1. Fujiwara (2010) presents evidence of a causal relationship between individuals’ income levels and their propensities to commit acquisitive crime. However, the voluntary nature of Job Finding Support participation means that this relationship cannot be used to obtain robust estimates of the programme’s effects on crime levels. ↩