Research and analysis

Cost benefit analysis technical report

Updated 17 May 2022

Applies to England

DWP research report no. 990

A research report carried out by a research consortium led by ICF on behalf of Department for Work and Pensions /Department of Health and Social Care (Employers, Health and Inclusive Employment directorate).

Crown copyright 2021.

You may re-use this information (not including logos) free of charge in any format or medium, under the terms of the Open Government Licence

Or write to:

Information Policy Team
The National Archives
Kew
London
TW9 4DU

Email: psi@nationalarchives.gov.uk

This document/publication is also available on our GOV.UK website

If you would like to know more about DWP research, email: socialresearch@dwp.gov.uk

First published July 2021.

ISBN: 978-1-78659-263-7

Views expressed in this report are not necessarily those of the Department for Work and Pensions or any other government department.

Acknowledgements

This study was commissioned by the joint Department for Work and Pensions and Department of Health and Social Care Work and Health Unit. We are particularly grateful to Pontus Ljungberg, Sian Moley, Lyndon Clews, Sarah Honeywell, Caroline Floyd, Rachel Shanahan, David Johnson, Mark Langdon, Anna Bee, Craig Lindsay and members of the DWP Policy Psychology Division for their guidance and support throughout the study.

Dr Adam P. Coutts would like to thank the Health Foundation for the three-year fellowship (Grant ID – 1273834) which enabled him to conduct the research. Many thanks to Liz Cairncross at the Health Foundation who provided support and advice throughout the fellowship.

We would also like to thank the Jobcentre Plus staff, Group Leaders, provider representatives and individual benefit claimants who gave their time to participate in the fieldwork.

Views expressed in this report are not necessarily those of the Department for Work and Pensions, Department of Health and Social Care, or any other government department.

Author’s credits

This report was prepared by Matt Rayment of ICF Consulting Services Ltd, Dr Susan Purdon of Bryson and Caroline Bryson of Purdon Social Research, and Professor Stephen McKay of the University of Lincoln.

Glossary of terms

Term	Definition
Active Labour Market Policy	Active Labour Market Policies (ALMPs) aim to increase the employment opportunities for job seekers and improve matching between jobs (vacancies) and workers (i.e. the unemployed). In so doing ALMPs may contribute to reducing unemployment and benefit receipt via increased rates of employment and economic growth.
Active learning techniques	Active learning techniques are based on actively involving participants in a learning activity rather than just requiring them to passively listen.
Benefit: Cost Ratio (BCR)	Ratio of estimated value of benefits compared to costs. If the BCR exceeds 1, this indicates that £1 of expenditure returns more than £1 of benefit. The DWP Social Cost Benefit Model estimates BCRs for society, employers, individuals and the Exchequer.
Caseness	A person is described as having suggested case level anxiety or depression if their scores on the Generalised Anxiety Disorder (GAD-7) and Patient Health Questionnaire (PHQ-9) scales suggests they would exceed the ‘caseness thresholds’ used by Improved Access to Psychological Therapies. Diagnoses of anxiety or depression respectively would be based on a clinical interview and would take account of additional evidence, to which the GAD or PHQ scores may contribute.
Cost Benefit Analysis	A cost benefit analysis (CBA) examines all the costs and benefits of the intervention and quantifies them in monetary terms as far as possible, in order to examine the balance of costs and benefits.
Employment and Support Allowance	Employment and Support Allowance (ESA) is a benefit for people who have an illness, health condition or disability that affects how much they can work. ESA offers financial support if people are unable to work, and personalised help so that people can work if they are able to.
Financial strain	Financial strain refers to when an individual’s financial outgoings start to exceed their income to a degree that psychologically threatens their sense of self, identity, relationships and/or self-esteem.
General self-efficacy	General self-efficacy is the strength of an individual’s belief that they are effective in handling life situations.
Group Leader	Group Leaders are the individuals who delivered the Group Work course, using active learning techniques, to participants.
Group Work	Group Work is a job search course designed to also enhance self-efficacy, self-esteem and social assertiveness among those looking for paid work. It aims to prevent the potential negative mental health effects of unemployment and help unemployed people back into work. The course is the application of the JOBS II model, originally developed by the University of Michigan, in the UK labour market.
Impact on Participants	Impact on Participants (IoP) refers to the analysis of the impact of an intervention based on comparing outcomes for individuals who participated in the intervention with a matched comparison group of individuals who did not.
Income Support	Income Support (IS) is an income-related benefit for people who have no income or are on a low income, and who cannot actively seek work. It is mainly for people who cannot seek work due to childcare responsibilities.
Initial Reception Meeting	All Group Work participants were invited to an Initial Reception Meeting (IRM) which preceded the course itself. The IRM was designed as an opportunity for participants to meet the Group Leaders who would deliver their course and learn more about what it would involve.
Intention to Treat	Intention to Treat (ITT) refers to the analysis of the impact of an intervention based on comparing outcomes for all individuals who were offered the opportunity to participate in the intervention with a control group of individuals who were not offered this opportunity.
Jobcentre Plus	Jobcentre Plus (JCP) is an organisation under which the Department for Work and Pensions offers working-age support services, such as employment advisory services. In the context of this report, ‘jobcentre’ refers to the physical premises in which Jobcentre Plus services are offered.
Job search self-efficacy	Job-search self-efficacy is the strength of an individual’s belief that they have the skills to undertake a range of job-search tasks.
JOBS II	JOBS II is the course originally designed by the University of Michigan, and the Group Work course is the application of JOBS II in the UK
Jobseeker’s Allowance	Jobseeker’s Allowance (JSA) is an unemployment benefit for people who are actively looking for work.
Latent and Manifest Benefits	Latent and Manifest Benefits (LAMB) are material and psychosocial benefits associated with being in work such as social interaction, social support, activity, identity, collective purpose, self-worth (Latent benefits) and the absence or lessening of financial strain (Manifest).
Learning and Development Officers	Individuals responsible for delivering training was provided to Work Coaches at the participating Jobcentre Plus offices.
Mental Health Issues	Mental Health Issue is a broad term that includes those who have: deteriorating mental health (for example, related to the experience of unemployment); elevated but not clinical levels of a symptom; mental health conditions; or are post-treatment; have symptoms but may not recognise they have a condition; or are aware of their condition/ situation but choose not to disclose. Many individuals with Mental Health Issues are found to struggle with their job search.
Psychosocial	Psychosocial indicators concern psychological and social factors that can influence health and wellbeing outcomes. Typical examples of such indicators include social support, employment status, job quality, poverty and marital status.
Self-efficacy	Self-efficacy is the strength of an individual’s belief that they have the skills to undertake a task and achieve an outcome.
Sensitivity analysis	An analysis to examine how the results could be affected by uncertainty regarding the inputs and assumptions of the model. A sensitivity analysis for this study examined the effects on the Cost Benefit Analysis results of possible changes in assumptions regarding the estimation of costs and benefits of Group Work.
Single Point of Contact	Single Points of Contact (SPoCs) were the designated point of contact in each of the Jobcentre Plus districts in which Group Work was trialled, involved in monitoring volumes, training and delivery.
Social Cost Benefit Analysis	Social Cost Benefit Analysis (SCBA) measures and weighs various impacts of a project or policy, comparing project costs (capital and operating expenses) with a broad range of social impacts.
Social Discount Rate	Discounting is used to compare costs and benefits occurring over different periods of time on a consistent basis, and is applied to all future costs and benefits to convert them to a present value, since generally people prefer to receive goods and services now rather than later. The HM Treasury Green Book specifies a social discount rate of 3.5 per cent.
Statistically significant	A statistic derived from a study, such as the difference between two groups, is said to be statistically significant if the size of that statistic has only a low probability of arising by chance alone. The probability of a statistic of that size occurring by chance alone is termed the ‘p-value’. By convention, if the p-value is less than 0.05 then it is stated that the statistic is ‘significant’.
Trial Integrity and Support Officers	Trial Integrity and Support Officers were designated DWP staff responsible for monitoring and supporting the fidelity of the DWP input to the Group Work trial.
Universal Credit	Universal Credit (UC) is an in and out of work benefit designed to support people with their living costs. Most new claims by people with a health condition or disability are now made to UC.
Well-being	Wellbeing is an individual’s self-report as to whether they feel they have meaning and purpose in their life, and includes their emotions (happiness and anxiety) during a particular period.
Work Coach	Work Coaches are frontline Jobcentre Plus staff based in jobcentres. Their role is to support benefit claimants into work through work-focused interviews.
Work and Health Unit	The Work and Health Unit (WHU) is a joint unit between the Department for Work and Pensions and Department of Health and Social Care. It leads on the Government’s strategy to support working-age disabled people or those with long-term conditions, to access and retain good quality employment.

Executive summary

ICF, in partnership with IFF Research, Bryson Purdon Social Research, Professor Steve McKay of the University of Lincoln and Dr Clara Mukuria of the University of Sheffield and Dr Adam Coutts at the University of Cambridge, were commissioned by the Department for Work and Pensions (DWP) to undertake a programme of research to evaluate a trial of the Group Work intervention in England.

This report presents a cost-benefit analysis (CBA) conducted as part of this research.

Group Work is a 20-hour job search skills workshop comprising five four-hour sessions delivered over the course of one working week designed to enhance participants’ self-efficacy, self-esteem and social assertiveness. The course is a UK version of the JOBS II programme that was originally developed in the United States by the University of Michigan and since been trialled in a number of countries.

A Randomised Control Trial (RCT) was conducted to test the effectiveness of the JOBS II intervention in a UK labour market context, targeting benefit claimants struggling with their job search and/or feeling low or anxious and lacking in confidence about aspects of their job search. The trial operated in five Jobcentre Plus districts between January 2017 and March 2018.

Costs of delivering Group Work

The total costs of delivering Group Work were estimated at £3.3 million, of which £1.7 million was spent on delivering the intervention, and £1.6 million on running the trial and evaluation. After deducting the costs of the evaluation elements, the costs averaged £656 per participant beginning the course and £886 per participant completing it.

Benefits of Group Work

Benefits of Group Work were expected to include:

• Improvements in participants’ mental health and wellbeing as a direct result of participating in Group Work; and
• Benefits to participants, the Exchequer and the economy where participation in Group Work leads to employment outcomes.

A central assumption of the CBA was that the intervention delivered benefits only if these were found to be statistically significant at the five per cent level. A higher bound estimate was also included in the CBA by assigning a value to benefits (positive employment outcomes for participants in comparison with the control group) even where these differences were not found to be statistically significant. This report draws upon data from the Impact on Participants (IoP) analysis (described in detail in Chapter 6 of the Technical Report on the Impacts of the Trial), which compared the outcomes of course participants with those from a matched comparison group. The IoP analysis isolates the impact on individuals participating in the course, rather than an Intention to Treat (ITT) analysis which measures the impact on all those offered the course (so including both participants and those who declined the Group Work offer).

Employment Benefits

A slightly higher proportion of Group Work participants were in work six months and 12 months after completing the course compared to members of the comparison group (20 per cent compared to 18 per cent at six months, and 23 per cent compared to 20 per cent at 12 months). However, these differences were not statistically significant.

An upper estimate of the benefits was made by valuing the observed differences in employment between Group Work participants and the comparison group, assuming that these are real differences even though they are not statistically significant.

Analysis of the administrative data found no significant difference in the value of out-of-work benefits paid to participants compared to the control group at either six or 12 months.

Benefits to Mental Health and Wellbeing

The IoP analysis detected some statistically significant impacts on participants’ mental health and wellbeing at six and/or 12 months after their participation in the course. These included a reduced likelihood of depression as measured on the WHO-5 scale and increased wellbeing on some of the ONS4 wellbeing measures. However, there is currently no evidence to enable the value of improvements in these measures to be quantified robustly in monetary terms. No statistically significant impacts were detected on the two health indicators for which it is possible to quantify benefits robustly in monetary terms: health service usage and overall health status.

The impact evaluation also detected some effects on participant confidence in finding work and general and job-search self-efficacy which could suggest that the course had moved them closer to the labour market, but these effects were stronger at six months than 12 months. These effects could not be valued in monetary terms.

Comparison of Costs and Benefits

CBA Model and Method

The estimated costs and employment effects of Group Work were input into the DWP Social Cost Benefit Analysis (SCBA) model in order to assess the overall balance of costs and benefits for all participants in the course. The model provides an analysis of costs and benefits at four levels: society; participants; participants’ employers; and the Exchequer. The CBA included a sensitivity analysis, to test the sensitivity of the findings to the assumptions and methods used to estimate costs and benefits.

The assessment was conducted at three levels:

• Main assessment - this applies the central estimates of costs and benefits, assuming no benefit for employment (on the basis that no statistically significant impact was found);
• Positive employment effect - this assessment applies the central estimate of costs but assumes a positive effect on employment (i.e. applying estimates of increased numbers of days in work, even though these were not statistically significant); and
• Positive employment effect, lower cost estimate - this assessment assumes a positive effect on employment and a reduced estimate of costs from the sensitivity analysis. It therefore presents a best-case scenario for both benefits and costs.

In each case the analysis focused on the intervention costs of Group Work and excluded the trial-related costs.

Overall CBA Findings

The analysis found that, for society as a whole, the costs of delivering Group Work outweighed the value of the monetised benefits, even employing best case assumptions for employment effects and costs. The societal benefit cost ratio ranged from 0 (main assessment) to 0.67 (assuming a positive employment effect and lower cost estimate) - in other words, for every pound spent the return was between 0 and £0.67. However, this analysis does not reflect the statistically significant benefits to mental health and wellbeing (as well as aspects of confidence and self-efficacy) detected in the impact evaluation, which could not be valued in monetary terms.

Subgroup CBA

Although the impact evaluation did not detect statistically significant employment benefits for the total population of Group Work participants, these were detected for two subgroups:

• Those with suggested case level anxiety^{[footnote 1]} (on the GAD-7 scale) at baseline; and
• Those with low general self-efficacy^{[footnote 2]} at baseline.

No significant employment effect was found for a third sub-group – those with higher levels of depression (on the PHQ-9 scale) at baseline.

The CBA assessed the benefits and costs for the two subgroups exhibiting statistically significant impacts on employment. The monetised benefits were found to exceed the intervention costs for both groups, with a benefit cost ratio of 1.76 for the subgroup with suggested case level anxiety at baseline, and 1.39 for the subgroup with low general self-efficacy at baseline (central estimates) – in other words, for every pound spent the return was £1.76 and £1.39 respectively. The sensitivity analysis estimated that benefits would exceed costs even if the assumed duration of employment effects was reduced and a higher estimate of programme costs used. Again, the significant effects on aspects of mental health and wellbeing, confidence and self-efficacy could not be valued in monetary terms so are not reflected in these benefit:cost ratios. This suggests that the intervention could deliver net benefits to society if targeted at these groups. This finding relies on the (strong) assumption that the costs of delivery to these groups can be held at the average intervention cost of Group Work, and that similar levels of benefits can be maintained.

1. Introduction

1.1. Group Work

Group Work is a 20-hour job search skills workshop comprising five four-hour sessions delivered over the course of a working week intended to enhance participant’s self-efficacy, self-esteem and social assertiveness. The Group Work course was the application of the JOBS II programme originally developed in United States by the University of Michigan and has been trialled in a number of countries. It has been adapted for the UK labour market by DWP policy psychologists. Further details of the course are provided in the Process Evaluation Technical Report.

Between January 2017 and March 2018, a Randomised Control Trial (RCT)^{[footnote 3]} was conducted to test and evaluate the effectiveness of the JOBS II intervention in a UK labour market context, targeting claimants of Jobseeker’s Allowance, Employment Support Allowance, Universal Credit Full Service and Income Support (Lone Parents with child(ren) aged three and over) who were struggling with their job search or feeling low or anxious and lacking in confidence in relation to their job search. The trial operated in five Jobcentre Plus districts. Work Coaches were responsible for recognising potential beneficiaries, who were then either randomised into the treatment group and offered the opportunity to go on the course or into a control group. The course was attended by 2,596 participants and completed by 1,922.

Group Work is one of several interventions being trialled by the Department for Work and Pensions (DWP) and Department of Health and Social Care (DHSC) Joint Work and Health Unit to build a strong evidence base on what interventions work best to help those with health issues move into or retain work.

Prior to its introduction, a report by RAND Europe (Van Stolk et al., 2014) recommended trialling JOBS II in the UK labour market and estimated that JOBS II would incur costs of around £875 per participant and could deliver one positive employment outcome per 4.48 participants, resulting in a potential benefit cost ratio of 1.07 (based on employment outcomes alone).

1.2. This report

ICF, in partnership with IFF Research, Bryson Purdon Social Research, Professor Steve McKay of the University of Lincoln, Dr Clara Mukuria of the University of Sheffield and Dr Adam Coutts of the University of Cambridge, were commissioned by the joint Department for Work and Pensions and Department of Health and Social Care Work and Health Unit in January 2017 to undertake a programme of research to evaluate the Group Work trial.

This report provides technical information and findings from one element of this research: the cost benefit analysis. The wider programme of research also included an impact evaluation which measured the impact of Group Work after six and 12 months, and a process evaluation focussing on the set-up and running of the trial and the perceptions of course participants and those declining to participate of Group Work. Details of these two strands, and their detailed findings, are presented in the Technical Reports on the Impacts of the Trial and the Process Evaluation Technical Report, with the combined findings from all three elements of the research being presented in the Evaluation Synthesis Report.

The evaluation of the Group Work trial aimed to identify ‘what works’ to improve employment and health outcomes for people who are out of work and struggling with their job search, with the primary research questions including:

• Did Group Work improve benefit claimants’ employment rates and wellbeing?
• For whom is this support most effective and why?
• Is the support cost effective?

The cost benefit analysis (CBA) aimed to address the third point and provide an assessment of the cost effectiveness of the Group Work course in order to inform decision making going forward.

The CBA examines all the costs and benefits of the intervention and quantifies them in monetary terms as far as possible, in order to examine the balance of costs and benefits. It brings together an analysis of the costs of the intervention, undertaken by ICF in conjunction with DWP, and an assessment of the benefits. The analysis of benefits builds on the evaluation of impacts undertaken and reported in detail in the Technical Report on the Impacts of the Trial (Purdon and Bryson, 2020) and values these in monetary terms where possible and appropriate.

The CBA was guided by a framework developed by ICF and agreed with DWP. The CBA report is structured as follows:

• Chapter 2 presents the cost benefit framework and methodology, including the range of costs and benefits of Group Work and the methods used in the analysis;
• Chapter 3 summarises the costs of the Group Work;
• Chapter 4 summarises the benefits of Group Work and values these as far as possible;
• Chapters 5 and 6 compare the costs and benefits of Group Work overall, and for specific sub-groups, respectively; and
• Chapter 7 presents overall conclusions from the CBA. The report also includes an appendix which sets out the assumptions used in the CBA.

2. Cost Benefit Analysis Framework and Methodology

The Cost Benefits Analysis (CBA) framework was used to plan and structure the work, identifying the range of potential costs and benefits that needed to be considered, and the approach to assessing and valuing them.

2.1 Costs and Benefits of Group Work

The main costs of the intervention were incurred by the Department for Work and Pensions (DWP) and include:

• Expenditure on contracted services delivered by the two providers running the course, and by the ICF-led research team;
• Staffing and administrative costs incurred by DWP in delivering the intervention, including project and contract management, training, support, expert advice and oversight, and monitoring; and
• The costs of time spent by Jobcentre Plus in selecting and referring participants.

The benefits of the intervention potentially included those to:

• Participants – including the net monetary benefits to those helped to find work, direct benefits to mental health resulting from attendance at the sessions, and indirect benefits to health and wellbeing resulting from changes in employment status;
• The Exchequer – including savings in benefit payments and associated administrative costs, increases in tax receipts, and reductions in other public expenditures such as the costs of healthcare; and
• Employers and the wider economy, through enhanced productivity.

2.2. Quantifying Beneficiaries and Impacts

The impact evaluation analysed the impacts of Group Work using an Intention to Treat (ITT) and an Impact on Participants (IoP) approach. The ITT approach compared outcomes for all those offered the opportunity to go on the course to the outcomes for the control group, while the IoP approach compared the outcomes of those actually participating in the course with those of a matched comparison group. Since 22 per cent of those offered the opportunity to participate in Group Work went on to take the course, the ability to detect an impact based on the ITT analysis was greatly reduced. As there were benefits only for those who participated in the course, it was logical to estimate costs and benefits per participant and to scale these up across the number of course participants. The CBA therefore examined the benefits of Group Work for participants, as measured by the IoP analysis, rather than the results of the ITT analysis. Chapters 5 and 6 of the impact evaluation report provide further details of the evaluation of impacts in the trial population and on participants respectively.

2.3. Valuing and Assessing Costs and Benefits

The CBA aimed to cover and assess all the costs and benefits of the intervention, and to value them as far as possible in monetary terms.

2.3.1. Quantifying the costs of Group Work

The analysis of costs identified the range of financial and human resources deployed to deliver Group Work. These included:

• Financial expenditures – including fees paid to contractors involved in delivering and evaluating Group Work; and
• Human resources – the time and cost of DWP and Jobcentre Plus staff involved in managing, delivering and evaluating the intervention. Financial expenditures were quantified in records kept by DWP. Staffing costs were estimated by:
• Quantifying inputs by different grades of DWP and Jobcentre Plus staff in different functions (including project management, training, advice, contract management);
• Valuing direct staff time using relevant pay rates (£/day); and
• Adding an allowance for expenses and overheads, in order to quantify the full costs of staff time.

The cost analysis distinguished between those resources that were spent in delivering the Group Work intervention (“intervention costs”) and those which were spent on evaluating Group Work (“trial costs”). This distinction recognised that most of the costs of trialling Group Work were one-off costs which would not be repeated if the intervention were to be delivered on an ongoing basis. These trial costs related to the selection of the comparison group and the management and implementation of the evaluation. The trial costs have been excluded when examining the cost-effectiveness of Group Work as an intervention.

The analysis estimated the total costs of delivering Group Work and the costs per participant.

2.3.2. Assessing and valuing the benefits of Group Work

The analysis examined the range of benefits of Group Work, quantifying and valuing them as far as possible, while recognising the potential value and significance of those that could not be quantified in monetary terms.

Not all potential benefits can be monetised. For example, some benefits for individual mental health and wellbeing have value, but evidence which enables their value to be quantified robustly in monetary terms is lacking.

Development of the CBA framework included a literature review to identify those benefits that can be valued in monetary terms. These include:

• Employment – the benefits to society, individuals and the Exchequer of increased hours worked and resultant earnings/output can be valued using the DWP Social Cost Benefit Analysis (SCBA) model (DWP, 2017);
• Benefit payments – savings in benefit payments to Group Work participants can be measured through DWP administrative data and are estimated through the SCBA model;
• Health service usage – savings to the NHS through reduced healthcare usage (GP visits and use of casualty and outpatient services) can be valued using appropriate unit costs, such as those in the DWP SCBA model or obtained from databases such as the Greater Manchester Combined Authority New Economy model database (GMCA, 2019); and
• Overall health status – any benefits to the overall health of Group Work participants, as measured through the EQ-5D scale (Euroqual Group, 1990)^{[footnote 4]}, can be translated into measures of quality adjusted life years (QALYs) which can be valued in monetary terms.

No monetary values could be found that enabled a robust valuation of other indicators used in the impact evaluation. These included measures of: job search outcomes (such as self-efficacy, confidence in finding work and job search activity); ONS subjective wellbeing; loneliness (UCLA Loneliness score); mental wellbeing and the likelihood of depression (measured using WHO-5 index); mental disorders, notably depression, using the PHQ-9 (Patient Health Questionnaire); anxiety, using the GAD-7 (General Anxiety Disorder) scale; and latent and manifest benefits from employment, using the LAMB scale.

The impact evaluation found that Group Work delivered statistically significant impacts on some of these measures of mental health and wellbeing for which monetary valuation evidence is lacking, without impacting on overall health (as measured by EQ-5D) or use of health services. Some of the benefits of Group Work could not therefore be valued in the CBA.

Benefits per participant were estimated by assessing differences in outcomes between the treatment and comparison group and multiplying these by an appropriate unit value per outcome. Benefits can then be aggregated across the number of participants.

Where benefits (such as those on mental health, wellbeing, general confidence and general and job search self-efficacy) could not be valued, they were assessed in qualitative terms (see Section 4.2).

2.3.3. Social, Exchequer and individual CBA

The DWP SCBA framework distinguishes between the benefits and costs of interventions from different perspectives:

• Society;
• Intervention participants;
• Employers of intervention participants; and
• The Exchequer.

Society level CBA assesses the overall benefits and costs for society as a whole, including benefits from enhanced output and reduced healthcare costs, less the costs of the intervention and additional employment related costs such as travel and childcare. Social Security benefits payments are treated as a transfer payment so are not included.

Intervention participants CBA examines net changes in income and expenditure for individuals, including income from employment and benefits, and costs of travel and healthcare.

CBA for employers of intervention participants examines changes in output and expenditure on wages and national insurance.

Exchequer level CBA examines net changes in income/expenditure for the Exchequer, including increases in taxes, changes in benefit payments, and costs related to the intervention.

2.3.4. Statistical significance of impacts

The impact evaluation measures differences in outcomes between Group Work participants and the comparison group and assesses the statistical significance of any differences observed at the five per cent level (with a p-value of less than five per cent being conventionally taken to indicate a statistically significant difference. For further details, please refer to the Technical Report on the Impacts of the Trial, Section 2.5).

A central assumption of the CBA was that the intervention delivered benefits only if these were found to be statistically significant. However, the impact evaluation found some positive differences in employment outcomes between the participant and comparison groups which were not significant at the five per cent level. The value of these differences was assessed as an “upper-bound” estimate of benefits in the sensitivity analysis – see section 2.3.7 for more detail.

2.3.5. Duration of impacts

Data was collected from Group Work participants and comparison group members at randomisation and baseline, and via a survey at two points in time – six months and twelve months after completion of the course – in order to assess outcomes. Assessment of the benefits of Group Work depends on the duration of any outcomes observed. This required assumptions to be made about how long these outcomes would last.

International evidence on JOBS II has detected impacts at up to two years after the intervention (for example, Vinokour, 2002; Vuori and Silvonen, 2005, see Chapter 2 of the Process Evaluation Technical Report for further detail).

Two alternative assumptions were applied in assessing benefits:

1. Benefits were assumed to last for eighteen months following the intervention (i.e. a further six months after the impacts found at 12 months); or

2. Benefits were assumed to last for two years following the intervention (i.e. a further year after the impacts found at 12 months).

Results based on the second assumption are presented in the analysis which follows in this report, as this is broadly consistent with the international evidence.

2.3.6. Analysis of Costs and Benefits

The CBA combined estimates of the value of the costs and benefits of Group Work annually over the years 2016/17 to 2019/20. The time profile of costs and benefits was estimated over this period. Costs were assigned to one of three phases (see Table 2.1).

Table 2.1: Profile of Group Work costs

Phase	Time period	Duration	Tasks
Design	August 2015 – December 2016	17 months	Design of intervention, trial and evaluation
Live Running	January 2017 – April 2018	16 months	Expenditure on delivery by contractors; DWP time on training, advice, monitoring, support, evaluation, contract management; Jobcentre Plus time on training and referrals; contracted evaluation work
Follow up	May 2018 – March 2020	23 months	Evaluation by DWP staff and contractors

Benefits were assumed to occur up to two years after completion of the course, i.e. between January 2017 and April 2020. The median start date was September 2017, when the 1,444th participant started the course. Evidence from the impact assessment of the effects observed at six months and 12 months was used to estimate the time profile of benefits over this period, estimating the changes in employment over time and the total change in each financial year.

The DWP SCBA model (DWP, 2017) was used to estimate the value of employment outcomes and related costs. While the SCBA model also estimates changes in healthcare costs, based on employment outcomes, these were considered separately using data from the impact evaluation. Since the survey of participants included questions about their actual use of health services, this approach was expected to deliver more reliable estimates than applying the assumptions in the SCBA model.

The analysis assumed no substitution effects (i.e. that any increase in employment by participants did not occur at the expense of others losing their jobs). This is potentially a strong assumption, although the size of any substitution effect would be difficult to estimate.

The present value of these costs and benefits was calculated using the 3.5 per cent social discount rate specified in the HM Treasury (HMT) Green Book (HMT, 2018) and applied by the DWP SCBA model. This enabled estimation of the net benefits and benefit: cost ratios for the intervention at each level (society, participants, employers and Exchequer).

2.3.7. Sensitivity analysis

A sensitivity analysis was included, examining how any assumptions or estimates employed affected the overall balance of costs and benefits. As well as the main CBA, providing the most robust estimates of costs and benefits, alternative estimates were made assuming lower levels of costs and higher levels of benefits.

The sensitivity analysis examined the effect on cost and benefit estimates of:

• Changes in assumptions regarding costs (including staff costs and levels of staff time, where there are uncertainties about these); and
• Inclusion of estimates of the benefits of employment outcomes where these are not statistically significant.

The sensitivity analysis assessed the effect of changes in these assumptions individually and collectively on overall cost and benefit estimates and benefit: cost ratios.

2.3.8. Sub-group Cost Benefit Analysis

As well as examining costs and benefits for participants overall, the analysis also provided an assessment for two sub-groups of individuals for which statistically significant effects on employment were found:

a) Those with suggested case level^{[footnote 5]} anxiety (as measured by the GAD-7 scale) at baseline; and

b) Those with low general self-efficacy at baseline (as measured using the General Self Efficacy (GSE) scale).

The benefits for these sub-groups were taken from the impact evaluation, while the average costs per participant were assumed to be similar to those for Group Work participants overall (in the absence of information to suggest otherwise).

No statistically significant effect on employment was found for a third sub-group, those with suggested case level depression (using the PHQ-9 scale) at baseline, so this sub-group was not analysed in the CBA.

3. The Costs of Group Work

3.1. Actual Costs

The costs incurred by Group Work are estimated to total £3.3 million (shown in Table 3.1 below), in line with the projections made in the Business case. Just over 50 per cent of these total costs are estimated to have been incurred in the delivery of Group Work, and the remainder on trial specific and evaluation elements.

Table 3.1: Estimated costs of delivering Group Work

Cost item	Details	Cost estimates Intervention- specific[footnote 6]	Trial- specific[footnote 7]	Total cost
Staffing costs: DWP and Jobcentre Plus staff	Including policy staff, Trial integrity and support officers, Psychological Services staff, Jobcentre Plus staff, evaluation specialists and contract management services	£660,280	£713,352	£1,373,632
Contractors: Intervention contract	Payment to two contractors for delivering intervention	£1,042,000	0	£1,042,000
Contractors: Evaluation contract	Payment to evaluators	0	£896,670	£896,670
Total contractor costs	-	£1,042,000	£896,670	£1,938,670
Total costs	-	£1,702,280	£1,610,022	£3,312,301

Table 3.1 includes all known costs related to developing, delivering and trialling Group Work, but excludes policy development costs not directly related to running the project.

The staff costs include pensions, National Insurance and overheads including estate costs, training, expenses and information technology.

All costs are assumed to be additional costs relative to a Business as Usual scenario involving no Group Work intervention. It is assumed that delivery of Group Work did not affect the other services delivered to participants and the comparison group by Jobcentre Plus.

The intervention was delivered over the period January 2017 to April 2018 and most costs relate to this period; however, some costs (e.g. evaluation) were incurred after this period. An estimate of the time profile of costs between each financial year from 2015/16 to 2019/20 is given in Table 3.2.

Table 3.2: Estimated time profile of costs, 2015/16 to 2019/20

Year	Intervention costs	Trial specific costs	Total costs
2015/16	£19,864	£57,047	£76,910
2016/17	£232,028	£123,239	£355,267
2017/18	£1,243,190	£610,473	£1,853,663
2018/19	£207,198	£427,890	£635,088
2019/20	£0	£391,373	£391,373
Total	£1,702,280	£1,610,022	£3,312,301

Further details of the assumptions employed in the cost assessment are given in Appendix 1.

The analysis that follows in this report focuses on the intervention costs and excludes the trial-specific costs, since only the intervention costs are directly related to the delivery of the project and would be incurred if the project were to continue in the future.

3.2. Unit costs per Participant

In total 2,596 participants began the Group Work course, and 1,922 completed it. Estimated unit costs are given in Table 3.3.

Table 3.3: Estimated costs of Group Work per participant

Total costs of Group Work	Number	Cost
Intervention costs (excluding trial-specific costs)	0	£1,702,280
Number of participants: Starting course	2,596	0
Completing course	1,922	0
Average cost per participant starting course: Average intervention cost	0	£656
Average cost per participant completing course: Average intervention cost	0	£886

If trial-specific costs are excluded, and only the costs of administering and delivering the intervention are considered, the costs of the project average £656 per participant starting the course, and £886 per participant completing the course.

3.3. Sensitivity Analysis

A sensitivity analysis was undertaken to assess the sensitivity of the cost estimates to key assumptions.

The assumptions tested were:

• The effect of excluding overheads from estimates of staff costs (i.e. including only salaries, pensions and National Insurance);
• The effect of the assumed allocation of time spent by certain DWP staff (Trial Integrity and Support Officers, responsible for monitoring the fidelity of the trial) between the intervention and trial-specific elements of Group Work. The main cost estimate assumed a 50:50 split for staff in this role, but an alternative estimate was that 10 per cent of time was related to the intervention and 90 per cent to the trial element of Group Work. These variations reflect the varying assessments of DWP staff involved in the project;
• The effect of reducing the estimate of the time spent by Jobcentre Plus Work Coaches per Group Work referral by one third from the best estimate of 24 minutes to 16 minutes. The 24-minute estimate is the mean estimate made by Work Coaches interviewed as part of the qualitative research for the evaluation. Since several interviewees gave lower estimates, a lower figure of 16 minutes was also included in the sensitivity analysis;
• The effect of increasing the estimate of the time spent by Work Coaches per Group Work referral by one third from the best estimate of 24 minutes to 32 minutes was also tested.

The effect on overall costs of varying these three assumptions is shown in Table 3.4.

Table 3.4: Sensitivity of costs of Group Work to changing assumptions

Assumption	Intervention specific costs	Trial specific costs	Total costs
A. Central estimate of costs	£1,702,280	£1,610,022	£3,312,301
B. Excluding overheads from staffing costs	£1,647,654	£1,515,189	£3,162,843
% change in costs	-3.2%	-5.9%	-4.5%
C. Assume only 10% of Trial Integrity and Support Officer time spent on intervention	£1,637,897	£1,674,404	£3,312,301
% change in costs	-3.8%	4.0%	0.0%
D. Assume Jobcentre Plus time per referral of only 16 minutes	£1,672,736	£1,559,245	£3,231,981
% change in costs	-1.7%	-3.2%	-2.4%
E. Combined effect of B + C + D	£1,567,378	£1,524,361	£3,091,740
% change in costs	-7.9%	-5.3%	-6.7%
F. Assume Jobcentre Plus time per referral of 32 minutes	£1,731,823	£1,660,798	£3,392,621
% change in costs	1.7%	3.2%	2.4%

Of these assumptions, the assumption that only 10 per cent of Trial Integrity and Support Officer time was spent on intervention has the greatest effect on estimated intervention costs, reducing them by £64,000, or 3.8 per cent.

The combined effect of changing the three sets of assumptions B, C and D would be to reduce the estimated intervention-specific cost of Group Work by 7.9 per cent to £1.6 million.

The effect of increasing the average time spent by Jobcentre Plus staff per referral would be to increase the estimated intervention-specific costs by 1.7 per cent to £1.73 million.

The overall cost estimates are therefore not greatly sensitive to these assumptions. We note that almost 60 per cent of the overall cost estimate and 61 per cent of the intervention cost estimate comprise payments to contractors, which are known with certainty and not subject to assumptions.

4. Benefits of Group Work

4.1. Potential Benefits of Group Work

Benefits of Group Work were expected to include:

• Improvements in participants’ mental health and wellbeing as a direct result of participating in Group Work; and
• Benefits to participants, the Exchequer and the economy where participation in Group Work leads to employment outcomes.

4.2. Evidence from the Process and Impact Evaluations

4.2.1. Findings from the Process Evaluation

In the survey conducted with participants six months after their participation in the course, 91 per cent of respondents reported that they had found it useful.^{[footnote 8]}

Participants identified a range of positive outcomes, including increased confidence, enhanced motivation, improved mental health and wellbeing, changes in job search behaviours, and progress towards and into work. Quotations included in the Process Evaluation Technical Report suggest that these outcomes were substantial for some participants.

Chapter 5 of the Process Evaluation Technical Report provides the findings from the six-month survey, which found that:

• The majority of participants said the course had a positive effect on different aspects of their job search. For example, nearly three-quarters (72 per cent) agreed they had a better understanding of what was needed to find and retain employment, and two-thirds (67 per cent) felt better equipped to write applications and CVs.
• Most participants said the course had improved their belief in their ability to find employment and their motivation and confidence (both in general and specifically in relation to their job search). Over half (56 per cent) also felt their resilience, i.e. ability to handle setbacks, had also improved.
• Around one in five participants said the course had improved their health (19 per cent) and nearly a quarter (23 per cent) said it had improved their ability to manage their health condition(s).

4.2.2. Findings from the Impact Evaluation (Impact on Participants, IoP)

Health and Wellbeing Outcomes

The Impact Evaluation report (Chapter 6) found the following effects on Group Work participants compared to the matched comparison group:

• Higher overall wellbeing and reduced likelihood of depression, as measured by the mean WHO-5 wellbeing score, at both six months and 12 months after baseline. This difference was statistically significant at six months but not at 12 months (because of a slight reduction in participant wellbeing as well as a widening of the confidence intervals);
• Higher overall wellbeing, as measured by the mean ONS life-satisfaction, life worthwhile and happiness scores, at both six months and 12 months. The difference in happiness scores was significant at both six and 12 months, while the difference in the life satisfaction and life worthwhile scores was statistically significant at six months but not at 12 months (largely due to improvements in these measures for the matched comparison group);
• Lower levels of loneliness, as measured by the UCLA loneliness score, at both six months and 12 months. Differences in the mean score were not found to be statistically significant at either six or 12 months. However, the proportion of participants classified as lonely was statistically smaller than the comparison group at six months (but not at 12 months);
• No overall difference in the mean score on the Latent and Manifest Benefits (LAMB)^{[footnote 9]} scale, which measures the latent and manifest benefits of employment, or in scales measuring psycho-social deprivation and financial strain;
• Lower levels of depression, as measured by the PHQ-9 depression scale, but these differences were not statistically significant at either six or 12 months;
• Lower levels of anxiety, as measured by the GAD-7 scale, but these differences were not statistically significant at either six or 12 months;
• No difference in the overall health of participants, as measured by mean EQ-5D scores. Participants registered a higher EQVAS index score, indicating better overall health than the comparison group, but this was not significant at either six or 12 months; and
• No significant difference in the use of health services (GP usage and casualty/outpatient visits in the past three months) at either six months or 12 months. Both measures of health service usage were actually higher for participants than the comparison group at 12 months, but not statistically significant.

The findings generally support those from the process evaluation that Group Work was beneficial to the mental wellbeing of participants. However, the evidence suggests that:

• The overall benefits on mental wellbeing were moderate;
• There are mixed results between indicators;
• Many of the differences are not statistically significant; and
• Where statistically significant differences were found, these tended to be at six months rather than 12 months, suggesting that differences compared to the comparison group reduced over time (for the reasons stated above).

Furthermore, the findings suggest that, while Group Work had some benefits for the mental health and wellbeing of participants, the scale of these was not large enough to affect the overall health of participants significantly (as measured by EQ-5D scores), or to reduce their use of health services.

Employment Outcomes

The impact evaluation found that Group Work participants were slightly more likely to be in paid work than members of the comparison group, at both six months (20 per cent compared to 18 per cent) and twelve months (23 per cent compared to 20 per cent). However, neither of these differences were statistically significant. There were similar findings in relation to the numbers of participants in paid work for 30 hours or more per week (Table 4.1).

Table 4.1: Working status of Group Work participants compared to comparison group

At 6-month follow-up

Working status	Participants %	Comparison group %	p-value
In paid work	20	18	0.442
In paid work 30+ hours a week	10	9	0.850
Base: all (number)	609	533	0

At 12-month follow-up

Working status	Participants %	Comparison group %	p-value
In paid work	23	20	0.445
In paid work 30+ hours a week	11	7	0.135
Base: all (number)	510	362	0

Source: Survey data

No statistically significant differences were found in job satisfaction or levels of earnings of those in work.

The survey found slightly lower levels of job search activity (as measured by the Finnish Institute of Occupational Health Job Seeking Activity Scale) among Group Work participants at six and twelve months compared to the comparison group, although a smaller proportion of Group Work participants than members of the comparison group reported no job search activity at all. None of these findings were statistically significant. However, the numbers of CVs submitted in the last fortnight was found to be larger among participants than the comparison group, and this difference was statistically significant at both six and 12 months.

The survey found limited evidence that Group Work participants were moving closer to the labour market. While a higher proportion than in the comparison group were found to be engaged in training courses (at six and 12 months), voluntary work (at 12 months) and work placements (at six and 12 months), none of these differences were statistically significant.

The impact evaluation did detect some effects on participant confidence and self-efficacy measures which could suggest that the course had moved them closer to the labour market, but again these effects were stronger at six months than 12 months. These included:

• Higher general self-efficacy of participants compared to the comparison group at six months and 12 months. This difference was only statistically significant at six months; as general self-efficacy was found to have improved in the comparison group at 12 months;
• Higher job search self-efficacy^{[footnote 10]} of participants compared to the comparison group at six months and 12 months. This difference was statistically significant at both six months and 12 months;
• A higher proportion of participants agreeing that their personal qualities will help them get work, and that their experience is in demand, at both six and 12 months compared to the comparison group. Both findings were statistically significant at six months but not at 12 months, because of both a slight decline among participants and an improvement in the comparison group.
• A higher proportion of participants stating that they were confident in getting a job compared to the comparison group, at six months and 12 months. However, this finding was significant only at six months, as confidence declined among participants and increased for the control group at 12 months.

Benefits Payments

DWP administrative data recorded benefits payments to Group Work participants and members of the comparison group which can be seen in Table 4.2.

Table 4.2: Benefits paid to Group Work Participants and the comparison group

Table 4.2: At 6-months

Benefit	Participants %	Comparison group %	p-value
Any benefit (other than Child Benefit)	86	84	0.107
Universal Credit, Jobseeker’s Allowance, Employment Support Allowance or Income Support	85	83	0.046
Mean amount (UC, IS, JSA, ESA) (£ per week)	73.6 (sd 45.8)	73.7 (sd 50.0)	0.919
Base: (number)	2,596	4,293	0

Table 4.2: At 12-months

Benefit	Participants %	Comparison group %	p-value
Any benefit (other than Child Benefit)	78	78	0.773
Universal Credit, Jobseeker’s Allowance, Employment Support Allowance or Income Support	77	76	0.315
Mean amount (UC, IS, JSA, ESA) (£ per week)	71.7 (sd 56.8)	74.0 (sd 62.6)	0.138
Base: (number)	2,596	4,293	0

Source: DWP administrative data

The administrative data found no difference in the proportion of Group Work participants receiving benefits at 12 months compared to the comparison group. However, at six months, the proportion of participants receiving Universal Credit (UC), Jobseeker’s Allowance (JSA), Employment Support Allowance (ESA) or Income Support (IS) (85 per cent) was statistically significantly higher than in the comparison group (83 per cent).

The mean level of these four benefits received by Group Work participants was £2.30 lower than for the comparison group at twelve months, but this finding was not statistically significant.

4.3. Valuing the Benefits of Group Work

Analysis of the evidence above indicates that, while Group Work was seen as being beneficial by the majority of participants, it has delivered limited benefits for participants overall that can be valued in monetary terms.

4.3.1. Health and Wellbeing Benefits

Group Work has resulted in some benefits to the mental health and wellbeing of participants but has not affected the two measures of health that can be monetised; the overall health of participants or their use of health services.

The intervention has therefore not delivered health benefits that can be valued in monetary terms.

4.3.2. Value of Employment Outcomes

A slightly higher proportion of Group Work participants were in work six months and 12 months after completing the course compared to members of the comparison group. However, these differences were not statistically significant.

Since Group Work was not found to result in a statistically significant increase in the proportion of participants in work, the most robust estimate of the value of employment outcomes of the intervention overall is zero.

As part of the sensitivity analysis, an upper estimate was made by valuing the observed differences in employment between Group Work participants and the comparison group, assuming that these can be attributed to participating in Group Work, even though they are not statistically significant.

The impact evaluation found that:

• The number of participants in work was 2.7 percentage points higher at six months and 3.2 percentage points higher at 12 months compared to the comparison group. Leaving aside the question of statistical significance, if these differences are assumed to be the result of Group Work, these figures suggest that, for every 1000 people completing the course, Group Work increased the number of people in work by 27 at six months and 32 at 12 months;
• The number of participants working 30 or more hours per week was 0.5 percentage points higher at six months and 4.3 percentage points higher at 12 months compared to the comparison group. Leaving aside the question of statistical significance, these figures suggest that, for every 1000 people completing the course, Group Work increased the number of people working more than 30 hours per week by 5 at six months and 43 at twelve months. The figures indicate that the number of participants working fewer than 30 hours per week at 12 months was 1.1 percentage points lower than in the comparison group.

For each additional person in full time work for one year in 2018/19, the DWP Social Cost Benefit Analysis (SCBA) model estimates that output is enhanced by £14,603 (present value, discounted from a 2016/17 baseline).

The SCBA model requires data to be input for the number of additional days in employment and off benefits as a result of the intervention. The time profile of additional days in employment was estimated by taking the percentages at month 6 and month 12 and using these to estimate incremental days in employment per month. A lower estimate was made based on an assumption that the effects on employment would last up to 18 months following completion of the course, and an upper estimate that these effects would last for up to 24 months. It was assumed that each additional person working more than 30 hours per week equated to one full time equivalent (FTE) job and that each person working fewer than 30 hours per week equated to 0.5 FTE jobs.

Table 4.3 estimates the additional effects in terms of the additional days that each participant spent in work per financial year, and the value of this additional employment in terms of enhanced output.

Table 4.3: Estimated effects of Group Work on days in work and value of output

Estimated additional days in work per participant per year

Financial year	Lower estimate	Upper estimate*
2017/18	1.7	1.7
2018/19	8.1	10.1
2019/20	0	1.4

Estimated present value of additional output per participant

Financial year	Lower estimate	Upper estimate
2017/18	£390	£528
2018/19	0	0
2019/20	0	0

*The difference between the lower and upper estimates reflects the assumed duration of employment effects (18 months in the lower estimate, 24 months in the upper estimate). The lower and upper estimates are therefore the same in the first year.

4.3.3. Value of Reduction in Benefits Payments

Analysis of the administrative data found that combined weekly levels of the four main employment benefits paid to Group Work participants were an average of £0.10 lower than those to the comparison group six months after completion of the course, and £2.30 lower at the twelve-month stage. These differences were not statistically significant at the five per cent level. The most robust estimate is therefore that Group Work caused no overall reduction in payments of benefits to participants.

As with employment outcomes, an upper estimate can be made by valuing the observed differences in benefits payments between Group Work participants and the comparison group, assuming these differences can be attributed to the Group Work intervention even though they are not statistically significant.

The time profile of savings in benefits payments (see Table 4.4) was estimated by taking the weekly savings at month 6 and month 12 and using these to estimate incremental savings per month. A lower estimate was made based on an assumption that the effects on benefits payments would last up to 18 months following completion of the course, and an upper estimate that these effects would last for up to 24 months.

Table 4.4: Estimated effects of Group Work on reductions in benefit payments

Estimated reduction in benefits payments per participant per year

Financial year	Lower estimate	Upper estimate
2017/18	£1.52	£1.52
2018/19	£60.88	£78.33
2019/20	£0	£12.46

5 Comparison of Monetised Costs and Benefits

The estimated costs and benefits of Group Work were inputted into the Department for Work and Pensions (DWP) Social Cost Benefit Analysis (SCBA) model in order to assess the overall balance of costs and benefits. The SCBA model provides an analysis of costs and benefits at four levels:

• Society;
• Project participants;
• Participants’ employers; and *The Exchequer.

5.1. Cost Benefit Analysis for Society

The overall estimates of the costs and benefits of Group Work for society are summarised in Table 5.1.

The assessment is presented at three levels:

A. Main assessment: This applies the most robust estimates of costs and benefits, assuming no benefit for employment (on the basis that no statistically significant impact was found);

B. Positive employment effect: This assessment applies the central estimate of costs but assumes a positive effect on employment (i.e. applying estimates of increased numbers of days in work, even though differences in work outcomes were not found to be statistically significant);

C. Positive employment effect, lower cost estimate: This assessment assumes a positive effect on employment and applies a reduced estimate of costs from the sensitivity analysis. It therefore presents a best-case scenario for both benefits and costs.

In each case the analysis focused on the intervention costs of the Group Work project, i.e. excluding costs that were specific to the trial and the evaluation (estimated at £1.7 million). The upper estimate of effects on employment is used – i.e. assuming that effects last for two years from completion of the course.

While the SCBA model provides estimates of potential reductions in healthcare costs resulting from employment outcomes, these were estimated at zero in the analysis, because no effect was found on use of health services by participants.

The results show a negative net benefit for society and a benefit cost ratio of less than one for all three assessments. Even if a positive effect on employment is included, the value of the extra work and resulting output is less than the costs of the intervention.

Under the most optimistic scenario (positive benefits, lower cost estimate, inclusion of intervention costs only, no substitution effect), the measurable benefits (in terms of enhanced output) are 67 per cent of the costs.

It should be noted that these numbers do not capture all of the potential benefits of Group Work – the research found some evidence of benefits to participants’ mental health and wellbeing, as well as certain job search related outcomes, which could not be valued in monetary terms. Other studies which have attempted to place monetary values on improvements to mental health (e.g. Fujiwara & Dolan, 2014) have found that these can be substantial, although the monetisation of such gains remains difficult and subject to much uncertainty.

5.2. Cost Benefit Analysis for Group Work Participants

The Cost Benefit Analysis (CBA) for Group Work participants compares the benefits of participation in Group Work (enhanced earnings and tax credits) with the costs (loss of welfare benefits, increased tax, travel and childcare costs).

The most robust estimate is that there is no net benefit to participants, because of the finding that Group Work resulted in no statistically significant increase in the number of participants in work.

However, if we assume that there is a positive effect on employment even though it is not statistically significant, the estimated net benefit to participants totals £206,000, with a benefit cost: ratio of 1.25 (Table 5.2).

The estimates of reductions in benefit payments are those derived by the DWP SCBA model, based on the increased number of people in work, rather than from the administrative data.

5.3.Cost Benefit Analysis for Participants’ Employers

Under the assumptions of the SCBA model, the estimated increase in employers’ output are offset by increased wages and National Insurance Contributions, so the estimated net benefit of Group Work to employers is zero (Table 5.3).

5.4.Exchequer Cost Benefit Analysis

Table 5.4 estimates changes in public finances resulting from Group Work, comparing benefits from increased taxes and reduced welfare payments with project costs.

In each case the project costs exceed any benefits through increased taxes and reduced welfare payments.

The estimates of reductions in benefit payments are those derived by the DWP SCBA model, based on the increased number of people in work, rather than from the administrative data.

Table 5.1: Society Cost Benefit Analysis for Group Work

Costs	A. Main estimate - No employment effect, central estimate of programme costs	B. Positive employment estimate, central estimate of programme costs	C. Positive employment estimate, lower estimate of programme costs
Benefits: Increase in output	£0	£1,013,850	£1,013,850
Reduction in healthcare costs	£0	£0	£0
Reduction in operational costs	£0	£29,079	£29,079
Social Cost of Exchequer Finance	Not applied	£0	£0
Total benefits	£0	£1,042,929	£1,042,929
Costs: Project costs	£1,646,463	£1,646,463	£1,515,892
Increase in social travel costs	£0	£37,726	£37,726
Increase in childcare costs	£0	£11,148	£11,148
Total costs	£1,646,463	£1,695,337	£1,564,766
Net benefit	-£1,646,463	-£652,408	-£521,837
Benefit Cost Ratio (BCR)	0.00	0.62	0.67

Table 5.2: Cost Benefit Analysis for Group Work participants

Costs	A. Main estimate - No employment effect, central estimate of programme costs	B. Positive employment estimate, central estimate of programme costs	C. Positive employment estimate, lower estimate of programme costs
Benefits: Increase in wages	£0	£950,007	£950,007
Increase in Tax Credits	£0	£76,079	£76,079
Total benefits	£0	£1,026,086	£1,026,086
Costs: Reduction in Universal Credit	£0	£0	£0
Reduction in legacy benefit	£0	£474,400	£474,400
Reduction in Housing Benefit payments	£0	£133,590	£133,590
Reduction in Carers Allowance	£0	£0	£0
Reduction in Council Tax Benefit payments	£0	£27,632	£27,632
Reduction in Free School Meals payments	£0	£0	£0
Increase in income tax	£0	£65,371	£65,371
Increase in employees’ NIC	£0	£54,233	£54,233
Increase in indirect tax	£0	£33,417	£33,417
Increase in travel costs	£0	£20,357	£20,357
Increase in childcare costs	£0	£11,148	£11,148
Total costs	£0	£820,148	£820,148
Net benefit	£0	£205,938	£205,938
Benefit Cost Ratio (BCR)	0.00	1.25	1.25

Table 5.3: Cost Benefit Analysis for employers of Group Work participants

Costs	A. Main estimate - No employment effect, central estimate of programme costs	B. Positive employment estimate, central estimate of programme costs	C. Positive employment estimate, lower estimate of programme costs
Benefits: Increase in output	£0	£1,013,850	£1,013,850
Employment subsidies	£0	£0	£0
Total benefits	£0	£1,013,850	£1,013,850
Costs: Increase in wages	£0	£950,007	£950,007
Increase in employers’ NIC	£0	£63,843	£63,843
Additional employer costs	£0	£0	£0
Total costs	£0	£1,013,850	£1,013,850
Net benefit	£0	£0	£0
Benefit Cost Ratio (BCR)	0.00	1.00	1.00

Table 5.4: Exchequer Cost Benefit Analysis for Group Work

Costs	A: Main estimate - No employment effect, central estimate of programme costs	B: Positive employment estimate, central estimate of programme costs	C: Positive employment estimate, lower estimate of programme costs
Benefits: Increase in income tax	£0	£65,371	£65,371
Increase in employees’ NIC	£0	£54,233	£54,233
Increase in employers’ NIC	£0	£63,843	£63,843
Increase in indirect tax	£0	£33,417	£33,417
Reduction in healthcare costs	£0	£76,957	£76,957
Reduction in Universal Credit	£0	£0	£0
Reduction in legacy benefit	£0	£474,400	£474,400
Reduction in Carers Allowance	£0	£0	£0
Reduction in Housing Benefit payments	£0	£133,590	£133,590
Reduction in Council Tax Benefit payments	£0	£27,632	£27,632
Reduction in Free School Meals payments	£0	£0
Reduction in operational costs	£0	£29,079	£29,079
Total benefits	£0	£958,523	£958,523
Costs: Project costs	£1,646,463	£1,646,463	£1,515,892
Employment subsidies	£0	£0	£0
Increase in Tax Credits	£0	£76,079	£76,079
Total costs	£1,646,463	£1,722,542	£1,591,971
Net benefit	-£1,646,463	-£764,019	-£633,448
Benefit Cost Ratio (BCR)	0.00	0.56	0.60

6.Sub-Group Analysis

6.1. Cost Benefit Analysis for Sub-Groups

The analysis also examined the costs and benefits for two sub-groups of Group Work participants for which the proportion of participants in work was statistically significantly higher than the comparison group at six and/or twelve months after the course:

• Those with suggested case levels of anxiety^{[footnote 11]} (as measured by GAD-7) at baseline; and
• Those with low levels of general self-efficacy at baseline.^{[footnote 12]}

The differences in the proportion of members of these sub-groups in work are summarised in Table 6.1.

Table 6.1: Working status of sub-groups of Group Work participants compared to comparison group

At 6-month follow-up

Working status	Participants %	Comparison group %	p-value
Those with suggested case level anxiety at baseline: In paid work	19.9	10.1	0.023
In paid work 30+ hours a week	9.0	2.7	0.023
Those with low general self-efficacy at baseline: In paid work	20.8	10.9	0.044
In paid work 30+ hours a week	7.8	1.7	0.030
Base (number)	609	533	0

At 12-month follow-up

Working status	Participants %	Comparison group %	p-value
Those with suggested case level anxiety at baseline: In paid work	24.3	13.1	0.054
In paid work 30+ hours a week	12.1	4.9	0.050
Those with low general self-efficacy at baseline: In paid work	18.5	11.8	0.207
In paid work 30+ hours a week	7.1	1.9	0.024
Base (number)	510	362	0

Source: Survey data

The impact evaluation also examined the effects on a third sub-group – those with suggested case level depression at baseline – but did not find a significant effect on the numbers in paid work at six or twelve months. The Cost Benefit Analysis (CBA) did not therefore estimate the benefits for this sub-group.

Using a similar method to that in Section 4.3.2, Table 6.2 estimates the additional effects in terms of the average number of additional days that each participant in each of the two sub-groups spent in work per financial year, and the value of this additional employment in terms of enhanced output.

Table 6.2: Estimated effects on days in work and value of output

Estimated additional days in employment per participant per year

Financial year	Lower estimate	Upper estimate
Suggested case level anxiety at baseline: 2017/18	8.6	8.6
2019/20	0.0	3.5
Low general self-efficacy at baseline: 2017/18	8.5	8.5
2018/19	16.9	20.1
2019/20	0	2.3

Estimated present value of additional output per participant

Financial year	Lower estimate	Upper estimate
Suggested case level anxiety at baseline: 2017\18	£1,262	£1,600
2018/19	£0	£0
2019/20	£0	£0
Low general self-efficacy at baseline: 2017/18	£1,018	£1,235
2018/19	£0	£0
2019/20	£0	£0

The estimated difference that Group Work made to the numbers of days worked by these sub-groups, and the effects on the value of output, are proportionally much greater than for participants in the project as a whole.

Table 6.3 presents a social cost benefit analysis per participant in these sub-groups. Costs and benefits are presented per participant, and it is assumed that the average costs per participant in these sub-groups is the same as those per participant in Group Work overall. This is a strong assumption, as we might expect that focusing on participants with these particular conditions might increase the costs of referral, and potentially also lead to a loss of economies of scale by reducing the size of the target population and possibly the numbers of participants in each session.^{[footnote 13]}

The central estimates in Table 6.3 assume that employment effects last for up to two years from the time of the intervention.

Table 6.3: Society Cost Benefit Analysis for each participant in two sub-groups

Costs	A. Suggested case level anxiety at baseline	B. Low general self-efficacy at baseline
Benefits: Increase in output	£1,600	£1,235
Reduction in healthcare costs	£0	£0
Reduction in operational costs	£46	£36
Social Cost of Exchequer Finance	Not applied	Not applied
Total benefits	£1,647	£1,271
Costs: Project costs	£857	£857
Increase in social travel costs	£60	£46
Increase in childcare costs	£18	£14
Total costs	£934	£916
Net benefit	£712	£355
Benefit Cost Ratio (BCR)	1.76	1.39

The estimated benefits for both sub-groups exceed the intervention costs, giving a benefit: cost ratio of 1.76 for those with suggested case level anxiety and 1.39 for those with low levels of general self-efficacy at baseline. This suggests that for every £1 spent to achieve an employment outcome, the benefits to society would total £1.76 for those with suggested case level anxiety and £1.39 for those with low levels of general self-efficacy at baseline. This is without other benefits, such as to confidence and wellbeing, being taken into account.

6.2. Sensitivity Analysis

The above analysis is based on the central estimates of programme costs per participant and assumes that additional impacts on employment last for up to two years from the time of the intervention.

Tables 6.4 and 6.5 present a sensitivity analysis around this central estimate of societal costs and benefits, using the following assumptions:

• Low benefit: cost estimate – Assumes higher programme cost estimate and that employment effects last up to 1.5 years from date of participation;
• Central estimate – Assumes central cost estimate and that employment effects last up to two years from date of participation (as above);
• High benefit: cost estimate – Assumes lowest programme cost estimate and that employment effects last up to two years from date of participation.

The estimated benefits exceed the costs for both sub-groups, even when the estimated effects on employment are assumed to last for up to 18 months instead of two years after participation, and where the higher estimate of programme costs is used. Under the three sets of assumptions, the estimated benefit: cost ratios range from 1.39 to 1.90 for those with case level anxiety at baseline (Table 6.4) and from 1.14 to 1.50 for those with low general self-efficacy at baseline (Table 6.5).

These findings are reliant on the important (and strong) assumption that the average costs per participant in these sub-groups is the same as for Group Work overall.

Table 6.4: Sensitivity analysis for Society Cost Benefit Analysis for sub-group with suggested case level anxiety at baseline (per participant, present values)

Costs	Low benefit: cost estimate	Central estimate	High benefit: cost estimate
Benefits: Increase in output	£1,262	£1,600	£1,600
Reduction in healthcare costs		£0
Reduction in operational costs	£36	£46	£46
Social Cost of Exchequer Finance	Not applied	Not applied	Not applied
Total benefits	£1,298	£1,646	£1,646
Costs: Programme costs	£871	£857	£789
Increase in social travel costs	£47	£60	£60
Increase in childcare costs	£14	£18	£18
Total costs	£932	£934	£867
Net benefit	£366	£712	£779
Benefit Cost Ratio (BCR)	1.39	1.76	1.90

Table 6.5: Sensitivity analysis for Society Cost Benefit Analysis for sub-group with low levels of general self-efficacy at baseline (per participant, present values)

Costs	Low benefit: cost estimate	Central estimate	High benefit: cost estimate
Benefits: Increase in output	£1,018	£1,235	£1,235
Reduction in healthcare costs	£0	£0	£0
Reduction in operational costs	£29	£36	£36
Social Cost of Exchequer Finance	Not applied	£0	£0
Total benefits	£1,047	£1,271	£1,271
Costs: Programme costs	£871	£857	£789
Increase in social travel costs	£38	£46	£46
Increase in childcare costs	£11	£14	£14
Total costs	£920	£916	£849
Net benefit	£127	£355	£422
Benefit Cost Ratio (BCR)	1.14	1.39	1.50

7. Conclusions

The Group Work project cost an estimated £3.3 million, very slightly less than budgeted in the Business Case. Of this, estimated costs of £1.7 million were incurred in administering and delivering the intervention, while costs of £1.6 million were incurred in activities related to trial and evaluation, most of which would not be incurred again if the project were continued or rolled out. The intervention costs were estimated at £886 per participant completing the course.

These cost estimates are considered to be fairly robust from the data available, and the sensitivity analysis estimated only a narrow range of costs (with intervention costs ranging from -8 per cent to +2 per cent around the central estimate).

The course was seen as beneficial by participants, with most respondents reporting benefits to their wellbeing and job search behaviour. This is reflected in positive effects in a range of indicators relating to mental health and wellbeing, although these were not all statistically significant, and not strong enough to affect the overall health of participants or their use of health services. As a result, no monetary value could be assigned to health effects.

Group Work participants were more likely to be in work, and in work for more than 30 hours per week, than members of the comparison group, both six months and 12 months after completing the course, but these differences were not statistically significant.

The costs of delivering Group Work as a whole were found to outweigh the value of the benefits to society, even employing best case assumptions for employment effects and costs. It should be noted that this analysis does not include benefits to individual mental health and wellbeing, which could not be valued.

However, Group Work was found to deliver significant employment outcomes for particular sub-groups, notably those with suggested case levels of anxiety and those with low general self-efficacy at baseline. If the costs of the intervention for members of these sub-groups are assumed to be similar to those for the project overall, the benefits in terms of enhanced employment and output were found to exceed the intervention costs. This suggests that the intervention could deliver net benefits to society if targeted at these particular sub-groups. This finding relies on the (strong) assumption that the costs of delivery to these sub-groups can be held at the average intervention cost of Group Work, and that similar levels of benefits can be maintained. For a third sub-group, those with suggested case level depression at baseline, no significant effect was found on numbers in work at either six or 12 months post-baseline, indicating that the costs of the intervention exceeded the benefits.

References

Bivand, P. and Simmonds, D. (2014) The Benefits of Tackling Worklessness and Low Pay. Joseph Rowntree Foundation.

Claxton, K., Martin, S., Soares, M., Rice, N., Spackman, E., Hinde, S., Devlin, N., Smith, P. C., and Sculpher, M. (2013) Methods for the Estimation of the NICE Cost Effectiveness Threshold. CHE Research Paper 81, University of York.

Curtis, L. A., and Burns, A. (2018) Unit Costs of Health and Social Care 2018. Project report. Personal Social Services Research Unit, University of Kent.

Department for Work and Pensions (2017) Social Cost-Benefit Analysis, version 2.2 User Guidance, June 2017. Central Analysis Division, 27 June 2017.

EuroQol Group (1990) EuroQol - a new facility for the measurement of health-related quality of life. Health Policy 16(3):199-208.

Fujiwara, D. (2010) The Department for Work and Pensions Social Cost-Benefit Analysis framework - Methodologies for estimating and incorporating the wider social and economic impacts of work in Cost-Benefit Analysis of employment programmes. DWP Working Paper No 86. London.

Fujiwara, D and Dolan, P. (2014) Valuing mental health. Policy, 4(2.1).

Glover, D., and Henderson, J. (2010) Quantifying health impacts of government policies: A how-to guide to quantifying the health impacts of government policies. Department of Health.

Greater Manchester Combined Authority (2019) Greater Manchester Cost Benefit Analysis Tool and Unit Cost Database

HM Treasury (2018) The Green Book – Central Government Guidance on Appraisal and Evaluation

Knight, T., Lloyd, R., Downing, C., Svanaes, S. and Coutts, A. (2021a) Group Work/JOBS II: Process Evaluation Technical Report, DWP Research Report No 989. London.

Knight, T., Lloyd, R., Rayment, M., Purdon, S., Bryson, C., Downing, C., Svanaes, S., Coutts, A., McKay, S., and Mukuria, C. (2021b) Group Work/JOBS II: Evaluation Synthesis Report. London. DWP Research Report No 991. London.

McCrone, P., Dhanasiri, S., Patel, A., Knapp, M. and Lawton-Smith, S, (2008) Paying the Price: The cost of mental health care in England to 2026. The King’s Fund.

Mallender, J., Tierney, R., Baah B. and Stavridou, M. (2017) Movement into Employment: Return on Investment Tool. Estimation of benefits from moving an individual from unemployment into sustainable employment. Public Health England.

NHS Digital (2018) Prescription Cost Analysis - England, 2017

NHS Improvement (2018) Reference Costs

NICE (2012) The guidelines manual - Process and methods [PMG6]. November 2012.

Purdon, S., and Bryson, C. (2021) Group Work/JOBS II: Technical Report on the Impacts of the Trial, DWP Research report No 988. London.

Trotter, L., and Rallings-Adams, M. K. (2017) Valuing improvements in mental health: Applying the wellbeing valuation method to WEMWBS. HACT. London, UK.

Van Stolk, C., Hofman, J., Hafner, M., and Janta, B. (2014) Psychological wellbeing and work: Improving service provision and outcomes. Department for Work and Pensions and Department of Health: London, UK.

Vinokur, A.D., Schul, Y., Vuori, J., and Price, R. (2002) Two Year after a Job Loss: Long Term Impact of the JOBS II program on reemployment and mental health. Journal of Occupational Health Psychology, 5 (1) 32-47.

Vuori, J., and Silvonen, J. (2005) The benefits of a preventive job search program on reemployment and mental health at 2-year follow-up. Journal of Occupational and Organisational Psychology, 78, 43-52.

Appendix: Assumptions used in Costs and Benefits Analysis

Costs

General assumptions

• All known costs specifically relating to developing, delivering and trialling Group Work have been included.
• Policy development costs have been excluded.
• Costs have been broken down into those that relate specifically to the trial and evaluation (one-off costs that would not be repeated if Group Work were rolled out) and those that relate to project delivery (and would be repeated if Group Work were rolled out).
• Unit staff costs include overheads (office, I.T, training, expenses etc.).
• All costs are assumed to be additional costs relative to a Business as Usual scenario involving no Group Work intervention; it is assumed that delivery of Group Work did not affect the other services delivered to beneficiaries and the control group.
• The intervention was delivered over the period February 2017 to May 2018 and most costs relate to this period; however, some costs (e.g. evaluation) were incurred after this period and benefits were expected to occur beyond the lifetime of the intervention. The Cost Benefit Analysis (CBA) applied a time profile of these costs and benefits and applied 3.5 per cent discount rate specified in HM Treasury (HMT) Green Book and Department for Work and Pensions (DWP) Social Cost Benefit Analysis (SBCBA) model.

Staffing

Policy staff

• 1 x Higher Executive Officers (HEOs) for one day per week to support live running.
• Assumed duration of 18 months.
• Costings based on average HEO salary including overheads
• It is assumed that all of this cost supported the delivery of the intervention, rather than being related to trial/ evaluation.

Trial Integrity and Support Officers

• 2x HEOs full time during trial period.
• Assumed duration of 18 months.
• Costings based on average HEO salary including overheads
• Role ensured trial fidelity, as well as supporting delivery of intervention.
• Assumed that 50 per cent of time and costs are intervention-specific and 50 per cent are trial-specific.

DWP Psychological Services Staff

• Staff provided advice, support, training and feedback related to aspects of trial design, procurement, evaluation, delivery. This included:
  • 300 days related to project delivery (51 Principal Psychologist, 99 Senior Psychologist, 150 Higher Psychologist).
  • 190 further days related to trial (8 Senior Principal Psychologist, 96 Principal Psychologist,86 Senior Psychologist).
  • 490 days total.
• Staff time was costed at daily rates.

Operations Staff

Work Coach time spent on referrals

• Time spent by Work Coaches on referrals is estimated by data given to ICF in qualitative interviews.
• Estimated that each referral took an average of 24 minutes of Work Coach time, including 12 minutes for survey (trial specific) and 12 minutes for administering referral (project delivery, included in the intervention costs).
• Project data show a total of 16,459 referrals (2,635 participants, 9,473 decliners, 4,351 control group).
• It is assumed that all time devoted to control group referrals is trial specific, and that 50 per cent of time spent on participants and decliners was trial specific and 50 per cent related to project delivery.
• Work Coach time was costed using a unit cost per productive minute, including overheads.

Training delivered to Work Coaches

• DWP data indicate that one day of training was provided to 696 Work Coaches at 53 Jobcentre Plus offices, delivered by Learning and Development Officers
• Assumed number of days involved – 696 for Work Coaches, 53 for Learning and Development Officers (assuming one Learning Development Officer provided training for one day at each job centre).
• Staff costings based on average salary per day.
• All of these training costs are assumed to relate to project delivery.

Single Points of Contact (SPoC)

• Single Point of Contact in an area, involved in monitoring volumes, arranging training and delivering the project.
• Assumed duration of 18 months.
• Costings based on monthly expenditure data, with assumption all of it related to project delivery.

Operations Executive Officer (EO) staff

• Operations EO staff supported Work Coaches and referrals.
• Costings based on estimated staff costs for 2017/18.

Evaluation Team

• Costs of DWP staff time spent on evaluation are included and treated as trial specific.
• Estimated time inputs are as follows:
  • Design stage: 18 months, 0.5 Grade 7, 0.5 FTE student (EO grade).
  • Live running: 15 months, 0.33 Grade 7, 1.2 FTE SEO, 0.5 student (EO).
  • Follow up: 7 months 0.33 Grade 7; 8 months 1 HEO; 4 months 0.1 Grade 6; 4 months 0.6 Grade 7, 1 HEO; 11 months 0.8 Grade 7, 0.5 SEO, 0.5 HEO.
• Costings based on average staff costs (including overheads).

DWP costs for contract management

• DWP staff in the Contracted Health and Employment team managed contract and payments to the supplier and performance management.
• Assumed 18-month duration.
• Costings based on estimated monthly staff costs, with assumption all costs relate to project delivery.
• All costs assumed to relate to project delivery.

Contractors

Intervention contract

• Payments made to two contractors.
• All of the costs assumed to relate to project delivery.

Evaluation contract

• Contract with ICF.
• All costs relate to trial rather than project delivery.

Participant costs

• Travel costs of 2,596 participants.
• Estimated 12,500 days attendance.
• Costings based on - weighted average daily commuting cost - £1.87 (Fujiwara, 2010) inflated to 2017 prices using consumer price indices (CPI) for personal transport.
• All of these costs assumed to relate to intervention delivery.
• These are not added to avoid double counting as they are included in provider costs.

Sensitivity analysis

A sensitivity analysis was undertaken to assess the sensitivity of the cost estimates to key assumptions.

The assumptions tested were:

• The effect of excluding overheads from estimates of staff costs (i.e. including only salaries, pensions and National Insurance);
• The effect of the assumed split of Trial Integrity and Support Officer time between the intervention and trial-specific elements of Group Work. The main cost estimate assumed a 50:50 split but an alternative estimate was that 10 per cent of time was related to the intervention and 90 per cent to the trial element of Group Work;
• The effect of reducing the estimate of the time spent by Work Coaches per Group Work referral by one third from the best estimate of 24 minutes to 16 minutes (with an equal split between trial and intervention costs).

Sub-group Costs

The costs of treating sub-groups were assumed to be the same as average costs per participant for Group Work as a whole. This is a strong assumption as it assumes no additional costs of referrals and no loss of economies of scale.

Benefits

Quantifying Beneficiaries and Impacts

The benefits analysis focuses on benefits to participants only, rather than the wider group engaged on an Intention to Treat (ITT) basis. This is because benefits are derived only as a result of participation in the intervention. Benefits are then scaled up across participants, rather than the overall membership of the Group Work group.

Significance of impacts

The impact evaluation measures differences in outcomes between Group Work participants and the matched comparison group and assesses the statistical significance of any differences observed at the five per cent level.

A central assumption of the CBA was that the intervention delivered benefits only if these were found to be statistically significant. However, the impact evaluation found some positive differences in employment outcomes between the participant and comparison groups which were not significant at the five per cent level. The value of these differences was assessed as an “upper-bound” estimate of benefits in the sensitivity analysis.

Duration of impacts

The evaluation surveyed Group Work participants at two points in time – six months and twelve months after completion of the course – in order to assess outcomes. Assessment of the benefits of Group Work depends on the duration of any outcomes observed. This required assumptions to be made about how long these outcomes would last.

International evidence on Jobs II has detected impacts at up to two years after the intervention (Vinokur et al., 2002; Vuori and Silvonen, 2005). Two assumptions were applied in assessing benefits:

1. Benefits were assumed to last for eighteen months following the intervention (i.e. a further six months after the impacts found at 12 months); and

2. Benefits were assumed to last for 24 months following the intervention (i.e. a further year after the impacts found at 12 months).

Modelling of Benefits

The DWP SCBA model was used to estimate the value of employment outcomes and related costs.

While the SCBA model also estimates changes in healthcare costs, based on employment outcomes, these were considered separately using data from the impact evaluation. Since the survey of participants included questions about their actual use of health services, this approach was expected to deliver more reliable estimates than applying the assumptions in the SCBA model.

Discounting

The present value of these costs and benefits was calculated using the 3.5 per cent social discount rate specified in the HMT Green Book and applied by the DWP SCBA model.

This enabled estimation of the net benefits and benefit: cost ratios for the intervention at each level (society, participants, employers and Exchequer).

1. A person is described as having suggested case level anxiety or depression if their scores on the Generalised Anxiety Disorder (GAD-7) and Patient Health Questionnaire (PHQ-9) scales suggests they would exceed the ‘caseness thresholds’ used by Improved Access to Psychological Therapies. Diagnoses of anxiety or depression respectively would be based on a clinical interview and would take account of additional evidence, to which the GAD or PHQ scores may contribute. ↩

2. General self-efficacy is the strength of an individual’s belief that they are effective in handling life situations. ↩

3. See Chapter 2 of the Technical Report on the Impacts of the Trial for detail the design of the RCT. ↩

4. For further detail on this, and the other outcome measures used in the evaluation, see Chapter 3 of the Technical Report on the Impacts of the Trial. ↩

5. A person is described as having suggested case level anxiety if their score on the GAD-7 scale suggests they would exceed the ‘caseness threshold’ (eight or more) used by Improved Access to Psychological Therapies. Diagnosis of anxiety respectively would be based on a clinical interview and would take account of additional evidence, to which the GAD scores may contribute. ↩

6. Intervention specific costs are the costs of delivering Group Work and are those that would be expected to be incurred if the programme were to be rolled out. ↩

7. Trial specific costs are one-off costs of actions concerned with trialling and evaluating Group Work and would not be incurred again if the programme were to be rolled out. ↩

8. 544 people participants responded to the survey, representing 21 per cent of participants starting the course and 28 per cent of those completing it. ↩

9. Latent and Manifest Benefits (LAMB) are benefits associated with being in work such as social interaction, self-worth and the absence or lessening of financial strain. ↩

10. Job search self-efficacy is a measure of an individual’s belief that they have the skills to undertake a range of job search tasks, measured using the Job Search Self-Efficacy (JSSE) Index Modified. ↩

11. Caseness is defined as the proportion of respondents whose score on the GAD-7 scale reaches the threshold (a score of eight or more) used by Increased Access to Psychological Therapies (IAPT) services to suggest that the person would probably receive a diagnosis of anxiety. Diagnoses of anxiety or depression respectively would be based on a clinical interview and would take account of additional evidence, to which the GAD score may contribute. ↩

12. General self-efficacy is measured using the General Self Efficacy (SE) scale, with a low level of general self-efficacy being defined as recording a mean score of 2.34 or above. ↩

13. It would only be possible to test this by designing and costing a new intervention focusing on participants with these conditions, which is beyond the scope of the current study. ↩