Health Transformation Programme Evaluation and Monitoring Strategy 2026

Question 1

Executive summary

Accepted Answer

This evaluation strategy updates DWP’s plans to develop the evidence base for the Health Transformation Programme (HTP), following the original evaluation strategy published in May 2023.

The HTP is modernising health and disability benefit services to improve customer experience and efficiency, build trust in our services and decisions and support people to enter or remain in work. It is transforming the Personal Independence Payment (PIP) service, including introducing the option to apply online; creating a new customer-focused Health Assessment Service (HAS); and supporting the development of a coherent employment support offer for disabled people and people with health conditions.

This high-level evaluation strategy provides a current overview of how we plan to evaluate the HTP. The core aims of the evaluation and monitoring remain:

• Establish a robust and detailed evidence base to inform decisions about service improvements and how the HTP can drive better customer experience

• Provide person-centred insight on what works for whom to ensure a broad range of needs are considered when transforming our services

• Provide assurance that the business case has been met

Our approach to monitoring and evaluating the HTP aligns to the agile approach to design and delivery in the Programme. This ensures we are consistently providing timely evidence and iterating our learning plans to feed into key decisions across the life cycle of the Programme, not just at the end. This approach facilitates evidence-based decisions and allows the Programme to adjust direction where necessary, based on up-to-date evidence.

The evaluation is built around four interchangeable layers that will help the Programme build a robust evidence base. It also includes a Theory of Change; an internal and externally commissioned research programme; a test and learn strategy; ongoing evaluation of services; and a performance framework.

We include new sections in this strategy to address how evidence and analysis will be and are already being used to assess whether the Programme is achieving its five strategic outcomes. This includes how we will measure customer trust and experience, cultural and behavioural change, efficiency, and employment outcomes, as well as understanding the impact of transformed IT and data and our approach to evaluating fraud and error.

We expect the evaluation strategy to continue to develop and adapt as the HTP further scales and develops in the future.

Question 2

Introduction

Accepted Answer

Updating the Health Transformation Programme Evaluation Strategy:

This evaluation strategy updates DWP’s plans to develop the evidence base for the Health Transformation Programme (HTP), following the original evaluation strategy published in May 2023. Since then, the HTP has continued to evolve in design and importantly, begun limited and small-scale service delivery, prompting adjustments to evaluation plans.

With this in mind, it is timely to review and update the HTP evaluation strategy to provide additional details, incorporate emerging evidence and data and outline ongoing and future research efforts.

Whilst the entire document reflects an updated view of the HTP, key updates include:

enhanced detail on the HTP Theory of Change (ToC), now guiding current and planned evaluations
expanded information on process, impact, and value for money evaluations as service delivery is better understood
new sections on the measurement of key outcomes outline how evidence and analysis will be and are already being used to assess whether the Programme is achieving its five strategic outcomes

Health Transformation Programme aims

The HTP is modernising health and disability benefit services to improve customer experience and efficiency, build trust in our services and decisions and support people to enter or remain in work. It is transforming the Personal Independence Payment (PIP) service, including introducing the option to apply online; creating a new customer-focused Health Assessment Service (HAS); and supporting the development of a coherent employment support offer for disabled people and people with health conditions.

The Programme’s strategic outcomes are:

increased customer trust in services and decisions
a more efficient service with reduced demand for health assessments
increased take up of wider support and employment
improved customer experience with shorter journey times
transformed in-house data and IT infrastructure, giving greater capability for innovation

Health Transformation Programme scope and approach

In this section, we provide more detail on the new services that the HTP is developing, the difference these new services will make, and the Programme’s approach to developing them.

The Programme is transforming the PIP service, from finding out about benefits and eligibility through to decisions and payments. As part of this, the Programme is improving existing ways to claim and introducing an optional online service, providing customers with a greater choice over the channels they use to interact with the Department for Work and Pensions (DWP).

The transformed PIP service will deliver a simpler application process for customers with more information and support available to those who need it, including helping them decide whether applying for PIP is right for them. It will provide greater transparency in how decisions are made, supporting customers by providing the right information at the right time, and using the most effective channels to communicate. Improved evidence gather will also enable the Department to better tailor the service to the customer’s circumstances.

As part of this, we are exploring a case management model: a personalised approach for customers from initial contact and throughout the application, including signposting to other benefits and services. This approach will help the Programme to better understand our customers, tailor their service, and help to build customer confidence and trust in the progression of their claim.

Simplifying and automating the PIP journey and tailoring how the customer is assessed will mean many customers get a decision on their claim much quicker. Customers will be able to apply, upload evidence, track progress and see payment details online. Communications and notifications will be simpler and easier to understand (and for those on the digital journey, they will be available to receive and submit electronically) so that customers understand their claim decision and how the evidence they provided supported it.

The Programme has launched a fully online claim service in a limited number of postcodes. Operating the service initially at small scale ensures protection of new and existing customers. These cases are being processed by staff in our New PIP Service Office.

The Programme is creating a more joined up health and employment journey for customers. The transformed service will better support disabled people and people with health conditions move towards and into sustainable work, where appropriate.

The Programme is also creating a single new Health Assessment Service that will replace the different systems and processes we and our assessment providers use to deliver functional health assessments for PIP and some specialist benefits, including new IT and streamlined processes.

The Health Assessment Service will be fully integrated with other systems, including the transformed PIP service, to create a seamless customer experience. By improving how we gather evidence and by enabling the re-use of information, the new integrated service will provide healthcare professionals and DWP staff with easier access to relevant information, and will provide customers with an improved, clearer and more joined up experience.

With around 2 million health assessments conducted each year, the nature and scale of the HTP’s work is complex and ambitious. The new services represent a major change to how the Department manages functional health assessments and how we support those who are applying for PIP. In recognition of these complexities, and the need to protect vulnerable customers, we are developing the new services at a small scale in a safe and controlled environment known as the Health Transformation Area and have begun testing the Health Assessment Service with providers. The Health Transformation Area is where we are exploring, adapting and learning from new ideas and processes. It provides the space to develop and build our services safely, before carefully and gradually growing the service.

“Business as usual”

The current Functional Assessment Service (FAS) represents the Department’s “business as usual” (BAU) approach to delivering health assessments. The new contracts, which have been operating since 2024, replaced the previous Health and Disability Assessment Services and PIP assessments, with joint services delivered under a single contract in each geographic area. The FAS contracts form an important step in delivering longer term reform by providing continuity for customers, supporting the gradual introduction of the new Health Assessment Service, and creating greater consistency across assessments.

Progress and achievements

The HTP is a long-term project running until 2029. The Programme’s roadmap has five stages: Building the Foundations, Improving the Service, Scaling the Service, Transitioning the Service, and Living the Service. We are currently in stage two, Improving the Service.

The Programme is taking a measured approach, moving away from the current ways of operating gradually and carefully, protecting new and existing customers.

To date, we have:

established test environments where we are developing the new Health Assessment Service and exploring health employment support
procured and transitioned to new assessment provider contracts
introduced options to apply for PIP online
taken the first live cases through our new PIP service and have continued to develop and improve the service as we enable it to handle a greater variety of customer cases from claim to payment and a greater volume of cases
enabled over 90% of PIP customers to submit their health information online after they call us to begin their claim. This is benefiting customers by removing postage times and increasing accessibility
launched Pathways to Work Exploration Hubs, pilot environments where we are exploring new ways to support disabled people and people with health conditions to move towards and into sustainable work
begun testing the new Health Assessment Service with all of our assessment providers
introduced a case management model; a personalised approach for customers from initial contact and throughout the application. We have begun exploring where it is appropriate for case managers to make entitlement decisions without a full health assessment

Evaluation and monitoring approach

Robust evaluation plays a crucial role in maximising the value of public spending, ensuring we understand the effectiveness, efficiency and impact of interventions. The Magenta Book, which sets out guidelines for evaluation of government interventions, suggests that evaluation should be built into design and delivery from the earliest stages. Our approach to evaluation and monitoring across four layers ensures our learning and evidence is proportionate to the complexity, approach, and length of the Programme.

The content of this strategy uses evaluation guidance from the Green Book and Magenta Book and builds on the DWP Evaluation Strategy which sets out the approach for ensuring that DWP evaluations deliver high quality evidence, on the right questions, at the right time.

We are drawing on advice from other major DWP programmes, to build a programme of evaluation and monitoring which supports the agile nature of the HTP across the life of the Programme. As well as the work detailed in this document, our evaluation and monitoring approach is complemented by ongoing day-to-day learning and iterating that forms an integral part of an agile Programme. This includes the insights and learning that we collect as part of our usual processes, such as that undertaken by user researchers, digital service designers and business analysts.

Evaluation and monitoring aims

The evaluation and monitoring approach for the HTP is developing over the life of the Programme to incrementally build a picture of:

the context, influencing factors and levers which relate to the outcomes of the HTP as set out in the strategic outcomes, ToC and Programme business case
how successful the HTP interventions are at achieving these outcomes and delivering on the HTP’s business case

Evaluation and monitoring activity provides evidence to inform the HTP’s decision-making and support the department’s wider health and disability strategic outcomes. The core aims of the evaluation and monitoring are to:

establish a robust and detailed evidence base to inform decisions about service improvements and how the HTP can drive better customer experience
provide person-centred insight on what works for whom to ensure a broad range of needs are considered when transforming our services
provide assurance that the business case benefits have been realised

Informing, monitoring and evaluating an agile programme

Our approach to monitoring and evaluating the HTP aligns to the agile approach to design and delivery in the Programme. We have developed an approach which ensures we are consistently providing timely evidence and iterating our learning plans to feed into key decisions across the life cycle of the Programme, not just at the end. This approach facilitates evidence-based decisions and allows the Programme to adjust direction where necessary, based on up-to-date evidence.

To deliver this approach, our evaluation strategy works across four layers interchangeably:

Figure 1 – a series of nested and overlapping ovals to show the interconnectedness of the four layers of the evaluation strategy: Planning and Scoping, Cross Cutting Insight, Test and Learn, Evaluation.

Planning and scoping

The planning and scoping layer ensures that we are developing and iterating our evaluation approach with consistent principles and understanding.

This involves:

developing a ToC (see the section on Theory of Change) that sets out the vision for the HTP by describing what changes could be delivered (inputs), how these changes could work (mechanisms), and how this will support delivery of HTP’s goals (outcomes)
iterating an evaluation and monitoring strategy that provides a clear forward-looking evidence plan
working with delivery teams to understand emerging evidence needs

Cross cutting insight

The cross-cutting insight layer ensures that we are continuously learning about the context, levers and influencing factors that relate to the HTP.

This involves:

conducting research about a broad range of issues that affect our customers and services to fill evidence gaps and provide up-to-date learning
reporting against a set of performance and results indicators to build a picture of whether the Programme is achieving what it set out to achieve

Test and learn

The test and learn layer ensures that when we test new ideas, we are testing the right things at the right times and feeding the learning into Programme to influence design and delivery decisions.

This involves:

designing a process for logging, prioritising, and scoping new ideas for testing
implementing tests that build our understanding of new ideas whilst protecting customers and staff
ensuring we get robust learning from our tests by implementing proportionate evaluation and reporting

Evaluation of services

The evaluation of services layer ensures that, as services develop, we can understand to what extent the HTP has successfully achieved its outcomes.

This involves:

conducting ongoing evaluation work which will scale as the Programme scales to give an indication of impacts and the effectiveness of delivery and inform key decisions

As required, evaluation activity will include:

impact evaluations which tell us what effect the service has on customers and outcomes
process evaluations which tell us whether the delivery of the service is successful and sustainable over time
value for money assessments which tell us if the service delivers good use of tax-payer money and delivers on the business case set out for the HTP

It is important to note that the strands of work in the HTP will not move linearly across these four layers but will move back and forth between layers as we build the evidence base about what works. For example, we might undertake some evaluation and then move back to test some new initiatives based on what we learnt.

Data protection

DWP takes data protection seriously. Section 3 of the Social Security Act 1998 allows DWP to use information held for purposes relating to social security, employment or training, private pensions policy or retirement planning for purposes relating to any of these functions. DWP processes personal data for research using the public task lawful basis, GDPR Article 6 (1) (e). This is because, as a government department, we need to deliver our functions that government has assigned to us.

When processing personal data using this lawful basis we do not have to rely on the legal basis of consent to process personal data. Section 3(2) of the Social Security Act 1998 allows information to be supplied to persons providing services to DWP to use for these purposes.

A person or organisation which delivers services for DWP under contract is acting for and on behalf of DWP for the contracted purpose. In data protection terms, DWP remains the Data Controller and is ultimately accountable for how the data is processed, with the contractor acting as its data processor. To find out more about our purposes, how we use personal information for those purposes please see Personal information charter - Department for Work and Pensions - GOV.UK

All research and analysis will be carried out according to these requirements. Research participants will be made aware that their participation is voluntary, confidential, not linked to any dealings with DWP and will not affect their PIP or any other benefit award and is for research purposes only. All information provided by participants will be treated in strict confidence under GDPR (2018). Information provided by participants will be stored securely in line with data protection legislation with any identifying details from interview records being removed prior to storage. Interview records will be deleted by DWP once the research reports have been finalised.

Equalities analysis

Under the Public Sector Equality Duty, we are committed to monitoring the impact of the Programme to ensure that it doesn’t cause any unlawful discrimination. Demographic information on customer groups and their outcomes will be used to review the impacts of the HTP, typically by age, sex, and primary health condition. Data on other protected characteristics is not relevant for a benefit claim, and therefore information on these is usually not available for analysis. We therefore would not be able to estimate the proportion of PIP customers from these groups or evaluate the impact of the Programme on these groups. As routine, we present information at an aggregate rather than at an individual level. If any characteristic group is too small and could risk identification, we may suppress this data.

The nature of the HTP is such that in early stages of proofs of concept or tests there is unlikely to be any data to inform equalities considerations. Indeed, the purpose of such work is to build up the evidence on the characteristics of those affected by a change in various ways, so unanticipated disproportionate impacts for particular groups can be detected. In designing such tests, mitigations will be taken to ensure that predictable discriminatory impacts are minimised.

External rationale for evaluation and monitoring approach

The HTP sits under the Governments Major Project Portfolio (GMPP) meaning it is considered one of the most complex and strategically significant projects in government. The Government Major Projects Evaluation Review sets out goals across the Evaluation Task Force (ETF) and His Majesty’s Treasury (HMT) to improve the quality and coverage of evaluations across major government projects. The review suggests that operational, cultural and resourcing barriers are often present in the evaluation of projects in the GMPP.

Within this strategy we demonstrate how our approach is supporting us to minimise those barriers. By integrating evaluation planning from the start of the HTP, we ensure timely and effective data collection for monitoring and analysis. Early support from senior staff has fostered a strong evaluation culture, while ringfenced resources enable us to provide robust and accurate evidence of the Programme’s progress toward its strategic outcomes.

The National Audit Office’s (NAO) review of the HTP made several recommendations relating to data, evidence and evaluation:

NAO Recommendation: Build a clearer view of what data and evidence will be required from the new services for evaluation
- How we have addressed this in the evaluation strategy: This can be seen in our updated ToC section and more detail in the process and impact evaluation sections
NAO Recommendation: Set out how the new Health Assessment Service (HAS) will be benchmarked and use comparative data from the interim Functional Assessment Service (FAS)
- How we have addressed this in the evaluation strategy: we have explained how FAS is now the BAU service being delivered in the HTP scope and approach section. We have outlined the approach for benchmarking HAS, as part of the transformed HTP services, against the BAU service, in the full evaluation scope and impact evaluation sections
NAO Recommendation: Be transparent about the Programme and its performance by publishing: metrics it will use to monitor delivery of the Programme, regular data on the performance of services
- How we have addressed this in the evaluation strategy: we have set out the key performance indicators (KPIs) and the publication approach in the measuring performance sections

Finally, guidance issued by HM Treasury explains that business cases are unlikely to receive approval without a robust and proportionate evaluation plan, properly aligned with the delivery context of the specific project, programme or intervention. This strategy sets out how the evaluation of the HTP will be feeding into the Business Case.

Question 3

Planning and scoping

Accepted Answer

Theory of Change

Use of Theory of Change in HTP

We are using ToC to demonstrate what changes we plan to deliver (inputs), what we expect these changes to involve (mechanisms), what we expect to see as a result of the change (outputs), and how this supports delivery of our overall goals (outcomes). By highlighting the links between these things, the ToC surfaces the assumptions that sit behind an activity or intervention. Understanding these assumptions allows the Programme to assess the evidence base for its approach to transformation and highlights where we have unevidenced assumptions that need to be tested. The ToC forms the basis for our evaluation strategy by guiding what we will test, how we will test it, and what we need to measure to understand if we are achieving our goals. The ToC is a live document that evolves and updates as the Programme develops.

Since the publication of our last evaluation strategy in 2023, we have updated the ToC wherever Programme decisions have enabled us to be more specific about planned transformations and the inputs and mechanisms that will deliver them. These updates are most visible in the addition of defined logic models for the new PIP service and HAS, as well as in the revised challenges and risks where thinking and decision-making has progressed. We have also highlighted where evaluation measurements will be aligned with the Programme’s KPIs.

The ToC is being used across the different stages of the Programme to guide transformation and implementation. In the Programme’s current stage, stage 2 “Improving the Service”, the ToC is demonstrating where we still need to build more evidence about what works before scaling changes more widely. It is also supporting evaluation planning by guiding what we will need to measure to understand whether the programme has achieved its goals, and whether the changes are safe and sustainable. Using the ToC in this way supports the Programme to make evidence-based decisions about what to take forward, and ensures evaluation provides relevant learning about what the Programme has achieved.

Theory of Change logic models

ToC uses logic models to visually summarise how complex changes are intended to occur. A more detailed version of the ToC for HTP can be found at Health Transformation Programme Evaluation and Monitoring Strategy 2026 - GOV.UK.

The elements of the logic models can be understood as follows:

Inputs: Specific transformative interventions the Programme is delivering
Mechanisms: Explanation as to how the input will operate
Outputs: What we expect to see as a direct result of each mechanism
Outcomes: Which of the five overall strategic goals of HTP this intervention is expected to contribute to
Measurements: What we will measure to determine whether the change has occurred as expected
Indirect outcomes: What we will measure to monitor potential unintended outcomes of the change

HTP Theory of Change logic model

Figure 2 – Logic model demonstrating the inputs (communication, job design, data, processes, estates, contracts, IT and digital tools, staff capability, policy) and mechanisms (intelligent use of data, transparency, information gathering, case manager and HCP behaviour, accessibility, person-centred approach, accuracy) the Programme will integrate in order to achieve its five strategic outcomes. It highlights the importance of policy change as a mechanism to facilitate these outcomes.

New PIP service logic model

Figure 3 – Logic model demonstrating how the new PIP service will work, using digital self-service, tailored support, and improved evidence gathering to create simpler, quicker journeys and improve customer experience. It also shows the measurements that will be used to indicate how the service is performing and if outcomes are being achieved: customer feedback, journey times, volumes of disputes, Departmental Expenditure Limit efficiency, volumes of health assessments.

Health Assessment Service logic model

Figure 4 – Logic model demonstrating how the new integrated Health Assessment Service will use transformed technology and data, changes to customer interaction and staff behaviour and capability to deliver fast, accurate assessments with reduced customer burden. It also shows the measurements that will be used to indicate how the service is performing and if outcomes are being achieved: customer feedback, journey times, volumes of disputes, unit cost of service, decision quality checks.

ToC challenges, risks and dependencies

The HTP ToC also highlights the assumptions and conditions that underpin delivery of the new service, including challenges, risks, and dependencies that need to be considered in evaluation. These include operational dependencies such as the digital build of secure IT systems and effective data sharing, alongside behavioural assumptions such as customer digital propensity and staff capability and collaboration. The Programme also faces other challenges, such as balancing improved efficiency with decision quality and accuracy, defining and designing appropriate measures, and being aware of any indirect outcomes. Additionally, we operate within a changing policy landscape, where shifts in policy or priorities could influence how and when changes are implemented. These factors will guide what we measure and how we interpret findings, ensuring evaluation provides meaningful insight and reflects the complexity of the Programme and its operating environment.

For example, since the last evaluation strategy in 2023, we have adapted our approach to measuring customer experience and trust by aligning with the metrics used across DWP. This ensures consistency with wider departmental measurement, allowing for comparability across services and supporting joined up learning across the department, as well as making use of an approach that is already developed and evidence-based. Further detail can be found in the measuring customer trust and experience section.

Question 4

Cross cutting insight

Accepted Answer

Research programme

Overview of approach

To ensure that transformation activity is informed by robust evidence, we deliver an ongoing programme of internal and external research that aligns to the Programme’s design and delivery plans. We have already accumulated a wealth of relevant evidence that we continue to add to and review.

Our research programme is run by professional Government Social Researchers. It’s developed iteratively through engagement with a broad range of stakeholders, both inside DWP and externally. Reflecting the agile nature of the HTP, it’s under constant review to ensure timely delivery against priorities.

We focus on inclusive research practices and work extensively with people with lived experience of disability as well as representative organisations to shape both research design and content.

We use conventional social research methods as well as conducting innovative behavioural insight studies. These rapid projects deliver focused evidence on specific elements of service design to inform development activity and are a key feature of the strategic programme of research.

We carry out research in DWP’s BAU services as well as in the test and learn areas and in experimental conditions, set up specifically for research purposes.

Examples

Research with PIP customers

We recently published two, complementary research reports, one examining the experiences of PIP customers with additional needs (May 2025) and one on the experiences of PIP customers with a pre-disposition to anxiety (October 2025). Both aimed to deliver evidence about stressors in the PIP process and ways we can make changes that benefit all customers.

Drawing across the studies we found:

PIP customers begin the application process with high baseline anxiety and stress. They often delay applying until the impact of their health condition or their financial situation has deteriorated
mental health and developmental conditions amplify challenges, creating cognitive overload and heightened anxiety throughout the process
many PIP applicants do not self-identify as disabled and perceive stigma in claiming benefits, which can feel disempowering and demotivating

There are challenges associated with different stages of the PIP journey.

Registering a claim:
- Uncertainty about eligibility and process triggers anxiety
- Fear of phone calls and unfamiliar interactions adds stress
- Lack of clarity on questions, how to answer and options for paper/digital forms
Completing the Health Information Gather (PIP2) application form:
- Length, complexity, and irrelevant questions (especially mobility-related for people who do not have physical conditions) cause stress
- Providing evidence for mental health conditions can be difficult
- Reflecting on traumatic experiences can be re-traumatising
- Applicants often lack awareness or access to practical support
Assessment:
- Applicants experience anxiety about being judged or disbelieved by assessors unfamiliar with their condition
- Uncertainty about format, timing, and questions heightens stress
- Emotional support needs peak here but are often met only through friends or family
Receiving a Decision:
- Full PIP Awards bring relief; partial or nil awards can cause frustration and confusion
- Customers feel decisions lack transparency and have poor understanding of how to appeal
- Award reviews for long-standing conditions create surprise and distress

The research identified a number of recommendations aimed at improving the experience of customers navigating the PIP application journey. These recommendations are being considered as part of ongoing transformational design activity to ensure future services better meet the needs of all applicants.

Behavioural Insight studies:

As HTP is a long, large and complex programme we have been carrying out targeted, small-scale tests and exploratory insight studies, alongside user research, to inform decisions around specific processes and design aspects of the PIP and health assessment journeys. The evidence generated through these Behavioural Insights studies has been critical ahead of formal evaluation activities taking place when scaling allows, as detailed in this Evaluation Strategy.

A compendium of Behavioural Insight studies undertaken to mid-2025 is being published alongside this Evaluation Strategy. We continue to deliver a rolling programme of Behavioural Insight studies, summaries of which will be published in due course.

Resourcing

We have a ringfenced budget for research and evaluation to support the HTP each year.

Since Spring 2025 we’ve had a call-off contract with the National Centre for Social Research (NatCen) which enables us to commission new research efficiently and at pace. NatCen have extensive social and behavioural research expertise and are adept at conducting research with disabled people to deliver to robust standards required in government. From 2022 to 2025 we had a similar call-off contract with Basis Social.

Some of our research may be competitively tendered outside of the call-off contract. For example, if we require a specific skill set, tool or technique we may issue an invitation to tender to ensure best value for money.

Performance monitoring

Measuring performance

Performance framework

As the HTP continues to evolve, it remains essential that we monitor progress and assess whether the Programme is delivering its intended outcomes. To support this, we have updated our performance framework, which links critical success factors to a robust set of performance indicators.

The indicators within the framework are regularly reviewed and developed as the Programme matures and new data sources become available, allowing us to provide greater detail, introduce additional measures where needed and remove some measures where plans and data availability have changed.

The aims of the performance framework are to:

provide a consistent approach to measuring progress over time and across all strands of the HTP
align with DWP-wide approaches to performance measurement, enabling robust comparisons with BAU services where appropriate
support agile delivery by feeding reliable and consistent data into the Programme regularly, helping to identify areas that require attention and drive forward decisions
enable the Programme to tell a clear and consistent story of progress against its strategic outcomes to internal and external stakeholders

Key Performance Indicators

Supported by additional metrics, there are 8 top-level KPIs in the performance framework. These KPIs are used to monitor the HTP performance against its strategic outcomes. We will continue to review these KPIs to ensure they remain the most relevant measures to monitor as the programme evolves.

Health assessment reports quality
Unit cost of service
Assessment provider capacity and demand
Productivity
Contracted staff attrition
Customer journey time by milestones
Cases cleared within benchmark
Starts to work programmes

KPIs and strategic outcomes are interrelated and can be mapped to each other; these relationships are often bi-directional, with one strategic outcome potentially linked to multiple KPIs.

Figure 5 – A mapping of the top 8 KPIs showcasing how they feed into the HTP strategic outcomes.

Our ability to publish these metrics is underpinned by ongoing work to ensure that our data design and collection is consistent, robust and relevant. This enables us to support our current performance monitoring, as well as future analytical requirements which may emerge as the Programme develops.

Publication approach

Current publication:

At present, two key metrics are published as part of the HTP’s quarterly Management Information (MI) release, with the latest edition published in March 2026:

Number of referrals for a PIP assessment in the Health Transformation Area (HTA) postcode groups
Number of customers applying for PIP via the new digital self-serve (DSS) GOV.UK channel

The latest publication is available at Health Transformation Programme Management Information to January 2026 - GOV.UK

The purpose of publishing these metrics is to provide transparency on service delivery and uptake, enabling stakeholders to monitor progress against strategic outcomes and to support continuous improvement. These metrics offer insight into the volume of assessments being processed in-house and the adoption of digital channels for PIP applications.

Future strategy for reporting performance metrics:

We plan to broaden the range of metrics and strands of the HTP we report on externally to ensure information remains robust, relevant, and aligned with the Programme’s evolving priorities. Additionally, when customer events such as registrations, clearances and mandatory reconsiderations within the new PIP service scale to reach 1% of total national new PIP claims, we will begin publishing metrics that align with the official PIP statistics. This is dependent on services stabilising and data quality improving.

The latest official PIP BAU statistics can be found at Personal Independence Payment statistics to January 2026 - GOV.UK

Business case

The Programme Business Case sets out the benefits and costs of different options under consideration in the programme. The appraisal process supports decision making by providing a framework to compare costs and benefits of policy options and indicate a preferred option, with guidance from the HMT Green Book.

Management of benefits ensures that the measurable value and other positive impacts resulting from the programme are identified, planned and tracked so that they can be realised in practice. The HTP Performance Dashboard metrics are used in this work.

The Programme expects to realise various types of benefits, including:

delivery costs (within the Departmental Expenditure Limit (DEL)) such as reducing IT expenditure, improving service efficiency, and enhancing service quality
wider societal benefits such as improved customer experience across the end-to-end journey and the sustainability benefits of digitised processes

The approach to appraisal of benefits is also applied to costs, the balance of which is expressed as the Programme’s Net Social Present Value (NPV). This modelling is based upon data from various areas within HTP and is quality assured using Aqua Book standards.

Evaluation and appraisal are closely integrated throughout the life of the programme, creating a continuous feedback loop as the Programme iterates. The Business Case includes details of the programme’s monitoring and evaluation arrangements, which will be reviewed and updated as part of regular reviews of the Business Case.

Question 5

Test and learn

Accepted Answer

Role of test and learn in agile programmes

The HTP has used test and learn guidance from other major DWP Programmes such as Universal Credit and the Government Digital Service to build on best practice when designing our approach to developing and testing new ideas in the HTP. The HTA provides a test and learn environment that enables the Programme to safely test new ideas. Given the expansive nature of HTP, we also test beyond the confines of the HTA to include BAU sites and other testing areas such as the Exploration Hubs (pilot environments where we are exploring new ways to support disabled people and people with health conditions to move towards and into sustainable work). Being flexible allows us to quickly provide insight about what works and build the evidence base that sits behind our ToC; this enables the Programme to make informed changes to the way we are designing and delivering transformation.

Test and learn principles

To ensure our test and learn activities produce high quality, innovative insight which drives behavioural, system, and process changes whilst protecting our customers and staff, we use 5 principles to guide the HTP test and learn activity:

Testing needs to be informed by the ToC
Build in engagement with a diverse range of stakeholders from the beginning
Identify and design tests with the transformed service in mind
Protect staff and customers from risk
Use a consistent and appropriate system of governance

Testing needs to be informed by the ToC

The ToC sets out the kinds of change needed to drive success and should remain the overarching reference point for all testing.

Build in engagement with a diverse range of stakeholders from the beginning

Engaging with a broad range of internal stakeholders and consultations with external stakeholders (including small numbers of customers) at an early stage ensures that we are driving the right behavioural and process changes and are building the right service for the people who need it.

Design tests with transformed service in mind

When designing tests, we keep the final transformed service in mind. This means planning how our test and learn activities can be utilised in a scaled-up service. Planning for the future and preparing to involve evaluation colleagues.

Protect staff and customers from risk

Staff involved in test and learn activities are trained to deal with difficult situations that could occur when working with customers, meaning that tests adhere to the same standards for supporting customers and staff that are applied to all DWP services. Our research will also adapt common safeguarding protocol in the DWP for ensuring the safety of customers and staff taking part in the test and for analysts conducting the research.

Use a consistent and appropriate system of governance

Sufficient oversight is in place to scrutinise all the strands of the Programme and give a clear picture of what we are learning. Testing is managed to maximise available capacity while avoiding any potential contamination caused by running multiple tests at the same time. We work closely with the Programme’s Design Authority to steer and scrutinise our testing prioritisation to deliver the Programme’s strategic outcomes.

Prioritising and designing test and learn

Alongside using the test and learn principles, our growing evidence base, policy priorities and operational deliverability help the Programme to prioritise what gets tested and when.

Our test and learn activities take different forms depending on the stage of intervention development and the scale of learning that is required. We use a range of methods, ranging from exploratory research to structured proofs of concept, and robust trials. The method of learning associated with any test is informed by the scale of the decision that the test will inform, the risks associated with the intervention, and the existing evidence base; all these factors are informed by the ToC.

The types of things we are exploring through test and learn include:

How should we clearly communicate with our customers to build their trust in our services and decisions?
How can we tailor the health consultation to improve customer experience, provide a more personalised service, and improve workflow for DWP staff?
How can we use data more effectively to decrease journey times and deliver better value for money?
How can we ensure our staff have the right skills and training to best support improved customer outcomes?
How could our processes be more person-centred to accommodate different needs and improve customer experience?
How can DWP’s IT systems and digital tools help the HTP services to be more flexible to future customer and service needs?

Note that these are simplified examples and not exhaustive.

The Pathways to Work Green Paper demonstrates the Government’s commitment to supporting disabled people and people with health conditions. The desire to help those who are able to work, whilst ensuring safeguards for those unable to do so, informs the prioritisation of test and learn activity in the HTP.

Role of evaluation in test and learn

In the HTP we use ‘test and learns’ to conduct evaluations on principles or ideas at a small scale, to allow us to refine our service design approach and provide confidence before expanding to larger scale evaluations. This means we do not waste taxpayer money and Departmental resources on developing the capability to deliver a system change that might ultimately fail to deliver the desired outcomes.

The learning from these tests feeds directly into our evaluation activity. Where a test shows that a principle or idea works well, and will be taken forward into the design of the Programme, it will be evaluated in full. In this way, test and learns inform both the future direction of the Programme, and the focus of long-term evaluation.

Question 6

Full evaluation of services

Accepted Answer

What is ‘full evaluation’?

Once the HTP services have been tested and reached a stable service design, they will be scaled to an appropriate volume to conduct robust formal evaluation and inform any decisions around further scaling. Evaluation of the HTP services in this context will determine:

whether the change is being implemented as intended and what is working well and less well for different user groups. This kind of evaluation is referred to as a process evaluation
what changes have occurred because of what the HTP has delivered, including any unintended outcomes. This kind of evaluation is referred to as an impact evaluation
a comparison of the benefits and costs of the changes the HTP has introduced. This is referred to as a value-for-money evaluation

Full evaluation scope

The full evaluation of the HTP will involve process, impact and value for money evaluation to determine whether the Programme is meeting its strategic outcomes and demonstrating cost-effectiveness. The focus will mainly be on the two primary delivery strands of the HTP: New PIP service and HAS. Throughout the evaluation the transformed PIP services will be compared where possible, to the BAU service. For example, the HAS service will be compared to the BAU FAS service.

In addition to this, certain specific interventions or clusters of interventions may have a full formal evaluation associated to them due to the risk they carry or because they are working to a different scaling timeline to the rest of the Programme. For example, a full evaluation of the fully digital channel to apply for PIP (Digital Self-Serve) has been conducted. See the full evaluation case study section for more detail on how this was conducted.

Full evaluation timelines

We will be conducting full evaluation at multiple timepoints to reflect the gradual scaling and maturity of the transformed services:

Learning on the early stages of the HTP will be gathered as part of the interim process evaluation to feed into decisions as the Programme transitions from stage 2 “Improving the Service” to stage 3 “Scaling the service”
The full process evaluation and impact evaluation of the HTP will begin and run throughout the course of stage 3. There will be an interim reporting point mid-way through and final reporting point at the end of stage 3
Publication of the final report will be in line with Government Social Research (GSR) protocols

At the time of publishing this strategy, we expect to have sufficient volume of customers receiving the new PIP service and HAS by the end of stage 2 of the Programme to conduct a robust evaluation in stage 3 of the fully transformed PIP customer journey.

Once a statistically sufficient volume of claims has gone through the service and received an initial decision, the evaluation will begin to report outcomes from the service in comparison to the BAU services. The length of the PIP customer journey means that these initial outcomes and comparisons will take several months to become available. Outcomes and findings for later stages of the journey such as disputes will take longer.

An outline of the measurements expected to be required for full evaluation of HTP against its strategic outcomes, including balancing measures of unintended consequences can be found in the ToC.

The following sections set out more detail on what work will be involved in the three kinds of evaluation and how they are planned to be delivered.

Process evaluation

Process evaluations tend to examine activities involved in an intervention and the pathways by which the change was delivered.

Examples of the kinds of process evaluation questions that will be addressed in the HTP evaluation include:

Do customers experience any barriers to accessing the services?
What worked well and less well for which customers when using services?
What is the role and perceived impact of enhanced digital products and systems?
Do staff have the right resources to deliver the service as intended?
How is collaboration across delivery partners and DWP working in the new transformed service?

Process evaluations cover a mix of subjective issues (such as perceptions and experience) and objective issues (such as processing times and clearance volumes) meaning that they employ a wide range of methods. This flexibility of methods mean that process evaluation can provide valuable learning throughout the life cycle of the Programme, including before services are operating at scale, or service design has been finalised.

Process evaluation of the new PIP service and HAS will involve combining MI and service level data with findings from research (e.g surveys, interviews and observations) with PIP customers and a range of DWP and contracted staff to build a rich understanding of how the transformed service is operating and how it is being experienced by different kinds of users. Where possible, we will look to compare with BAU services using existing measures of customer and DWP staff experience. We may also include, if appropriate, PIP telephone claims line contracted staff to gather their experiences of delivering the BAU service.

Given the resource intensity of conducting process evaluation, we will look to externally commission large parts of this work through evaluation funding.

Below we provide an overview of our plans for the first interim process evaluation stage, as an example.

Interim process evaluation

We will conduct interim process evaluation during stage 2 of the Programme “Improving the Service” as services develop to provide learning about how the HTP is delivering against its plans.

The interim process evaluation will utilise administrative data from the services (provided via the HTP performance dashboard and other sources of MI) and qualitative data from staff and customers. The aim is to build a clear understanding of how the HTP services are being developed and understand any barriers and enablers to delivery. The learning will provide the Programme with the opportunity to learn and adapt its delivery plans and increase likelihood of success against its plans.

The current proposal for interim process evaluations is as follows:

HAS: Process evaluation focused on contracted assessment staff, taking place once all provider sites have gone live and stabilised, reporting findings ahead of further scaling. This will be qualitative work focusing on the behaviours of provider staff; their experiences delivering the transformed service; perceptions of the new digital systems and how this relates to HAS achieving its proposed efficiency value releases.
New PIP service: Both a customer and DWP staff-focused process evaluation. For customers this will take place once the majority of case types are being retained in the transformed service (when the service is operating at c.1% of national volumes), before PIP expands into a second site processing claims. This qualitative work will focus on the customer experience of the new PIP service in relation to how tailored and supportive they find it, increased levels of self-service and journey times. For DWP staff we will focus on their readiness for, and experiences of, delivering a transformed service, perceptions of the new digital systems, and whether the anticipated cultural and behavioural changes are being demonstrated.

Customer and staff experience metrics surveys

As part of the HTP process evaluation customer and staff experience metrics surveys are being conducted, though with different timings and objectives.

The HTP ToC highlighted the importance of understanding the experiences and trust of PIP customers, DWP staff and contracted provider staff. We aim to gather robust data over three waves of fieldwork using commissioned customer surveys and internal staff surveys and will compare against BAU findings for each. More information as to the purpose and method of researching customer trust can be found in the measuring customer trust and experience section and the evaluating cultural and behavioural change sections of this strategy.

The customer and staff experience metrics and the wider process evaluation activities are for distinct purposes: the metrics surveys are to provide a set of quantitative comparisons to BAU over three points in time. The interim and full process evaluations are to explore experiences of transformed services through a focus on the new services being delivered, there may be some comparison to BAU but it is not the main focus.

See Figure 6 for how the different evaluation activities fit together.

Figure 6 – Illustrates how the Customer Experience Metrics Surveys feed into and align with wider HTP evaluation activities. The initial wave of surveys and the Stage 2 process evaluation will report at the end of Stage 2. The subsequent two survey waves, as well as the two waves of the Stage 3 process evaluation and the impact evaluation, feed into Stage 3 of the Programme. Interim findings will be reported mid‑way through Stage 3, and final findings reported at the end.

The HTP process evaluation risks/dependencies

There are a number of risks and dependencies for the process evaluation which we recognise could hinder our work and timings:

Process evaluation will require access to both DWP and contracted assessment provider staff for research purposes. The expectation of the evaluation teams is that this resource will be facilitated by Programme and the operational sites involved. The resource and access requirements also need to be included in any change requests made to assessment providers.
Commissioning evaluation work carries significant lead-in times. To ensure timely delivery of process evaluation we require the Programme to give a clear view on timing and scope of full evaluation with a minimum of 6 months’ notice to facilitate necessary social research procurement and planning.
With the new PIP service and HAS scaling on separate trajectories it means that the volume of customers receiving the fully transformed service, and our priority focus for evaluation, remains smaller for longer. This will have a consequence for our quantitative research planned, especially the customer experience metrics surveys. We will work closely with Programme colleagues to mitigate challenges to sampling for the first waves of the survey until volumes have increased, but we acknowledge that the early findings may be limited in their representativeness.
Comparable data between transformed and legacy services. In some instances, it may not be possible to simply compare HTP and BAU outcomes. For example, if a new quality framework for health assessments is introduced along with a new audit procedure, the new and old audit scoring systems are unlikely to be comparable.

Impact evaluation

Impact evaluations focus on the changes caused by an intervention and assess the measurable achievements that drive or contribute to the objectives of the intervention.

Examples of the kinds of impact evaluation questions that will be addressed in the HTP evaluations are:

Do customers have shorter claim journey times when using the HTP service than when using DWP BAU services?
What is the impact of greater accessibility of the HTP services on overall demand, compared to DWP BAU services?
Is there any difference in outcomes for customers using the HTP services compared to DWP BAU services?

In the HTP, impact evaluation will assess whether the changes caused by the HTP interventions result in delivery of the strategic outcomes and whether there are any unintended outcomes of the changes that need to be considered before services can scale further. Unintended consequences could include fiscal implications of the changes such as changes in the award levels, or deliverability implications such as increased demand for a service.

To provide this measurable and objective information, the HTP impact evaluation will utilise experimental and/or quasi-experimental approaches. This kind of approach infers the impact of an intervention through statistical comparison to a group or time period which is unaffected by the intervention. This unaffected group acts as a proxy for what would have happened to the affected group if the intervention hadn’t occurred. It is referred to as the counterfactual and can be measured using a control group. It is important that the control group and the group that receives the intervention (also known as the treatment group) are comparable in that they have similar characteristics and can be expected to perform similarly if given the same treatment, so that we can attribute any differences in outcome to the intervention.

There are certain requirements of an intervention or service that need to be met before an impact evaluation can produce robust conclusions about its causal effects. The requirements are:

The design is reflective of the end stage design or delivery model
The service is stable and not changing during the evaluation window
The service is operating at sufficient scale to meet sample size requirements for statistical analysis and to mimic conditions of full roll out
Those in scope of both the treatment and control group are representative of the wider population so that evaluation results are generalisable

Implementation and planning considerations for impact evaluation

Given the requirements set out above, impact evaluation will take place once the HTP services have been designed, tested and iterated. Implementation planning for the Programme is accommodating impact evaluation requirements as needed to ensure robust evaluation can take place at the right time.

The key planning and implementation considerations for ensuring effective impact evaluation are:

Site selection:
- Where geography plays a role in the allocation of cases to control or treatment groups (i.e. certain claims have to be processed within certain geographic restrictions), it is important that the geographies of sites selected are not likely to significantly skew results. For example, consideration of rural vs. urban areas, socio-economic factors etc.
- Operational factors such as performance also need to be considered. For example, particularly high or low performing operational sites would impact the generalisability of results. And where part of a site is utilised, the staff involved are a representative selection e.g in terms of experience.
Customer journey times dictate the length of time it takes to gather evaluation findings. Impact evaluation evidence cannot be delivered faster than it takes for a sufficient volume of claims to progress through the claim journey in both the treatment and control groups. Planning timelines need to account for the time from evaluation beginning to the evidence being available to inform scaling decisions. In order for us to be confident our sample is representative we need 90% of the customer cohort to have reached a decision. This is because the percentage awarded (of those cleared so far) increases as more of a cohort is cleared. At the time of publication, the median clearance time (where 50% of a customer cohort has been cleared) is around 5 months for an initial decision but it takes around 8 months to receive sufficient evaluation data up to initial decision and upwards of 20 months to receive equivalent data on appeals. If we used the median clearance time as our benchmark we risk using evaluation data that is not representative of the full cohort.
Wider operating environment: to attribute the outcomes we see in the evaluation to the changes delivered by HTP, we need to be confident that the outcomes are occurring because of what HTP has delivered, not other external factors. Therefore, the operating environment of the control and treatment groups needs to be consistent during the evaluation window. For example, no major operational changes are happening in other parts of the organisation that would affect the operation of the service being monitored in the evaluation.

Counterfactuals and baselines in the HTP impact evaluation

Counterfactuals

A counterfactual outcome is what would have happened if the treatment was not delivered. For example, a HTP customer’s award outcome if they had gone through a BAU version of the service. Counterfactuals are not observed so they need to be estimated using a control group which experienced the service you are comparing to.

The HTP is transforming services over a long period of time and supporting ongoing development of non-HTP policy and practice throughout the life of the Programme. This means that the broader operating and policy environment will change over the life cycle of the Programme. This makes it challenging to simultaneously compare outcomes of the HTP services to a completely non-HTP control group and thus demonstrate the full impact of the HTP. The counterfactuals selected for the HTP evaluation of new PIP service and HAS will therefore be as close as possible to the pre-HTP operating environment but will likely have already adopted some HTP-facilitated change.

Baselines

In addition to having a robust comparable counterfactual in the form of a control group for impact evaluation, it is also valuable to have a time series of baseline data for key measurements. A time series of baseline data demonstrates trends and fluctuations prior to an intervention being delivered and gives greater confidence that the findings of the impact evaluation are a result of the intervention, rather than any external factors. This kind of data is available from DWP administrative data sources for many of the key measurements in the HTP evaluations (for example: journey times, referrals to assessment provider, award rates) and will be utilised during evaluation.

The HTP impact evaluation risks/dependencies:

There are a number of risks and dependencies for the impact evaluation which we recognise could hinder our work and timings.

Access to data about transformed services. It is imperative that data from new digital platforms is fed through to DWP administrative data systems and can be used by analysts for comparison to legacy services as mentioned in the Data Development and Future Proofing section. However, the data markers required for evaluation may change over time and as such may not be fully designed into the data infrastructure. This could limit the level of detail evaluation can produce.
The roll-out of the new PIP service and HAS progressing over stage 2: “Improving the Service” to agreed delivery plans, timelines and areas, is key to ensuring we have representative data at a sufficient volume to allow impact evaluation in stage 3: “Scaling the Service”.
Customer journey times remaining stable, to ensure a sufficient proportion of a cohort of claims (based on registration date) reach decisions to give representative data to evaluate the impact of new PIP service and HAS on outcomes within the required timeframe.

Value for money evaluation

Value-for-money evaluation methods compare benefits to the costs of interventions, including adverse and unintended aspects. Quantified estimates of the outputs or impacts of alternative interventions are usually derived from experimental, quasi-experimental or synthesis methods. Without a quantified estimate of the net effect, theory-based methods can be used to assess whether the impact is likely to be of the size consistent with breaking even.

The costs of the alternative interventions under consideration must be explored in detail to demonstrate which will provide greater return on investment. These costs should include social value and social costs, such as employment, health, wellbeing and productivity where possible. Distributional analysis should be used to consider costs and benefits on different population groups.

For the HTP value for money evaluation we have set the groundwork to deliver this by collecting and analysing data as part of the impact evaluation – customer journey times, overall demand for the service, DSS options and transformed organisational design can all be quantified into costs and compared against the BAU service. There will be supplementary data from the process evaluation such as efficiency of resource use, operational challenges and identifying areas for improvement. Finally, value for money is assessed in the HTP Programme Business Case, in line with HMT Green Book guidance and principles.

As value for money evaluation happens at the end of a Programme, we will continue to develop our plans as the HTP continues to scale, the process and impact evaluations start to produce findings and the HTP Business Case guides the value for money framework.

Full evaluation case study: Apply for PIP digital self-service

A recent example of impact and process evaluation within the Health Transformation Programme is the evaluation of the fully online application route, sometimes described as the Apply for PIP digital self-service (DSS). This application route is one of the ways that HTP is transforming PIP, by introducing greater channel choice and enabling customers to digitally self-serve throughout their claim journey. A key part of this offer is the ability to start a PIP application online. The HTP made the DSS PIP application route available to postcodes that accounted for approximately 8% of volumes in England and Wales. A multi-stage process and impact evaluation was designed to assess the new online service and understand its impacts on customers and DWP.

To provide a holistic evaluation of DSS the process and impact evaluations worked closely together. When the impact evaluation highlighted an increase in claimants scoring zero points across both daily living and mobility components and revealed a gap in evidence, the process evaluation designed targeted qualitative research with staff to explore this further, providing timely insights for the Programme.

The evaluation found that demand for PIP increased in DSS rollout areas, with registrations estimated to be 17% higher than in non-DSS areas. DSS customers reported greater flexibility and reduced mental burden when using the online application route. Referrals to assessment providers were 18% higher in DSS areas. DSS cases were also more likely to be disallowed or withdrawn, with disallowances 35% higher. Staff acknowledged the benefits of digital applications but emphasised the need for non-digital application routes for customers without digital skills, connection or access.

DSS claims scoring zero points were 32% higher than in non-DSS areas. Staff identified limited differences in specialist input and medical evidence in digital zero-point claims and expressed concern about early identification of such cases. Mandatory reconsiderations (MRs) post-assessment were 14% higher in DSS areas, although DSS customers were less likely to register an MR. Customers were receptive about a future digital MR service but felt their decision to register an MR would not be influenced by the availability of an online route. Appeals following MR were 12% higher in DSS rollout areas.

Qualitative research showed that customers across both DSS and non-DSS routes found initial questions in the PIP1 form to be straightforward, though many described the health questionnaire as lengthy, emotionally demanding and often requiring breaks due to fatigue or the need to gather evidence. Customers with mental health conditions or certain physical conditions reported difficulties with the relevance of the activity questions and delays in obtaining medical evidence were a common barrier. Despite these shared challenges, DSS customers reported clearer benefits which included faster application starts and submission, reduced mental burden compared with the phone or paper routes and greater flexibility and convenience, particularly due to being able to start and submit applications at any time.

DSS continues to operate at the same small scale and it will be rolled out to all new PIP customers nationally by the end of the Programme, in 2029. In 2024, we published a summary of evidence to date, including process and impact evaluation findings. More recently, a published report provides an updated view with more detail of the evaluation findings (see ‘Apply for PIP Digital Self-Serve: Evaluation Findings’, March 2026).

Question 7

Measurement of key outcomes

Accepted Answer

This chapter sets out current thinking on how we will approach measurement against HTP’s five strategic outcomes as well as some other potential unintended or enabling outcomes.

It is important to understand that HTP transformation is being landed in a complex landscape of policy and operational delivery as well as being delivered alongside several other transformational change programmes. Decision making about scaling HTP services will ultimately be considered alongside this wider context. To support this, HTP evaluations will utilise existing measurement frameworks wherever possible, building comparability of measurements that internal stakeholders are familiar with.

Measuring customer trust and experience

The strategic outcomes ‘Increased customer trust in our services and decisions’ and ‘Improved customer experience’ will be measured in alignment with the Customer Experience and Trust metrics used by across DWP. This approach is based on a set of evidence-based Customer Experience drivers that define what contributes to a positive experience for individuals engaging with DWP services.

‘Get it Right’, ‘Ease of Use’, ‘Communicate Clearly’ and ‘Professional and Supportive’ are the four key drivers of customer experience. Therefore, any transformations targeting improved customer experience should be making improvements in one or more of these areas in order to expect to achieve the outcome. As set out in the approach, being trusted is seen by customers as a potential consequence of delivering against the four other drivers. If these four drivers are met, it’s more likely that customers will have improved trust in DWP’s services and decisions. It’s important to note that some factors that influence a customer’s trust in DWP sit outside of the scope of the HTP. These may include experiences with other benefit lines, the eligibility criteria for benefits, media coverage, and second-hand accounts from others.

We will utilise these drivers to measure customer experience and trust in the HTP evaluations. For example, in the full evaluation of services, we will use surveys of the treatment (customers receiving the new service) and control (BAU customers) areas to measure differences between the two groups against these drivers.

The Customer Experience Survey (CES) is a quarterly survey of DWP BAU services which has a series of questions used to measure these drivers. We will use the same questions in the HTP evaluation surveys to ensure consistency of measurement. When one or more of the four drivers shows a statistically significantly more positive response in the treatment group than the control group, we will consider an improvement in customer experience to be attributable to the HTP service. Where each of the four key customer experience drivers demonstrates a statistically significantly more positive response in the treatment group than the control group, we will infer that it’s likely that customer trust has also been improved.

This work will also inform future evaluation plans and measurement. In qualitative evaluation and service testing we will map customer responses or themes emerging from those responses to the relevant experience and trust drivers. This will help us understand which elements of customer experience are being influenced by the interventions we are testing or evaluating. The planned qualitative research will provide additional depth to the test and learn activities. This will build consistency across strands of evaluation and help to build a framework for understanding what works.

Measuring efficiency

The evaluation will assess whether the Programme is creating a more efficient service for taxpayers, using changes in unit cost as the headline measure and tracking supporting indicators to evidence improvement over time.

The Programme will measure efficiency through a combination of headline and supporting measures that demonstrate whether the transformed service is delivering improved value compared to BAU.

The primary measure will be the change in unit cost, calculated as the cost to operate divided by the number of claims received. This provides a clear indicator of whether the transformation is achieving cost-effectiveness at scale.

Underneath this, a set of more specific measures will be monitored, including:

transaction times across customer journeys to understand end-to-end journey times as well as identify areas for improvement
percentage of automated journeys, to assess the extent to which digital solutions reduce manual intervention
number of staff interventions per case, to track operational efficiency and consistency in case handling

Efficiencies will also be monitored and reported through the benefits management process. Over time, this approach will provide robust evidence of efficiency gains and inform continuous improvement.

Measuring employment outcomes

Increased take-up of wider employment support is a strategic outcome for the Health Transformation Programme and as such the Programme is exploring which levers it has at its disposal to provide a health employment offer for PIP customers and better join-up with existing employment support offered in DWP.

The shape of this offer and level of integration is dependent on testing. There will be learning locations known as Exploration Hubs, strategically positioned to align with existing service delivery locations and all sites reflect a broad spectrum of demographic and regional diversity. These Exploration Hubs allow for Test and Learn activities which will be able to report on the delivery and experience of these employment support initiatives and feed into iterations of the design. Following this, if any of the initiatives are viable for inclusion in the new PIP service they will consequently be in scope for the Full Evaluation of Services (Process and Impact Evaluation).

We will evaluate whether the strategic outcome of increased take-up of wider employment is being met by measuring customer and colleague experience, awareness, and engagement through a mixed methods approach of quantitative and qualitative research with PIP new service customers and a range of DWP and specialist staff delivering the transformed PIP service, as well as through monitoring outcomes via our performance frameworks and impact evaluation.

Evaluating cultural and behavioural change

As demonstrated in the ToC, a number of the Programme’s strategic outcomes are dependent on and facilitated by cultural and behavioural changes. This requires some significant developments to the way that DWP and its people operate to improve the way benefits are claimed and improve the way customers interact with the department.

Staff

The Programme aims to achieve strategic outcomes such as:

more efficient service with reduced demand for health assessments
increased take up of wider support and employment
increased customer trust in services and decisions

For these outcomes to be realised, the HTP is working to transform the culture and behaviour of DWP delivery staff. Staff will receive regular training and have accessible resources enabling them to deliver their roles, and to use new tools and systems confidently. They will also further develop soft skills such as empathy, innovative thinking and adaptability to provide clear communication to customers. Staff are also expected to be engaged with the wider DWP vision and values. Measuring these initiatives is essential to confirm they are producing the desired behavioural changes.

The HTP evaluation will deliver data on what is working well, areas for improvement and the contextual environment in which these changes take place, such as the operational conditions of individual DWP sites. We acknowledge that there may be difficulties in measuring behavioural changes due to their inherent subjectivity. Therefore, we need to have a considered approach to measuring behavioural change, to ensure our findings are robust, substantive and will effectively improve the Programme as a whole.

We aim to explore staff behavioural change in the interim and process evaluations. For DWP delivery staff e.g case managers, qualitative research will be conducted with in line with key Programme scaling points. We will use interviews and focus groups to gather feedback on their experience of delivering the new service and their understanding of the expected cultural and behavioural change. We also plan to conduct interviews with senior Programme colleagues to understand their expectations of cultural and behavioural change. Quantitative research will be gathered via a HTP staff experience metrics survey investigating staff skills, behaviours and experiences in delivering the new service.

We will use established methods, such as metrics from the Civil Service People Survey, to assess behaviours relevant to the HTP strategic outcomes. Relevant metrics include ‘use of new technologies’, ‘collaboration across roles’, ‘provision of holistic journey to customers’, and ‘effective in-house decision making’. The HTP Staff Metrics Survey will incorporate questions from these categories to track behavioural change over time and benchmark against the Civil Service People Survey.

For contracted HAS provider staff we will explore cultural and behavioural change through a case study approach via qualitative research to analyse the differences between the providers and their operating environments. The qualitative research will complement the staff metrics collected within DWP and provide a broad understanding of how the Programme’s intended changes and new ways of working are embedding in practice. We will remain flexible in our approach and may consider quantitative research in Stage 3 if it would add value.

DWP customers

We will observe and evaluate any behavioural changes exhibited by DWP customers when they interact with the transformed PIP service. This will provide evidence on the strategic outcomes: ‘Increased customer trust in services and decisions’ and ‘Improved customer experience with shorter journey times’.

To achieve these outcomes, the Programme expects to see behavioural changes such as a deeper understanding of the PIP decision-making process, increased levels of trust in the department and its staff, and the development of sufficient digital skills allowing for confident navigation of the new self-service online portal. These behavioural changes allow for enhanced freedom and choice, improving the customer experience and reducing application journey times.

As mentioned previously, the customer experience metrics survey will collect repeated time series quantitative data of HTP and BAU customers at different stages in the claim journey, utilising existing drivers to develop an understanding of the customer experience for both the groups of customers, allowing for robust comparisons. The survey will also draw on questions from the quarterly DWP BAU Customer Experience Survey, ensuring consistency of measurements and results.

The interim process evaluation will conduct small-scale qualitative research to explore new PIP service customers’ application experiences and the support received from DWP staff. The full evaluation will expand to larger-scale research with new PIP service customers at various claim stages, providing broader insights into customer behaviour and engagement with the HTP service.

Understanding impact of transformed IT and data

Digital services and strategic data are central to the HTP and enable every strategic outcome rather than acting as a standalone objective. The transformed IT infrastructure and data capabilities underpin improvements in efficiency, customer experience and employment outcomes by supporting automation, better decision making and accurate service delivery.

Evaluation activities will therefore embed measures of digital uptake, data quality and system reliability across outcomes, reflecting their role as an enabler, and the foundation for transformation, rather than a standalone objective.

Approach to evaluating fraud and error impacts of HTP

As the HTP scales and develops we will need to be aware of any implications this will have on Fraud and Error (F&E) in the Department. The Programme needs to ensure that any new systems or ways of working that are introduced are delivering the service with rigour and accuracy. For example, improved use of data, and a focus on ‘getting it right’ for the customer first time will support accuracy and better decision making, which may have a positive impact on F&E. There are multiple opportunities across the early stages of the HTP development to ensure that the data collected is robust and sufficiently detailed to allow us to measure possible F&E. Over the longer term, the process and impact evaluation will integrate measures and indicators of F&E into scheduled evaluation activities, with the objective of providing robust long-term evidence regarding F&E in the new PIP service.

Question 8

Conclusion and next steps

Accepted Answer

This updated evaluation strategy sets out the plans for evaluating the Health Transformation Programme to deliver learning and provide accountability. There is a need to provide evidence to determine whether the Programme is meeting its five strategic objectives and identify any wider, potentially unintended, outcomes.

The HTP is an agile programme that continues to evolve and improve. Because of this, the evaluation strategy will need to be revisited over time to make sure it meets both Policy and Programme requirements. This strategy explains how layers of planning and scoping, combined with cross-cutting insights like performance monitoring and work on the HTP business case, testing and learning, and comprehensive service evaluations - including process and impact assessments - will provide practical learning for the Programme and inform future policy-making within DWP.

In line with the Office for Statistics Regulation Code of Practice for Statistics, we will continue to publish MI and plan to expand the range of metrics we report on externally as the service evolves and data quality improves. Research reports from projects commissioned as part of the HTP’s research programme to address key gaps in our knowledge will be published in line with the GSR Publication Protocol.

Cookies on GOV.UK

Executive summary

Introduction

Updating the Health Transformation Programme Evaluation Strategy:

Health Transformation Programme aims

Health Transformation Programme scope and approach

“Business as usual”

Progress and achievements

Evaluation and monitoring approach

Evaluation and monitoring aims

Informing, monitoring and evaluating an agile programme

Planning and scoping

Cross cutting insight

Test and learn

Evaluation of services

Data protection

Equalities analysis

External rationale for evaluation and monitoring approach

Planning and scoping

Theory of Change

Use of Theory of Change in HTP

Theory of Change logic models

HTP Theory of Change logic model

New PIP service logic model

Health Assessment Service logic model

ToC challenges, risks and dependencies

Cross cutting insight

Research programme

Overview of approach

Examples

Research with PIP customers

Behavioural Insight studies:

Resourcing

Performance monitoring

Measuring performance

Performance framework

Key Performance Indicators

Publication approach

Current publication:

Future strategy for reporting performance metrics:

Business case

Test and learn

Role of test and learn in agile programmes

Test and learn principles

Testing needs to be informed by the ToC

Build in engagement with a diverse range of stakeholders from the beginning

Design tests with transformed service in mind

Protect staff and customers from risk​

Use a consistent and appropriate system of governance

Prioritising and designing test and learn

Role of evaluation in test and learn

Full evaluation of services

What is ‘full evaluation’?

Full evaluation scope

Full evaluation timelines

Process evaluation

Interim process evaluation

Customer and staff experience metrics surveys

The HTP process evaluation risks/dependencies

Impact evaluation

Implementation and planning considerations for impact evaluation

Counterfactuals and baselines in the HTP impact evaluation

Counterfactuals

Baselines

The HTP impact evaluation risks/dependencies:

Value for money evaluation

Full evaluation case study: Apply for PIP digital self-service

Measurement of key outcomes

Measuring customer trust and experience

Measuring efficiency

Measuring employment outcomes

Evaluating cultural and behavioural change

Staff

DWP customers

Understanding impact of transformed IT and data

Approach to evaluating fraud and error impacts of HTP

Conclusion and next steps

Is this page useful?

Help us improve GOV.UK

Help us improve GOV.UK

Protect staff and customers from risk