Research and analysis

HMRC Evaluation Framework

Published 25 November 2021

Foreword

I am delighted to introduce our HM Revenue and Customs (HMRC) evaluation framework.

HMRC is the UK’s tax, payments and customs authority and we have a vital purpose: we collect the money that pays for the UK’s public services and give financial support to people. Our vision is to be a trusted, modern tax and customs department. We will achieve this by ensuring we have a resilient and agile tax administrative system that helps protect society from harm. It’s our job to make it easy for customers to get tax right, and hard for anyone to bend or break the rules.

Our evaluation framework sets out our approach to monitoring and evaluation and the activities we are planning to do so that we can be transparent. It reflects how we gain an understanding of what works, when, why and what doesn’t work so that we can be accountable for what we do. This guides our activities and facilitates our understanding of whether policies, programmes and projects have been effective and if they have achieved expected outcomes.

We will continue to work closely with colleagues across the department and other government departments to build and refine our approach for the future.

Justin Holliday, Chief Finance Officer and Tax Assurance Commissioner

Executive summary

The evaluation framework sets out our approach for achieving HMRC’s evaluation vision of good quality monitoring and evaluations of policies, programmes and projects in line with government good practice. Through this framework we will deliver proportionate and systematic evaluations, transparency in our evaluation decision-making and build a stronger evidence base.

The following key elements of the evaluation framework will be applied across the department:

• deliver proportionate and systematic monitoring and evaluation using our evaluation criteria to help bring transparency to our decisions
• continue to embed monitoring and evaluation into our governance framework and provide guidance and tools
• ensure HMRC has the right capability for monitoring and evaluation and builds a positive culture by working with colleagues across the department and other government departments
• continue to embed lessons learnt from monitoring and evaluation already delivered to maximise efficiency in our departmental processes

Roles and responsibilities

HMRC is the UK’s tax, payments and customs authority. Our purpose is to collect money that pays for the UK’s public services and give financial support to people. HM Treasury (HMT) and HMRC work together through a policy partnership. HMT leads on strategic tax policy and policy development, whereas HMRC leads on policy maintenance and implementation.

HMRC has 5 strategic objectives:

• collect the right tax and pay out the right financial support
• make it easy to get tax right and hard to bend or break the rules
• maintain taxpayers’ consent through fair treatment and protect society from harm
• make HMRC a great place to work
• support wider government economic aims through a resilient, agile tax administration system

Our evaluation vision

HMRC’s vision is for good quality monitoring and evaluation delivered proportionately and systematically. This will support transparency in our evaluation decision-making and help us to build on our evidence base.

To achieve our evaluation vision, HMRC’s evaluation framework sets out that we will:

• deliver proportionate and systematic monitoring and evaluation using our evaluation criteria to help bring transparency to our decisions
• continue to embed monitoring and evaluation into our governance framework and provide guidance and tools
• ensure HMRC has the right capability for monitoring and evaluation and builds a positive culture by working with colleagues across the department and other government departments
• continue to embed lessons learnt from monitoring and evaluation already delivered to maximise efficiency in our departmental processes

Our evaluation vision

The 4 elements of HMRC's evaluation vision - Proportionate and systematic, Governance, Capability and culture, Learning.

Importance of evaluation in HMRC

The importance of good evaluation is recognised across government. We have a responsibility to maximise public value and outcomes delivered for taxpayers’ money from government activity. By integrating a proportionate approach into our governance structure, we can target investment to ensure our activity is useful and credible and that we are maximising value by embedding learning from our experiences.

The Major Projects Authority (MPA) provide independent assurance on major projects. Additionally, the Public Accounts Committee (PAC) scrutinise public spending for Parliament drawing on the work of the National Audit Office (NAO). Monitoring and evaluation is not new to HMRC. This framework formalises how we will continue to build on evidence to drive our learning and help us to improve effectiveness and efficiency.

The Magenta Book provides guidance on what to consider when designing an evaluation. It recommends using a systematic evaluation to assess how change such as policies, programmes and projects is implemented, understand if its delivery is as expected and assess if it represents value for money.

Key approaches and models

Monitoring and evaluation is important, and we have embedded it at various stages from design to post-implementation of a policy, programme and projects. However, it is essential that we are proportionate and reflect the needs of decision makers and those scrutinising activities.

We use a range of approaches across the department and follow government guidance on best practice. We have presented some examples.

From the Magenta Book good-practice guidance, monitoring is the collection and analysis of information about an intervention to track progress against its objectives. Whereas, evaluation is a process of understanding how an intervention was implemented and the effects it had for whom. Monitoring and evaluation are related, and evaluation will usually rely heavily on monitoring data.

There are 3 main types of evaluation, each with a different focus:

• process - what can be learned from how the intervention was delivered?
• impact - what difference did the intervention make?
• value for money - was this a good use of resources?

Typically, a combination of all types of evaluations are used to understand the effect of the policy, programme or project.

The Green Book includes the rationale, objectives, appraisal, monitoring, evaluation and feedback (ROAMEF) framework. The ROAMEF framework is used to express a complex process in a simple way. It is useful for thinking about the key stages in the development of a proposal through to feeding back evidence to inform future change.

We use the Public Value Framework and apply when appropriate to our policies, programmes and projects. This is important as our role is to pay for the UK’s public services. This is a tool to help quantify inputs and outputs and help establish the relationships between them.

The department has drawn on best practise when developing the Tax Consultation Framework. Across the department we have implemented a 5-stage policy process when we consider the implementation of a tax policy.

The 5-stage policy process

1. Setting out objectives and identifying options

2. Determining the best option and developing a framework for implementation

3. Drafting legislation to effect the proposed change

4. Implementing and monitoring the change

5. Reviewing and evaluating the change

We endeavour to investigate best practice models and approaches. We will also develop bespoke approaches across the department to improve the way we develop and deliver policies, programmes and projects to maximise public spending.

The evaluation framework

The evaluation framework outlines how our evaluation vision will be achieved, by enhancing our current system of good quality monitoring and evaluation.

We have outlined below how we are embedding our vision into the development of policies, programmes and projects across the department so that we can deliver more good quality evaluations where funding permits. In 2021 to 2022, HMRC received additional funding for evaluation capability building and research.

A proportionate evaluation programme

It is important to have a proportionate approach to understand what works, how, why and what doesn’t work. It would not be an effective use of public funds if we undertook evaluation of all activities. A proportionate approach allows us to balance the needs of good quality monitoring and evaluation against other priorities and helps us ensure we maximise public value and outcomes delivered for the taxpayer. We will do this by using our evaluation criteria to inform evaluation decision-making.

By implementing a proportionate and systematic approach to deciding what should be monitored and evaluated we ensure we are continuing to build a strong evidence base that supports our strategic objectives.

Our evaluation criteria

HMRC has developed a set of evaluation criteria following an internal review of best practice guidance as well as drawing on cross-government expertise and resource. The 5 criteria will help determine what is in and out of scope and help bring transparency to our decisions. All the following 5 criteria do not have to be met to evaluate:

• whether there is a knowledge gap to be filled by an evaluation
• whether there is an untested or a new strategy, policy or operational intervention
• whether there is an opportunity to gather timely and useful evaluation outputs
• whether there is an opportunity to improve future change (e.g. strategy, policy, transformation programme)
• to demonstrate accountability and transparency of the use of public funds

The use of these criteria will help us to provide evidence-based advice to Ministers. Due to our wide remit, in some instances additional criteria have been applied to ensure proportionality and help ascertain what is appropriate to evaluate. For example, tax reliefs have additional specific criteria to prioritise the future evaluation of specific reliefs.

Governance

To support proportionality, we are embedding monitoring and evaluation into our governance process. The criteria will help inform if the change should be evaluated during early design stages. This will provide early assurance if the design of policies, programmes and projects align with HMRC’s long-term strategic objectives prior to delivery. The iterative nature of the governance and strategic assurance process enables reconsideration of decisions, moving back through the process as appropriate.

In addition to having their own quality assurance procedures, HMRC’s specialists are continuing to enhance the quality of evaluation plans by engaging with stakeholders early in the process. Early evaluation thinking will support collaboration with experts to ensure proposals are of a good standard. This will ensure the lessons learned remain high-quality and continue to build a strong evidence base.

Enhancing capability and building a positive culture

HMRC will continue to deliver a programme of training to build organisational evaluation capability and embed a culture that incentivises good quality monitoring and evaluation. This will allow us to use internal and external knowledge to inform decision-making and maximise learning about what works for whom, when and why.

A new centre of expertise has been established to support the delivery of these activities and drive the delivery of this evaluation framework. The new centre will co-ordinate and promote good practice across the department.

This will be achieved through a combination of activities to train staff, providing access to the information, advice and guidance they need to deliver good quality evaluations; learn lessons from existing projects; network with the evaluation community across government to leverage opportunities such as the Cross-Government Evaluation Group; benefit from the new Cabinet Office Evaluation Taskforce as envisaged in the Declaration on Government Reform; as well as external expertise when appropriate. For example, working with the Economic and Social Research Council to deliver a joint Tax Research Programme.

Learning from evaluations delivered

By understanding what works, in what context and what does not work, we can maximise lessons learnt across the department and ensure learning is used to inform thinking for new activities and enhance practices and processes. Even on occasions where activities do not deliver the outcomes expected, learning is important and will inform thinking behind future activities.

This will be done by collaborating with colleagues across the department as well as other government departments to ensure we maximise public value. The PAC recommendation has been taken on board on the management of tax reliefs to “ensure that the results of internal… evaluations are published and are easily accessible to Parliament and the public”. This will support embedding lessons learnt from the NAO audits.

We have invested in HMRC’s Data lab which allows external researchers to access HMRC data that is anonymised for research that contributes to HMRC activity. Additionally, we are exploring how we can expand access to provide a wider public benefit as defined by the Digital Economy Act.

Next steps to deliver the evaluation framework

The evaluation framework has set out the department’s current approach and the activities it is planning to do for monitoring and evaluation to achieve our evaluation vision.

To further embed all aspects of the framework we will continue to build evaluation into the governance process, enhance our evaluation capability across the department and continue to learn and build a positive learning culture. We will continue collaborating with colleagues across government departments as well as external stakeholders as appropriate.

The Tax Information and Impact Notes (TIINS) for tax policy changes when the policy is final or near final is already published. More detail will now be added into the monitoring and evaluation section of future TIINS, particularly for key large and impactful policies based on our evaluation criteria.

To ensure the department’s approach to evaluation is transparent we will publish more evaluation activity and continue to publish a wide range of statistics as part of our monitoring.

We have committed to the publication of our external research programmes in the HMRC annual report and accounts and will now make the same commitment for evaluation. Additionally, a list and a brief description of our research reports as well as the final research report is available on GOV.UK. We will now make the same commitment for evaluation.

Our approach goes beyond the PAC recommendation which was focused on the management of tax reliefs. We believe it is important that we publish and ensure evaluations from across the whole department are easily accessible to the public.

It will take time to deliver our evaluation vision in full and we want to use the evaluation framework as a platform for engaging with and learning from our stakeholders. It reflects some areas are more developed than others and outlines our aspiration for the future.

Comments and suggestions are welcome and should be directed to evaluationframework@hmrc.gov.uk.