Resources for evaluating policy in government (HTML)
Updated 17 January 2023
© Crown copyright 2023
This publication is licensed under the terms of the Open Government Licence v3.0 except where otherwise stated. To view this licence, visit nationalarchives.gov.uk/doc/open-government-licence/version/3 or write to the Information Policy Team, The National Archives, Kew, London TW9 4DU, or email: psi@nationalarchives.gov.uk.
Where we have identified any third party copyright information you will need to obtain permission from the copyright holders concerned.
This publication is available at https://www.gov.uk/government/publications/guidance-and-resources-for-evaluating-policy-in-government/guidance-and-resources-for-evaluating-policy-in-government-html
Introduction to evaluation
Magenta Book, Annex A. Analytical methods for use within an evaluation
This guidance contains:
- analytical methods for use within an evaluation
- theory based methods for impact evaluation
- experimental and quasi-experimental methods for impact evaluation
- methods for value-for-money evaluation
- methods for synthesis of existing evidence
- generic research methods used in both process and impact evaluation
Type of guidance: Methodological
Sector: All
Who it’s for: Evaluators, analysts
Organisation: HM Treasury
Keywords: Analytical; impact; value-for-money
The Green Book
Central government guidance on appraisal and evaluation.
Category: Methodological
Sector: All
Who it’s for: Evaluators, policymakers
Organisation: HM Treasury and Government Finance Function
Keywords: Introduction; appraisal; evaluation
Good Practice Guidance
This good practice guidance is for commissioners, practitioners and participants to establish good practice in the conduct of evaluation.
The guidance is underpinned by the relational and consequence theories of ethics and democratic principles and values.
Category: Overview, introductory
Sector: All
Who it’s for: Commissioners, practitioners, evaluators
Organisation: UK Evaluation Society
Keywords: Introduction; guidelines; ethics
Introduction to Evaluation in Health and Wellbeing
This guidance aims to help public health practitioners when conducting evaluations.
This section provides an overview of what evaluation is, when it should be undertaken, and different types of evaluation. These are described in more detail on other pages.
There is also a glossary defining the important terms and a guide to other resources.
Category: Introductory
Sector: Health
Who it’s for: Evaluators, social researchers
Organisation: Public Health England
Keywords: Outcome; process; economic
An 8-step guide to better evaluation
A series of linked blogs that identifies the key principles to successful evaluation and key considerations. The ‘how to’ evaluate covers topics such as:
- Starting early
- How long?
- What to evaluate?
- Find a control group
- Collect data
- Plagiarise
Category: Overview, introductory
Sector: Local government
Who it’s for: Commissioners, practitioners, evaluators
Organisation: What Works Centre for Local Economic Growth
Keywords: Impact; summary; policy
Measuring wellbeing guide
Practical guidance for designing and conducting evaluations focused on wellbeing. This guidance includes:
- What is wellbeing?
- Introduction to wellbeing evaluation
- Planning your wellbeing evaluation
- Wellbeing measures and data
- Choosing wellbeing measures
- Gathering qualitative data
- Analysing and interpreting your results
- Wellbeing measures bank
Category: Overview, introductory, methodological
Sector: Wellbeing
Who it’s for: Practitioners, evaluators
Organisation: What Works Centre for Wellbeing
Keywords: Public sector; qualitative; analysis
Impact evaluation
Magenta Book: Quality in policy Impact Evaluation
A guide for the design and assessment of impact evaluations of government policy.
- Chapter 1: Introduction
- Chapter 2: Quality in policy impact evaluation
- Chapter 3: Strong research designs in the measurement of attribution
- Chapter 4: Weaker/riskier research designs in the measurement of attribution
Category: Methodological
Sector: All
Who it’s for: Analysts, economists, evaluators
Organisation: HM Treasury
Keywords: Impact; policy; research design
What Works for Children’s Social Care Evaluator resources
A series of detailed guides and templates to support evaluators conduct robust studies. These include:
- WWCSC RCT & IPE TP Template
- WWCSC Pilot Protocol Template
- WWCSC Systematic Review Protocol Template
- WWCSC Systematic Review Report Template
- WWCSC QED Template
- REC Checklist and Application
Category: Methodological
Sector: Children and young people
Who it’s for: Evaluators
Organisation: What Works Centre for Children’s Social Care
Keywords: Impact; randomised control trials; quasi-experimental; systematic review
What Works for Children’s Social Care Randomised Controlled Trial Statistical Analysis Guidance
Read What Works for Children’s Social Care Randomised Controlled Trial Statistical Analysis Guidance.
This document outlines the approach to statistical analysis of randomised controlled trials recommended by What Works for Children’s Social Care (WWCSC).
This guidance is based on the existing methodological literature in related fields, particularly education, which deals with similar cohorts.
Category: Methodological
Sector: Children and young people
Who it’s for: Evaluators; Analysts
Organisation: What Works Centre for Children’s Social Care
Keywords: RCT; impact; statistical analysis
Statistical analysis guidance for Education Endowment Fund (EEF) evaluations
This document outlines the EEF’s policy on statistical analysis and effect size calculations. This guidance has been developed in association with the EEF’s Evaluation Advisory Group, other experts, and some members of EEF’s panel of evaluators
Category: Methodological
Sector: Children and young people
Who it’s for: Evaluators; Analysts
Organisation: Education Endowment Fund
Keywords: Impact; statistical analysis; randomised control trials
Evaluate: An introduction to evaluating age-friendly successes and challenges.
Tools and resources to support those evaluating age-friendliness programmes.
Category: Introductory, framework, methodological
Sector: Age
Who it’s for: Evaluators
Organisation: Centre for Ageing Better
Keywords: Impact; framework; measures
The Centre for Transforming Access and Student Outcomes in Higher Education (TASO) Evaluation guidance
TASO’s evaluation guidance resources focus on a range of topics including:
- Correlation vs Causation
- Randomised Controlled Trials (RCTs)
- Quasi-experimental Designs
- Theory of Change
Category: Overview, introductory, methodological
Sector: Children and young people, Education
Who it’s for: Evaluators
Organisation: TASO
Keywords: Impact; research methods; theory of change
Youth Endowment Fund resources for Evaluators
A series of detailed guides and templates to support evaluators conduct robust studies. These include guidance on: feasibility studies, pilot studies, analysis, commissioning and outcomes measures.
Templates available include: statistical analysis plans, protocol plans, feasibility and pilot plans.
Category: Methodological
Sector: Children and young people, Crime
Who it’s for: Evaluators
Organisation: Youth Endowment Fund
Keywords: Feasibility Studies; analysis; commission
Impact Evaluation: A Guide for Commissioners and Managers
This report sets out to support those evaluating and implementing evidence from impact evaluations across 5 chapters:
- Introduction and scope
- What is impact evaluation?
- Frameworks for designing impact evaluation
- What different designs and methods can do
- Using this guide
Note that this report assumes a level of familiarity with evaluation and socio-economic development.
Category: Overview, methodological, impact
Sector: International development
Who it’s for: Evaluators, Policymakers, Commissioners, Managers
Organisation: Bond
Keywords: Impact; framework; research design
Theory of change
Theory of Change in ten steps
Ten step guide to developing theory of change for policy evaluation. Ten steps include:
- Step 1: Situation analysis
- Step 2: Target groups
- Step 3: Impact
- Step 4: Outcomes
- Step 5: Activities
- Step 6: Change mechanisms
- Step 7: Sequencing
- Step 8: Your theory of change diagram
- Step 9: Stakeholders and ‘enabling factors’
- Step 10: Assumptions
Category: Overview, introductory, methodological
Sector: All
Who it’s for: Practitioners, evaluators
Organisation: NPC
Keywords: Theory of change; impact; outcomes
Implementation and process evaluation
Implementation and process evaluation guidance for EEF evaluations
This paper recommends a set of key principles for EEF evaluators planning, conducting and reporting IPE, in line with recommended best practice for pragmatic mixed-method and qualitative research.
Category: Introductory, methodological
Sector: Children and young people
Who it’s for: Evaluators
Organisation: Education Endowment Fund
Keywords: Implementation; process; mixed method; qualitative
Test, Learn, Adapt
Test, Learn, Adapt: Developing Public Policy with Randomised Controlled Trials
A document introducing what RCTs are and how they can be used to support public policies. It highlights the case for evaluation and provides practical guidance on how to conduct RCTs to support this.
Category: Overview, introductory, methodological
Sector: All
Who it’s for: Evaluators, policymakers, practitioners
Organisation: Behavioural Insights Team
Keywords: Implementation; process; randomised control trials; policy
Cost-benefit
Cost benefit tool
A range of guidance focused on how to undertake cost benefit analysis of crime prevention interventions.
Category: Methodological
Sector: Crime and policing
Who it’s for: Evaluators, Economists
Organisation: College of Policing
Keywords: Cost-benefit; economic; tool
Cost evaluation guidance for EEF evaluations
Guidelines for cost-evaluation
Category: Methodological
Sector: Youth
Who it’s for: Evaluators
Organisation: Education Endowment Fund
Keywords: Cost-benefit; methodology
Tools, frameworks and evidence maps
Government Analytical Evaluation Capabilities Framework
A resource drawn from the Magenta Book. The Government Analytical Evaluation Capabilities Framework is addressed in four sections:
- Introduction to the framework
- Scoping
- Methods
- Use and dissemination
Category: Methodological, framework
Sector: All
Who it’s for: Analysts, economists, evaluators
Organisation: HM Treasury
Keywords: Impact; process; value-for-money
Evidence and Gap Map
A resource mapping evidence on what works to keep children safe from involvement in violence.
Category: Overview
Sector: Children and young people, crime
Who it’s for: Practitioners, evaluators
Organisation: Youth Endowment Fund
Keywords: Evidence map; young people
Evidence and Gap Map
The map contains 399 studies that evaluate the effectiveness of youth employment and skills interventions. Of these, 378 are impact evaluation studies and 21 are systematic reviews.
Category: Overview
Sector: Children and young people, employment
Who it’s for: Practitioners, evaluators
Organisation: Youth Futures Foundation
Keywords: Evidence map; young people
Ageing Better Measures Framework
The Measures Framework allows individuals or organisations to quickly find a measure, scale or survey related to their topic.
Category: Overview / Measures & Framework
Sector: Age
Who it’s for: Practitioners, evaluators
Organisation: Centre for Ageing Better
Keywords: Measures; scales; surveys
Choosing Appropriate Evaluation Methods Tool
This tool can be used for 3 main purposes: general learning about evaluation methods; support design of planned interventions; and help users understand the best fit between evaluation questions, stakeholder interests and programme features or attributes.
Additionally, a link is available to book onto a training course: ‘Tools and methods available for monitoring, evaluation, accountability and learning’
Category: Methodological
Sector: International Development
Who it’s for: Evaluators, Policymakers
Organisation: Bond
Keywords: Methods; intermediate
Evaluating complex policies
Analysis in Government Sharing Webinar: Evaluation of Complex Policies
A webinar introducing evaluation of complex policies in government and understanding how outcomes of policies can support government functioning.
Category: Overview / Introductory
Sector: All
Who it’s for: Evaluators
Organisation: Government Analysis Function
Keywords: Complex; introduction; policy
Handling Complexity in Policy Evaluation
A supplementary guide developed by CECAN to support evaluation of complex policy through the Magenta Book
Category: Methodological
Sector: All
Who it’s for: Commissioners, Evaluators, Managers
Organisation: CECAN and Magenta Book
Keywords: Complex; policy; commissioners
Developing Strategic Approaches to Infrastructure Planning
This report documents trends in the use of strategic planning in a range of countries and the associated establishment of independent infrastructure advisory bodies. It documents practices in the areas of project identification, appraisal and selection.
It considers the appropriate scope of impact assessments, including the use of wider economic impacts and the role of ex post evaluation in contributing to better appraisals of future investments. Finally, it looks at how the effective stewardship of existing infrastructure assets can maximise their ongoing productivity.
Category: Research, Methodological, Overview
Sector: Infrastructure
Who it’s for: Commissioners, Evaluators, Managers
Organisation: OECD and International Transport Forum
Keywords: Complex; infrastructure; commissioners
Developing and Evaluating Complex Interventions (online)
This course provides participants with an understanding of the complex intervention research process.
Category: Research, Methodological, Overview
Sector: All
Who it’s for: Evaluators, Researchers
Organisation: NCRM with University of Glasgow
Keywords: Complex; interventions; development; evaluation