Guidance

Government analytical evaluation capabilities framework – accessible version (HTML)

Updated 2 July 2025

Introduction

What is this framework for?

1. To provide a description for Government analysts of the knowledge and skills which enable the effective delivery of quality evaluations.
2. It is a learning and development tool allowing people to assess their evaluation skills in order to identify gaps they might want to fill (e.g. through training).
3. It promotes a shared understanding of evaluation roles and skills.
4. It can be used when setting up evaluation projects and teams as a tool for considering the functions and roles required.
5. It can also be used more strategically as a checklist to assess the balance of skills needed in a Department as a whole and the need for training to support Government evaluation managers.
6. This tool has been developed for analysts. Other Government professions, including the policy profession, may find this useful. The policy profession competency framework includes evaluation.

How do I use this framework?

Identify the most relevant part for your needs which are initially broken down into four categories:

1. Scoping
2. Leading and managing
3. Methods
4. Use and dissemination

This document will go through each section in turn.

What if I am unsure about my current skill level?

Review the full framework and use feedback from line managers, peers and mentors.

Where do I go if I want to develop further?

Each section in this document has its own set of unique resources, however the cross-government Magenta and Green books are good resources to start with.

Frequently asked questions

What is a capabilities framework?

This capabilities framework describes the skills, attitudes and practices which enable effective and high quality Government evaluations. This is for the purpose of learning, personal development and training.

How does it link with existing analytical competency frameworks?

It complements other specific analytical competency frameworks. Where the skills in the evaluation capabilities framework overlaps with a skill in analytical competency frameworks, it focuses on the specific evaluation dimension of that skill. The evaluation framework also differs from the individual competency frameworks in terms of its use and purpose as a learning and development tool. It is a tool for all analytical disciplines and can also be used by teams, as well as individuals, to identify skills required in an evaluation project team. In general evaluation in government is not a distinct profession, it therefore draws on relevant professional guidance (e.g. GSR, GES, civil service code) where appropriate.

How does it link with other evaluation competency and evaluation capability frameworks

This framework is designed to guide analysts working within government departments. Other frameworks exist focusing on other audiences. A notable example is the UK Evaluation Society Framework of Evaluation Capabilities, this covers much of the same content but applies to the full UK evaluation community.

Does it mean I need all the capabilities listed in the framework to be an effective evaluation manager?

The framework outlines the skills which help ensure high quality, impactful evaluations but these do not need to be held by a single evaluation manager. Evaluations will often be delivered by multidisciplinary analytical teams and/or external consortiums. In terms of technical skills in particular, it is likely that different analytical disciplines (and individuals within them) will have different strengths and expertise. The frameworks will also be useful to identify the additional skills needed in a team or project.

I am not a badged analyst, is this of use to me?

Currently the framework is focused on evaluation managers who are often analysts. However, if you are interested in evaluation or are a policy official looking to better understand these processes, there may be aspects of this that you can use.

Scoping

1. Understanding and communicating the rationale for evaluation

a. Sells the concept of evaluation, championing the role of evaluation in policy making;

b. Understands the value of evaluation and how evaluation can support: accountability, measuring impact, learning, and programme development;

c. Understands the evidence, research and evaluation needs and priorities of policy customers;

d. Understands the evaluation processes used within their department, and across Government;

e. Ensures evaluation is considered early in the ROAMEF policy cycle and that enough thought is given to monitoring and evaluation requirements (including resources needed) at the appraisal stage e.g. in the business case;

f. Demonstrates knowledge and experience of a range of types and approaches to evaluating programmes and policies;

g. Defines evaluation questions to address policy needs, and re-defines where necessary;

h. Is aware of different types of evaluation questions: e.g. exploratory; descriptive; normative; causal; explanatory; developmental;

i. Can provide a hierarchical structure for evaluation questions and prioritise amongst questions.

2. Understanding the intervention and constructing a theory of change

a. Understands how evaluation questions relate to the underlying logic of the intervention;

b. Can create a theory of change, linking elements of the intervention with outcomes;

c. Can plan and run a logic/theory mapping workshop with key stakeholders;

d. Uses the theory of change with stakeholders to identify priority areas for evaluation;

e. Considers the original appraisal for the intervention (including any business case and/or impact assessment);

f. Uses the theory of change to guide evidence collection, analysis and interpretation of findings (e.g. monitoring data and gaps and using it as a tool to drive analysis and in particular synthesis);

g. Can identify the key metrics or performance indicators which need to be measured and approaches to measuring them;

h. Can define their role alongside that of benefits management approaches;

i. Ensures that consensus is reached and there is a shared view regarding the underlying logic of the intervention.

3. Identifying the right evaluation design for the policy

a. Works closely with clients to identify evaluation purpose and requirements and ensures conscious and informed choices are made about level and type of evaluation design;

b. Understands the different roles of process, impact and value for money evaluation and can communicate and explain these to evaluation clients and stakeholders;

c. Understands the range of approaches which can be used to assess whether an intervention was effective in meeting its objectives, and can determine the most appropriate approach and methods for a particular evaluation;

d. Is aware of frameworks for assessing the complexity of a programme and the distinctions between: ‘simple; complicated and complex’;

e. Devises appropriate evaluation questions and designs;

f. Can make the case for investing in good quality / proportionate evaluation;

g. Understands the role of monitoring and evaluation as part of the life cycle of project and programme design;

h. Where there is a programme delivered by other partners, ensures roles and responsibilities are clear for delivery of the evaluation, building these in from the start;

i. Ensures arrangements for access to monitoring data are in place (via the necessary data sharing agreements) to facilitate use of the data for monitoring and evaluation (e.g. linking and re-contact);

j. If designing an evaluation of a communications intervention, takes into account the Government Communication Evaluation Framework.

4. Producing a proportionate and appropriate evaluation plan

a. Designs evaluations for in-house or commissioned projects and procures external evaluations where appropriate;

b. Designs programmes of evaluation (including those containing multiple research projects over multiple years) including consideration of:

1. evaluation questions,
2. programme attributes,
3. context,
4. timescales,
5. and resources needed

c. Able to develop, and consult on, high quality specification/ Terms of Reference to underpin evaluation;

d. Works with stakeholders/delivery partners to agree and align timeframes;

e. Understands the trade-offs between timeframes for observing impacts and policy timeframes.

Resources to improve your scoping capability

1. Magenta Book & Supplementary Guidance - the key Government guidance on evaluation. Early chapters focus on designing an evaluation and the range of considerations for scoping.
2. DfT Logic Mapping Guidance (PDF, 208KB) - Guidance produced by DfT to provide hints and tips for carrying out logic mapping.
3. HIVOS Theory of Change Guidelines – a detailed guide to Theory of Change. Link
4. Better Evaluation guidance on using logic models and theory of change.
5. NPC Theory of change in ten steps. A guide to developing a theory of change.
6. NPC Theory of Change beginnings. An overview of theory of change and how to use it. Link
7. Sport England Logic Model (PDF, 707KB). A logic model worksheet for those beginning to look at this form of framework.
8. Choosing appropriate evaluation methods tool. A tool developed by BOND to help guide evaluators identify and chose appropriate methods for different evaluation questions.
9. Public Health England introduction to evaluation including links at the end to evaluation podcasts and videos

Leading and managing

1. Leading and managing

a. Demonstrates leadership and champions evaluation to maintain momentum and impact;

b. Works collaboratively, manages relationships and supports others (e.g. contractors and colleagues);

c. Can proactively influence stakeholders, and ensure buy in from less engaged evaluation users;

d. Is aware of and adapts to changing circumstances, feeding this into evaluation delivery;

e. Displays integrity and resilience, and maintains standards when challenged;

f. Builds a culture of evaluation, learning and transparency into the policy and analytical teams they work in;

g. Works effectively across analytical disciplines;

h. Is aware of key departmental procurement/commissioning procedures;

i. Identifies and sets up appropriate governance structures for input and quality assurance of evaluation products;

j. Ensures all parties are fully aware of their roles and responsibilities and key goals/deadlines are met;

k. Considers and implements relevant Government guidance that impacts on evaluations (including equality, transparency, ethics) and applies behaviours in the civil service code to evaluation management decisions;

l. Is prepared to invest time and energy in personal development and is willing to work with new methodologies to meet challenges;

m. Demonstrates evaluative thinking. In particular: the investigation of one’s own as well as others’ assumptions, motivations, and biases; understanding the relevance of context dependency; and the ability to navigate uncertainty (e.g. of outcomes), ambiguity (e.g. in data and evidence), and complexity.

Resources to improve your leading and managing capability

1. Magenta Book & Supplementary Guidance

This is the key Government guidance on evaluation. It provides information how to plan, manage and deliver evaluations in Government.

2. GSR Professional Guidance: Ethical Assurance for Social Research in Government (PDF, 720KB)

Full guidance produced by the GSR and applicable to evaluation managers which covers the ethics involved in research and evaluation

3. UK Evaluation Society: Good Practice in Evaluation

A set of guidelines designed to support those managing and conducting evaluation to produce something of high quality.

4. Civil Service Code

The core values (integrity, honesty, objectivity and impartiality) to which civil servants are committed in order to support good government and ensure highest possible standards.

5. Public Sector Equality Duty

Outlines the requirements created under the Equalities Act 2010 that public sector bodies have due regard to advancing equality.

6. Evaluation Failures: 22 Tales of Mistakes Made and Lessons Learned

Described as a candid collection of stories from seasoned evaluators from a variety of sectors sharing professional mistakes they have made in the past, and what they learned moving forward.

Methods

1. Use of monitoring and administrative data

a. Recognises the importance of monitoring data and influences how programmes collect monitoring data;

b. Understands the role of monitoring data in an evaluation, and when and how to use it;

c. Works with others (e.g. delivery partners) to ensure that correct data sharing agreements are in place early in the process;

d. Works with delivery partners to ensure high quality monitoring data is built into the programme;

e. Is aware of legal, ethical and practical issues in ensuring programme monitoring (and other) data is collected and used effectively;

f. Identifies sources and weighs up strengths and weaknesses of data and key metrics to help answer key evaluation questions;

g. Considers the links with project management information and reporting especially key delivery metrics and any benefits realisation plan;

h. Manages risks and issues with data monitoring and governance, escalating if necessary;

i. Considers the long-term potential of data sources;

j. Engages data science experts to explore innovative use of data, for example approaches to data management, use of data technology and visualisation.

3. Methods of primary research and analysis

a. Has a robust knowledge of qualitative and quantitative data collection and data analysis methodologies, knowing how to design and apply specific methods;

b. Understands strengths and weaknesses of a range of data collection methods, and when and how to apply them;

c. Has good knowledge of ethical issues in research and evaluation and can apply these appropriately to evaluation specifications and projects;

d. Collects and manages data (in particular personal data) in line with the GDPR/Digital Economy Acts and data handling protocols;

e. Ensures necessary quality assurance processes are built into data collection and analysis methods at appropriate stages;

f. Considers opportunities for the evaluation to analyse existing social media data, and to use social media as a tool to collect new data.

3. Process evaluation

a. Understands the value and role of process evaluation, and its link to monitoring and to impact evaluation, to understand how a programme is being delivered and why;

b. Times process evaluation deliverables to coincide with best opportunities to have impact and identifies and builds in suitable interim products;

c. Works collaboratively with programme managers to draw conclusions from process evaluation, alongside other insight and stakeholder engagement, and feeds these into policy design in a timely manner;

d. Understands the wider learning and value from process findings, alongside impact findings, for appraisals, and to build the evidence base in an area;

e. Can use results from process (and impact) evaluation in one project, to evidence potential optimism bias in other policies;

f. Understands limitations of process evaluations in measuring and determining impact.

4. Theory based approaches to impact evaluation

a. Understands the role of theory based approaches to evaluating complex interventions;

b. Recognises the limitations of experimental and quasi experimental approaches in some evaluation contexts;

c. Ensures theory based methods have robust approaches to testing hypotheses and causal claims;

d. Is familiar with specific theory based methods (e.g. contribution analysis, realist evaluation, process tracing, qualitative comparative analysis) and their appropriateness for addressing different evaluation questions;

e. Understands the value of a theory based approach in establishing and understanding causation, and enhancing other methodological approaches.

5. Experimental approaches to impact evaluation

a. Understands the conditions required for experimental and quasi-experimental approaches to be useful and applicable;

b. Can assess the robustness of different impact evaluation designs (including consideration of the ‘Maryland Scale’);

c. Familiarity with the ‘Test- Learn- Adapt’ framework and the positive case for Randomised Control Trials (RCT)– understands the variants of RCT design;

d. Appreciates the importance of the counterfactual and control groups in determining attribution;

e. Can apply experimental or quasi-experimental approaches effectively to attribute impacts;

f. Able to identify risks of using weaker designs (e.g. before and after) while being able to apply and interpret these effectively if they are the only approach available;

g. Can make appropriate use of emerging impact data (including managing the risks of use of early findings);

h. Understands the limitations as well as strengths of experimental and quasi-experimental designs and the challenges to implementing them.

6. Value for money evaluation

a. Has a good understanding of the HMT Green Book guidance on economic appraisal, cost benefit analysis, and requirements for consideration of monitoring and evaluation throughout the project cycle;

b. If not an economist, works jointly with economists to scope and deliver value for money evaluation;

c. Links impact evaluation back to objectives of the policy and initial appraisal, to ensure comparability;

d. Understands economic approaches to project appraisal and evaluation, including different ways of measuring efficiency, equity and impacts;

e. Ensures that programme costs are tracked to allow the necessary analysis of costs and benefits;

f. Considers the distributional impacts in the value for money and impacts of the programme;

g. Is aware of different methods which can be used in the valuation of social benefits and wellbeing;

h. Uses value for money evaluation to help improve assumptions and methods in Cost Benefit Analysis being developed for appraisal of new policies.

7. Research synthesis

a. Understands range of approaches to synthesis and meta-analysis and the role of triangulation of evidence;

b. Brings evaluation evidence from different sources together to identify overarching messages and findings;

c. Quality assures evidence to assess how it has been used in producing an evaluation synthesis.

Resources to improve your methods capability

1. Magenta Book & Supplementary Guidance

This is the key Government guidance on evaluation. It provides details on approaches and methods that can be used for process, impact and value-for-money evaluation.

2. BetterEvaluation

A collaboration that aim to share information and support evaluation Worldwide. It includes a lot of evaluation resources including overviews of different evaluation methods.

3. BOND Impact Evaluation A Guide for Commissioners and Managers

Guide on how to approach, design and deliver impact evaluations, including the methods that can be used.

4. Evalsed Sourcebook: Methods and techniques

Produced by the European Commission for looking at socio-economic development. It describes a range of methods and techniques for evaluation.

5. Broadening the Range of Designs and Methods for Impact Evaluations

A report looking at existing and alternative Impact Evaluation methods, in particular applied to development – but also applicable to other policy areas. Covers theory based approaches.

6. Treasury Board of Canada. Theory-Based Approaches to Evaluation: Concepts and Practices

A useful overview of theory based evaluation approaches.

7. Test, Learn, Adapt (PDF 3MB)

A guide to RCTS.

8. The Maryland Scientific Methods Scale (SMS)

An introduction and scoring guide on the Maryland Scale produced by the What Works Centre for Local Growth.

9. Research Methods for Policy Evaluation

National Centre for Social Research guide covering process and counterfactual impact evaluation methods.

10. MRC guidance on process evaluations

11. The Green book

The central government guide to economic evaluations.

12. DfT guidance on how to strengthen the links between appraisal and evaluation (PDF, 723KB)

13. Guidance for transport impact evaluations (PDF, 287KB)

A document produced by DfT with guidance on impact evaluations including which one may be the most appropriate.

14. Guidance on evaluating the impact of interventions on business focussing on counterfactual approaches

15. Information Commissioner’s Office Guide to the Data Protection Act

A guide to better understand the “day to day” principles and implications of the data protection act.

16. Using Social Media for Social Research: An introduction (PDF, 926KB)

A document produced by GSR that outlines some of the basics of social media research in a government context.

17. Quality in Qualitative Evaluation (PDF, 748KB)

A framework for assessing the robustness of qualitative research in evaluations.

18. A Rapid Evidence Assessment (REA) toolkit containing guidance on what is a REA, planning a REA and how to do a REA.

19. The Production of Evidence Assessments: A How to Guide (PDF, 864KB)

A DEFRA set of guidance on scoping reviews and rapid evidence assessments which are forms of evidence synthesis.

20. Systematic reviews and meta-analyses: a step-by-step guide

A guide produced by the University of Edinburgh with links to other resources.

Use and dissemination

1. Reporting and presenting data

a. Has clear knowledge of the format and style required to report evaluation results; can report evaluation information clearly in various formats (orally and in writing);

b. Can use charts, figures and infographics appropriately to add value to interpretation and presentation of data;

c. Is clear about the limits of the evaluation and communicates these clearly and positively;

d. Makes use of regular reporting where appropriate to enable ‘real-time learning’ for example using multiple evaluation outputs built around key reporting and decision-points.

2. Considering policy implications and sharing findings

a. Builds use and impact of findings into the evaluation design from the beginning;

b. Develops a use and dissemination plan based on which groups require what information at which point in time for what purpose;

c. Identifies opportunities throughout the evaluation process for effective dissemination and enhanced impact;

d. Builds relationships and demonstrates leadership in order that evaluation findings have impact;

e. Tailors reporting and communication of findings to meet the needs of different audiences, improving the usability of findings by showing how they relate to specific audiences’ areas of interest;

f. Identifies and explores creative ways of disseminating findings;

g. Is familiar with a variety of communication modes and tools (e.g. one page summaries, video outputs, infographics, data sharing, newsletters, social media posts, conference presentations and seminars) to maximise impact;

h. Identifies policy implications of results and feeds these into policy design including future appraisal to influence decision making and for future learning;

i. Identifies how findings apply to areas beyond specific scheme or policy being evaluated;

j. Takes forward publication of evaluation output including datasets, technical annexes and research tools where relevant, in accordance with relevant guidance.

Resources to improve your use and dissemination capability

1. Magenta Book & Supplementary Guidance

This is the key Government guidance on evaluation. It provides useful advice on use and dissemination of evaluation evidence.

2. GSS Communicating Statistics

A wealth of resources on communicating and disseminating official statistics, including guidance documents and case studies from across the Government Statistical Service (GSS).

3. ONS infographic guidelines

A document that covers infographics and how to use them.

4. GSR publication guidelines

The document outlining the publication of any research and analysis including evaluation.