Form

Producing post-implementation reviews: principles of best practice

Updated 6 August 2021

Introduction

This guidance was produced by the Department for Business, Energy, and Industrial Strategy (BEIS) Better Regulation Executive (BRE) to assist colleagues across government in completing Post-Implementation Reviews (PIRs).

It is not a collection of binding rules. Instead, it provides overarching principles of best practice to be used in conjunction with departments’ own guidance when completing PIRs.

Additionally you should refer to the guidelines set out in the Treasury’s Green Book, Aqua Book, and Magenta Book. In particular, the Magenta Book supplement and the Regulatory Policy Committee’s (RPC) Guidance for Conducting Regulatory Post Implementation Reviews, which provides analytical guidance for conducting PIRs.

Further guidance on the Better Regulation Framework, including information on the process for developing policy for regulatory measures, engaging with the RPC, seeking collective ministerial agreement, and laying legislation in Parliament is provided in the Interim Better Regulation Framework Guidance.

For more information on the legislative requirements for undertaking PIRs, please see sections 28-31 of the Small Business, Enterprise and Employment Act 2015 (SBEE Act 2015) and the PIR Statutory Guidance under s.31 of the SBEE Act, which sets out the factors departments need to consider in determining whether it is appropriate to include a review clause in secondary legislation:

Departments should also consider how the evidence collected in the PIR could be used for other purposes, such as for post-legislative scrutiny.

Your Department’s Better Regulation Unit (BRU) should in the first instance be approached for advice and information on preparing a PIR, particularly in terms of proportionality and other areas of the PIR process mentioned in this guidance: read the RPC proportionality guidance.

Overview

Overview of the policy making process.

The diagram below outlines the stages for regulatory policy making. This can broadly be broken down into three stages:

  • Pre-consultation – This refers to the initial development of policy proposals, prior to (or in preparation for) formal or informal consultations.
  • Pre-Implementation – This refers to the stage following initial consultation prior to the introduction or laying of legislation. PIR statutory guidance under s.31 of the SBEE Act 2015, to which a minister must have regard in determining that it is not appropriate to make provision for review, states that if the regulatory provision has an impact below +/- £5m EANDCB (Equivalent Annual Net Direct Cost to Business) then this is a significant factor in considering whether a statutory review clause is the appropriate way of keeping the regulations under review. In such circumstances the department will be required to publish a statement as to why it is not appropriate to make provision for review of the legislation. In the absence of a review clause [the PIR statutory guidance under s.31 of the SBEE Act 2015 provides that] the policy should be subject to appropriate monitoring and evaluation. Review clauses are also included in primary and secondary legislation beyond these requirements, for policy and other reasons.
  • Post-Implementation Review – Once the proposals have come into force, this stage offers the opportunity to review whether the regulation has met the intended objectives of the legislation.

*Independent scrutiny is optional for all measures at pre-consultation and is not required for measures below the threshold of + /- £5m EANDCB or for building safety regulations; for other measures departments would usually seek independent scrutiny before seeking Cabinet clearance to agree the final policy, although there may be circumstances where this is not appropriate. Your BRU should be asked to advise on this.

The PIR rationale and process

A PIR provides evidence that can inform the various stages of the policy development cycle (ROAMEF)[footnote 1]:

Source: The Magenta Book, HM Treasury guidance on evaluation

It is important however that none of the stages within the ROAMEF cycle are considered in isolation by departments: the diagram should be viewed as an iterative rather than a linear process, not least because of the feedback loop. As such, monitoring and evaluation should inform thinking throughout the ROAMEF cycle - before, during and after implementation - which maximises the potential for learning to inform future policy interventions.

The requirement for departments to complete a PIR is triggered by a statutory review clause within the legislation, or an equivalent administrative commitment to carry out a review. A statutory review will normally need to be completed within five years of the regulatory measure coming into force and on a repeating five-year cycle thereafter, or according to any statutory deadlines.

Having in place an effective Monitoring and Evaluation (M&E) plan will be crucial when the time comes for a department to complete a PIR on a specific policy and/or measure. In particular, M&E plans will be invaluable in helping to ensure that evaluation is of appropriate quality and will assist in drawing out lessons learnt to inform the implementation of future policy. Departments are therefore strongly encouraged to develop such a plan for each regulatory measure at the very outset of the policy cycle and before the measure is implemented. The plan should be a key feature of the Regulatory Impact Assessment (RIA) for the proposed measure, which is produced at the pre-implementation stage. It should set out how the PIR will be completed when the time comes, particularly in terms of how the collection of relevant data will be compiled and how the impacts of the policy will be monitored to inform if the measure is meeting its policy objectives.

M&E plans are not just about measuring effects that feed into PIRs but should be designed to inform departments about the effects and impacts of their policies as they are being implemented. For example, there may be developments in new technologies and market structures that departments should be taking account of. The Plans should not be viewed as a purely administrative exercise as they can and should help the policy evolve in ‘real time’ where possible.

While M&E Plans should set out intentions and commitments, they should not be static and should be adaptable and flexible, especially if different outcomes or unexpected effects and consequences take place: as ‘living documents’ M&E Plans should be amended as necessary to capture these things. As just mentioned, best practice is always to set out the M&E Plan in the RIA accompanying the original proposal. Please refer to The Regulatory Impact Assessment Template for further steers on what departments should consider for their M&E plans.

Wherever practicable, work on the PIR should be built into any wider evaluation work for a policy so that all relevant evidence can be considered, and any duplication of effort avoided, particularly if there are several related policy measures. This will be important if the measure is integral to wider departmental policy, in which case there will be a strong case for producing the PIR to coincide with outputs of the wider evaluation. Any relationship between the measure and the wider policy, and the proposed next steps for each, should be explained in the PIR.

The PIR template provides suggested headings for the findings of a PIR and can be used for both statutory and non-statutory reviews. The use of the PIR template is not mandatory but is considered best practice.

The revised Statutory Guidance under s.31 of the SBEE Act 2015 encourages departments to take a proportionate approach in considering the inclusion of statutory review clauses pursuant to the SBEE Act, as well as updating the better regulation framework requirements for PIRs under the Act. These include removing the need for completed statutory and non-statutory PIRs to be laid before Parliament, and the introduction of the de minimis route, where measures below +/- £5m EANDCB are not required to be scrutinised by the RPC. All PIRs including those not subject to RPC scrutiny still need to be published, however.

In the context of this guidance, proportionality means taking an approach where the costs of the data collection, analysis and review activities are appropriate to the expected benefits or levels of uncertainty associated with the measure. The level of analysis and resource required will depend on a case-by-case basis from a light-touch desktop review to significant evaluation projects with data collection.

The Magenta Book is the principal source of guidance for policy makers and analysts in central government on conducting an evaluation. It includes supplementary guidance specifically aimed at supporting analysts in preparing a PIR, which includes advice on how to assess proportionality. The Green Book provides guidance on the economic principles that should be applied to both appraisal and evaluation.

For retained EU law where there is a provision for review by ministers, the commitment for review must be honoured as this is a statutory duty. For amendments to, or replacements of retained EU law, the relevant requirements in SBEE would apply as normal irrespective of whether the amendment or replacement is of retained EU law. If the retained law makes or amends a regulatory provision in relation to a qualifying activity, then ministers will need to:

  • Make provisions for reviewing the regulations in question.
  • Publish a statement in the explanatory memorandum (EM) that a review is not appropriate by reference to the qualifying exceptions listed in the revised Statutory Guidance under s.31 of the SBEE Act 2015 (e.g., that it would be disproportionate when taking into account the economic impact of the measure and/or that it would be undesirable for particular policy reasons).

The exceptions to the above are where: (i) the measure is made under powers in the European Union Withdrawal Act 2018 (EUWA), in which case the SBEE review requirements are disapplied; or (ii) the measure is not made under EUWA powers but could have been, in which case, this is a significant factor in supporting a decision to not include a review clause. In such cases, ministers will still need to publish a statement in the EM that a review clause is not appropriate. Overall, decisions taken on whether a PIR is necessary for legislation that amends or replaces retained EU law should be dealt with on a case-by-case basis, with departments having regard to the requirements in the SBEE Act 2015, the revised Statutory Guidance under s.31 of the SBEE Act 2015 and this Principles of Best Practice guidance.

Assessing Impacts

As set out in the principles of cost benefit analysis within HMT’s Green Book, the role of appraisal and evaluation is to provide objective analysis that supports decision making (p.5). As such, the principles of cost benefit analysis should be used at all stages of the development and appraisal of policy [see ROAMEF cycle above]. Where government intervention requires a legislative or policy change to be made, departments are expected to conduct a proportionate assessment of the impact of the change on the groups affected – which should take the form of an impact assessment. For small regulatory changes, standalone impact assessments may not be required, though any analysis included to support these changes should be in line with the Green Book methodology. Where a policy change impacts business, the Better Regulation Framework will be of relevance. Accordingly, each PIR should look to evaluate a given measure’s actual effects on business, including comparing these impacts with those that were anticipated at the RIA stage.

The scope and proportionality of the PIR should take into consideration the expected impacts as set out in the RIA, although evidence should not be restricted to these expected impacts of the measure but should also be capable of capturing unintended effects and consequences. Outlined below is a non-exhaustive list of considerations and commitments for some key areas:

  • UK business impacts: business impacts that occur within the UK economy should be assessed as part of a PIR (this reflects Green Book guidance). In some cases, it may be appropriate to discuss effects on non-UK businesses, but any such discussion must be clearly distinct from UK impacts.
  • Trade and Investment impacts: Trade and Investment impacts should be assessed as part of the PIR. This should include an assessment of whether the measure is more trade restrictive than necessary, based on relevant international standards where possible, whether it introduces different requirements for domestic and foreign businesses or different requirements for businesses from different foreign countries and should demonstrate how the measure is consistent with the UK’s international obligations for equal treatment.
  • Commitments under Free Trade Agreements (FTAs): Following EU Exit, the UK has entered into the UK-EU Trade and Co-operation Agreement and the UK-Japan Comprehensive Economic Partnership and is negotiating Good Regulatory Practice and Regulatory Cooperation chapters in other FTAs. The UK has agreed to international obligations to undertake post-implementation reviews (retrospective evaluation) of regulatory measures, where appropriate and to have mechanisms and processes in place to promote periodic review of measures. These reviews must consider whether there are opportunities to achieve the policy objectives more effectively and to reduce unnecessary regulatory burdens, including the impact on small and micro-sized business. In addition, the UK is required to publish any existing plans for post-implementation reviews (retrospective evaluation) and the results of such reviews.
  • International impacts and regulatory cooperation: the PIR should review whether the measure is consistent with other approaches internationally which seek to address the same issue. This can include, for example, international instruments, standards, and the approaches of other countries. Policy makers should consider whether evidence on the efficacy of these international approaches could be applied in the UK, either through aligning regulatory approaches to reduce barriers to trade and investment or identifying whether greater regulatory cooperation may be necessary to achieve the intended outcome (such as where a measure may displace a problem from one jurisdiction to another). It should also be noted that under the SBEE Act where the regulatory provision being reviewed implements an international obligation of the UK the review must so far as is reasonable have regard to how the obligation is implemented in other countries which are subject to the obligation. It should analyse the costs (and benefits) of diverging from international practice if such a choice was made ex ante and identify the unintended divergences (in design and enforcement) that may be source of friction.
  • Retained EU legislation: for legislation that is going to be retained in UK law and has a statutory PIR commitment, the commitment to undertake a PIR will continue. This will also apply to non-statutory commitments. Your BRU should be contacted in the first instance for further advice.
  • Direct and indirect impacts[footnote 2]: both are of potential concern for the PIR. For smaller interventions however, when conducting the initial RIA, it may not have been judged to be proportionate to attempt to assess indirect impacts. The PIR should therefore consider significant wider impacts, even if not included in the original RIA.
  • Small and micro businesses: given that the burden of regulation can fall disproportionately on small and micro businesses, the impacts on them should be specifically considered. In the international context, consideration of these impacts on small and micro-sized businesses is required where PIRs are appropriate under the UK-EU Trade Co-operation Agreement.
  • Assessment of compliance and enforcement: it is standard practice to assume at the RIA stage that there will be 100% compliance with the measure unless there is evidence to the contrary. At the PIR stage, it is important to test this assumption empirically where it is feasible and proportionate to do so. The effectiveness of any enforcement regime should be assessed against the Hampton Principles (find more information) and the regulators’ code.
  • Market structure impacts: a PIR should include an assessment of the market structure before and after the measure, for example whether the measure has affected the number of firms in the market or market dynamics.
  • Innovation Impacts: the PIR should include a proportionate assessment of whether the measure has enabled or restricted innovation. For example, if the measure has facilitated the creation of new processes or products or has locked-out innovation.

Departments should consider the timing of the expected impacts of the regulation, and how reviews can be factored in to match these. Where feasible, co-ordinating the reviews of related regulations should be considered.

Departments could develop their own internal checklists to support meaningful peer reviews and aid policy makers and analysts in deciding that an appropriate and proportional PIR has been completed.

With regards to unintended effects and consequences (those not included in the original RIA because of analytical methodology, flaws in implementation or unforeseeable changes in the external environment), it is important for the PIR to both capture and assess the reasons why they have occurred, insofar as possible.

Departments are encouraged to use the following flowchart as a structure for assessing impacts for a PIR:

  1. To what extent is the existing regulation working?

    – To what extent has the policy achieved its objectives?
    – To what extent have the success criteria been met?
    – Have there been any unintended effects?
    – What have been the actual costs and benefits of the policy? How do these compare with the estimated costs and benefits?
    – What have been the impacts on small and micro businesses (where relevant)? How do these compare with the estimated impacts?

  2. Is government intervention still required?

    – What would happen if you removed the regulation (has the problem been solved either because of the policy, because the market has changed, or due to other factors)?
    – Are the objectives of the regulation still valid / relevant?

  3. Is the existing form of government regulation still the most appropriate approach?

    – What are the likely costs and benefits going forward?
    – How likely are unintended effects in the future?
    – How effective is the implementation / enforcement mechanism for the policy?
    –– To what extent do compliance levels indicate that the enforcement mechanism chosen is inappropriate?
    –– What are stakeholders’ views on implementation?
    – To what extent would non-statutory measures achieve the same benefits?

  4. If this regulation is still required what refinements could be made? (What scope is there for simplification, improvements?)

    Could refinements be made:
    – To improve enforcement / compliance levels.
    – To simplify implementation to reduce burden on business (including small and micro businesses where relevant) and reduce costs.
    – To reduce the scope of the policy to take organisations out of the regulatory obligations.
    – To otherwise improve implementation to enhance benefits.

  5. If this regulation is not required, but government intervention in some form is, what other regulation or alternatives to regulation would be appropriate?

Writing the PIR

For transparency and consistency of reporting, a summary of the findings, conclusions and recommendations of a review should be set out in the PIR template.

It is for departments to decide on the best way to present the findings. It is recommended however that all PIRs should include a summary of the PIR methodology (including the approach to economic, impact and process evaluation where applicable) and a critical review of the coverage and quality of the evidence and data collection that has informed the PIR where appropriate. Departments should also record any significant lessons for the RIA process arising from the evidence, for example where costs or benefits had been found to be wrongly estimated.

The process of finalising and publishing a PIR should follow the five steps outlined below:

  1. Departmental approval process

    Departments may find it useful to peer review their PIR, particularly where there is centralised coordination of PIRs. Departmental BRUs should be asked to provide further advice on the internal approval process in the first instance. The PIR should be signed off by the department’s chief economist and relevant Minister, or in line with the department’s own internal process.

  2. Formal quality assessment by the RPC

    For PIRs pursuant to review clauses in statute, as well as non-statutory reviews the PIR should be sent to the RPC if the direct impacts of the measure to business are above +/- £5m EANDCB. The RPC will undertake a review of the quality of the evidence base and analytical elements of the PIR. A “fit for purpose opinion” from the RPC should be obtained before moving to step 3. Further guidance on how the RPC assess PIRs can be found in its case histories document which can be found on the RPC’s Whitehall Portal. RPC validation is not required if the measure has an impact below the +/- £5m EANDCB threshold but will however still require collective clearance from the Domestic and Economy Implementation Committee (DEI).

  3. Obtaining Cabinet Committee clearance

    Government publications, including a PIR setting out the conclusions of a statutory review, may require collective agreement and clearance through a Cabinet Committee. If you are due to publish a PIR, please contact the Economic and Domestic Affairs Secretariat (EDS) in the first instance so that they can advise you if clearance and, therefore, a write round, is required on a case-by-case basis. EDS will advise you on whether collective agreement is required, how to navigate the write round process, and which committee to go through (usually DEI). Your Private Office can advise you on who to contact in EDS.

    Generally, PIRs and other publications will be subject to clearance where:

    – There is a level of cross-department interest in the policy or measure concerned.
    – The policy area is controversial or novel.
    – There are proposed changes to the measure arising from the review.

    PIRs occasionally do not meet the above criteria, especially if there are no proposed changes to the measure arising from the review. In such cases where EDS have decided that a write round is not required, they can advise you on alternative handling (i.e., producing a ministerial ‘for information’ letter). For information letters do not need to go through EDS, and it would be for your departmental CAB correspondence team to issue them cross-government.

    If a WR is required, EDS will ask you to engage thoroughly with relevant departments, especially HMT and perhaps No.10, ahead of WR to ensure interested parties are sighted. EDS will ask you to share your draft write round letter and pro forma in advance (usually a couple of weeks) of issuing the write round. This ensures that EDS have ample time to advise on the draft before SCS/ministerial clearance is sought. When writing round, the PIR should be attached to the write-round letter seeking collective clearance, alongside the draft report and the RPC opinion (if applicable).

  4. Publish the PIR

    For both statutory and non-statutory reviews, departments must publish the PIR after securing clearance from the DEI: the PIR would normally be published alongside the RPC opinion (if applicable). The PIR should be published on legislation.gov.uk, linked to the original legislation, under the ‘more resources’ tab to which it relates. Departmental BRUs should be asked to advise on the process for doing this.

  5. Use and dissemination of the PIR findings

    After the publication of the PIR the department should seek to undertake any recommendations made in the review. Chapter 6 of HMT’s Magenta Book covers the use and dissemination of evaluation findings. As outlined in the ROAMEF cycle, the findings of the review should be used as feedback and a reference point to inform new policies and ensure lessons learnt are applied.

    Another PIR will be required within five years if the statutory obligation has not been removed.

Annex 1: Acronyms

Term Definition
BEIS Department for Business, Energy and Industrial Strategy
BIT Business Impact Target
BRE Better Regulation Executive
BRU Better Regulation Unit (within Government Department)
DEI Domestic and Economy Implementation Committee
EDS Economic and Domestic Affairs Secretariat
EANDCB Equivalent Annual Net Direct Cost to Business
EM Explanatory Memorandum
EUWA European Union Withdrawal Act 2018
Japan CEPA UK-Japan Comprehensive Economic Partnership: The Agreement between the United Kingdom of Great Britain and Northern Ireland and Japan for a Comprehensive Economic Partnership
PIR Post-implementation Review
RIA Regulatory Impact Assessment
RPC Regulatory Policy Committee
SBEE Small Business, Enterprise and Employment Act 2015
UK – EU TCA Trade and Cooperation Agreement between the European Union and the European Atomic Energy Community, of the One Part, and the United Kingdom of Great Britain and Norther Ireland, of the Other Part.
WR Write Round
  1. The ROAMEF cycle (Rationale, Objective, Appraisal, Monitoring, Evaluation, Feedback) is expected lifecycle for all policies. More information can be found in the Green Book

  2. Direct impacts broadly mean any impacts from a policy whose effects will be immediate and unavoidable. Indirect impacts broadly mean any subsequent impacts, often where there have been explicit behavioural changes.