Research and analysis

5GIR and SIPP scoping and baseline report: executive summary

Published 13 May 2026

Executive summary

1.1 Introduction

To support the Department for Science, Innovation and Technology’s (DSIT) ambition to deliver world-class digital infrastructure across the UK, drive innovation and unlock opportunities for economic growth, and as part of its Wireless Infrastructure Strategy ^{[footnote 1]}, DSIT is currently in the process of delivering 2 wireless infrastructure programmes: 5G Innovation Regions (5GIR) and the Smart Infrastructure Pilots Programme (SIPP). Details of these programmes can be found in Section 2.1.1 of this report.

In line with the expectation of government departments to undertake comprehensive, robust and proportionate evaluations of their policy interventions^{[footnote 2]}, DSIT has commissioned KPMG to undertake programme-level evaluations of 5GIR and SIPP. The evaluations will consist of process, impact and economic evaluations for each programme:

• The process evaluations will examine the application process (running from May to July 2024) and the delivery of the 5GIR and SIPP programmes (running from April to August 2025).
• The impact evaluations will seek to identify and measure observed outcomes and impacts and assess the extent to which such outcomes and impacts are caused by/ attributable to the 5GIR and SIPP programmes. This will run from April 2025 to August 2025.
• The economic evaluations will seek to assess the value of outcomes realised at the point of the evaluation to determine the extent to which the 5GIR and SIPP programmes have realised benefits in excess of the costs of delivering the programmes. This will run from April 2025 to August 2025.

In delivering the evaluations we will work closely with the DSIT evaluation project team ^{[footnote 3]} and the Steering Group^{[footnote 4]} (hereafter referred to as the DSIT evaluation team and Steering Group respectively) that has been set up as part of the 5GIR and SIPP evaluations. The purpose of this scoping report is to set out the result of initial scoping and baselining phases of work for the evaluations, including the planned

1.2. Approach to scoping the evaluations

The approach taken to scoping and designing the evaluations follows the approach detailed in the HM Treasury Magenta Book^{[footnote 5]}.

The work undertaken for this report has been informed by a range of activities, including:

• Engagement with DSIT evaluation team and Steering Group to understand the purpose and priorities for the evaluations.
• Review of programme documentation.
• 2 familiarisation interviews, one with SIPP policy leads, technical advisors, and officials leading on the collection of benefits information; and 1 with 5GIR technical advisors and officials leading on the collection of benefits information. These were used to obtain information and understanding of the 2 programmes.

Based on this consultation and review of existing documentation, a comprehensive theory of change (ToC)^{[footnote 6]} for each programme was developed. These provide an overview of how the activities undertaken in both of the programmes are expected to feed through to the intended outcomes and end impacts using causal analysis of available evidence. Details of the ToC developed for each programme is included in Section 3.3.

The understanding gained as to the purpose of the evaluations and the programme ToCs informed the development of evaluation research questions. The research questions were developed in consultation with the DSIT evaluation team and the Steering Group. The final evaluation research questions agreed with these stakeholders are set out in Section 1.3 below.

This initial scoping work provided the basis for the subsequent development of appropriate evaluation methodologies. These methodologies, for each of the process, impact and economic evaluation were then tested and agreed with the DSIT evaluation team and the Steering Group. Detail on the methodologies is presented in Section 4.

1.3. Evaluation research questions

The evaluation research questions presented in Table 2 below reflect the original draft research questions set out in the Invitation to Tender (ITT)^{[footnote 7]} and the outcomes/impacts that are expected to be achieved – and that can be measured – within the timescales of the evaluations.

Evaluation research questions

Process evaluation

1) What went well and what could be improved with regard to the 5GIR/SIPP programmes, specifically:

a. in relation to the application process (from the point of view of Local Authorities (LAs) and DSIT)?
b. in relation to the design and delivery (from both a DSIT and Local Authority (LA)/partner organisation perspective)?

Impact evaluation

1) To what extent did the 5GIR programme achieve the outcomes it set out to achieve within the timescale of the evaluation and to what extent can these outcomes be attributed to the programme – specifically:

a. To what extent did the 5GIR interventions provide evidence of the realisable benefits of advanced wireless technologies?
b. To what extent were the 5GIR interventions sustainable post-government funding (ie networks continued after funding)?
c. To what extent were the benefits of advanced wireless technology adoption from the 5GIR programme documented and disseminated?
d.To what extent has the 5GIR programme fostered an emergent 5G ecosystem by enabling learning by doing?
e. To what extent is there evidence of the 5GIR programme encouraging and/or increasing long-term adoption of advanced wireless technology?

2) To what extent did the SIPP programme achieve the outcomes it set out to achieve within the timescale of the evaluation and to what extent can these outcomes be attributed to the programme – specifically:

a. To what extent has SIPP provided evidence of the benefits of utilising street furniture and other assets for network deployment?
b. To what extent have lessons learned from SIPP around the procurement and installation of technology been documented and disseminated?
c. To what extent has SIPP improved the understanding within the intervention LAs of the benefits of utilising street furniture and other assets for network deployment and how this can be delivered?

Economic evaluation

1) Is there evidence that the benefits from some of the use cases delivered by the 5GIR and SIPP programmes to date outweigh the costs?

Longer term impact and economic evaluation research questions (not within scope of this evaluation)

1) To what extent was the 5GIR programme able to achieve the outcomes it set out to achieve – specifically:

a. To what extent has the 5GIR programme driven economic growth?
b. To what extent has the 5GIR programme accelerated commercial investment in 5G and other advanced wireless technologies?
c. To what extent has the 5GIR programme fostered a 5G ecosystem?

2) To what extent has the SIPP programme improved the understanding and realisation of the benefits of utilising street furniture and other assets for advanced wireless network deployment?

3) To what extent do the benefits of the 5GIR and SIPP programmes outweigh the costs?

1.4 Data and evidence collection

The 5GIR and SIPP evaluations will rely on a breadth of data and evidence to inform evaluation analysis and the findings and conclusions in relation to each of the research questions for the evaluations.

A detailed exercise has been undertaken to identify what indicators, information and evidence will be needed, linking back to the ToC developed, and the research methods that will be employed to gather this evidence. More details on the approach that was taken to identifying data and evidence requirements is provided in Section 4.2.1 of this report.

A brief summary of the main research methods that will be used to collect data/information for the evaluations is set out below.

• Benefits tracker: All LAs participating in both the 5GIR and SIPP programmes are expected to return information on the activities and outputs delivered by their intervention along with associated outcomes and impacts. This will provide data and information for all aspects of the evaluations and the 6-monthly reporting.
• Interviews and focus groups:
  
  
  • Representatives from LAs, use case providers and network/tech providers involved in both the 5GIR and SIPP programmes respectively will be invited to participate in focus groups. The focus groups will be used to gather primarily qualitative evidence to inform the process, impact and economic evaluations.
    
  • Interviews with DSIT officials involved in the delivery of the 2programmes and UK Telecoms Innovation Network (UKTIN) officials will be undertaken to gather, again primarily qualitative evidence, to support the analysis of each of the process, impact and economic evaluations.

In total, using the above approach, it is expected that around 80 individuals will be invited to participate in focus groups or interviews for the evaluations.

• Surveys: A number of surveys will also be run over the course of the evaluations to help answer the evaluation research questions, including surveys of:
  
  

  • LAs (incl. successful and unsuccessful applicants) regarding the application process; and
    
  • use case providers and network/tech providers to better understand the commercial viability of both use cases and network provision as a result of the 5GIR programme.

• Review of documentation: We will review a wide range of documentation to help answer the evaluation research questions. Documentation including official and internal DSIT documentation; existing literature; and, documentation produced as a consequence of the interventions will all be used to support the analysis underlying the delivery of the process, impact and economic evaluations.

Sources of information that will be used to inform each element of the evaluation across both programmes.

Elements of evaluation	Published data	Benefits tracker	Focus groups/interviews: LAs	Focus groups/interviews: use case providers	Focus groups/interviews: network/tech providers	Focus groups/interviews: DSIT policy	Focus groups/interviews: DSIT comms/UKTIN	Survey	Documentation
Baseline	Yes	Yes	No	No	No	No	No	No	Yes
6-monthly reporting	No	Yes	No	No	No	No	No	No	Yes
Process evaluation: application	No	No	No	No	No	Yes	No	Yes	Yes
Process evaluation: design and delivery	No	Yes	Yes	Yes	Yes	Yes	Yes	No	Yes
Impact evaluation - 5GIR: benefits of advanced wireless technology	No	Yes	Yes	Yes	Yes	Yes	No	Yes	No
Impact evaluation - 5GIR: sustainable post funding	No	Yes	Yes	Yes	Yes	Yes	No	Yes	No
Impact evaluation - 5GIR: value documented/disseminated	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No
Impact evaluation - 5GIR: ecosystem	No	Yes	Yes	Yes	Yes	Yes	No	Yes	No
Impact evaluation - SIPP: benefits of using LA assets	No	Yes	Yes	Yes	Yes	Yes	No	Yes	No
Impact evaluation - SIPP: improved understanding of benefits	No	Yes	Yes	Yes	Yes	Yes	No	Yes	No
Impact evaluation - SIPP: lessons learned disseminated	No	Yes	Yes	Yes	Yes	Yes	Yes	Yes	No
Economic evaluation: benefits outweigh costs	No	Yes	Yes	Yes	Yes	Yes	No	Yes	No

Further details of the data collection methods which will be used over the course of the evaluations are provided in the main report and Appendix 3.

1.5 Analytical approaches to the evaluations

1.5.1 Process evaluations

The process evaluations will examine how the processes employed in both the application process and the design and delivery of the 2 programmes contributed to the 2 programmes’ outcomes.

Given the application process has recently concluded and to avoid lessons from this process being lost with the passage of time, the process evaluations will be conducted in 2 phases. The first phase, to be conducted between May 2024 and July 2024, will consider the application process and include aspects of the design of the programme. Then, following the conclusion of the programme in March 2025, the second phase covering the delivery of the programme will be conducted.

Data and evidence drawn from the range of sources detailed above will be brought together through thematic analysis to identify key themes and insights from the various stakeholders and data sources. Through this process we will assess how the specific processes and approaches adopted for both the 5GIR and SIPP programmes have affected the overall delivery of the programmes.

1.5.2 Impact evaluations

The purpose of impact evaluations is to assess what changes occur as the result of an intervention and the extent to which such changes can be attributed to the intervention, over and above what would have happened had the intervention not taken place.

As part of the scoping, a range of methods for the impact evaluations were considered, including experimental and quasi-experimental methods (QEM) as well as theory-based approaches (as set out in more detail in Section 4.3.2). This concluded that experimental methods would not be suitable and the timing of the evaluations precluded QEM being employed at the programme level (though a programme level QEM might be possible longer term). Moreover, whilst QEM might be possible for individual use cases, these would not be generalisable at the programme level and would carry notable resource requirement and risks around the availability and quality of data collection.

As a result, we propose a theory-based approach to the evaluations, specifically contribution analysis.

To deliver this approach we will combine evidence of the activity, outputs and outcomes reported through the benefits tracker with qualitative and quantitative evidence drawn from focus groups, interviews and surveys of various stakeholders in the programme. This combined evidence base will be used to test the causal links identified in the ToC for both programmes, from the delivery of activities and outputs through to the realisation of outcomes.

Whilst this approach will not allow us to precisely estimate the exact size of impact from the intervention, it will enable us to test the extent to which the pathways identified in the ToC materialised (or not). In this way, the approach will also provide useful contextual information around how and why such outcomes were achieved.

1.5.3 Economic evaluations

To evidence the scale and nature of costs and benefits for 5GIR in a proportionate way, a cost benefit analysis will be undertaken of 3-5 (ideally stackable) use cases as part of the evaluation. These will be selected based on the availability of data required to allow costs and benefits to be monetised robustly, and will seek to obtain coverage across a range of sectors and types of use case.

Following HM Treasury’s Green Book guidance ^{[footnote 8]}, the economic evaluations will monetise costs and benefits where possible. They will draw on data from intervention LAs as well as from other alternative information sources to inform the analysis and assumptions, e.g. drawing on existing literature. The analysis will account for additionality and all costs and benefits will be discounted in line with Green Book guidance.

For the SIPP programme, we will investigate the costs of the Publicly Available Specification (PAS) 191 standard multi-purpose poles. These costs will be compared against the potential benefits from the use cases deployed on the poles. The objective will be to produce a robust understanding of the nature and scale of benefits that can be expected by LAs installing PAS 191 poles as compared to their costs.

1. See: UK Wireless Infrastructure Strategy ↩

2. See: About us - Evaluation Task Force - GOV.UK ↩

3. The DSIT evaluation project team has overall responsibility for the evaluations and provides ongoing input and direction in relation to the evaluation such that it meets DSIT’s requirements; the team is comprised of DSIT officials. ↩

4. The Steering Group provides advice, guidance, scrutiny and challenge to the evaluations, with the aim of supporting the evaluations and ensuring the findings are robust and provide useful insights to build the evidence base in relation to digital infrastructure. It consists of DSIT officials covering the areas of policy, analysis, benefits realisation and technical expertise with respect to the 5GIR and SIPP programmes, together with a member of the Cabinet Office’s Evaluation Taskforce. ↩

5. HM Treasury 2020; Magenta Book ↩

6. 6 The ToC identifies the changes an intervention is seeking to make, how it is expected to happen and the measurable outputs, outcomes and impacts associated with the intended change. ↩

7. UK Shared Business Services Ltd (2023) Mini Competition against an existing Framework Agreement (MC) on behalf of Department for Science, Innovation and Technology Subject: 5G Innovation Regions and Smart Infrastructure Pilots Programme Evaluation. Sourcing Reference Number: PS23394 ↩

8. HM Treasury (2022) The Green Book: Central Government Guidance on Appraisal and Evaluation ↩