Research and analysis

Project Gigabit Evaluation Plan

Published 15 August 2025

1. Introduction

Building Digital UK (BDUK), an Executive Agency within the Department for Science, Innovation and Technology (DSIT) commissioned Ipsos UK and partners to undertake an evaluation of Project Gigabit, which includes the Gigabit Infrastructure Subsidy (GIS) interventions and the Gigabit Broadband Voucher Scheme (GBVS). This report sets out the recommended approach to the evaluation of Project Gigabit, and both the GBVS and GIS interventions project individually.

1.1 Description of the programme

Project Gigabit is a government-funded programme to enable To what extent do the changes in market position of beneficiaries differ by beneficiary type and size hard-to-reach communities to access fast, reliable gigabit-capable broadband. It targets homes and businesses that are not included in broadband suppliers’ commercial plans, reaching parts of the UK that might otherwise miss out on upgrades to next-generation speeds. There are several mechanisms through which Project Gigabit aims to provide this connectivity, including:

• the Gigabit Infrastructure Subsidy (GIS) interventions– which awards contracts which provide subsidies to network providers to make it commercially viable to deliver gigabit capable networks (those offering downloads speeds of at least 1,000 Mbps) to premises which are not expected to benefit from commercial deployments
• the GBVS – which aims to provide communities with rapid access to gigabit capable connections through providing vouchers, worth up to £4,500, to businesses and homes to cover the cost of installing gigabit capable networks in their area
• GigaHubs – which provides funding for public sector buildings such as GP surgeries, libraries and schools - in hard-to-reach parts of the UK to access gigabit capable connections

The GigaHubs element of Project Gigabit is not covered in this evaluation plan.

GIS Interventions

The intended procurement model was designed to ensure that rollout to those with lower connections speeds was provided in parallel to roll out to those with higher connection speeds.

The delivery of the GIS interventions builds on the Superfast Broadband programme model, which prior evaluations have shown to be highly effective in accelerating the deployment of faster broadband networks in rural areas. The processes to be used in the delivery of the GIS interventions are described in detail in the early process evaluation report, which also explains how these differ from the processes used in the Superfast Broadband programme.

As of Autumn 2024, according to Gigabit Infrastructure Subsidy programme Management Information, provided by BDUK, procurement activity (the award of contracts) had been completed in 36 local areas. A further 5 are in the tendering process. In total, it is expected that approximately 50 GIS contracts will be issued. There are three types of contracts issued through the GIS interventions:

• Type A contracts: smaller, local contracts for intervention areas of up to 10,000 premises, which are tenders that any network provider which meets the eligibility criteria can bid to deliver
• Type B contracts: Larger, regional contracts for larger intervention areas of up to 150,000 premises. These contracts have more restrictive eligibility criteria for network providers. A restricted procurement procedure is used to award contracts for larger intervention areas
• Type C contracts: The largest, cross regional contracts, where no appropriate market interest has been received through market engagement steps for proposed Type A or Type B procurement areas. These are called off from a single supplier framework agreement

So far, most contracts awarded have been Type A contracts (11 awarded, 4 in procurement), with eight Type B contracts awarded and 14 in procurement, with one Type C contract in procurement. The contracts issued so far cover approximately 586,100 premises at a cost of £942 million.

Gigabit Broadband Voucher Scheme (GBVS)

The GBVS provides funds to consumers, which network providers can use to allocate funds to projects to install gigabit capable infrastructure in areas that are unlikely to receive commercial rollout. As a first step of the GBVS, BDUK developed a list of premises and UPRNs which were eligible for support through the GBVS. This list is reviewed on an ongoing basis by BDUK. All network providers have access to the list of UPRNs.

Network providers can review the list and identify where they may want to locate a GBVS project (a collection of UPRNs in a specific location). Proposed projects can vary in size from tens of premises upwards, with no upper limit, although the largest current project passed 30,000 premises. The network providers develop a project specification and submit this for review by BDUK, which includes location, number of premises passed, network provider financial input and value of BDUK required for the project. BDUK review the project specification and can approve the project or reject it if the project does not meet eligibility criteria.

Once a project is approved, a network provider will advertise to premises in the project area that they are planning to build a gigabit capable network, utilising BDUK support. The process can also work in the opposite way, where a network provider advertises to the premises in a planned project area, and subject to them obtaining sufficient support they submit an application to BDUK. This is to generate interest in the network and to try to get potential customers to express an interest in taking a gigabit voucher for their premises. The customer expresses an interest to BDUK, who will then email a link to the customer to validate their interest and to check their eligibility. For eligible customers, a voucher is issued.

A network provider is given 12 months to begin installing the connection to the premises following the issuing of the first voucher for a project. Network providers can apply for an extension from BDUK if there are specific challenges in delivering the project.

Following the network provider building the gigabit capable network and providing an installation to the customer, the customer needs to agree to take the enhanced connection. Once they have agreed this and the connection is provided, the network provider informs BDUK, who then contact the customer to validate that the connection has been provided. With this confirmation, BDUK will provide up to the £4,500 voucher value to the network provider.

Network providers are paid via the vouchers up to the maximum BDUK contribution agreed in the project specification. If customers submit vouchers for a project which have a cumulative value higher than agreed BDUK contribution, then some of the vouchers do not trigger a BDUK payment to the network provider. If the cumulative value of vouchers claimed by customers is lower than the agreed BDUK contribution, then the network provider has to make up the financial shortfall.

Programme timeline

The timeline of the delivery of Project Gigabit was intended to be staggered. This was due to design of the individual programmes making up Project Gigabit. The large scale and complex nature of the GIS interventions, involving collecting detailed information about current and future network provision , identifying premises which will not receive commercial roll out, running large scale procurement and contracting processes, designing the engineering programme of work and then the delivery of large civil engineering projects has meant it has taken longer to roll out. Therefore, delivery of the civil engineering work as part of the GIS interventions is expected to run from 2024 to at least 2030.

By comparison, GBVS projects were expected to be small in scale, agile and to be delivered at a quicker pace, both in terms of designing and delivering projects. Therefore, the delivery of these projects is expected to be completed earlier than the GIS contracts, predominantly between 2021 and 2025 – although they can continue to be delivered alongside the GIS interventions.

As a result of this staggered timing of delivery, and the additional time taken for take-up and benefits to be realised, the evaluation will initially focus on the impacts achieved by the GBVS, before moving towards a focus on the processes and impacts of the GIS interventions and Project Gigabit as a whole towards the end of delivery.

Evaluation objectives

The objectives of the evaluation, set out in the Invitation To Tender and adapted in line with updates to the commission, are as follows:

• to undertake scoping activities to develop future research and evaluation design
• to develop a baseline to measure the impact of Project Gigabit, GIS procurements, and GBVS against
• to conduct process and impact and economic evaluations to explore the effectiveness of BDUK programme design and implementation

The evaluation is also designed to provide evidence to support:

• the Department’s communications around the impact from improved connectivity, including the outcomes of GBVS and GIS interventions individually, and Project Gigabit as a whole
• planning and where applicable monitoring in relation to Trade and Cooperation (TCA) subsidy control (future subsidy control law) and Infrastructure Projects Authority (IPA) reporting
• preparations for future scrutiny such as from external bodies i.e., Public Accounts Committee, National Audit Office, DSIT Select Committees, MPs and other external stakeholders

1.2 Objectives of this report

This evaluation plan aims to address objective 1, providing a framework for the evaluation of Project Gigabit which sets out a programme of research will meet objectives 2-5.

The table below sets out the key evaluation questions which must be answered to meet these objectives, and where in the report the approach to answering them can be found.

Table: key evaluation questions and relevant sections of the report which describe approach to answer questions

Key evaluation questions	Question detail	Project Gigabit	GBVS	GIS	Relevant section
Overall scope of evaluation	Provide evidence to address the outcomes of GBVS and GIS interventions individually, and Project Gigabit as a whole	Yes	Yes	Yes	All sections
What are the impacts of Project Gigabit on broadband infrastructure?	What is the extent to which, Project Gigabit, GBVS and GIS interventions have increased access to gigabit capable networks for non-commercial premises in the UK?	Yes	Yes	Yes	All sections
What are the impacts of Project Gigabit on broadband infrastructure?	To what extent have the Project Gigabit, GBVS and GIS interventions changed the speeds available and at what price?	Yes	Yes	Yes	All sections
What are the impacts of Project Gigabit on broadband infrastructure?	How did GIS, GBVS and GigaHub interventions interact to enhance outcomes achieved?	Yes	Yes	Yes	Section 4 and 5
What are the impacts of Project Gigabit on broadband infrastructure?	What effect has GIS had on the market position of contracted beneficiaries (suppliers)?	No	Yes	No	Section 5
To what extent do the changes in market position of beneficiaries differ by beneficiary type and size?		No	Yes	No	Section 3 and 5
What are the impacts of Project Gigabit on broadband infrastructure?	To what extent do Project Gigabit, GBVS and GIS have any spill-over, crowding out or crowding in effects (infrastructure and supplier market)?	Yes	Yes	Yes	Section 5
What are the impacts of Project Gigabit on broadband infrastructure?	Does the GIS intervention led to any changes to parameters of competition?	No	Yes	No	Section 5
What are the impacts of Project Gigabit on broadband infrastructure?	Were Project Gigabit, GBVS and GIS interventions proportionate (infrastructure only)?	Yes	Yes	Yes	Section 3 and 5
What are the impacts of Project Gigabit on broadband infrastructure?	Are the funding models efficient; and were sustainable networks created? (networks that are still in use to deliver broadband connectivity and will continue to be used in the next five years)	No	Yes	No	Section 3 and 5
What are the impacts of Project Gigabit on broadband infrastructure?	What are the environmental impacts of construction and maintenance of the publicly funded infrastructure?	Yes	Yes	Yes	Section 3
To what extent has Project Gigabit achieved its intended outcomes for end users, and what are the broader economic, social, and environmental impacts of the programme?	To what extent have the Project Gigabit, GBVS and GIS interventions affected end user take-up of services and at what speeds?	Yes	Yes	Yes	Section 4 and 5
To what extent has Project Gigabit achieved its intended outcomes for end users, and what are the broader economic, social, and environmental impacts of the programme?	What is the effect of Project Gigabit, GBVS and GIS interventions on end users of the rollout of gigabit coverage?  How has the outputs of the programme affected: Local economies (job creation and business growth (new businesses, size of business, turnover and productivity)? Quality of life for residents in the targeted areas? Provision and use of public services such as improved education and healthcare services? Environment; including the use of connectivity through changes in commuting and energy use and the resultant carbon emissions?	Yes	Yes	Yes	Section 4 and 5
To what extent has Project Gigabit achieved its intended outcomes for end users, and what are the broader economic, social, and environmental impacts of the programme?	How has the behaviour of individuals or organisations changed for those outcomes to come about?	Yes	Yes	Yes	Section 4 and 5
To what extent has Project Gigabit achieved its intended outcomes for end users, and what are the broader economic, social, and environmental impacts of the programme?	To what extent do Project Gigabit, GBVS and GIS interventions have any spill-over or agglomeration effects (local businesses, services and citizens), including: business and population agglomeration effects; new businesses and households entering upgraded areas or no longer leaving the area  Spill-over effects from agglomeration; for example changes in business offerings / public services	Yes	Yes	Yes	Section 5
To what extent has Project Gigabit achieved its intended outcomes for end users, and what are the broader economic, social, and environmental impacts of the programme?	Were Project Gigabit, GBVS and GIS interventions proportionate? (benefits)	Yes	Yes	Yes	Section 5
How effectively and efficiently has the programme been implemented, and what processes have contributed to achieving the intended impacts?	Was the delivery of the GIS intervention effective and efficient?	No	Yes	No	Section 3 and 5
How effectively and efficiently has the programme been implemented, and what processes have contributed to achieving the intended impacts?	What are the lessons for future policy designs and implementation?	No	Yes	No	Section 3
How effectively and efficiently has the programme been implemented, and what processes have contributed to achieving the intended impacts?	What processes have been most effective in achieving the intended impacts of Project Gigabit?	Yes	Yes	Yes	Section 3 and 5
How effectively and efficiently has the programme been implemented, and what processes have contributed to achieving the intended impacts?	What barriers have been encountered in realising the impacts beyond the outputs of programme delivery?	Yes	Yes	Yes	Sections 4 and 5
To what extent has the programme delivered value for money, considering the economic, social, and environmental impacts in line with the Business Case?	Provide evidence to address the cost-effectiveness of the investments made. What is the value of the direct and indirect economic benefits of the Project Gigabit for:  Local businesses? Local labour markets? Property values Environmental impacts? (Existing modelled benefit used in the Programme Business Case)  What are the other outcomes which can be substantiated as additional and monetised, through the output of Project Gigabit?	Yes	Yes	Yes	Section 5

1.3 Methodological approach

This report is based on the following research activity:

• a review of programme management information, including the business case, intervention area data, OMR responses, GBVS monitoring data and commercial data
• a review of wider, contextual published information, such as news items, Connected Nations data, and regulatory decisions
• a review of potential data sources which could be used to support the evaluation
• stakeholder interviews with nine stakeholders involved in the GIS interventions and GBVS delivery at BDUK

1.4 Structure of the report

The remaining sections of this report are structured in the following way:

• a presentation of the theory of change for Project Gigabit
• an outline of the process evaluation required for GIS contracts
• a description of the data sources which could be used to support an impact evaluation
• a discussion of potential approaches to an impact evaluation
• details of how the evaluation can be implemented

2. Theory of change

This section presents the theory of change for Project Gigabit, including the background and context for the evaluation. The theory of change will provide the overall analytical framework for the study.

2.1 BDUK benefits framework

Table: driving growth in the economy

Benefit type	Outcome / Impact	Description
Productivity growth	Increased business productivity	Helps workers to become more efficient and utilise time and skills. Usually measured through GVA.
Greater competition	New businesses established	Allows new business opportunities, such as the creation of start-ups through cloud computing.
Human Capital	Increased ICT skills and wider educational attainment	Gives workers the skill set to become more productive and utilise new technology.
New Markets	Increased international trade	Allows businesses to access international markets and expand globally.
Employment	Employment (safeguarded or new)	Helps create new job opportunities from increased skill sets and removes physical barriers through remote working.

Table: enabling public sector efficiency

Benefit Type	Outcome / Impact	Description
Public sector efficiency	More efficient use of public finances and resources	Public sector take-up of improved connectivity allows public services to offer greater efficiency in management, administration and through the reallocation of tasks and work hours due to digitalisation. Examples include online GP booking systems, libraries, or schools being upgraded to fast speeds which in turn can lead to efficiencies in teachers’ workloads etc.
Public sector digitalisation	Improved public access to digital service	Better access to public sector services, such as applying for a new passport or driver’s licence online.
Central government learning	Cross-government learning for large procurement Programmes	Future government programmes can learn from existing programmes and replicate successful features.
Demographic change (potential disbenefits)	Pressure on local services	The improved broadband infrastructure may attract higher migration to areas which in turn may increase demand for public services beyond what can be provided and may result in higher dissatisfaction in quality of service.

Table: reducing the digital divide and providing public value

Benefit Type	Outcome / impact	Description
Technology	More energy efficient technologies	Technologies which enable individuals and firms to be more energy efficient. For example, increased numbers in online conferences rather than in person reduces demand for travel. Also, businesses switching to less energy intensive cloud computing from physical servers.
Infrastructure	More efficient infrastructure	Fibre networks are less energy intensive than copper networks and also require less maintenance.
Behaviour change (potential disbenefits)	Household energy consumption	A reduction in the commuting demand for transport reduces carbon dioxide and nitrogen oxides (NOx) emissions. A future way of working enabled by gigabit networks could also have a significant impact on noise and NOx emissions.

Table: stimulating the broadband market

Benefit type	Outcome / impact	Description
Stimulating the broadband market	Stimulated private sector partnerships and investment	Public sector intervention can stimulate the private sector to invest in previously commercially un-viable areas.
Stimulating the broadband market	Market failure addressed through appropriate intervention	Reduces barriers to entry such as costs, location and monopoly power by giving businesses the ability to access new and international markets.
Stimulating the broadband market	Increased competition in the market, including small suppliers	Addresses market failure by reducing barriers to entry, creating increased competition and gives consumers more choice.
Stimulating the broadband market	Innovation and knowledge of new technologies	Helps support the reach of next generation technologies, whilst increasing efficiency and harnessing potential savings.
Distorting the broadband market (potential disbenefits)	Crowding out commercial build investment	Programme build areas may divert resources from commercial build areas, therefore resulting in a delay in coverage to other areas.
Distorting the broadband market (potential disbenefits)	Consolidating market power	Market distortion resulting in strengthening of the position of monopolist which could result in negative impacts on customer service, infrastructure and price.

Overarching theory of change

This section sets out an overarching theory of change for the programme. This builds on the Theory of Change developed for the previous Superfast Broadband programme, which preceded Project Gigabit and aimed to achieve similar objectives.

Infrastructure effects

The first order effects of the Project Gigabit will primarily be in terms of the additional gigabit capable broadband infrastructure brought about by subsidies by BDUK. The extent of these effects will be determined by a number of factors:

1. Supplier behaviour during the tendering process: The underspend and take-up clawback mechanisms of the contract should, in principle, have eliminated systematic incentives for suppliers to overstate the gap funding requirement (since any overstatement of costs at the tendering stage would be recovered via the underspend clawback mechanism , and any understatement of future revenues would be recovered via the take-up clawback mechanism). Understating expected costs or overstating take-up expectations would result in the supplier ultimately taking a loss. However, these assumptions only hold true if there is competition for the GIS interventions contract, or the supplier believes there is competition for the contract, in the absence of competition suppliers could understate take-up or overstate the gap funding requirements to shift commercial risk from the private to the public sector (although the take-up clawback should still minimise this risk).

2. Crowding out: The provision of subsidies for gigabit capable broadband investment may have had a negative impact on other areas if suppliers face resource constraints – either in the labour market or in financial markets (for smaller suppliers). If firms are not able to expand their overall capacity to deliver the programme of subsidised infrastructure improvements, then this may result in delays or abandonment of schemes planned without subsidy in ‘grey’ or ‘black’ areas (partly offsetting the effects of the programme in ‘white’ areas). This may be more likely for the Project Gigabit than for the previous Superfast Broadband programme due to the areas being delivered to (targeting less commercially viable areas meaning network builds will be more expensive) and the current level of vibrancy in the market.

3. Variability across space and time: The pattern of effects of the programme could be expected to vary across areas with different characteristics. On the presumption that the supplier investment decision can be simplified to a basic Net Present Value (NPV) criterion, then it might be expected that local variability in the effects of the programme will be driven largely by factors determining the cost of installation and maintenance (such as local topography, characteristics of the local network – such as the prevalence of exchange only lines, distance from regional population centres) and demand for the technology (e.g. incomes of households, the industrial structure of the local economy, or the speed of existing broadband services). These factors are also unlikely to be static over time (technological progress may result in reductions in the cost of installation, while demand for higher speeds may rise as more data intensive applications emerge).

4. Additionality: Making subsidies available for infrastructure delivery involves a risk that private sector providers face an incentive to seek public funds for investments that they would have made anyway (enabling them to earn higher rates of return). Network providers also have a commercial incentive to invest in all projects where their expected rate of return exceeds the cost of their capital. The extent to which the Project Gigabit will produce an increase in gigabit capable broadband availability will depend on the effectiveness of the mechanisms used to allocate public resources to areas that would not have received commercial rollout (i.e. upgrades to networks that would not have been deemed by suppliers to be commercially viable without a public subsidy), and that contractual arrangements make the rate of return equal the cost of capital. The programme involves a number of processes that are designed to maximise the extent to which public funding is directed at marginal activity:

Allocation of subsidies: Indicative subsidies were assigned to areas on the basis of an ex-ante assessment (by BDUK) of the number of non-commercial premises. This budget could be revised once the contract was agreed.

Open Market Review and public review in the GIS interventions: BDUK manages an Open Market Review (OMR) and public review process, which became a national, rolling OMR process in 2022 (the public review process was not continued for the national, rolling OMR). The OMR process involves BDUK requesting network providers to describe their commercial plans to roll out gigabit capable coverage over a defined time window, and this request is issued to network providers three times a year. This resulted in an identification of premises where there were no commercial plans to roll out gigabit capable broadband (‘white’ premises), premises where one provider was offering or expected to offer gigabit capable broadband services (‘grey’ premises), and premises where multiple providers were offering or expected to offer gigabit capable broadband (‘black’ premises). Some premises were placed ‘under review’, where the premise had been marked as potentially having commercial gigabit capable networks provided, but these plans were not certain.

Review of eligibility of GBVS projects: The GBVS team reviews the projects submitted by network providers to ensure that the projects are covering only eligible premises, and that they do not duplicate other public funding through the wider Project Gigabit. This is to ensure that the GBVS projects only cover areas that are not going to receive commercially funded networks. The eligibility criteria were amended by BDUK so that eligible premises are categorised as ‘white’ premises from the rolling, national OMR.

Tendering in the GIS interventions: BDUK provided a list of ‘white’ premises for the tendering exercise for each contract in which BDUK sought to procure additional investment in upgrading the local telecommunications infrastructure. Competing suppliers outline which premises they proposed to cover for the available funding. Suppliers were required to provide estimates of the overall costs associated with delivering the project, the number of premises they aimed to cover, and take-up assumptions (determining expectations of future revenues), which determined the overall level of subsidy to be offered.

Underspend: Protections for the public sector against the risk that suppliers overestimated their delivery costs were put in place through the introduction of a mechanism to recover any underspend. For the GIS interventions, the principle underlying contracts was that the supplier paid first, then BDUK. In the event of any underspend, funds are returned to BDUK through a clawback mechanism. In GBVS, vouchers are paid up to a maximum value, and any vouchers which take the value of the voucher project above the agreed value are not paid by BDUK, and where insufficient vouchers are claimed for a project the network provider pays the difference.

Take-up clawback: In GIS interventions contracts, further protections for the public sector were introduced through take-up clawback clauses in contracts. If take-up proved to be higher than anticipated at the tendering stage, then suppliers are required to return a share of the excess revenues to BDUK.

Vibrancy of the broadband market: Stakeholders engaged in familiarisation consultations and a review of the broadband market have highlighted current challenges in the broadband market. Over the preceding years (up to 2021), the number of network providers in the UK had increased significantly, due to regulatory encouragement, access to finance and venture capitalists identifying potential long-term returns from providing gigabit capable infrastructure in the UK (as coverage in the UK lagged behind other developed countries). More recently, supply chain issues, increases in the cost of finance and the cost-of-living crisis (which has reduced the expected take-up of gigabit capable connections) has led to a contraction in the market, and some market consolidation taking place. There is a risk that the reduction in network providers could reduce interest in rural deployment of gigabit capable networks (with providers concentrating on areas where there is a higher commercial return) and potentially limiting interest in Project Gigabit contracts if subsidies are insufficient. There is also a further challenge in that challenges in the broadband market may mean that Project Gigabit deployment (subsidised deployment) is in competition with, and prevents, commercial roll out of gigabit capable networks. Commercial deployments would expect to have higher returns on the investment than the areas receiving subsidised coverage, as they are in more commercially viable areas (and do not require public subsidy).

Supplier behaviour during OMR process: Some suppliers may see an incentive to understate their commercial plans during the OMR process to increase the likelihood they are able to secure a subsidy for activity they saw as not commercially viable without subsidy. Discussions with BDUK and network providers suggested, however, there may be disincentives for this type of behaviour in that an understatement of commercial plans may risk the emergence of a subsidised competitor. A further risk is that suppliers use the OMR process to overstate their commercial plans to protect themselves from a subsidised competitor. The assessment of OMR n responses aimed to minimise this risk as the practicalities of OMR responses were also assessed (whether a network providers proposed build plans were achievable), and where BDUK felt commercial plans were unrealistic areas were marked as ‘under review’ and could be changed to ‘white’ by BDUK.

The different strands of Project Gigabit can interact with one another to generate outcomes and impacts. These interactions include:

Increase in size and scale of network providers: Network providers benefiting from subsidies could grow in size, in terms of the number of customers they have, their turnover and the number of premises their networks pass. This would include network providers benefiting from GBVS, GigaHubs and GIS interventions contracts. With the enhanced size, these network providers may subsequently be able to access further Project Gigabit funding – through Type C or Type B GIS interventions contracts. This could then further enhance the size of their business.

Increase in geographical coverage of network providers: Network providers benefiting from subsidies could also increase their geographic coverage, entering area that they previously had no networks, or expanding the coverage of their existing network. This enhancement of their existing networks could also increase their ability to compete for subsequent further Project Gigabit funding. This could lead to enhanced competition for contracts – improving the value for money for BDUK, but also further enhance the size of the network providers business.

This process is summarised in the logic model below.

Project Gigabit Logic Model – Infrastructure Effects of Project Gigabit

Economic impacts

In addition to the infrastructure effects described above, Project Gigabit may result in second order effects through raising the performance of the economy. A range of economic impacts might be anticipated at the local and national levels:

Productivity gains: A large number of studies have shown that faster broadband stimulates productivity growth. Adoption of gigabit capable broadband could raise the productivity of local firms in a number of different ways (though such improvements may take time to arise and require complementary investments to take advantage of higher speeds):

Adoption of complementary technology: The availability of gigabit capable broadband may enable the adoption of complementary data intensive technologies that would not have been viable at lower speeds, enabling them to produce higher levels of output (GVA) with the same resources.

Product or process innovation: Firms may also be able to use the technology to drive forward product or process innovations that increase the value of their output or reduce the use of factor inputs. The availability of broadband has been shown (in the short term) to stimulate research and development activity and the introduction of new processes and products.

Raising teleworker productivity: Firms may also benefit from any effects through which the availability of gigabit capable broadband enables workers to work from home more effectively. Such effects could arise from the reallocation of commuting time to productive activities (an effect assumed in the BDUK’s benefits model) or raise the efficiency of their working hours.

Turnover and employment growth: The adoption of gigabit capable broadband may also aid firm growth directly by opening new channels to market (e.g. through enabling them to integrate into global supply chains), or indirectly if any productivity gains resulting from the adoption enable them to lower output prices or raise quality and displace market share from competitors.

Sorting effects: The programme may also result in local economic benefits via the spatial reallocation of economic activity. A number of studies have illustrated that the availability of broadband weakens the pull of agglomeration and makes activities viable in less central locations, with the employment impacts associated with the availability and adoption of broadband often found to be stronger in rural or less central locations than in metropolitan urban areas. This suggests the programme could lead to ‘sorting effects’ in which areas benefitting from the programme attract firms located in other areas, resulting in positive local economic impacts (though little, if any, change at a national level). Such a process could also trigger the in-migration of skilled labour, encouraging further agglomeration of economic activity in the programme areas (and positive spill-overs in the form agglomeration economies). Clearly, attraction of firms from other areas also has the potential to place pressure on local prices (encouraging lower productivity firms to reduce their output or relocate to lower cost locations). Many of these effects could be expected to play out over the medium term (three to five years) and may not yet be visible given the time that has elapsed since the start of the programme.

Safeguarding of economic activity in previously low connectivity areas: Improved broadband infrastructure may help some areas retain economic activity that would have otherwise been lost to other high connectivity areas. While many workers may be able to adjust to such local economic shocks by relocating, retraining or commuting to more buoyant local economies, some may be unable to do so (as a consequence of frictions in the housing market, for example). These types of ‘fixed factor’ problems could produce local issues of long-term unemployment and permanent losses of productive capacity (i.e. hysteresis effects).

Labour market participation impacts: The enablement of gigabit capable broadband in low connectivity areas could also have further economic benefits though increasing labour supply. The BDUK benefits framework suggests that these effects could arise through enabling carers or those with disabilities to enter the labour market through teleworking. However, it is plausible that labour supply effects could occur through other mechanisms. For example, those in (or on the verge of) retirement may re-enter the labour market if they are able to telework from the location in which they chose to retire. Equally, if gigabit broadband enables previously unviable economic activities to be provided in rural or other types of low connectivity areas, then the jobs created may have features (higher wages, greater flexibility, better working conditions) that are attractive to residents that are economically inactive.

Project Gigabit Logic Model – Economic Impacts

2.2 Wellbeing impacts

The previous section describes the potential impacts of the programme on workers. However, households may also benefit from the technology through their consumption of the technology (though there are also a range of possible disbenefits that may arise), as outlined below:

Consumption benefits: Improved access to faster broadband may produce a range of consumption benefits for households arising through improved choice, quality and time savings. Most obviously, faster broadband speeds will allow consumers to access a range of entertainment and media services that depend on high bandwidths (e.g. streaming services or smart devices). Benefits may also arise from access to more extensive on-line marketplaces that allow consumers more choice or to obtain savings – and potentially free up time that would have otherwise been spent travelling to retail or other centres. It should be noted that a shift to on-line consumption patterns could be accompanied by disbenefits if it reduces the commercial viability of in-store retail services. The loss of retail outlets may reduce the vibrancy of town centres (reducing the well-being of residents of those communities) as well as produce digital exclusion issues amongst those that are unable to take advantage of increased digitalisation (because they are unable to pay or because they do not have the skills to do so). Such effects may not be permanent if town centres can adjust to changing consumption patterns - in the long run, such effects could be expected to lead to reduced commercial rents, encouraging the redeployment of those spaces for alternative uses. The COVID-19 pandemic has clearly accelerated these trends as the closure of non-essential retail has forced households to shift their consumption on-line (and there are signals that this shift may have some permanence).

Teleworking and leisure time: Greater opportunities for teleworking may produce benefits that exceed any effect on the productivity of the worker and associated wage income. Households newly able to work remotely may derive additional benefits from extra leisure time gained from reduced commuting times and travel costs. The well-being gain may not always be positive, however, if gigabit connectivity encourages workers to engage with work outside of normal working hours. These types of issues are being explored by BDUK in on-going work to understand the public value impacts of the programme.

Social interaction and public sector engagement: Faster broadband may also open new modes of communication between residents. While use of email and social media may not be dependent on higher bandwidths (and can be straightforwardly used via mobile telecommunications networks), the COVID-19 pandemic has popularised the use of video conferencing (previously used for remote meetings in a business context) as a mode of interpersonal communication. This technology requires greater bandwidths and subsidised coverage has the potential to improve well-being by supporting more extensive social interactions within and beyond the communities in which residents live (potentially reducing social isolation for some). In addition to this, the enhanced connectivity can help individuals with public sector engagement, such as securing GP and other health service appointments, engagement with other public sector organisations (such as libraries, claiming benefits, making payments etc.).

Social costs: Greater on-line social interaction may not always be positive. There is evidence that for some groups, greater use of social media is associated with lower levels of self-esteem. Internet addiction (i.e. compulsive desire to use the internet) has also been an area of recent clinical investigation – and has been found to be associated with depression and low self-esteem. The direction of causality is unclear – i.e. internet addiction may be a symptom of underlying emotional disorders, rather than a cause – but it should be at least acknowledged that improved broadband connectivity has the potential to produce negative subjective well-being effects in some users.^{[footnote 1]}

Perceptions of inequity: Project Gigabit also has the potential to address perceptions of inequity relating to the locations of major investments in infrastructure. For example, focus groups undertaken by University College London^{[footnote 2]} revealed a perception that recent investments in infrastructure have exacerbated disparities in amenities and mainly benefitted those that were already affluent. Although clearly the programme cannot tackle these issues in their entirety, bringing gigabit broadband coverage to rural areas that would not have otherwise been covered by commercial deployments has the potential to at least alleviate these types of public concerns. However, consideration may need to be given to the possibility that the programme exacerbates these perceptions in some areas (e.g. in cases where communities have not been included in the build plans of local schemes, or certain groups being able to benefit from enhanced connectivity). The evaluation can explore how Project Gigabit contributes towards these perceptions of inequality in the areas that have received enhanced connectivity through interviews with residents of impacted areas.

Technology induced disagglomeration: As highlighted above, improved gigabit broadband connectivity may encourage the relocation of firms to rural areas. This may require their workforces to make relocation decisions to avoid episodes of unemployment, maintain their incomes, or reduce commuting times. In these cases, the well-being impact of gigabit broadband coverage may not be positive (and may indeed be negative).

Rural population growth: Migration of population to rural areas could also lead to pressures on local housing markets. This could also have a negative impact on the well-being of residents for example, if it increases equilibrium rents or stimulates house building activity on previously undeveloped land (creating disamenities for existing residents). Additionally, rural population growth could feed through into pressures on public services (if supply does not expand to meet demand, as discussed below) or create other negative externalities such as greater congestion on rural road networks (and associated impacts on air quality).

Composition of local populations: Finally, while increased social connectivity may promote greater community cohesion, migration of population to rural areas could have the opposite effect if it disrupts settled patterns of community life.

Project Gigabit Logic Model - Wellbeing impacts of Project Gigabit

2.3 Environmental impacts

A core assumption of this report is that upgrading to faster internet speeds changes social behaviour by increasing the feasibility of working from home (videoconferencing, online events etc.), which consequently results in fewer commutes to the individual’s office / workplace leading to reductions in carbon emissions and air pollution, and positive impacts on human health and biodiversity. 

In the longer term this may lead to reduced demand for road transport, with an associated impact on net emissions. However, rebound effects may also exist if there is an increase in trip demand or a shift to less frequent but longer distance commuting,^{[footnote 3]} which would reduce the net impact on transport induced emissions. More working from home may also result in rebound effects in household energy consumption, with higher net energy demands across multiple households working from home, compared to the energy economies of scale in an office setting (noting that offices may not immediately close or downsize in response to working from home, meaning that workers will be contributing to energy demands both at home and in the office during the period where firms adjust to increased levels of remote workers).

In the longer-term, there may also be net emissions impacts from the increased demand for rural/greenfield land away from the cities, as people are able to work from home and base themselves further from geographic clusters of offices in cities. This could have impacts on emissions, as well as habitat loss and biodiversity.

There are also emission impacts from the infrastructure works required for the installation of fibre cables, leading to local transport disruption and a negative impact on transport induced emissions in the short-term.

However, the shift to new infrastructure materials can be expected to reduce the number of repair trips (based on evidence that fibre networks require fewer repair trips than copper cable networks, and a switch to new technology (e.g., cloud computing) which has been found to be more energy efficient). However, this could lead to rebound effects if higher usage of data and technology follows, with implications for the net impacts on emissions. It is also necessary to account for the embedded carbon from the value chain.

Project Gigabit Logic Model - Environmental Benefits

3. Process evaluation

This section presents the approach to a process evaluation of the GIS interventions. It sets out the key evaluation questions and how evidence will be collected to analyse the effectiveness of the GIS interventions. It should be noted that BDUK do not require a process evaluation of the GBVS project.

3.1 Process evaluation framework

The processes used to deliver the GIS interventions are set out in the early process evaluation report. The table below presents the process evaluation questions which can be used to assess how effectively the project is being delivered, where information could be collected from to assess the key process evaluation questions in order to identify areas where processes could be improved. The process evaluation questions in the first three sections of the table have been assessed in the early process evaluation which was completed in 2023. However, a revisiting of these questions may be required following the implementation of more GIS interventions contracts.

Table: early process evaluation framework

x = evidence available

Process	Evaluation questions	Metrics	Management information	BDUK delivery staff interviews	Network provider interviews	Local body interviews
Programme design	How effective was the market engagement exercise to discuss programme design?	Number of network providers invited / attended	x	x	x
Programme design	How effective has the programme design been in ensuring non-commercial premises are supported?	Proportion of premises in build plans that were originally modelled as uncommercial	x	x
Programme design	How effective has the programme design been in ensuring interest from a wide range of network providers?			x	x
Programme design	How effectively have the different elements (GIS interventions, GBVS and Gigahubs) of the programme been designed to complement other aspects of Project Gigabit		x	x	x
Identification of intervention areas	How accurate was the F20 data in identifying non-commercial areas?	Proportion of premises in “” included in commercial roll out plans (via OMR)	x	x
Identification of intervention areas	Did BDUK have the required knowledge of locations and premises to accurately identify intervention areas?			x		x
Identification of intervention areas	How frequently have intervention areas had to be changed due to changes in commercial roll out plans?		x	x	x
OMR and public consultation	How effectively have BDUK engaged with network providers to collect coverage data?	Number of network providers providing data	x	x	x	x
OMR and public consultation	Did all relevant suppliers respond to the OMR and public consultation process?	Number of network providers providing data	x	x
OMR and public consultation	Did the information and guidance published by BDUK provide sufficient information for network providers to complete the OMR process				x
OMR and public consultation	Did the invitation to participate clearly state the requirements of the OMR process?			x
OMR and public consultation	How appropriate is the frequency of the data request?			x	x	x
OMR and public consultation	What resources are required to provide data to BDUK?	Time required to complete OMR response			x
OMR and public consultation	Did BDUK and their contractors have the required capacity, skills and knowledge to evaluate the validity of the information received?			x	x	x
OMR and public consultation	Were the assessment criteria used to assess the validity of roll out plans and coverage appropriate?			x	x	x
OMR and public consultation	Did BDUK provide clear and timely feedback to network providers that had roll out plans rejected / partially accepted?	Time between submission and feedback	x	x	x
OMR and public consultation	How effective was the feedback in supporting network providers to resubmit their OMR response?	Number of rejected OMR responses which were subsequently accepted	x	x	x
Tendering and procurement	How effectively were suppliers engaged with to ensure interest in the programme procurements?			x	x	x
Tendering and procurement	How accurate was the procurement pipeline in indicating when network providers would be expecting to undertake procurement activity?		x	x	x
Tendering and procurement	How effective was the procurement activity in securing bids for programme procurements?	Number of bids submitted for each procurement Scores of tenders submitted	x	x	x	x
Tendering and procurement	How far did the tendering advice provided improve the effectiveness of the tendering process?	Scores of tenders submitted	x	x	x
Tendering and procurement	How effectively did the tender documents explain the requirements of the tendering process?	Number of queries submitted about tenders	x	x	x
Tendering and procurement	Was sufficient time provided to enable suppliers to prepare a bid?				x
Tendering and procurement	What resources are required to provide a bid to BDUK?				x
Tendering and procurement	Did the tendering process yield sufficient information on all relevant aspects to make effective decisions?			x
Tendering and procurement	How appropriate were the assessment and scoring criteria used in the procurement?			x	x	x
Tendering and procurement	Were there any significant differences in procurement outcomes between the different contract types?	Cost per premises upgraded by different
Tendering and procurement	Was the process completed in a timely manner?			x	x	x
Tendering and procurement	To what extent did participation in other Project Gigabit interventions support network providers ability to tender?			x	x
Tendering and procurement	To what extent did participation in other Project Gigabit interventions impact the quality of tenders received?		x	x
Contracting	Were the terms of the contract correctly specified to ensure delivery of the network build and required reporting (both timeliness and quality)?			x	x
Contracting	Did the clawback arrangements in the contract work protect the public sector sufficiently from possible threats to value for money?	Value of contract taken through underspend and take-up clawback mechanisms	x	x	x
Contracting	To what extent did the staggering of the contracts affect efficiencies in the delivery of the contract?	Proportion of contracts delivered to planned timescales	x	x	x
Contracting	To what extent did the contract ensure that premises with lower speeds were connected at the same time as those with higher speeds?	Connection date by commerciality level	x	x	x	x
Contracting	How effectively did the ongoing OMR process interact with the contracts with network providers?			x	x
Project delivery	How effectively did local authorities support the delivery of the GIS interventions contracts?			x		x
Project delivery	To what extent did the project management approach ensure the project was delivered on time, to budget, and in line with the project aims?	Proportion of projects delivered in line with initial expectations	x	x	x	x
Project delivery	Were change requests responded to a timely manner?	Average time to respond to change requests	x	x	x	x
Project delivery	Were the changes to contracts due to change requests made at appropriate times to support the delivery objectives?			x	x
Project delivery	To what extent di the change requests lead to appropriate premises being included in the intervention area?		x	x	x
Project delivery	How effectively were risks and managed?		x	x	x	x
Project delivery	Did suppliers have sufficient (financial and operational) capacity to deliver at the scale anticipated? Did the programme have any adverse effects on parallel programmes of investment?			x	x
Project delivery	Did suppliers have sufficient (financial and operational) capacity to deliver at the scale anticipated? Did the programme have any adverse effects on parallel programmes of investment?			x	x
Project delivery	What challenges did suppliers face in delivering the contracts and how did BDUK support overcoming the challenges?			x	x	x
Project delivery	How effectively did network providers supply data to BDUK (both timeliness and quality)?			x	x
Project delivery	Were there differences in project delivery by type of contract?	Proportion of contracts delivered in line with initial expectations by type of contract	x	x	x	x
Project delivery	How effectively did the Local Area Reviews interact with the contract delivery?			x	x

3.2 Approach to assessing process

Stakeholder interviews

It is proposed that the process evaluation will include a series of qualitative interviews with project participants and wider stakeholders. The type of stakeholders to be interviewed, and the information to be collected from them, is described below:

BDUK delivery staff: These individuals will be able to provide information about the design of the project, engagement with local organisations, the OMR, public consultation and tendering and contracting processes, alongside project delivery. The staff may also be able to provide details about the Programme context and how the Programme fits with other Government strategies. The interviews will mainly collect qualitative information; however, they may also be used to fill gaps in the Management Information and monitoring data or to clarify inconsistencies. 

Programme beneficiaries (network providers that have received funding from BDUK): These interviews will primarily provide information about the OMR process, tendering, contracts and project delivery. 

Non-beneficiary network providers: The individuals will represent network providers that did not deliver any GIS interventions contracts. This will include both network providers that unsuccessfully bid for contracts, and those that decided not to tender for opportunities. The interviews will explore the OMR process, motivations for tendering/not tendering, the tendering process and the effect non-participation has had on their organisation. 

Local authorities / Devolved Administrations: Although local authorities  are not responsible for the delivery of GIS interventions contracts, they are involved in the oversight of contracts and support with the delivery where possible. Therefore, interviews with local authorities and Devolved Administrations will be used to explore how effective the relationship is between BDUK and local authorities and Devolved Administrations, how data and information is shared and how local authorities and Devolved Administrations support the delivery of projects. 

It is proposed that over the course of the evaluation, process evaluation interviews are undertaken with 50 stakeholders:

All project beneficiaries: It is anticipated that this will include up to 10 interviews.

A selection of network providers that have not benefitted from the Project Gigabit: It is recommended that 10 interviews with non-beneficiaries are completed – with six interviews with unsuccessful applicants and four with non-applicants.

BDUK stakeholders: The research team would need to speak to BDUK staff involved in each stage of the design and delivery, and this is anticipated to include 20 interviews with BDUK staff involved in the design and delivery of the programme.

Local authorities / Devolved Administrations: A selection of 10 local authorities / Devolved Administrations would be interviewed. These would need to include local authorities / Devolved Nations with each type of GIS interventions contract in the area – with at least three local authorities / Devolved Administrations with a Type A, Type B and Type C contract in their area. 

Management Information

The process evaluation will draw on different types of information and data. The information collected from these data sources will be triangulated with information provided in the stakeholder consultations to provide consistent answers to the key process evaluation questions. 

The Project Gigabit already collects a large volume of information and data, through project monitoring and the application process, the OMR and public consultation processes. The evaluation will use the information already collected. This information and data will include:

• OMR responses – number of responses, results of responses

• public consultation responses – number of responses and result of responses made 

• list of premises included in the successful bidders application

• financial profiles included in the successful applicants bid

• example contracts used in the project 

• change requests – a log of change requests for contracts and responses from BDUK

• quarterly project monitoring data and reports – covering rollout, take-up and wholesale usage

• information showing Internet Service Provider (ISP) FTTP rollout plans (prior to Programme inception and current)

• project / programme meetings attendance and minutes

Where information required for the process evaluation is not included in these reports or data sets, or the data collection process is not providing sufficient / timely information, the evaluation team will work with BDUK to expediate the data collection process or explore alternative data sources.

Outcomes data, data collection plan and descriptive analysis

This section sets out a review of the available data sources of relevance to an evaluation of Project Gigabit. Sources have been reviewed with a view to establishing the extent to which they could be used to explore the outcomes and impacts defined in the Theory of Change. It also presents the descriptive analysis that will be used to assess the subsidy control evaluation questions.

Unit of analysis

An evaluation of the Project Gigabit needs to, as far as practicable, establish the outcomes at four key levels:

1. Direct impacts on the availability of gigabit capable connections: This includes the availability of connections, take-up of connections and analysis of speeds available.

2. Direct impacts on the finances of the Programme beneficiaries: This will include an assessment of the financial implications of providing GIS interventions contracts on the Programme beneficiaries.

3. Indirect impacts on local areas benefitting from the programme: This would include spill-over effects of the connectivity Project Gigabit provides on downstream economic, social, public service provision and environmental outcomes in the relevant areas are also of interest here.

4. Wider impacts: Wider impacts on the telecommunications sector as a result of the Project Gigabit (market shares etc.) and on subsequent and complementary Government schemes (see Annexes). 

Overview

The evaluation will involve the collection of the following data:

Management Information (MI) providing information on progress in delivery of the Programme and to support an assessment of the impact of the Project Gigabit on beneficiaries. 

Primary research (depth interviews, group interviews etc.) with stakeholders involved in the delivery of the Programme and with network providers to obtain strategic views on the impact of the Programme and to support the interpretation of results.

A Quantitative survey to collect information about households in areas benefitting from subsidised coverage.

Secondary data to explore the connectivity, economic, social and environmental impacts of the programme.  

The table below presents a summary of how the data collection plan relates to the analytical work packages for the evaluation.

Table: Data collection summary table                        

Work Package	Management Information	Stakeholder and business consultations	Secondary data sources	Quantitative survey
Availability of cable connections	Yes	Yes	Yes	No
Economic impacts	Yes	Yes	Yes	Yes
Social and public value impacts	No	No	Yes	Yes
Environmental impacts	Yes	Yes	Yes	Yes
Direct impact on beneficiaries	Yes	Yes	No	No

3.3 Management Data

The evaluation will make use of information collected by the Project Gigabit, including Programme level MI and internal documentation. In particular, the research team will need access to the following information sources:

Project Gigabit Data on GBVS projects: A detailed description of each GBVS proposed project, including proposed premises passed, area, supplier, project status (completed, rejects, in progress etc.).

Data on proposed passed premises: UPRN level data which details the premises that each project planned to pass in the project, including which project each UPRN relates to. This includes details on rejected and withdrawn projects. This information will be used to identify treatment and comparator areas for the impact evaluation and analysis of the wider telecoms market.

Actual premises passed: A list of UPRNs where the GBVS has provided upgraded connectivity (premises passed by the intervention build and the premises which claimed a voucher). This information will be used to identify treatment and comparator areas for the impact evaluation and analysis of the wider telecoms market.

Data on change requests: A list of all change requests made by network providers about GBVS projects, which can be used to help identify potential counterfactual areas for the GBVS.

Detailed reports on premises passed by GIS interventions contracts: A list of UPRNs where the GIS interventions contracts have provided upgraded connectivity. This information has not been seen by the evaluation team due to the current stage of Programme delivery. This information will be used to identify treatment and comparator areas for the impact evaluation and analysis of the wider telecoms market.

List of ‘white’ UPRNS/postcodes: A list of premises that have been identified as ‘white’ in the OMR process for GIS interventions contracts and therefore eligible for subsidised infrastructure. This is a rolling list of UPRNs, therefore this data will need to be shared on an ongoing basis. This information can be used to support the identification of treatment and comparator areas. This information has been made available to the research team. As the OMR process is ongoing, the research team would take the most recent OMR results as the basis for the analysis when it takes place.

BDUK Commerciality data: This is a data source which was developed by BDUK to assess the commercial viability of each premises in the UK. This data has been provided to the research team.  This data can be used to support the identification of comparator areas for the impact analysis.

3.4 Stakeholder consultations

Telecommunications providers and Government stakeholders

In order to understand the outcomes achieved by the Project Gigabit (GBVS and GIS interventions acts), it will be essential to undertake primary research with network providers and ISPs. This will be to support the assessment of impacts on Project Gigabit beneficiaries (how the contracts have affected the contract holders), an understanding of the nature of the broadband market at the time that the contracts are being delivered, for example challenges faced by the market or opportunities, to get a better understanding of the additionality of the intervention and the crowding out and crowding in of investment. The stakeholder consultations will also be essential in collecting information about the potential environmental impacts of the GBVS and GIS interventions, and in particular the benefits of fibre or other technical solutions over copper networks. The interviews will also need to explore the interactions between the different programmes within Project Gigabit – for example whether the provision of GBVS projects impacted the GIS interventions contract areas and the extent to which it impacted network providers ability to participate in GIS interventions contract procurement.

Interviews with these types of organisations will help to contextualise the findings from the analysis of Management Information and secondary data sources. It is recommended that of the 30 stakeholder consultations budgeted for, 15 are undertaken with telecommunications companies, including providers that have delivered GIS interventions and GBVS contracts. 

The remaining interviews will be undertaken with BDUK staff and Ofcom, to verify the findings from the telecommunications providers and probe on further issues raised in the interviews. The table below provides a summary of the topics the stakeholder consultations should cover. 

Table: Qualitative interviews: coverage of topics

Topic	Supplier	Other network providers	ISPs	BDUK	Ofcom
Impact of the programme on suppliers	Yes	No	No	Yes	No
Interactions between different elements of Project Gigabit	Yes	Yes	No	Yes	No
Market Conditions	Yes	Yes	Yes	No	Yes
Delivery of contracts	No	No	No	No	No
Build plans	Yes	Yes	No	Yes	Yes
Supply chain capacity	Yes	Yes	No	No	No
Pricing	Yes	Yes	Yes	No	Yes
Wholesale access	No	No	Yes	No	Yes
Environmental Impacts	Yes	Yes	No	No	No

It is anticipated that interviews will have a duration of 60 minutes and will be undertaken using MS Teams. Interviews will be recorded and transcribed with the consent of the respondent. The sample for these qualitative interviews will be agreed with BDUK, and it is assumed that BDUK will support the recruitment process by leveraging their relationships with Suppliers (a process that was invaluable in securing engagement during the delivery of the evaluations of the Superfast Broadband Programme), network providers and ISPs. Interviews will be coded and analysed thematically using the NVIVO qualitative analysis software package.

Businesses

A series of in depth interviews with businesses that have taken up an enhanced connection through the vouchers should be undertaken. These businesses will be drawn from a sample of businesses which have claimed a voucher and have agreed to take part in research. The interviews will need to cover a variety of sectors, business sizes and geographic areas. The interviews should cover:

• how the business uses their internet connection

• what barriers they faced with their previous connection, and whether these barriers persist

• what other infrastructure was upgraded alongside the connection and why

• what has changed in terms of their business delivery since upgrading their connection

• what future plans they have for their business

• employee performance and flexible working arrangements.

Local area case studies

A series of case studies will be used to collect evidence of the impact of Project Gigabit (and broadband deployment more generally) in local areas. The case studies will be used to provide more details of how enhanced broadband can support local communities. In particular, the local case studies will focus on:

How enhanced connectivity has supported local businesses – the types of businesses operating in an area, how connectivity has supported their operations, output and employment. Different case studies could focus on different types of business, for example agriculture, hospitality or retail.

How enhanced connectivity has supported the local authority to implement digital strategies – have local authorities been able to implement strategic changes as a result of the connectivity provided by the programme. 

How have public services altered as a result of enhanced connectivity – whether schools, local authorities, health and social care provision has altered as a result of enhanced connectivity, and what difference this makes for public service providers and residents. 

How enhanced connectivity affects how people feel about where they live: How local residents feel about the area they live in; how enhanced connectivity alters their perception of the local area.  

The case studies will explore how broadband has supported local communities and can be used to explore differences in the experiences of areas that have gigabit capable technologies and those that have not yet received enhanced coverage. 

The research team will work with BDUK to identify areas which would be suitable for local case studies. This will include:

• identifying areas which have received BDUK funding (through GIS interventions and GBVS) - ideally this would include specific sub areas which have received a high concentration of BDUK support

• exploring the business and employment characteristics of areas to ensure the case studies include a variety of economic characteristics

• utilising the household survey responses – if there are a large number of people agreeing to be recontacted within a single area this can be selected as a case study location

• a review of local digital strategies, to identify areas which are altering their public service provision

The research team would initially make contact with the local authority selected and hold an initial workshop with the local authority to identify the types of topics the case study should focus on, and the types of stakeholders who should be contacted for the research. The research team would then develop a short, two-page plan for the case study, and share this with the local authority and BDUK. We would recommend researchers go in person to the local authority area to conduct research face to face with stakeholders. Each case study will comprise a series of ten depth interviews with the following groups of stakeholders (not all types of stakeholders would be interviewed for all case studies):

Local authorities / Devolved Administrations: To understand how its approach to providing public services has altered due to enhanced connectivity, how enhanced connectivity has supported the development and delivery of local digital strategies, how connectivity has impacted on the local economy and perceptions of the area. 

Local public service providers: To explore similar issues around the delivery of public services, and any challenges faced due to users of the service not having access to gigabit capable broadband connections.

Local businesses or business groups: These could be the local chamber of commerce or sector bodies, alongside businesses, to explore how connectivity in the local area has supported their business.

Residents of areas with and without access to gigabit capable networks: To understand their perspectives on the local area, access to public services and employment. These could be individuals that have completed the household survey and agreed to participate in further research, or individuals identified by the local authority. 

It is anticipated that interviews will last for 60 minutes with local authorities / Devolved Administrations and public service providers and 30 minutes with businesses and residents. We will offer interviews via MS Teams, but we would ideally undertake interviews face to face in the local area, to allow researchers to visit the area and generate a better understanding of the locality and how connectivity will support the area. 

In addition to the primary research, the case studies will also involve a suite of desk research. This will include:

• an analysis of household survey responses for the selected areas

• a review of connectivity related announcements and policies within the area (for example announcements of the introduction of digital services, roll out of gigabit capable networks etc.)

• a review of changes in gigabit coverage over time

Each case study will have a standalone two page report – the format of which will be agreed with BDUK in advance.

3.5 Primary research quantitative surveys

There is the potential to include quantitative surveys as part of the data collection for the evaluation of the GBVS. These include a survey with households or residents, and surveys of businesses. These are discussed in more detail below.

Household survey

A household survey, sampling households that have claimed a voucher should be undertaken. The advantage of using households that have claimed a voucher is that they have agreed to take part in further research for the GBVS (as a condition of them claiming the voucher), which will enhance response rates. Additionally, it will be known that the household has upgraded their internet connection, therefore the survey can probe around what changes this has brought about. 

The survey should utilise a push-to-web methodology to maximise the number of responses that can be collected. The purpose of the survey will be to collect evidence of how households are utilising the new connection that we know they have taken up.  

The survey should also include some businesses in the sample, collecting some evidence of how businesses utilise the broadband connection. Given the number of businesses that have taken vouchers, we propose that these represent a very small proportion of the household survey. The businesses included in the survey should be different businesses from those included in the business interviews described above.

The survey will cover the following topics: 

• household / business characteristics (age, number of residents, occupations, income, size of business, sector etc.)

• devices used in the house/ business (laptops, phones, tablets, Smart TV, other smart devices)

• description of how they use the internet (e.g. working from home, education, gaming, streaming, smart devices, administration, frequency of use etc.)

• barriers faced with their previous connection

• connection speed

• satisfaction with current connection

• leisure activities and whether these have altered

• journeys taken (e.g. commuting, travelling to shops etc.) and mode of transport and whether these have altered

• satisfaction with current connection

• level of wellbeing and how this has altered

3.6 Project Gigabit Secondary data sources

There are a wide range of secondary and administrative datasets that could be utilised in the evaluation of the Project Gigabit (GBVS and GIS interventions). Many of these were used in the previous evaluations of the Superfast Broadband Programme, however scoping research has identified some additional data sources which will also be used for the evaluation. The data sources are presented in the table below and linked to the outcomes and impacts described in Section 2. 

Secondary data sources to be used in Project Gigabit evaluation

Connectivity and broadband market data

Data source	Description and access	Key data points	Use in the evaluation
Ofcom Connected Nations	Ofcom Connected Nations data is produced annually detailing the current state of connectivity in the UK. It is publicly available through the Ofcom website.	Postcode data on Gigabit capable coverage, download/upload speeds, take-up, data used.	This dataset will underpin the analysis of how the Programme has provided additional coverage and will support the process evaluation workstreams. This principal issue with this data is that in 2020, Ofcom replaced its measure of FTTP availability with a measure of gigabit capable availability (which includes both FTTP and other technologies capable of delivering gigabit speeds). This may produce inconsistencies in measures of coverage over time.   Consultations with Ofcom and BDUK have suggested that the difference between the two measures will be largely driven by Virgin Media’s rollout of DOCSIC 3.1 technology, which was negligible prior to 2020. This may mean that it is sufficient to assume that FTTP and gigabit capable availability are equivalent prior to 2019. BDUK has access to additional data on Virgin Media’s footprint that can be used to test the validity of this assumption.  BDUK may be able to provide more granular, UPRN level data. If this is possible, UPRN level data will be used in place of postcode level data.
Think Broadband	Thinkbroadband collect data on a number of variables relating to broadband connectivity in the UK, such as coverage by suppliers, take-up data and download speeds. While this data is made available to the public at no charge through their online platform, the complete dataset will need to be purchased from ThinkBroadband.	Postcode data on broadband coverage by connection speed and supplier, speed test data by supplier.	This dataset will be used to underpin the analysis of the market position of beneficiaries.

Economic outcomes data

Data source	Description and access	Key data points	Use in the evaluation
Business Structure Database	This provides vital data for enterprises and local units on key variables such as employment, turnover, Standard Industrial Classification and output area. Data is provided from 1997.	Turnover and employment for businesses by output area.	Local units / businesses in an output area which have been subject to Project Gigabit interventions can be collated and turnover per worker can be calculated to assess the changes in labour productivity.
Annual Business Survey	The Annual Business Survey is a large-scale survey of enterprises in the UK, which includes a financial enquiry aiming to establish levels of capital investment, turnover, GVA, employment, and other financial measures. The survey is mandatory for all large firms (250 employees or more) and as a consequence, provides a potentially useful longitudinal panel dataset that could be exploited.	GVA, factor cost data per business by output area.	The availability of the ABS is subject to some lags. Sample sizes are likely to cause issues in the use of this dataset, as there will be a limited number of businesses with 250+ employees within Project Gigabit treatment areas. This means this source is unlikely to provide useful information for the analysis of Project Gigabit.
Claimant Count	NOMIS provides longitudinal data on the number of claimants (combining Jobseekers Allowance and Universal Credit claimants) at a small area level (LSOA).	Out of work benefits claimant’s by LSOA.	This can be used to explore the effects of the Programme in reducing unemployment and exposure to long-term problems of unemployment as well as benefits dependency.
Annual Survey of Hours & Earnings	The Annual Survey of Hours and Earnings can be accesses at individual level through ONS SRS to analyse real hourly wage changes. Data is available from 1997, and covers earnings, hours worked, age, gender, industry and occupation.	Wages, hours worked by output area.	This is a possible alternative to measuring productivity changes, although the relationship between wage growth and productivity growth is tenuous and can be influenced by exogenous variables separate from productivity growth.

Social and public value outcomes data

Data source	Description and access	Key data points	Use in the evaluation
Annual Population Survey	The Annual Population Survey can be accesses at individual level through ONS SRS. It includes data points on subjective well-being, as well as age, employment status and occupation.	Subjective well-being by output area.	This dataset can be used to undertake further secondary data analysis to estimate the impact of the Programme on subjective well-being.
Understanding Society Survey	The Understanding Society survey is a UK household Longitudinal Study which surveys each adult member of a nationally representative sample about subjects such as health, work education, income, family and social life, all of which would serve to understand the general well-being of the UK population.	Subjective well-being by LSOA.	This dataset can be used to undertake further secondary data analysis to estimate the impact of the Programme on subjective well-being.
Land Registry	Data on median house prices are publicly available through the ONS available as house price statistics for small areas in England and Wales. This dataset is updated quarterly and details the mean or median price paid for houses by MSOA in that quarter. The dataset also includes variables for the type of property.	Mean and median house prices by MSOA.	This dataset could be used to explore property market effects of the Project Gigabit as a proxy measure for changes in subjective well-being. The one drawback of this is that the geographic granularity (MSOA level) has the potential to dilute the extent to which effects might be visible should effects be very localised (within a small number of postcodes from an intervention site). This dataset only covers England and Wales.
VOA ratings list	The data used to determine the rateable value of non-domestic properties used to calculate business rates. First produced in 2010, there are now two data points within this data corresponding to 2010 and 2017 with actual data collection dating to two years prior to these years.	Mean non-domestic rateable value by output area.	This dataset was updated in 2023, which would provide a useful baseline for the evaluation. However, it would require a further update before the end of the evaluation to provide useful evidence for the assessment of the impact of the Project Gigabit. This is expected to happen in 2028, which would be outside the current evaluation timeline, and therefore will not be useful.

Public sector outcomes data

Data source	Description and access	Key data points	Use in the evaluation
Patient Online Management Information Programme Data	This tracks individual GP practices and its total number of online patients registered to use an online Patient Recorded Access Service. Starting in 2015-16, data is collected on an annual basis.	E-health activity by GP practice and postcode.	This data can be matched to Programme delivery areas to explore any changes in e-health activity in areas that have been upgraded.
GP patient survey	The GP Patient Survey is a postal survey providing practice level data on the views of GP patients on their experiences of primary care and will be used to refresh estimates of the impact of the Programme on the efficiency of public service delivery.	Patient satisfaction, use of online services by GP practice and postcode.	This data can be matched to Programme delivery areas to explore any changes in patient views or usage in areas that have been upgraded.
DfE School performance tables	Information pertaining to the performance of schools across England can be accessed via the DfE website. The data here covers educational attainment at both primary and secondary school level measured through exam results, teacher assessments and attendance. Information about school costs, expenditure and income also reported. Data is updated annually.	School attainment, attendance, income and expenditure by postcode.	This dataset could be used to explore the effects of the Project Gigabit on schools which have been upgraded.

Environmental

Data source	Description and access	Key data points	Use in the evaluation
DfT road traffic statistics: Basemap countpoint data	DfT road traffic data provides street-level traffic data for road-links on the motorway, ‘A’ road and minor road network in Great Britain up to 2020.	DfT publish manual and automatic vehicle counts for major and minor roads. Local road network traffic counts can be obtained independently from each local authority.	Traffic count data will be analysed using difference-in-difference techniques to estimate the causal effect of the Programme in terms of the number of journeys made in the rollout areas compared to control group areas. This data can be applied to DfT TAG values for air pollution and Greenhouse Gas emission reductions associated with these journey savings.
TRACC travel time analysis	Traffic speed data is more readily available via TRACC imported satellite data providing street-level traffic data for road-links on the motorway, ‘A’ road and minor road network in Great Britain.	TRACC uses data to run multi-modal journey time calculations, including walking, cycling, public transport and car.	In the absence of traffic count data, traffic speed data will be analysed using difference-in-difference techniques to estimate the causal effect of the Programme in terms of the number of journeys made in the rollout areas compared to control group areas.
TrafficMaster congestion Data	TrafficMaster data is licensed by the DfT and provided free to local authorities by DfT	Data on bi-directional traffic speeds and congestion levels on relevant and adjacent streets.	To analyse traffic behaviour and congestion impacts caused by civil works.
WebTRIS Highways England data	The Highways England WebTRIS website provides flow data for Strategic Road Network (SRN) roads.	Data on local traffic speeds on relevant and adjacent streets.	To analyse traffic behaviour and congestion impacts caused by civil works.
QUADRO	QUADRO (Queues And Delays at Roadworks) provides a method for assessing the total cost of major road maintenance works.	Data on the congestions (and resultant carbon) costs associated with broadband deployment funded by the programme.	Required to analyse the environmental impacts of congestion caused by civil works.
DESNZ Lower and Middle Super Output Areas electricity and gas consumption	These tables provide the following statistics at Middle Layer Super Output Area (MSOA) and Lower Layer Super Output Area (LSOA) level in Great Britain: MSOA level statistics are provided for both domestic and non-domestic meters, while LSOA level statistics are only provided for domestic meters.	Household and non-domestic electricity and gas usage data at the LSOA, MSOA and Intermediate Geography Zone (IGZ) electricity data.	Data is updated annually allowing the before and after comparison of energy consumption between the treatment and control areas.

4. Impact Evaluation

This section sets out the options available to evaluate the impacts of Project Gigabit and understand the key factors that determine success. This section examines the quantitative approaches that could be used to explore the effectiveness of the Programme and provides a recommendation as to the most appropriate ways in which the exercise could be approached. The approaches described below can be used to estimate the impact of both the GBVS and GIS interventions, either separately or together.

4.1 Aims of the impact evaluation

The impact evaluation of Project Gigabit has the following aims:

Provide a robust assessment of the extent to which the outcomes of interest (as defined in the theory of change) would have occurred without BDUK’s investments.

Understand how the different strands of Project Gigabit have interacted to generate outcomes.

Understand the key procedural and contextual factors that determine the degree of success achieved to support on-going learning and aid improvements to the development and implementation of broadband coverage. 

4.2 Issues for impact evaluation

There are several key issues that need to be addressed in the design of the impact evaluation: 

Level at which impacts can be understood: An impact evaluation will need to consider two types of impact:

Local impacts: The primary direct impacts of Project Gigabit will be felt at the local level in terms of the gigabit capable coverage and the social, environmental and economic outcomes resulting from making gigabit capable broadband services available. These outcomes can be more straightforwardly attributed to the investments made by BDUK using quantitative methods as the anticipated outcomes are largely tangible and can be tied to an intervention that is well defined. However, there will also be a need to go beyond a purely quantitative approach to ensure the factors driving or inhibiting the desired outcomes are well understood.

Wider impacts: However, there will also be set of wider channels through which the Programme influences the broader decision-making processes of broadband suppliers and other actors (such as decisions around wider, commercial rollout of gigabit capable networks in other locations). For connectivity, it may be possible to link the outcome to Project Gigabit due to the ongoing nature of the OMR process, identifying nationally which areas are receiving commercial rollout. However, for social, economic and environmental impacts, the outcome will be more challenging to attribute to the Project Gigabit owing to the variety of other factors likely to influence those decisions and in some cases the less tangible nature of the anticipated effects. 

Counterfactual: The build-out of gigabit capable networks depends on the investment decisions made by private suppliers, and there is a possibility that expansions in gigabit capable networks resulting from the public expenditure which would happen in the absence of Project Gigabit happened at a slower pace. To provide a credible quantitative assessment of what would have been achieved without the programme, it is necessary to compare areas affected by the Programme to a comparison group of areas that did not benefit or have not yet benefitted but are otherwise equivalent to those that did. The selection of this comparison group involves several complications. Areas which are receiving Project Gigabit funding may systematically differ from other ‘white’ premises that do not receive subsidised coverage.  For example, premises supported through the Project Gigabit may be in areas that are more commercially viable than ‘white’ premises that are not included in build plans. 

Project / Contract type: There are differences between the contracts awarded through the GBVS and GIS interventions. GBVS contracts are smaller in financial size and premises included in the footprint, and are expected to be completed at a faster pace than GIS interventions. This may influence the types of premises which are included in GBVS projects, and these may differ from those included in GIS interventions. In addition, there are three types of contracts which the GIS interventions are being delivered through. There may be differences between the areas which use each type of contract. These differences were largely determined by market interest in each area (the number of network providers interested in providing coverage in each area) – however, these differences in market interest may reflect underlying differences between the local areas. As GBVS premises are likely to be generally delivered to in advance of GIS interventions premises, and Type C GIS interventions contract areas are being delivered to later than Type A and Type B contracts, this may present some challenges for the selection of a counterfactual area. Local contextual factors, such as local geographical factors (proximity to main roads, railways, rivers, population density, and socio-economic factors such as income and willingness of residents to pay for gigabit capable services) could influence market demand in an area and also the propensity for outcomes to be achieved. Different counterfactual areas may need to be selected to assess the impact of Type C contract areas than for GBVS, Type A and Type B contract areas.  

Crowding out: Provision of gigabit capable networks as a direct consequence of Project Gigabit will absorb resources (e.g. labour or equipment) that will become unavailable to support gigabit capable deployment in other areas. As such, there is a risk that the Programme ‘crowds-out’ gigabit capable build in other areas. It is possible that these effects are complex, involving diversion of investment from areas elsewhere in the UK. These effects will be more challenging to quantify and will require exploration through broader research, such as qualitative interviews with suppliers.

Project Gigabit Timescales: Some of the key outcomes and impacts of Project Gigabit are dependent on future build and take-up of connections.  Some of these impacts may develop beyond the current timescale for the evaluation, and therefore will not be captured by any impact analysis. Therefore, it is important to note that the impacts measured will be the impacts as of the year of the evaluation. The impacts which are measured using secondary data sources could be revisited in future years to explore if the outcomes and impacts have further evolved over time.

Types of impact that can be assessed quantitatively: Not all the outcomes and impacts highlighted in Section 2 will be able to be assessed quantitatively. This is due to a lack of data being available to support the evaluation, or an inability to form a suitable counterfactual group. It will be feasible to quantitatively assess the impact of Project Gigabit on gigabit capable coverage, take-up and connection speeds, and downstream economic impacts. In addition, it will be feasible to assess quantitatively the environmental impact of Project Gigabit on energy consumption and public sector impacts such as GP appointments and satisfaction and school level outcomes. However, it will not be feasible to measure other public sector impacts, such as use of online services for payments / benefit payments, library usage etc. due to a lack of secondary data sources. It will also not be possible to assess the environmental impacts of changes in commuter patterns and working from home due to a lack of credible data sources. Finally, for GBVS, because of the geographically smaller nature of intervention areas, it will not be feasible to assess any environmental or public sector impacts quantitatively using secondary data sets. Therefore, these impacts will be assessed using a theory-based evaluation approach, using data from the qualitative interviews and quantitative survey of households.

4.3 Quantitative evaluation

This section focuses on exploring how a quantitative evaluation of Project Gigabit, and a separate evaluation of GBVS and GIS interventions could best be achieved. It is assumed that the quantitative evaluation would explore the impacts achieved by the Project Gigabit (or GBVS and GIS interventions) as a whole, rather than looking at a contract by contract or project by project analysis. This section first focuses on how a counterfactual set of areas not benefitting from the Project Gigabit or commercial rollout could be established before providing an overview of the analytical methods that could be applied to estimate the effects of interest as robustly as possible. A summary of the recommended quantitative evaluation approaches is presented in the table below:

Outcome	Approach	Data used	Project Gigabit	GBVS	GIS
Enhanced coverage	Using non-upgraded areas from GBVS and GIS interventions contracts  identified in the OMR process.  Utilise Difference-in-Difference approach to identify impact of enhanced coverage.	Management information and OMR responses Connected Nations dataset	Approach not recommended	Recommended approach	Can theoretically be pursued, but further information on rollout plans required
Enhanced coverage	Using areas to be upgraded later in Project Gigabit as comparator areas for earlier delivery areas. Utilise Difference-in-Difference approach to identify impact of enhanced coverage.	Management information and OMR responses Connected Nations dataset	Recommended approach	Recommended approach	Recommended approach
Downstream economic and social impacts	Identify a counterfactual case using areas to be upgraded later in Project Gigabit as comparator areas for earlier delivery areas.  Utilise Difference-in-Difference approach to identify impact on downstream impacts.	Management information ONS datasets (BDS, ASHE, house price data etc.)	Recommended approach	Recommended approach	Recommended approach
Contribution analysis	Use a Contributions framework to explore claims about how and why outcomes have been achieved and utilise qualitative and quantitative data to explore the robustness of these claims.	Qualitative findings Management information ONS datasets Connected Nations data	Recommended approach	Recommended approach	Recommended approach
Cost Benefit Analysis	Compare the expected economic and social benefits to the total costs of the programmes	Management information	ONS datasets	Recommended approach	Recommended approach

4.4 Counterfactual selection - Project Gigabit

The options available to identify a counterfactual for the evaluation of the impact on gigabit capable coverage, take-up and download speeds are presented below for Project Gigabit. The discussion below includes considerations about the robustness of the counterfactual case, the availability of data to form counterfactual cases and how the selection will allow comparisons between local projects and delivery mechanisms. As the Programme has already begun delivering, and due to the nature of the intervention, a Randomised Control Trial is not feasible for this evaluation. Further, because the Project Gigabit is operating throughout the UK, there are no regions which could be used as a counterfactual case. The research team have identified two potential approaches to developing a counterfactual case, which are presented below.     

Option A1: ‘White’ areas not supported through Project Gigabit 

There will be groups of postcodes (UPRNs) which have been categorised as ‘white’ in the OMR process for GIS interventions, meaning they are not going to receive commercial rollout, which are also not included in GIS interventions contract build plans or GBVS delivery plans (or a Gigahubs intervention). This is because there is a certain amount of funding available in each contract area for GIS interventions (and a limit to the GBVS budget), and suppliers bid for contracts by stating how many premises they can deliver to for the budget. Therefore, not all ‘white’ premises will be delivered to. In theory, these premises (and groups of premises) could be used to form a counterfactual case for Project Gigabit.  

However, as Project Gigabit continues to evolve over time to meet its stated aim of providing coverage to 99% of premises in the UK by 2030, the research team understands further interventions could be used to provide broadband coverage to these ‘white’ UPRNs in the future. This would mean that most ‘White’ areas that are not upgraded through existing Project Gigabit interventions will be built to using public funding, which reduces the scope to use these as a counterfactual case.  

Project Gigabit will still leave some premises without gigabit capable networks. These could be used as a comparator area for Project Gigabit. However, these final 1% of premises in the UK are likely to be the least commercially viable premises to deliver to and are unlikely to share similarities with those premises delivered to during Project Gigabit.  

Therefore, this approach to forming a counterfactual group is unlikely to feasible for Project Gigabit as a whole. 

 Option A.2: Pipeline methods 

There is a staggering of rollout of subsidised coverage through Project Gigabit. For example, most GBVS contracts are expected to complete delivery before many of the GIS interventions begin delivery – and the GIS interventions contracts are being awarded in a phased manner. Up to Autumn 2024, 36 GIS contracts have already been awarded, 15 being in procurement and a further nine expected to be awarded in the future. The contract award dates range from December 2022 to mid 2025 (these are not the dates where new networks begin being delivered), although these are targeted award dates and there may be slippage in when contracts are awarded. Contract completions are expected to run until the 2030. There were: 

• four contracts awarded in 2022, which aim to provide coverage to 73,700 premises 

• 15 contracts awarded in 2023, which aim to provide coverage to 512,400 premises 

• 22 contracts are expected to be awarded in 2024, with the number of contracted premises expected to be around 570,000 

Further contracts are expected to be awarded in 2025, with an unspecified number of targeted premises.   

In addition to this, GBVS projects began delivery in 2020, and most are expected to be delivered by 2024. 

Due to the phased nature of contract awards and delivery of Project Gigabit, a pipeline approach to developing the counterfactual case can be considered. This approach uses later contract areas as a comparator case for earlier contracts awarded through Project Gigabit. The approach avoids the problems normally associated with selection bias because no areas that are not receiving Project Gigabit support are included in the analysis. As such, all areas can be assumed to share unobserved features that influenced their selection into the programme.  

In order to utilise this approach, an intention to treat analysis would need to be used. This means the analysis focusses on the ‘white’ premises in an intervention area for GIS intervention or within an initial project design for GBVS, rather than the premises included in the delivery. The reason for this is that including only contracted premises in the analysis would exclude any commercial delivery on ‘white’ premises occurring between the start of the Programme and the tendering exercise. This would be commercial deployment that would have come forward in the absence of the programme.  

The analysis would therefore explore the changes in gigabit coverage across all ‘white’ UPRNs in an intervention / project area at the time of the first contract award for Project Gigabit in December 2022. The changes in gigabit coverage, take-up and connection speeds in the intervention areas of early contract areas would then be compared to the intervention areas of later contract areas.  

The analysis would need to include a matching exercise, involve a matching exercise between the ‘white’ premises included in each area, to enhance the level of comparability between the areas (see analytical approaches below). This would ensure that analysed later ‘white’ premises shared similar characteristics to those included in earlier projects and contracts. The matching approach would need to consider factors such as: 

• Supply side factors: distance / speed from exchange, commercial viability , number of suppliers operating in the area, baseline average download speeds etc. 

• Demand side factors: premises and population density, average household income, employment etc. 

The strength of the results depends on the assumption that there are no systematic differences between areas receiving investment earlier and later that are also correlated with the outcomes of interest. Further, it would require that the outcomes of interest, namely coverage, take-up and download speeds, were comparable, or at least moving in the same direction prior to the intervention taking place. This can be done by comparing the trends of Connected Nations data for all ‘white’ postcodes in each intervention / project area. However, the research team do not currently have the information for all potential intervention areas. Therefore, this will need to be tested once all intervention area data is available.  

It is recommended that this pipeline approach is used to explore the impact of Project Gigabit on coverage, download speeds and take-up. 

Counterfactual selection - GBVS 

As with Project Gigabit as a whole, two main options for assessing the impact on connectivity appear viable – ‘White’ areas not supported through GBVS, and a pipeline approach. For GBVS, it is recommended that both approaches are used to measure the impact of the programme on connectivity outcomes. 

Option A1: ‘White’ areas not supported through GBVS 

There will be groups of postcodes (UPRNs) which have been categorised as ‘white’ in the OMR process for GIS interventions, meaning they are not going to receive commercial rollout, which are also not included in GIS interventions build plans or GBVS delivery plans (or a GigaHubs intervention). These premises (and groups of premises) could be used to form a counterfactual case for Project Gigabit.  

For GBVS this should be a suitable approach due to the timing of the delivery of GBVS and GIS interventions. Most GIS interventions contracts will not have started to construct the subsidised networks but will have build plans, meaning ‘white’ premises not included in the build plans could be identified and used as a comparator group.   

A further enhancement of this approach would be to use ‘white’ premises which were not included in the network providers final build for a completed GBVS project but were included in other network providers unsuccessful or abandoned projects. For GBVS projects, a total of 675 projects were cancelled or rejected or withdrawn, covering over 20,000 UPRNs. The evaluation will examine the profile of premises included in the withdrawn projects to explore the level of comparability between delivered to and withdrawn projects. However, as the projects may have been withdrawn due to commercial considerations there may be systematic differences between delivered and withdrawn projects, which would limit the use of withdrawn projects as a comparator group.   

To ensure that there is comparability between ‘white’ premises included in the GBVS projects and other ‘white’ premises, a matching approach, as described above for Project Gigabit, would need to be used.   

Option A.2: Pipeline methods 

Due to the phased manner of the awarding of GBVS projects and GIS interventions (as described above), it will be feasible, using later GIS interventions contracts as a comparator area for GBVS projects. In order to utilise this approach, an intention to treat analysis would need to be used. This means the analysis focusses on the ‘white’ premises in a GIS contract intervention area, rather than the premises included in the GIS contract award. The reason for this is that including only contracted premises in the analysis would exclude any commercial delivery on ‘white’ premises occurring between the start of the GBVS programme and the tendering exercise. This would be commercial deployment that would have come forward in the absence of the programme.  

The changes in gigabit coverage, take-up and connection speeds in the intervention areas of early contract areas would then be compared to the intervention areas of later contract areas.  

As with the approach described above, the analysis would involve a matching exercise between the ‘white’ premises included in each area, to enhance the level of comparability between the areas. 

GIS contracts interventions 

As with Project Gigabit as a whole, two main options for assessing the impact on connectivity appear viable – ‘White’ areas not supported through GIS interventions, and a pipeline approach. For GIS interventions, it is recommended that both approaches are used to measure the impact of the programme on connectivity outcomes – although the approach using ‘White’ areas not supported through the programme will be dependent on the timing of further interventions launched by BDUK through Project Gigabit to enhance gigabit coverage in the UK. 

Option A1: ‘White’ areas not supported through GBVS 

There will be groups of postcodes (UPRNs) which have been categorised as ‘white’ in the OMR process for GIS interventions, meaning they are not going to receive commercial rollout, which are also not included in GIS interventions contract build plans. These could be used as a comparator group for the premises included in the GIS interventions contracts. 

Currently, there are still uncertainties around the feasibility of this approach to developing a counterfactual case for GIS interventions.  As the delivery of GIS interventions is still at an early stage, few GIS contracts having finalised the premises that they will deliver enhanced connectivity to and the ‘white’ premises that are not included in the build plan. Therefore, it is not yet known if there will be a large enough pool of ‘white’ premises to use as a comparator group. Once a significant number of contracts have finalised the build plans the premises included in the build plans and the ‘white’ premises that are not included will need to be explored in detail to ensure there are a sufficient number of UPRNs in each sample and that these are comparable. For any analysis relating to GIS interventions, all GBVS UPRNs will need to be removed from the analysis. 

To ensure that there is comparability between ‘white’ premises included in the build plans for GIS interventions contracts and those that are not, a matching approach would need to be undertaken as described above.  

A further enhancement of this approach would be to use ‘white’ premises which were not included in the successful network providers application, but were included in other network providers unsuccessful applications to provide the contract. This would require data on the proposed UPRNs included in each of the successful and unsuccessful applications. Of the 29 areas that the research team have information about the tendering exercise for, 12 had no unsuccessful applicants (a sole application was received). Further, Type C contract areas would have no unsuccessful applicants. The lack of unsuccessful applicants may limit this potential approach to developing a counterfactual case. Again, once a significant number of contracts have finalised the build plans, the feasibility of this approach can be explored in more detail. 

Option A.2: Pipeline methods 

As the GIS interventions are being awarded in a phased manner (as described above), a pipeline approach to developing the counterfactual case can be considered. This approach uses later contract areas as a comparator case for earlier contract awards. The approach avoids the problems normally associated with selection bias because no areas that are not receiving Project Gigabit support are included in the analysis. As such, all areas can be assumed to share unobserved features that influenced their selection into the programme.  

In order to utilise this approach, an intention to treat analysis would need to be used. This means the analysis focusses on the ‘white’ premises in an intervention area, rather than the premises included in the contract award. The reason for this is that including only contracted premises in the analysis would exclude any commercial delivery on ‘white’ premises occurring between the start of the Programme and the tendering exercise. This would be commercial deployment that would have come forward in the absence of the programme.  

The analysis would therefore explore the changes in gigabit coverage across all ‘white’ UPRNs in an intervention area at the time of the first contract award for GIS contracts, December 2022. The changes in gigabit coverage, take-up and connection speeds in the intervention areas of early contract areas would then be compared to the intervention areas of later contract areas.  

As with the approach described above, the analysis would involve a matching exercise between the ‘white’ premises included in each area, to enhance the level of comparability between the areas. 

The strength of the results depends on the assumption that there are no systematic differences between areas receiving investment earlier and later that are also correlated with the outcomes of interest. Further, it would require that the outcomes of interest, namely coverage, take-up and download speeds, were comparable, or at least moving in the same direction prior to the intervention taking place. This can be done by comparing the trends of Connected Nations data for all ‘white’ premises in each intervention area. However, the research team do not currently have the information for all potential intervention areas. Therefore, this will need to be tested once all intervention area data is available. It is anticipated that given the likely size of the later contracts in terms of premises delivered to, that there should be sufficient data points available that do not differ systematically from those receiving subsidised coverage earlier in the GIS interventions. 

Counterfactual selection – downstream economic, social and environmental impacts (indirect impact of Project Gigabit) 

For Project Gigabit as a whole and GBVS and GIS interventions individually, one feasible approach to measuring indirect and downstream impacts has been identified. This is discussed in detail below and is the recommended approach to measure the impact for all three programmes.  

Option B.1: Pipeline Design 

A pipeline design could utilise the staggered rollout of the Project Gigabit to estimate the downstream economic, social and environmental impacts of the scheme (both within contracts and across GBVS projects / GIS interventions with different start dates).   

The basis of the pipeline design for the downstream impacts is that areas that have received BDUK investment first are likely to experience the impacts of those investments first. The approach has the following features: 

Feasibility: This approach is only feasible with longitudinal panel data (i.e. annual observations of the same individuals, firms, or areas) to exploit staggering in the rollout of the programme. This does not present an issue for analysis focusing on the secondary data sources identified in Section 3.  

Selection bias: As mentioned above, the strength of the results is partly dependent on an assumption that there are no systematic differences between areas receiving investment earlier and later that are also correlated with the outcomes of interest. For example, if subsidised broadband is rolled out to areas experiencing higher productivity growth first, then this will overstate the impact of the programme. This assumption would need to be examined prior to the impact analysis being undertaken. 

Nature of results: Like the instrumental variable approach, this approach identifies the impact of broadband infrastructure but does not account for whether the infrastructure would have come forward without the programme. As such, results would need to be combined with estimates of the likelihood infrastructure would not have come forward in the absence of the Programme (from the analysis on coverage described above) to provide an estimate of its net effects. 

Implementation of a pipeline approach would need to be alert to the following issues: 

Differences between earlier and later delivery: The analysis would need to be aware of any differences between the areas receiving subsidies in the earlier stages of the Programme and those in later stages of the Programme. If there are substantial differences between areas receiving contracts in earlier and later years of the programme, the downstream impact analysis may need to be separated by contract award year.  

Differences within contracts: There could also be systematic differences in the order in which suppliers chose to rollout subsidised infrastructure within schemes. This should be addressed by the contract terms ensuring that the most commercially viable areas are not connected before less commercially viable areas, but this will need to be checked for analytical purposes. This concern can be mitigated by allowing for unobserved but time invariant characteristics common to areas benefitting from investment in different years. However, the issues cannot be fully eliminated.  

Defining a ‘treatment’ variable at an area level: For analyses at an area level, an appropriate measure of the treatment variable will need to be defined, as the number of premises receiving BDUK investment is likely to vary substantially. Assuming that the scale of the impact will be proportional to the number of premises enabled with gigabit capable broadband, it may be appropriate to reflect this in the modelling (by defining the treatment variable as the number of premises enabled). 

Analytical approaches 

The potential quantitative approaches which could be used to measure the impact of the Project Gigabit as a whole, and GBVS and GIS interventions individually, are discussed below. 

Before and after comparison (Level 2 on Maryland Scientific Methods (MSM) Scale) 

A before and after comparison methodology involves analysing the outcomes of interest over time for the treatment group. It does not involve the selection of a counterfactual group. This includes time periods both prior to and after the introduction of programme. It can include simply observing the changes over time or including control variables in a regression analysis (Interrupted Time Series). However, a before and after methodology can struggle to robustly identify the impact of a Programme if there are important external factors also likely to influence the outcomes of interest. 

In the context of the Project Gigabit, a basic before and after methodology should be possible for all the outcomes of the programme. However, this is unlikely to be a robust approach to estimating the impact of the Project Gigabit. This is because the approach will not be able to fully control for external factors which could affect the outcomes, such as supplier behaviour in the absence of the Project Gigabit and business and consumer responses to current economic conditions, which will limit the robustness of any findings. Therefore, the more robust approaches described below are recommended. 

Matching approaches (Level 3 on MSM) 

A matching approach involves refining a counterfactual group (made up of areas which are yet to benefit from the programme) to ensure it has comparable pre-intervention characteristics to the areas that have benefitted from the Project Gigabit. This can be done by estimating the probability that each UPRN/postcode/LSOA in the treatment and comparator group were proximate to one another, based on their initial characteristics. This probability is then used to match postcodes in the treatment group to similar postcodes in the comparator group (an approach known as Property Score Matching). Areas that cannot be matched are excluded from the analysis. The outcomes achieved by the refined treatment and comparator groups are then compared to estimate the impact of the intervention.  

The benefits of a matching approach are that it helps to reduce any selection bias between the treatment and comparator group. If all factors which could influence the selection of actors for the Programme can be observed and measured, then matching methods have the potential to provide estimates of impact that are unbiased. The table below presents an initial assessment of the characteristics the research team would look to include in a matching approach, and the characteristics which it would be beneficial to include but may be difficult to access data for. 

Table: potential matching characteristics

Category of characteristics	Characteristics used
Supply side	Superfast broadband coverage Gigabit capable broadband coverage Distance from exchange Distance from cabinet Rural/urban indicator Premises density
Demand side	Population density Employment Economic activity Earnings Digital employment

Matching approaches only provide robust results to the degree that it is possible to observe all factors that determine both the likelihood a particular postcode benefits from funding through the Programme and the outcomes of interest. Clearly, this cannot be assured in this case as there are likely to be a number of important aspects that are unobservable (such as expectations of future local economic growth). While matching is likely to be helpful in refining samples for analysis, the application of further techniques will be needed to increase the robustness of results to unobserved confounders as described in the following section. 

Difference in Difference / Fixed effects (Level 3 on MSM)

The availability of longitudinal data from the administrative and national survey data sources allows for the creation of a longitudinal panel covering areas that the Project Gigabit has delivered to and those it expects to deliver to in the future. Longitudinal panel techniques (such as fixed effects models or difference-in-differences) extend a standard regression model so as to control for unobserved variables. Where particular characteristics remain invariant across time or invariant across observation units (or groups of observation units) the effect of the characteristic on the outcome can be estimated. This approach can be used with a matched sample (as described above), where data for the comparator case is included only for the matched sample.  

Using the annual data that tends to be available from administrative datasets, it will be feasible to allow for a broader range of fixed effects such as unobserved time, sector or location specific shocks. However, this methodology would not be able to account for changes in the unobserved characteristics of individual firms or households included in the analysis. This has the potential to reduce the robustness of the findings, as changes in the outcomes could be due to the unobserved characteristics (such as outsourcing or changes in local consumer preferences) rather than the Project Gigabit. 

It is feasible to use Difference in Differences and fixed effects approaches in the evaluation of the Project Gigabit for all outcomes which draw on secondary data sources. These approaches can also be combined with the matching approaches described above, to improve the comparability of the treatment and comparator groups.  This approach is recommended for the analysis of the impacts of Project Gigabit, GBVS and GIS interventions. 

4.5 Theory based approaches 

In addition to the quantitative analytical approaches described above, the impact evaluation should also utilise theory-based approaches, to support an assessment of how and why outcomes have been achieved. This will also allow the evaluation to explore the effect the Project Gigabit has had on outcomes and impacts that cannot be measured quantitatively – for example environmental impacts for GBVS and the impact on the wider telecommunications market. One approach which would be appropriate to use for the evaluation of the Project Gigabit is a Contributions Analysis approach. 

A contributions analysis approach is an analytical framework which aims to identify how the expected (or unexpected) causal pathways have contributed to the achievement of an outcome or impact in a rigorous manner. This analysis is heavily linked to the Theory of Change for an intervention or set of interventions.  

The research team will develop a contributions, or performance story, for outcomes expected to be achieved in areas with gigabit broadband coverage. This contribution story will be linked to the Theories of Change. The research team will then collect qualitative and quantitative data to test the performance story. This would utilise data from the household survey, the qualitative interviews and data from secondary data sets to test the performance story. During the data collection process, the research team will reflect on the performance story and make amendments to this based on the evidence collected and then collect further evidence to test the revised performance story. This approach is recommended for the analysis of the impacts of Project Gigabit, GBVS and GIS interventions. 

4.6 Crowding out / spill-over effects 

Providing an assessment of the extent of any crowding out is likely to be challenging. However, one potential approach to investigating crowding out is to use econometric models known as ‘distance-decay’ models. This would assume that the likelihood that gigabit coverage comes forward in ineligible areas (those not receiving a Project Gigabit support, for example London, or areas which are in existing build plans and have been designated ‘grey’ or ‘black’, but do not yet have gigabit capable networks) in a given year will be negatively influenced by the volume of subsidised connections coming forward in nearby areas. The expectation would be that the strength of this influence would decline with distance, and allowances can be made to estimate separate effects at increasing distances from the ineligible areas. The model could also be adapted to examine similar effects on average and maximum download speeds.  

This approach may not be effective if suppliers operate in national labour markets or if resources can be transferred easily from one area to another. This issue merits further investigation during the interviews with network providers, to explore how many operate at a national level and how easily resources can be moved between areas, and whether the public expenditure has led to a reduction in marginal costs in some areas leading to more infrastructure build. 

4.7 Cost Benefit analysis and Value for Money assessment 

The economic evaluation of the programme, providing an assessment of its lifetime costs and benefits and its value for money (the ratio of public sector costs to the benefits involved), is linked to the type of impact evaluation, which is undertaken, will be the final analytical task. If during the course of the evaluation it is feasible to quantitatively assess the impact of the Programme on social, economic and environmental outcomes, and conduct the financial analysis, a cost-benefit analysis will be feasible. The cost-benefit analysis will consider costs and benefits to date and over a period extending to 2042 (a twenty-year time horizon from the launch of the scheme in 2022).  

The analysis will include: 

Costs: Estimates of the present value of expected costs over the lifetime of GIS interventions will be taken directly from the financial modelling exercise, if this can be completed within the evaluation contract timeframe, and the costs associated with GBVS taken directly from the Management Information. The underlying models will provide annual estimates of the net costs to the public sector and will allow for future income from the underspend and take-up clawback mechanisms. 

Benefits: Estimates of the benefits of Project Gigabit will be derived from the results of the econometric analysis and are expected to cover: 

• Productivity gains derived from improved efficiency of businesses. These will be developed from estimates of the effects of gigabit capable deployment on the turnover per worker of firms that do not change their location (to avoid the risk of overstating the net impact of the Programme by including effects driven by the relocation of firms, which would involve offsetting effects for other areas). This will also need to allow for any decay in these impacts over time. 
• Labour supply impacts driven by (a) reductions in long-term unemployment and (b) any increases in economic activity rates. These impacts will be monetised in line with approaches adopted in the appraisal of other forms of infrastructure investment projects.  
• Social benefits - These will be estimated either by using estimates of changes in subjective wellbeing (if a household survey is assessed to be feasible) or will use (as was used in the Superfast Broadband evaluations) a hedonic pricing methodology (assuming the social value of the Programme was capitalised into house prices).  
• Environmental impacts – The CBA will also be augmented with estimates of the environmental costs and benefits of the scheme.  

Additionality: All estimates will need to be adjusted in light of the estimated additionality of gigabit capable connectivity provided by Project Gigabit over time (how much of the coverage provided would have been delivered in the absence of Project Gigabit in each year after the build), alongside additionality measures for the entire Project Gigabit (and GBVS and GIS interventions separately). This will be done by estimating the impact of enhanced connectivity for each year individually following the build. 

4.8 Financial analysis 

Project Gigabit aims to subsidise commercial suppliers’ broadband provision in areas where it would not be commercially viable to roll out a broadband network in the absence of a subsidy. Contracts are made between BDUK and suppliers. For the GIS interventions, an analysis of the financial implications of the gap funding model is required to answer subsidy control questions. This analysis is not required for the evaluation of GBVS. 

For GIS interventions, funding is provided through a gap funding model, which makes it possible for BDUK to ensure that the subsidy is equal to the minimum amount necessary to deliver the project whilst ensuring an adequate IRR on the project. Under the model, the winning supplier finances, designs, builds, owns, and operates the network.  

When bidding for a contract, the supplier provides financial information which allows BDUK to assess the expected delivery and take-up profile of a project (i.e. the number of premises receiving coverage) as well as an estimation of costs, revenues, Weighted Average Cost of Capital (WACC), and Internal Rate of Return (IRR). During the deployment phase and after contract closure, the actual financial performance of the contract is monitored to ensure that correct amount of subsidy is paid to the supplier, and that any excess profits are clawed back.  

Methodology 

To address the Subsidy Control questions, an analysis of the financial implications of the Project Gigabit will be required. The analysis will need to compare the network provider’s expected IRR per contract against their cost of capital (WACC) to determine the extent to which there was an incentive effect. In principle, if the IRR associated with a project exceeds the network providers’ WACC, there should be sufficient incentive for the network provider to have brought the project forward without public subsidy.  

The table below sets out the IRRs which were calculated for the State aid evaluation of the Superfast Broadband Programme. We propose to use a similar approach for the evaluation of the Project Gigabit, focussing on the same IRRs. This will allow for an assessment of the effectiveness of the contracting model in avoiding suppliers earning supernormal returns, as well as comparing the IRRs against the WACC. 

Table: IRRs for comparison

IRR Number	Description	Interpretation
IRR 1	the original IRR before state aid (baseline)	Estimated by the supplier at the time of bid. This IRR would demonstrate whether the project would be commercially viable without the assistance of public funding.
IRR 2	The original IRR after state aid (estimated by the supplier at the time of bid).	Estimated by the supplier at the time of bid, this IRR would demonstrate the project’s viability with the help of public funding, and the profitability of such a project, to the extent that a supplier would be incentivised to deliver the contract.
IRR 3	The updated estimate of IRR before state aid (modelling exercise based on latest available data and/or evidence-based assumptions).	A modelled IRR based on latest available information to inform whether the IRR before state aid (as above) was accurate in determining the likelihood that the project would have been unprofitable for the supplier had public funding not been made available. Where the estimate of IRR before state aid is above the WACC, for example, this would imply that state aid wasn’t required for the delivery and sustainability of the contract.
IRR 4	The updated estimate of IRR after state-aid and before clawback (modelling exercise based on latest available data and/or evidence-based assumptions).	A modelled IRR based on latest available information to estimate the IRR post state aid but pre clawback post contract closure.   This IRR tests the accuracy of the supplier’s IRR estimate post public funding (as per IRR 2 above).
IRR 5	The updated estimate of IRR after state-aid and after clawback (modelling exercise based on latest available data and/or evidence-based assumptions).	A modelled IRR based on latest available information to estimate the IRR post state aid and clawback post contract closure.  This IRR assesses the extent to which the clawback mechanism is effective is recouping over subsidy (i.e. in limiting the amount of profit that the Supplier has made on a project whilst having a significant incentive effect on the beneficiary).

Calculating the IRR

The IRR for each project determines its economic viability. The IRR is the discount rate that makes the net present value (NPV) of a project zero.^{[footnote 4]} In other words, it represents the discount rate that makes the initial investment in a project equal to the present value of future expected cash flows. The IRR is also sensitive to the overall time frame of the investment and the timing of expenses and revenues.

For all completed contracts, we will need to calculate the IRRs in the same way as the suppliers use to estimate their own IRR in their application to BDUK for consistency and estimated based on revised cashflows. In order to understand exactly what data is required to undertake this modelling, we will need to review a selection of suppliers’ financial submissions to establish what has been used to form their IRR calculations. However, we expect that the following data will be required: 

Take-up: Cumulative number of premises connected per quarter for gigabit capable connections – this can be taken from the connection and take-up reports provided by suppliers every quarter.

Revenue: Rental price, Average Revenue per User (ARPU), installation price, migration cost, service cessation cost, churn rate. We anticipate that this will be included in the initial financial information provided by suppliers at the point of bidding for contracts. 

Operating expenditure (Opex): Average unit cost per quarter, and maintenance cost. We anticipate that this will be included in the initial financial information provided by suppliers at the point of bidding for contracts and in ongoing contract management documents.

Capital expenditure (Capex): Build unit cost, connection unit cost, per quarter, and project management costs per quarter. We anticipate that this will be included in the initial financial information provided by suppliers at the point of bidding for contracts and in ongoing contract management documents.

Public Funding and private funding: Funding per quarter. We anticipate this will be reported in ongoing contract management documents.

Clawback: Capital (implementation) clawback, take-up clawback, per quarter. We anticipate this will be reported in ongoing contract management documents.

A separate model for each contract will be developed to derive project level IRRs and subsequently averaged to derive a portfolio level IRR for each supplier. 

Analysis

The first stage of the analysis will revisit the previous methodology to calculate IRRs for the Superfast Broadband Programme and make amendments to this to tailor the approach for completed GIS interventions contracts. The methodology will be checked for accuracy, appropriateness, and relevance in the current context. This will include any areas of uncertainty around contracts in scope (we propose to focus on completed contracts) and data availability, and a description of any assumptions which need to be used for the analysis.

Following a review of the methodology and above assumptions, the calculation of IRRs 1-5 will be conducted for the portfolio of contracts in scope. Results will be collated into a report, which will present observations for all major components of the modelling (take-up, build costs, modelled revenues, and clawback). The report will also present an updated view on the overall profitability of Phase 3 contracts to support a response to the subsidy control questions.

A key challenge for this analysis will be the timing of the contract completion. For the analysis to be robust, we would focus on contracts that have completed their network build. However, examining the expected contract end dates suggests that at the time of this evaluation, few contracts will have been completed. This will severely limit the scope of the analysis and limit the opportunity to compare financial outcomes by the type of contract and supplier. A second challenge is the availability of data. For the evaluation of the Superfast Broadband Programme, we discovered inconsistencies in the type of financial data being provided by different suppliers – with insufficient information being provided by some suppliers to allow a complete financial analysis to be completed. As with the Superfast Broadband evaluation, we will only be able to fully analyse the financial implications of a contract where sufficient information is available. The availability of data will need to be reviewed on an ongoing basis.

Assessment of market impact

For the evaluation of Project Gigabit, in order to answer subsidy control questions, an assessment of the impact of the contracts on the parameters of competition, both nationally and in the local areas where the Programme has been delivered will be required..

This analysis will utilise the ThinkBroadband dataset, as it has been identified that as with the evaluation of the Superfast Broadband Programme, Ofcom data will not provide supplier level data to support the evaluation. One further alternative considered was using the responses to the national, rolling OMR process run by BDUK. However, this would require substantial work on the part of BDUK to prepare the data and data sharing considerations that have been agreed between BDUK and the network providers. Further, the OMR responses include fewer network providers than are included in the ThinkBroadband dataset. 

Methodology

The analysis undertaken in the previous evaluation will be repeated as a first step. This will involve matching the ThinkBroadband data (which is provided at a postcode level) to the delivery data collected by the Programme and the ‘white’ premises included in intervention areas. 

The next step will be to use this matched dataset to identify:

The number of network providers and ISPs which offer broadband services nationally, in the Project Gigabit treatment areas, and in ‘white’ areas that are yet to receive subsidised coverage. 

The market share of network providers and ISPs nationally, in the GIS interventions treatment areas, and in ‘white’ areas that are yet to be delivered to utilising the speed test data from ThinkBroadband. 

This analysis will be undertaken towards the end of the evaluation period, and we propose using four time periods – 2022 (the baseline period), 2024, 2026 and 2028 (see Section 5 for discussions around the project timeline). The research team will compare how the market share of network providers and ISPs change in Project Gigabit areas, comparator ‘white’ areas, and nationally to draw inferences on how far subsidies have enabled network providers to acquire a competitive advantage. 

The analysis will explore which Internet Service Providers are present in the Project Gigabit treatment areas, to support an assessment of whether the publicly funded networks have been made available for wholesale access, whether this access has been taken up by ISPs and the impact on consumer choice.

We will also explore the extent to which the GBVS and GIS interventions have supported changes in geographic coverage of network providers, by identifying beneficiaries geographic footprint prior to any contracts being delivered. This will be compared to their geographic coverage in 2024, 2026 and 2028 and the location of their contracts / projects (and timing of the contract / project delivery) to establish if Project Gigabit has had an impact on geographic coverage in the UK. Regression analysis will also be used to explore the extent to which the provision of GBVS and GIS interventions has led to network providers being able to successfully bid for further BDUK contracts.

5. Implementation

This section sets out how the Project Gigabit evaluation should be implemented. It draws on the information provided in the preceding sections of the report. The section includes the focus of the evaluation, the timelines for each evaluation stage and the key risks for the evaluation. 

5.1 Delivery timeline 

Given the stage that the GIS interventions is  at, in terms of delivery (36 contracts awarded, 15 in procurement and at least a further 5 yet to be procured as of Autumn 2024), it is recommended that the evaluation of GIS interventions takes place after the current evaluation contract expires (in 2026). However, this activity is included in the delivery timeline below.  

Project Gigabit 

The timeline for the evaluation of Project Gigabit includes three main stages. These are: 

A baselining exercise: This should take place in 2025, using data from secondary sources for 2020. This will provide a description of the economic, environmental and social situation in the areas where the Project Gigabit is being delivered prior to the delivery of any subsidised build. At this stage the counterfactual areas will be finalised. This exercise can be undertaken separately for GBVS and GIS interventions. This exercise can then be used to compare changes in outcomes to in subsequent evaluation activity.  

An interim evaluation stage: This stage should include a synthesis of the findings from the evaluation of the GBVS (see below), alongside an assessment of the processes used to deliver Project Gigabit and GIS interventions, and an assessment of the impact Project Gigabit has had to date on connectivity outcomes.  This should take place in 2028 

A final evaluation stage: This stage should include a synthesis of findings from an evaluation of the GBVS and GIS interventions, provide further findings of the impact of Project Gigabit on coverage and take-up, and provide evidence of downstream economic and social impacts. This should take place in 2032, or when all Project Gigabit contracts have been completed. 

GBVS evaluation 

It is recommended that the current evaluation focusses on GBVS, and is delivered in two main stages (building on the baselining exercise described above for Project Gigabit). These would be: 

An interim evaluation report: This report should include the findings from the quantitative survey of GBVS recipients, demonstrating the impacts the enhanced connectivity has had on households. This evaluation will also include an assessment of the additional connectivity the GBVS has delivered. This report will provide a summary report of key findings and two technical annexes, presenting more detailed findings from the survey and analysis of coverage.  

A final impact evaluation report: This evaluation should be provided in 2026, allowing time for downstream impacts to be realised and to be visible in secondary data sources. This report would provide further findings of the impact of GBVS on coverage, take-up, and provide evidence of downstream economic and social impacts. Project Gigabit This report will be split into a higher-level findings report, presenting a summary of the findings of the evaluation, and a series of technical annexes, presenting more detailed findings and details of the methodological approach. 

GIS interventions 

The timeline for the evaluation of GIS interventions includes four main stages (building on the baselining exercise described above for Project Gigabit). These are: 

An early process evaluation: This took place in 2024, and explored processes around the design and early implementation of the GIS interventions. This included an assessment of Management Information and a series of qualitative interviews with BDUK staff, network providers and local authorities. 

A process evaluation: This should take place in 2026/2027, when delivery of the GIS contracts is taking place. The process evaluation should focus on the key processes being used in the delivery of contracts, what has worked well and less well and identify key lessons for the future delivery of the contracts. The process evaluation will include an assessment of Monitoring data and secondary data sets, and findings from a suite of qualitative interviews with key stakeholders.  

An interim evaluation stage: This should take place in 2028 (aligning with the interim phase for Project Gigabit) and include an assessment of the connectivity outcomes to date and qualitative findings from a suite of case study research. 

A final evaluation stage: This should take place towards the end of the GIS interventions delivery (around 2030), and include a complete impact and economic evaluation, an assessment of the impact on the broadband market and a complete financial analysis of GIS interventions. 

5.2 Timeline  

For the Project Gigabit evaluation, the baselining activities are expected to take place in July 2024, and then between December 2024 and February 2025. The connectivity outcomes analysis is expected to take place in 2028, and to be repeated in 2031 and 2032. The economic and social impact analysis is expected to take place in 2031 and 2032. The interim evaluation report is expected in 2028 and the final evaluation report is expected in 2032.

For the GBVS evaluation, the vouchers household survey will be conducted between October 2024 and February 2025. A review of the project management information will take place between September and October 2024, and will be repeated in January and February 2025. An analysis of the impact of enhanced coverage will take place in January and February 2025. The interim evaluation report is expected in March 2025. Interviews with business who have received vouchers will take place between March 2025 and June 2025, as well as stakeholder interviews. The analysis of downstream economic and social impacts and an economic evaluation will take place between September 2025 and November 2025. The final evaluation report is due in March 2026.

For the GIS evaluation, a review of the management information and stakeholder interviews will take place in 2026 and 2027. The process evaluation report is due at the end of 2027. An analysis of the impact of enhanced coverage will take place in 2028 as well as local area case studies. The interim report is due when these are completed.  An analysis of downstream economic and social impacts is due in 2030, as well as a financial analysis, a market analysis, and an economic evaluation. The final report is due in 2030 when these activities are completed.

Project Gigabit Logic Model – Infrastructure Effects of Project Gigabit

5.3 Risk register

The table below presents the key risks to the successful delivery of the evaluation of the Project Gigabit, and mitigating steps which could be taken to ensure that the evaluation provides robust answers to the key evaluation questions: 

Table: Key risks for the successful delivery of the evaluation

Risk	Likelihood	Impact	Mitigating action
Technical
Issues identifying a suitable counterfactual	Low	High	Scoping work completed to date has identified two potential options for counterfactuals to subsidised premises.  Further analysis, once Programme delivery has progressed, will identify most appropriate approach for counterfactual case.
Evaluation findings do not stand up to external scrutiny	Low	High	Clear audit trail of all evidence collected and analysed. Internal and external peer review process in place. Evaluation steering group’s role in reviewing the approach, analysis, and findings of the evaluation. Quality assurance protocols in place and sign-off required.
Impacts are too small to be detectable with analytical approach	Medium	High	Calculate scale of impact required to be observable given sample size for each impact Undertake theory-based evaluation alongside quantitative analysis
Data availability
Data sharing agreements between BDUK and Ipsos prevent relevant information being shared	Medium	Low	BDUK hold a lot of data, provided by network providers which could be used to support an evaluation but data sharing agreements may prevent use in evaluation. Ipsos identified alternative sources of information which can be used.
Conversion of UPRN level delivery data to higher geographic area levels for analysis	High	Low	Delivery data is provided at UPRN level, whereas connectivity data is provided at postcode level, social, economic and environmental data is available at higher geographic levels.  UPRN data will be transposed to other geographic levels, as has been completed for the Superfast Broadband Programme.
Primary research
Low participation rate in qualitative research and surveys	Low	High	Monitor response rate throughout fieldwork. Over sample to ensure there are additional individuals to undertake research in if struggling to meet targets. Utilise tested approach to making contact.
Lack of understanding of questions	Low	High	Pilot questionnaires and topic guides, and make alterations where questions do not work
Programme delivery
Delayed programme delivery squeezes timeline for evaluation	Medium	Medium	Mitigation – bring forward activities to be delivered later in the evaluation programme where possible, such as some environmental analysis Monitor achieved rollout with BDUK
Slow progress means lack of completed contracts to be included in the financial analysis	Medium	High	Monitor project completion with BDUK Postpone financial analysis until sufficient contracts have completed
Resources / capacity
Resourcing issues in terms of staff, fieldwork, or technical capacity	Medium	High	Business continuity and quality assurance secured through use of extensive and agile internal supply chain at Ipsos Ability to draw on wide pool of researchers within Public Affairs at Ipsos Staff commitments monitored throughout the evaluation Reallocation of taks to ensure staff focus on a small number of tasks within the evaluation to reduce chances of internal conflicts
Staff turnover at BDUK	Medium	Medium	Staff turnover at BDUK presents risk of lost institutional knowledge BDUK to provide detailed handover notes when staff member leaves team Ipsos UK to provide detailed video conference briefing to new team member
Capacity and resource levels at BDUK	Low	High	Three analysts in the BDUK team now who work on multiple projects Ipsos to provide sufficient time for staff at BDUK to review documents due to competing priorities within BDUK

For more information:

3 Thomas More Square

London E1W 1YW

t: +44 (0)20 3059 5000

www.ipsos.com/en-uk

About Ipsos Public Affairs

Ipsos Public Affairs works closely with national governments, local public services and the not-for-profit sector. Its c. 200 staff focus on public service and policy issues. Each has expertise in a particular part of the public sector, ensuring we have a detailed understanding of specific sectors and policy challenges. Combined with our methods and communications expertise, this helps ensure that our research makes a difference for decision makers and communities.

1. Pantic (2014) Online Social Networking and Mental Health, Cyberpsychology, Behaviour and Social Networking ↩

2. Natarajan et al (2020) Civil Society Perspectives on Inequality: Focus Group Research Finding, Submission to UK2070 Commission ↩

3. O’Brien, W. and Aliabadi, F. (2020) Does telecommuting save energy? A critical review of quantitative studies and their research methods, Energy and Buildings, 225(15) ↩

4. Corporate Finance Institute (2020). Internal Rate of Return (IRR). An Analyst’s Guide to IRR. Accessed at: https://corporatefinanceinstitute.com/resources/knowledge/finance/internal-rate-return-irr/. Accessed 7 August 2025 ↩