Policy paper

Plan for Digital Regulation: developing an Outcomes Monitoring Framework 2022

Updated 10 October 2023

A. Summary

Through the Plan for Digital Regulation, government launched a conversation about how to set the right rules for governing digital technologies. We want to drive innovation and growth while also making sure we tackle the most serious downsides associated with digital technologies.

One of the key principles set out in the Plan for Digital Regulation was to actively promote innovation by regulating digital technologies through an outcomes-focused approach, supported by robust evidence. We set out a commitment to improve the evidence base for digital regulation, strengthen our understanding of the impact of our interventions and establish an appropriate monitoring approach to assess performance and inform learning. We want to make sure our approach is streamlined, proportionate and effective.

A diverse range of stakeholders across business, academia and civil society welcomed this commitment. As set out in the summary of views, they stressed the need for government to be clear about how it will track progress against the vision it set out and what good looks like in practice. Multiple stakeholders acknowledged significant evidence gaps across the landscape and welcomed government’s commitment to develop evidence to inform our overall regulatory approach.

In response, we are developing a monitoring and evaluation approach to inform progress against the Plan for Digital Regulation. This will be composed of three main elements:

1. An outcomes monitoring framework to monitor trends on key areas of the digital ecosystem that government is seeking to influence, through regulation alongside broader policy measures.

2. Evaluations of specific governance or regulatory measures.

3. Research projects to provide an overarching view of impacts and learning on measures’ implementation.

This document shows our progress in developing the first of these elements: an outcomes monitoring framework. It sets out the objectives and outcomes of regulating digital technologies, an initial set of indicators and our planned next steps to strengthen the evidence base. It represents an initial step to develop a holistic approach to monitoring progress against government’s overall priorities and objectives, understanding the impact of our interventions and identifying any gaps in our approach.

We welcome stakeholder views on our approach and our next steps. At the end of this document, we explain how you can get in touch to provide your input and feedback.

B. Building on our vision for regulating digital technologies

Through the Plan for Digital Regulation, we set out government’s overall objectives for regulating digital technologies. Our approach will be nimble and proportionate but — where absolutely necessary — we will regulate in order to:

• drive growth, we will promote competition and innovation across the digital sector
• ensure this growth and innovation does not harm citizens or businesses, we will keep the UK safe and secure online
• protect our fundamental rights and freedoms, we will shape a digital economy that promotes a flourishing, democratic society

In the Plan for Digital Regulation, we also set out clear principles for how we design and implement digital regulation to create proportionate, agile, flexible measures — summarised below:

Actively promote innovation:  We will seek to remove unnecessary regulations and burdens where possible. Where we judge intervention is strictly necessary, we will first consider non-regulatory measures like technical standards. Where regulation is needed, it should be designed to minimise unnecessary burdens on businesses. It should be outcomes-focused, backed by robust evidence, and consider the effects of proposed policies on innovation.

Achieve forward-looking and coherent outcomes: Policymakers must make sure that new regulation minimises contradictions, undue burdens, or overlaps and gaps with existing frameworks. Regulatory interventions should address underlying drivers of harm rather than symptoms, and new regulations should be designed with a clear understanding of the links to our wider regulatory regime. Regulators will take a collaborative approach through engagement with businesses and other regulators.

Exploit opportunities and address challenges in the international arena: Policymakers will build in international considerations from the start, taking account of our existing international obligations, likely future agreements, and the impact of regulations and standards developed by other nations. International regulatory cooperation should be a first order consideration for digital regulation.

Theses principles describe how we seek to regulate. Below we lay out more granular and detailed outcomes, which describe why we would seek to regulate, following on from the objectives described above. Each of the outcomes we set out is influenced by a complex interplay of factors, of which regulation is likely only one, alongside other factors such as training, investment, and economic factors.

Promoting competition and innovation

Objectives

• Digital markets are competitive and drive growth across the economy
• Consumers trust they are treated fairly online and can exercise choice over the services they access
• The UK has a vibrant digital economy where startups and scaleups can prosper (alongside other firms)
• Businesses and other organisations are able to use data innovatively

Outcomes

• Enable data sharing to support competition, in line with privacy laws
• Increase consumer choice and the quality of digital services
• Increase investment in UK digital companies and grow the UK supplier base
• Increase business incentives to innovate
• Enable wider digital adoption across industries and regions and support UK AI exports
• Ensure data is used in the public interest
• Ensure data is used to support scientific research and development
• Encourage digital business to locate and grow in the UK, and make the UK a highly attractive jurisdiction for digital investment

Keeping the UK safe and secure online

Objectives

• Citizens are empowered to be safe online, and trust they are protected from online harms beyond their control
• Organisations have the capabilities and resilience to preserve their digital security, and security is factored into new products and services from the outset
• The security of UK networks and critical infrastructure is protected

Outcomes

• Increase user safety online through reduced incidents of online harms, especially with respect to children
• Improve management of cyber risk within UK organisations
• Work with global stakeholders to shape the development of global digital technical standards in priority areas
• Improve law enforcement’s ability to tackle illegal content online
• Ensure consumer connectable products meet essential cyber security requirements and providers of digital services follow better cyber security practices
• Improve users’ ability to keep themselves safe online through greater platform transparency and non-legislative support measures

Promoting a flourishing democratic society

Objectives

• Digital technologies support democratic engagement, preserve freedom of expression and human rights
• UK’s media and press sectors are able to flourish online
• UK citizens’ data will be used responsibly and their privacy preferences will be upheld

Outcomes

• Preserve and enhance freedom of speech online
• Ensure higher transparency for voters about those trying to influence their views when viewing digital campaigning material
• Ensure an online environment which limits the spread of, and user exposure to, misinformation and disinformation and the harms caused by this content
• Ensure platforms remain open to diverse viewpoints and allow for unencumbered circulation of press content
• Enable the growth of digitally-adapted sustainable business models for the press sector
• Increase citizens trust when sharing data and protect individual’s data rights

C. Developing a set of indicators to track our approach

Monitoring progress against the objectives and outcomes we are committed to will be critical to tracking the impact of our overall approach. We need clear, measurable indicators to understand progress - but identifying these indicators is difficult. Competition, safety, and democracy are complex issues, and there are many factors which shape these outcomes in addition to regulatory interventions.

The next section brings together an initial suite of high-level tracking indicators to understand the areas that will be impacted by digital regulation. These will help us understand the current health of these areas, and allows us to better determine whether and where regulation might be required. We are not seeking to use this set of indicators to demonstrate causality in how regulation impacts government’s objectives of interest, but rather to build a holistic picture of the digital landscape. The data we have selected for our indicators do not map precisely to specific government policies - for example, in some cases, multiple policies will contribute to a single outcome, and in other cases, government does not yet have a mature policy response for a given outcome.

The indicators will help us to gather insights against government’s overall objectives in this space as we consider evidence holistically. In some cases the indicators are only loosely linked to the relevant objective due to the limited metrics available, while in other cases some indicators are relevant to more than one objective. Therefore each area of objectives contains a pool of metrics which we will monitor and consider the degree to which they, on balance, provide evidence that government’s objectives are being met.

Our initial thinking on indicators for the objectives is set out below, along with sources for the data, a confidence assessment for the indicator, and measurement gaps.

Promoting competition and innovation

Competition, growth and innovation are all complex phenomena which are difficult to precisely measure, especially in digital environments^{[footnote 1]}. Measures of market concentration, firm entry and exit, profitability and mark-up rates are difficult to determine when the definitional boundaries of digital markets are themselves hard to pin down^{[footnote 2]} and digital markets exhibit structural conditions that reinforce tipping effects^{[footnote 3]}. Many of these measures are influenced by a variety of factors beyond regulation and it is not possible to separate out the impact of regulation alone. The overall evidence is mixed regarding the relationship between market structure and innovation^{[footnote 4]}. DCMS has commissioned research on appropriate indicators of competition, innovation and choice in this area, including to develop a monitoring and evaluation framework for the pro-competition regime for digital markets.

Objectives

• Digital markets are competitive and drive growth across the economy
• Consumers trust they are treated fairly online and can exercise choice over the services they access
• The UK has a vibrant digital economy where startups and scaleups can prosper (alongside other firms)
• Businesses and other organisations are able to use data innovatively

Indicators identified	Source	Confidence assessment of indicators identified	Gaps identified
UK Gross Value Added of digital sector (£150.6 bn: 2019)	DCMS Economic Estimates	Low - many factors influence GVA	Measures of in-market competition and concentration
UK tech sector private investment (£29.4 billion: 2021)	Dealroom/Tech Nation Report	Low - many factors influence investment decisions	Reporting lag in data from announced deals/investment
Percentage of population who feel they have no choice but to sign up to services despite concerns (47%: 2020)	DotEveryone Digital Attitudes Report (updated data on user choice to be set out in forthcoming DCMS research)	Low - self-reported attitudes-based survey responses do not give an objective measure of consumer choice and treatment online	Measures on switching and multihoming
Percentage of businesses who use digital data who think data has become more available (54%: 2020)	UK Business Data Survey	Medium - proxy indicator	Definitions of innovative data use

Keeping the UK safe and secure online

Safety and security are distinct but interrelated issues covering many areas of digital regulation, many of which have robust data sources available. However, measurement challenges remain; for example, there is a difference between the perception of safety online, and the actual incidence rate of online harms. These are difficult to accurately evidence when much of online safety reporting comes from company transparency reports (which only show harms that have been noticed by the firm), or where users self-report experiences of harm in surveys. It should also be noted that while there are thirteen critical national infrastructure sectors, this framework has provided a snapshot view of the telecoms sector specifically.

Objectives

• Citizens are empowered to be safe online, and trust they are protected from online harms beyond their control
• Organisations have the capabilities and resilience to preserve their digital security, and security is factored into new products and services from the outset
• The security of UK networks and critical infrastructure is protected

Indicators identified	Source	Confidence assessment of indicators identified	Gaps identified
I. Percentage of adult internet users experiencing harms online in last 12 months (62%: 2020) II. Percentage of 12-15-year-old internet users experiencing harms online in last 12 months (81%: 2020) III. Number of reports of unauthorised access to personal information (including hacking)[footnote 6] reported in UK (1,467: 2020-21)	I. Ofcom-ICO Internet users’ experience of potential online harms[footnote 5] II. Ibid III. Crime in England and Wales Survey	Medium - online harms are vast in scope and range The Ofcom statistics rely on broad self-reported measures of harm.	Breakdown of specific types of harms experienced Efficacy of different approaches to prevent citizens from harm
Percentage of Operators of Essential Services who increased prioritisation of security at a senior management level within their organisation as a result of the Network Information Systems (NIS) Regulations  (60%: 2020) Percentage of relevant Digital Service Providers who updated or strengthened security governance processes as a result of the NIS Regulations (47%: 2020)	DCMS Post- Implementation Review of the NIS Regulations	Medium - directly surveys organisations impacted by NIS Regulations on their views of its impact - though may not be a representative sample of organisations	Product security measures
Number of reported security incidents from telecoms providers (532: 2020)	Ofcom Connected Nations	Medium - telecoms providers only	Measures of infrastructure resilience across other critical national infrastructure sectors

Promoting a flourishing, democratic society

This cluster of objectives is more difficult to measure and metrics are often indirect. For example, we lack robust indicators to measure how digital technologies support democratic engagement, freedom of expression and human rights, or to measure the extent to which the UK’s press sector can flourish online. We have had to rely on proxy indicators for this pillar that provide measures of related concepts.

Objectives

• Digital technologies support democratic engagement, preserve freedom of expression and human rights
• UK’s media and press sectors are able to flourish online
• UK citizens’ data will be used responsibly and their privacy preferences will be upheld

Indicators identified	Source	Confidence assessment of indicators identified	Gaps identified
Freedom of Internet Index UK country score and ranking (78/100 - Free: 2021)	Freedom House Freedom on the Net Index	Low - proxy indicator on an aggregate rank score (/100)	Specific measures of democratic engagement, freedom of expression, or human rights, or of the impact of digital technologies on these
Percentage of: • UK adults accessing news online, including social media (74% -2021) • UK adults paying for news online (8% - 2021)	Reuters Digital News Report	Medium - proxy indicator that measures readership levels, not business model sustainability	Measures of plurality within online media Measures of business model sustainability
Percentage of people who ‘highly’ trust companies and organisations storing and using their personal information (27%: 2020)	ICO Information Rights Strategic Plan: Trust and Confidence	Medium - proxy indicator that measures attitudes, not experiences	Measures of data ethics and data use

Confidence assessment key:

• High = indicator represents the objective well, based on methodological considerations, regularity of collection, and its direct fit to the area in question
• Medium = indicator represents the objective well, but only meets two of the three criteria that satisfy the ‘high’ confidence assessment (specified in table)
• Low = indicator represents the objective partially (further detail in the table)

D. How we are strengthening the evidence base

We know that gaps remain in our evidence base, that there are complex linkages between our regulatory objectives, that other non-regulatory interventions also impact on these objectives, and that the causal effects of digital regulation are difficult to measure.

We have begun to fill the evidence gaps we have identified, drawing on the work of partners and commissioning new research and surveys. We have focused initially on developing new evidence sources that will help us identify emerging issues in the landscape. This includes:

• DCMS commissioned survey data on public attitudes to digital regulation
• DCMS commissioned research on defining and assessing the prevalence of digital consumer harms
• DCMS commissioned research to establish the evidence base around the commercial value to business of safer online environments
• DCMS commissioned research on developing a monitoring and evaluation framework for the pro-competition regime for digital markets and the Digital Markets Unit that will oversee it
• Ongoing research by the Office for National Statistics to measure the impact of regulatory compliance on digital businesses
• The Digital Regulation Cooperation Forum’s new digital research portal, which brings together recent research on emerging and future digital developments from eight regulatory bodies

Over time, as government introduces new digital regulation, there will be further work to develop bespoke monitoring and evaluation approaches for each area, including impact assessments and evaluation plans where relevant. Alongside this, we will also be keen to understand the interactions and aggregate impact of digital regulation including on consumers and digital businesses.

We will ensure this overall framework links to these specific monitoring and evaluation frameworks as they are developed, and we will continue to review our framework as more information becomes available. We will use this framework to draw out overall lessons and findings, as well as fill in any gaps.

E. Call for evidence and our next steps

We are keen to hear views on what indicators we could feed into this work and address the evidence gaps. We may be able to fill in certain evidence gaps by commissioning additional survey research as above. But we also know that the landscape of digital regulatory issues is fast-evolving and there may be metrics and research we are not aware of. We are therefore opening this monitoring and evaluation framework to public input.

For the purposes of this call for evidence, we are defining an indicator as a measure which is recorded on an annual or regularly recurring basis, from a reputable source, and which is available in the public domain.

We will consider inputs from this call for views and publish an update on our thinking as we learn more.

Based on that context:

1. Do you have any comments on the indicators which we have suggested?

2. Do you have any views on how the digital regulation indicator suite is structured? In particular, we would like to gain a better understanding of any underlying drivers and root causes that give rise to interactions between the Outcomes and Objectives and their aggregate impact.

3. Are you aware of any metrics which currently exist which we could use as indicators for the Objectives or Outcomes? If so, please provide a hyperlink or further information on where they are recorded. We would particularly value input where we have no or low confidence measures.

4. How would you prioritise among the gaps we have identified for commissioning further research?

5. Do you have any views on how to best measure the impact of digital regulation on digital businesses?

Annex A: How to respond to the call for evidence

The call for evidence will be open for 12 weeks, closing on 5 September 2022, to allow time for your consideration and response.

You can send your views on this to: digitalregulationplan@dcms.gov.uk. You can also write to us at:

Digital Regulation Team
DCMS
100 Parliament Street
London
SW1A 2BQ

If you are submitting a hard copy response, please include your name, organisation, and a contact email.

1. See e.g. European Commission Competition Policy for the Digital Era (2019) ↩

2. See e.g. CMA State of UK Competition (2020) ↩

3. PwC Digital opportunities and harms (2020) ↩

4. BEIS Competition and Innovation in Digital Markets (2021) ↩

5. This included victims’ details being compromised via large-scale data breaches, and victims’ email or social media accounts being compromised. ↩

6. This survey has been discontinued. A new survey tracking experience of harm will be included in Ofcom’s Online Nation Report 2022 which is expected to be published at the end of May and is expected to inform the indicator going forward. ↩