Independent report

Centre for Data Ethics and Innovation: Two year review

Published 5 July 2021

1. Foreword from Roger Taylor, outgoing Chair of the CDEI

I started my term as Chair of the Centre for Data Ethics and Innovation (CDEI) with the firm belief that there is no inherent contradiction between the necessity of ethics and the drive for innovation. Data and data-driven technologies (including AI) can grow our economy and help to tackle deep-seated societal challenges. But to harness this potential and build public trust over the long-term, the UK will need to develop effective governance that incentivises responsible innovation.

I remain as committed to this view now as I was then - but the wider debate about data ethics has matured a great deal in the last three years. If in 2018 the focus was on developing the right normative principles to govern AI, policymakers and civil society organisations are now increasingly focused on how to operationalise concepts like “transparency” and “accountability” in the real world. In short, the challenge is how to make these values a reality on the ground.

I am proud of the work the CDEI has already done to advance thinking and practice about the responsible use of data and data-driven technologies. We have delivered internationally recognised policy reviews into two of the most challenging cross-cutting issues in this field, online targeting and bias in algorithmic decision-making; and addressed a range of pressing challenges, from trustworthy public sector data sharing to the use of data-driven technologies in response to the pandemic.

Increasingly we have adopted a new approach, working with partners to help them to operationalise responsible approaches to innovation in the real world. At the height of the pandemic, we supported NHSX and the Department for Health and Social Care (DHSC) to ensure a trustworthy rollout of the UK’s contact tracing app - an app which has now been voluntarily downloaded by over 20 million people and is estimated to have averted almost 600,000 infections.^{[footnote 1]} We have also been working with public sector organisations across the length and breadth of the UK. We have helped local authorities to develop frameworks to enable them to use data in a way that earns the trust of their citizens. We are also currently working with security-focused organisations, including helping Police Scotland to develop inclusive governance for the use of data-driven technologies in policing, and supporting the Ministry of Defence to develop ethical principles for the use of AI.

This work has been made possible by a multidisciplinary team that is plugged into global policy conversations, from the Council of Europe to the Global Partnership on AI. We have built expertise in data policy, public engagement, and technical assurance, using a range of experimental methods to inform our advice.

But there is still much to be done. First, there is a huge opportunity for the public sector to use data to better serve citizens. In the last year, we have seen how data has been used in unprecedented ways to manage the impact of the pandemic, from predicting infection rates to supporting the delivery of furlough. Our public engagement suggests that there is an expectation for the government to use data to effectively deliver services, and to do so in a way that meets high ethical standards.^{[footnote 2]} For public services to responsibly innovate after the crisis, organisations will need to have the toolkit to enable them to meet this demand.

Second, enabling access to data for innovation in a trustworthy way is a significant barrier to responsible innovation.^{[footnote 3]} The UK has set global benchmarks in some of the institutions and mechanisms it has created - Genomics England, the National COVID-19 Chest Imaging Database (NCCID) or Open Banking to name three - but these are the exception, not the rule. To harness the benefits of increased data use, while mitigating the risks, we need to speed up the development of new models that enable data to be used in a trustworthy and privacy-preserving way.

Third, the UK needs to develop an ecosystem for AI assurance products and services, as well as lead globally on the governance of AI. Assurance, which is a common feature of mature markets from accountancy to cyber security, enables people to understand whether systems are trustworthy. As such, it is a critical enabler of increasing market confidence in AI adoption, while ensuring that these technologies are serving societal values and are legally compliant. At present, this ecosystem is fragmented and in a relatively nascent stage. The UK - with its strengths in research, law, and professional services - has the opportunity to create a world-leading ecosystem, but this requires a multi-stakeholder effort and effective coordination. Our team is now working closely with the Office for AI to support the development of the National AI Strategy.^{[footnote 4]} Through our policy reviews, we have argued that sectoral regulators have a key role to play in helping organisations to better understand what responsible and legal use of AI looks like in their sectors. This will also require greater coordination, and I have been encouraged by the establishment of a new Digital Regulation Cooperation Forum.

As more and more governments start to develop policies and regulations, the need to engage and shape the international debate will become more pressing. No country could claim, at this point, to have cracked the problem. But there are important differences in the direction of travel, and a need to ensure that approaches to responsible AI can be delivered in the Global South. The more that like-minded countries can coordinate a response, the more effective it will be.

The CDEI is helping to tackle many of these challenges, and working at an increasingly operational level to do so. The team is at work piloting new models and technical approaches to enable trustworthy data sharing; supporting public sector partners to responsibly innovate with data and AI; and developing a roadmap for an algorithmic assurance ecosystem in the UK.

There is a great deal of work to do, but the prize is real: an opportunity for the UK to lead the way in harnessing the transformative power of data-driven technologies, in a way that benefits and earns the trust of our whole society.

Roger Taylor, Chair of the Centre for Data Ethics and Innovation (July 2018-July 2021)

2. Insights from the CDEI’s first two years of operation

This review is published as the tenure of the first Chair and Board of the CDEI comes to an end, and we take stock of what the Centre has achieved in its first few years. This has been an opportunity to reflect on how the world has changed since the CDEI was set up, what has been learned from the research and projects we have led, and how these shape where the Centre now needs to focus to continue delivering its mission. The government’s response to the National Data Strategy consultation pointed to strong support from stakeholders for the CDEI and our work to date, as well as support for the proposed future functions and a desire for more clarity around upcoming projects.

Box 1. Key facts

Number of staff: From 3 in 2018 to 46 in 2021

Annual budget: £2.5m in 2019/20 to £4.5m in 2021/22

Published reports: 12. These include major reviews into online targeting (February 2020) and bias in algorithmic decision-making (November 2020), an AI barometer covering the opportunities, risks, and governance challenges associated with AI and data use in the UK across five key sectors (June 2020), a report on public sector data sharing which explored how to address the issue of public trust (July 2020), and a six month analysis of novel AI and data use-cases implemented to counter the COVID-19 pandemic, with a corresponding analysis of public attitudes (March 2021).

Blogs posted: 38

Events hosted: Over 20 events hosted, including international events focused on our review into bias in algorithmic decision-making, with partners in the US, EU, Germany and Canada. This is in addition to presenting at over 40 domestic events and international engagements, including London Tech Week, CogX Festival, techUK’s Digital Ethics Summit, international forums such as POLITICO’s AI Summit and RightsCon, and evidence meetings hosted by Select Committees and All-Party Parliamentary Groups (APPGs).

2.1 The changing data ethics policy landscape

The establishment of the CDEI was first announced in the 2017 Autumn Budget - with a mandate to enable ethical innovation in AI and data‑driven technologies. This was an early move by the UK that has since been followed by other nations. Since then the sector has expanded, the debate has matured, and public interest has grown.

In these years, the data and AI governance landscape has experienced rapid growth. The 2018 AI Sector Deal committed a £1 billion investment to the sector, and included the creation of three new AI new bodies. In addition to the CDEI, a new Office for AI was set up as a joint unit of the Department for Digital, Culture, Media and Sport (DCMS) and the Department for Business, Energy and Industrial Strategy (BEIS) to oversee the implementation of the AI and Data Grand Challenge. The AI Council was established to advise the government on AI policy. Outside of government, the wider ecosystem of organisations working on data governance has matured, including existing bodies like the Open Data Institute, and newer organisations like the Ada Lovelace Institute and Alan Turing Institute. Internationally, more than 30 countries have created bodies tasked with supporting and piloting national responsible AI and data efforts.

The policy debate has also matured. At the time of the AI Sector Deal, many government teams and civil society organisations working on data ethics were focused on the development of AI principles. Dozens of statements have been developed, with significant normative overlap.^{[footnote 5]} The Organisation for Economic Co-operation and Development’s (OECD) AI principles have been adopted by more than 50 countries, including the UK. With this high level grounding of normative goals, policymakers and organisations are increasingly focused on the greater challenge of how to put these principles into practice in a variety of contexts.

Public interest in data and AI has substantially increased. Data policy has become the topic of evening television news - for good, as in the case of a range of data-driven innovations used to tackle the pandemic, and for ill, such as the Cambridge Analytica scandal. The CDEI’s own analysis found a 54% increase in UK media coverage on data and AI during the first year of the pandemic. Research suggests public concern about a range of issues relating to the governance of data-driven technologies, but also an expectation that technology is part of modern life which should be used by governments.^{[footnote 6]}

2.2 Consistent insights from a range of projects

The CDEI was commissioned to lead two major reviews, the first into online targeting, exploring in depth the role personalised advertising and content recommendation systems play in three main areas: autonomy and vulnerability, democracy and society, and discrimination. The second review explored the risks of bias in algorithmic decision-making in four sectors where significant decisions about individuals are increasingly informed by algorithms: policing, local government, financial services, and recruitment, and made cross-cutting recommendations on how government and regulators should act to manage these risks.

In addition to these major pieces of research, the Centre has produced analysis which has significantly informed data ethics discussions across government and industry, including the AI barometer (a first of its kind analysis of the most pressing opportunities and challenges presented by AI and data), a report on public sector data sharing (analysing a series of government data sharing projects to identify the common barriers to data sharing in the public sector, as well as steps taken to address them), and a six month analysis of novel AI and data use-cases implemented to counter the COVID-19 pandemic, with a corresponding analysis of public attitudes towards the use of data-driven technologies in the UK’s COVID-19 response.

All of our work has been underpinned by public engagement. Engagement with the public helps decision-makers to understand the elements of governance most important to building a trustworthy environment. Citizens’ priorities and the appropriate mechanism to meet them will depend on the context: in our review of online targeting, we found that citizens expect to have meaningful control over targeting systems on online platforms; whereas in our review into bias in algorithmic decision-making, we found citizens were most concerned with ensuring human accountability over decision-making, with a human in the loop.

We must engage widely: there is no one ‘public’, and it is important to explore the perspectives of people who are at risk of being disproportionately affected or at risk of not being heard in government decision-making processes. For instance, in our review of online targeting, as well as engaging widely with the public, we took particular effort to engage the views of people who might be disproportionately affected by targeting systems, including people with poor mental health.

Emerging from our work are consistent and recurring challenges that government, industry and the regulatory ecosystem will need to tackle as a priority. Tackling these challenges effectively is a critical part of delivering on the fourth pillar of the National Data Strategy, which commits to ensuring that increased innovation is trustworthy, with data being used in a way that is lawful, secure, fair, ethical, sustainable and accountable.

The first challenge is a need to develop and maintain accountability when deploying data-driven technologies. Our review of online targeting found that there were insufficient mechanisms in place to make major online platforms accountable for the societal impacts of the targeting systems used to recommend content and show advertisements. We recommended regulatory oversight of the targeting systems on social media platforms, with a focus on governing systems and processes rather than categories of content; clear information-gathering powers for the regulator, including powers to audit algorithms; and greater coordination among the major regulators. The spirit of these recommendations has been adopted in the government’s draft Online Safety Bill.

In our review into bias in algorithmic decision-making, we highlighted the risk of algorithms obscuring the accountabilities and liabilities that individual people or organisations have for making fair decisions. The review underlined the need for organisational decision-makers to be clear that they retain accountability for decisions made by their organisations, regardless of whether an algorithm or a team of humans is making those decisions on a day-to-day basis. It pointed to the need for an ecosystem of industry standards and professional services to enable organisations to develop and deploy algorithms fairly.

Our research, including our review into bias in algorithmic decision-making, has drawn attention to the need for regulators to build and share capabilities to effectively regulate AI in their sectors. As a general purpose technology, AI will become commonplace across the economy, posing different challenges depending on the context in which it is being applied. This means that efforts are better focused on supporting regulators to regulate AI in their sectors rather than establishing a single new cross-sectoral regulator for AI. However, to enable this ecosystem to operate more effectively, there is scope for more consistent approaches among regulators to techniques such as algorithmic auditing. In our review of online targeting, we advised that Ofcom, the Information Commissioner’s Office (ICO) and the Competition and Markets Authority (CMA) should establish coordination mechanisms. Since then these regulators have established a new Digital Regulation Cooperation Forum.

The second is a need to address the transparency and explainability of data-driven systems. This transparency must be meaningful to the people affected. In the context of online targeting, we highlighted the need for social media platforms to make it clearer to their users why they are seeing targeted content, and to enable them to make meaningful choices over how data about them is used. To be useful to the public this information would need to be compressible and relevant, and following the review, we began a research project with the Behavioural Insights Team to find ways to enable users to make active choices on social media platforms, which reflect their preferences, and are based on an understanding of the likely consequences. But for researchers to better understand the impacts of social media platforms on society, it would be necessary to have a more granular level of transparency, with access to wider datasets to enable computational analysis. In our review of online targeting, we recommended that Ofcom should be given powers to enable researchers investigating issues of significant public interest secure access to platform data, and the draft Online Safety Bill will require Ofcom to investigate how to improve researcher access.

Our review into bias in algorithmic decision-making highlighted the need for transparency when using algorithmic decision-making in the public sector, particularly as adoption accelerates. Technology should not reduce the accountability of public institutions to citizens. In fact, it offers opportunities to improve accountability and transparency, especially where algorithms are used when making significant decisions about individuals. Following publication of our review, we began working with the Cabinet Office’s Central Digital and Data Office (CDDO), which is developing a standard for algorithmic transparency in the public sector. We worked with CDDO and BritainThinks to conduct a deliberative public engagement exercise to explore what meaningful transparency about the use of algorithmic decision-making in the public sector could look like in practice.

The third challenge is the question of improving access to high quality data, which is crucial to the development, deployment and evaluation of data-driven technologies. Often decision-support systems require bringing together different datasets, but physical barriers, such as poor infrastructure, and legal complexity or confusion - such as insufficient knowledge of how and when to share data in line with data protection legislation - inhibit data from being shared and used even when there are clear benefits, which hinders innovation.^{[footnote 7]} Going forward, it will be critical to develop new data stewardship models that enable data to be handled safely and responsibly, as well as to pilot new technical solutions that preserve privacy and confidentiality. By managing and mitigating some of the risks involved in sharing and using data, new approaches such as these have the potential to unlock avenues to innovation, which would bring about significant benefits. As part of the government’s National Data Strategy, we have been working with DCMS and the ICO to explore the potential for greater adoption of privacy enhancing technologies (PETs), such as homomorphic encryption, trusted execution environments and synthetic data, and will soon be publishing a guide to assist individuals and organisations working on data initiatives in considering how they might use PETs in their projects.

3. How do we create the conditions for responsible innovation?

Over the next year, we will prioritise three themes in our work, to help foster responsible innovation at pace, and address the challenges highlighted above. The themes are:

1 - Data sharing: We will facilitate responsible data sharing across the economy, including piloting new forms of data stewardship and governance.

2 - Public sector innovation: We will support and facilitate the responsible development, deployment and use of AI and data across the public sector, with a focus on the most high impact use-cases.

3 - AI assurance: We will help lay the foundations for the development of a strong AI assurance ecosystem in the UK, helping organisations to have confidence to innovate responsibly with AI and data, by fostering an emerging industry in AI assurance services.

This work will be informed by multidisciplinary capabilities, with expertise in data policy, public engagement, and technical understanding.

3.1 Data sharing: Supporting new approaches to facilitate trustworthy data sharing and improve data access

The potential benefits of increased data sharing are significant, yet data sharing also poses risks and tests our existing governance systems. We need to widen understanding and adoption of new approaches that help to encourage data sharing for the public benefit. There is an increasing focus on the role of data intermediaries to manage different data rights and support data sharing for defined purposes. This might include developing mechanisms designed to make it easier for competing organisations to share data in order to drive growth by enabling greater innovation. Similar arrangements could make it easier for individuals to donate data for research purposes.

As highlighted in our review into bias in algorithmic decision-making, good use of data presents an opportunity to enhance fairness. If an organisation has hard data about differences in how it treats people, it can build insight into what is driving those differences, and seek to address them. Going forward, evaluating datasets for bias and discrimination will be of more importance to organisations across the economy, to prevent data-driven tools from amplifying historic biases, or creating them anew. Here, data intermediaries could provide a mechanism that would allow third parties greater access to data for auditing purposes, which would help to improve the development and deployment of new data-driven tools.

There is also an opportunity to deploy more technical solutions to enable greater access to data. PETs have the potential to be disruptive. In the broadest sense, a privacy enhancing technology is any technical method that protects the privacy of personal or sensitive information. This definition includes relatively simple technologies such as ad-blocking browser extensions, as well as the encryption infrastructure we rely on every day to secure the information we communicate over the internet. Of particular interest to the CDEI is a narrower set of emerging PETs. This is a group of relatively young technologies which are being implemented in an increasing number of real world projects to help overcome privacy and security challenges, and could increasingly begin supporting more secure data processing, trustworthy data sharing, and privacy-preserving machine learning.

3.2 Public sector innovation: Enabling public sector partners to innovate responsibly

It is crucial that the public sector uses data and data-driven technologies in a way that is trustworthy. This includes striving to be as transparent as possible and meeting the highest standards of accountability. Democratically-elected governments bear special duties of accountability to citizens. An individual has the option to opt-out of using a commercial service whose approach to data they do not agree with, but they generally do not have the same option with essential services provided by the public sector.

As a major developer, buyer and user of data-driven technologies, the public sector has an opportunity to set an example of what responsible use of these technologies looks like, and there is a strong desire across the public sector to do so. In its National Data Strategy, the government set out an ambition to secure a trusted data regime. When engaging with local authorities, we have found that good data governance is front-of-mind for data and information governance leads, although they find it difficult to translate theoretical frameworks into practical steps. We have also been struck by the strong interest in and momentum on data ethics in the policing community.

A renewed focus for the CDEI is to partner with government departments and public sector bodies across a range of sectors, to help them make good decisions about the responsible use of data and data-driven technologies. Through these collaborations, we apply, test and refine governance approaches, support partners in designing frameworks for responsible data use in their specific context, as well as to identify and address barriers to responsible innovation. To inform our advice, we engage across partner organisations, as well as with civil society, industry, academia and the public. We seek to understand the specific contexts in which the different organisations are operating, as well as the expectations of citizens.

As we progress with our partnership work, we will compare findings and methodologies, and consider whether governance approaches can be replicated elsewhere, sharing insights which emerge to support other organisations with the ambition of delivering similar projects. This will ensure that individual projects have a wider systemic impact, and reduce duplication of effort. Cross-cutting issues could include the role of independent oversight, approaches to transparency in different contexts, as well as broader insights relating to trustworthiness.

3.3 AI assurance: Establishing a world-leading ecosystem

Maintaining the accountability of organisations and individuals using data-driven technologies has been, as previously described, a recurring theme throughout our work. However, there is not yet clarity or consensus about how to assess whether the use of AI and data-driven technologies is fair, safe or otherwise acceptable. Meaningful, consistent information about how these systems perform is also often lacking, which prevents the organisations using these technologies from assessing whether they can be trusted. In order to hold organisations and individuals fairly and consistently accountable, there is a need for the UK to develop a robust ecosystem of AI assurance.

‘Assurance’ covers a number of governance mechanisms for third parties to develop trust in the compliance and risk of a system or organisation. As well as delivering better societal outcomes, an ecosystem will improve market confidence in AI, as it will reduce the regulatory and reputational risk of using AI tools. When applied to the context of AI, an effective AI assurance ecosystem will mean that regulators, developers, executives and frontline users will be able to rely on a collectively agreed set of standards through the supply chain, which gives them the confidence that the AI tool they are using is safe, with clarity on how it has been developed.

An ecosystem of assurance tools and approaches is beginning to emerge, with a range of companies starting to offer these services. But the picture is fragmented, with insufficient consensus among organisations and within sectors on what it means to be accountable and transparent when, for example, relying on algorithmic decision-making - making it harder than it should be for organisations to do so confidently and responsibly. AI assurance is likely to become a significant economic activity in its own right and with strengths in research, legal and professional services, the UK is well placed to take a leadership role globally.

Our programme of work on AI assurance aims to assess how assurance approaches used in other sectors could be applied to AI, the current maturity and adoption of these assurance tools in addressing compliance and ethical risks in AI, as well as the role of standards to support this. We will publish an AI assurance roadmap that sets out our view of the current ecosystem, as well as how it should develop to enable organisations to innovate with confidence, while minimising the risks. In doing so, we hope to help industry, regulators, standards bodies, and government, think through their own roles in this emerging ecosystem.

Going forward, our multidisciplinary team of specialists will develop, test and refine approaches to trustworthy data and AI governance, working in partnership with organisations. We will help to drive responsible innovation, enabling the UK to capitalise on the societal and economic opportunities posed by data and data-driven technologies.

1. The epidemiological impact of the NHS COVID-19 app, Nature, May 2021 ↩

2. Polling data on data sharing, Centre for Data Ethics and Innovation, May 2021 ↩

3. AI barometer, Centre for Data Ethics and Innovation, June 2020 ↩

4. New strategy to unleash the transformational power of Artificial Intelligence, Department for Digital, Culture, Media and Sport, March 2021 ↩

5. Principled Artificial Intelligence, Berkman Klein Centre, January 2020 ↩

6. The Citizen’s Perspective on the Use of AI in Government, Boston Consulting Group, March 2019 ↩

7. Addressing trust in public sector data use, Centre for Data Ethics and Innovation, July 2020 ↩