Data and AI Ethics Framework

Question 1

Introduction

Accepted Answer

This framework provides a set of principles and activities to guide the responsible development, procurement and use of data and artificial intelligence (AI) in the public sector. It helps public servants understand ethical considerations and how to address these in their work.

Data and AI ethics is about using data and data-driven technologies responsibly, including:

respecting privacy
promoting fairness
protecting individuals, communities and society from harm

This includes ensuring that data is collected, shared and used in appropriate, fair, safe, sustainable and transparent ways.

The framework applies to any project that involves:

data (collection, sharing or use)
data-driven technologies
AI
automated decision-making and algorithmic tools

In this guidance, this is what we mean when we refer to ‘data and AI’.

Who this guidance is for

The framework is for people who work in government and other public sector organisations. It’s mainly written for people who are designing, building, maintaining, using or updating projects that use data and AI.

This may include:

developers
project managers
analysts
statisticians
policy makers
commercial or procurement professionals

We also encourage anyone who works directly or indirectly with data and AI to read this guidance and apply it to their work. This includes operational staff and anyone helping to produce data-informed insight.

How to use this guidance

Your team should work through the framework together as you plan, implement and evaluate a project. You should regularly revisit it throughout your project, especially when you make changes to your work or project.

The framework covers a broad range of use cases and challenges that you might encounter. Some projects may be limited to certain aspects – for example, data collection and processing – while others may cover a full AI development life cycle. This means you might find certain parts of this guidance more or less relevant to your project.

Data and AI Ethics Self-Assessment Tool

If your team is working on an AI or data-driven technology project, then as well as reading this guidance you should use the Data and AI Ethics Self-Assessment Tool.

The tool will help you capture information, and share learnings, challenges and progress with colleagues. We recommend that you maintain it along with your other project documentation to record decision making across the life cycle of a project.

Why this guidance is important

Government guidance needs to address the rapidly evolving ethical challenges and risks posed by data and AI technologies. We’ve updated the previous version of this guidance to meet this need.

The Data and AI Ethics Framework bridges the gap between high-level ethical principles and practical actions. It considers themes such as safety, security and privacy through an ethical lens. It does not replace technical or regulatory guidance in these areas, but instead complements and connects to them.

It complements existing tools, standards and guidance, including the:

This guidance does not replace:

existing ethics guidance and tools for statisticians and social researchers, such as the UKSA’s Ethics Self-Assessment Tool
bespoke guidance provided by individual departments, devolved administrations or for specific professions

Principles

This is an overview of the main principles everyone should follow when working with data and AI in the public sector. The rest of the guidance explores each principle in more detail.

Transparency

Transparency means that information about your project, actions, processes and data are communicated to relevant parties in an understandable, easily accessible and free way. Transparency includes the concept of explainability, which refers to the ability to understand and communicate how a system makes decisions, including how it arrived at an individual decision.

Accountability

Accountability means that data and AI projects have appropriate and effective governance, oversight, and routes to challenge decisions in place, where this is necessary. It involves setting clear roles and responsibilities, and being transparent about the system and the processes around it.

Fairness

Fairness means avoiding unfair biases and impacts, and addressing accessibility barriers, while actively maximising positive impacts. You should assess data representativeness, bias, and the different potential impacts that can result from data use.

You should mitigate model biases and aim for outputs and decisions generated with the help of automated processes to:

be non-discriminatory
be just
respect dignity
align with the public interest, human rights and democratic values
comply with the Equality Act 2010 and the Public Sector Equality Duty (PSED)

Privacy

Privacy means that personal data is respected and handled responsibly, ensuring it’s used only for intended purposes with an appropriate legal basis. It also means adhering to UK data protection regulation, including principles of data minimisation, purpose limitation and informed consent.

Safety

Safety refers to the use of data or development of a system without causing harm to people, organisations, wider social institutions or the environment. Data-driven systems such as AI should be robust, secure and safe at every stage of their life cycle. This means ensuring that they operate properly and reliably, without causing unnecessary safety or security risks.

Societal impact

Societal impact is about how the development and use of data and data-driven technologies impact wider society. For example, by helping to solve societal challenges and deliver public good.

Environmental sustainability

Sustainability calls for responsible design and use of data and AI that minimises environmental impact, and supports long-term wellbeing for people and the planet. This means considering biospheric consequences when scoping potential AI and data projects, and building tools and systems that are robust and energy efficient. Biospheric consequences are the effects a project can have not only on the planet’s ecosystems but also its climate and the entire biosphere.

Trade-offs

There may be instances where you find that 2 or more of the ethical principles are in tension with one another. For example, using more demographic information may help to identify biases and promote fairness, but this may come at the expense of an individual’s privacy. Here, you might need to consider trade-offs. Ethical trade-offs are inherently complex and you must treat them with care.

Here are some approaches which can help you establish suitable trade-offs:

include stakeholder perspectives by engaging with affected communities to understand values and impacts
consult legal and human rights frameworks to ensure that decisions align with national and international standards
balance proportionality and necessity
ensure meaningful and inclusive deliberation
establish mechanisms to monitor outcomes, and adjust if necessary

Question 2

Transparency

Accepted Answer

Transparency is foundational to all other ethical principles. Being transparent allows for scrutiny of actions, decisions and processes. It cultivates accountability and can help to build trust in the use of data and technology.

Explainability applies specifically to AI systems and the ability to describe how a system arrives at a given output or decision. For example, this could include explaining what factors led to a benefit application being granted or denied. Explainability can help to build trust by making it easier to understand decision-making processes, and easier to identify risks, inaccuracies and harms.

A lack of transparency might lead to:

solutions being designed suboptimally – at worst leading to harmful outcomes
insufficient accountability and no clear ways to appeal decisions of automated systems – which may result in public distrust

The first data protection principle in the UK General Data Protection Regulation (UK GDPR) is that of ‘lawfulness, fairness and transparency’. In the context of data protection, transparent processing is about being clear, open and honest with people from the start about:

who you are
how and why you use someone’s personal information

What transparency means in practice

It can be helpful to think about:

why you’re being transparent
what you’re being transparent about
to whom you’re being transparent

For why you’re being transparent, consider these questions:

are you following a particular mandate or requirement to be transparent?
are you trying to address topics such as safety, trust, accountability, internal or external scrutiny, reproducibility, transferability or robustness?
what are the potential consequences of not being transparent?
how will your organisation balance the resources required to make material transparent with other competing operational priorities?

For what you’re being transparent about, consider:

the tools you’re using – for example, for data analysis or automated processes – and explaining the process or context for how you’re using them
publishing open data or source code, where this is safe and complies with internal governance
explaining the factors that influence a system’s output
providing contact details to get in touch with your team

In terms of who you’re aiming to be transparent to, consider:

internal transparency within your organisation – for example, record-keeping for handover and continuity purposes
cross-organisational transparency – for example, sharing case studies with other government departments
transparency as part of oversight requirements – for example, being transparent to regulators, auditors or evaluators
transparency with directly affected individuals or defined groups – for example, explaining algorithm-assisted decision making
public transparency – for example, publishing information about a project on GOV.UK

Depending on who you’re trying to be transparent to, you should engage with these people and organisations. Ask them what type and level of transparency would be most meaningful and useful to them.

This might involve:

talking to users about what level of explainability is useful
talking to civil society about what they need to know to effectively hold you to account
talking to researchers about what information would help them evaluate the system
talking to other teams in government about what information they would like about what you’re doing

These considerations will help you identify the right content and level of transparency to use in each specific case, across relevant stakeholder groups. As an example, you might be the owner of a data set that others might benefit from, and which does not contain personal data. You might consider publishing it openly on data.gov.uk. However, where personal data is concerned, you may have to provide transparency information in response to a data subject access request via official channels.

As another example, if you want to ensure a smooth handover of an in-house AI system, you might choose to record and share its technical details and source code with colleagues. This will ensure it remains safe, functional and easy to maintain. Or, if you’re generating explanations of a system’s outcome to affected parties, you may adapt transparency information to include simple, non-technical and user-accessible explanations where possible.

Transparency in the context of data

Data transparency is about communicating:

where the data comes from
how the data is collected, cleaned, used, stored, shared, analysed and interpreted
the limitations of the data
how you’ve complied with applicable laws – for example, data protection and intellectual property law

This applies to the data itself, but also to metadata (such as file format or author) and paradata (data about the data collection process itself).

Being transparent about data can involve the following elements.

Open data

Making data openly available can support trust and transparency by allowing others to replicate and scrutinise published results. If data is non-sensitive, non-personal, and if data sharing agreements with the supplier allow it, you should make the data open and assign it a digital object identifier (DOI).

You can publish government data sets on data.gov.uk, following the Open Data Charter.

Documentation about the provenance (collection or generation) of data

If you’re collecting or generating data, it’s important to be open about why, what and how (by which methods) data is being collected.

Documentation about any pre-processing to address limitations of the data

Discuss what pre-processing steps you’ve taken to ensure adequate data quality. Disclose any limitations of the data, such as quality issues, missing or incomplete data, gaps in representativeness, or known errors.

Documentation about how the data is used in a specific project

When using data in a specific project, clearly communicate the problem domain, user need, project aims and value to relevant stakeholders. When starting data analysis, you should be able to clearly explain the purpose and methodology of your analysis.

Be open about who has access to the data, what data sharing agreements are in place, and what the conditions for reuse are, such as licensing.

Where personal data is being processed, you must comply with relevant laws: You need to carry out a Data Protection Impact Assessment (DPIA) before you process personal data when the processing is likely to result in a high risk to the rights and freedoms of individuals. It’s good practice to publish your completed DPIA to demonstrate that you’re taking the appropriate precautions to protect personal data and ensure your system is fair.

When processing personal data, you must provide a privacy notice that is easy to locate and explains in clear, accessible language:

why you need the data
how long you’ll keep the data
whether you’ll share the data with others
how people can exercise their rights or raise complaints

You should also explain the safeguards that achieve effective data minimisation, such as anonymisation or pseudonymisation, in plain language.

Concepts such as data sheets or data set cards can be helpful in recording some of this information. Good examples include data sheets for data sets and variations of data set cards that have been developed based on this approach, such as Google’s data cards.

For more information, refer to the UK GDPR guidance and resources on the Information Commissioner’s Office (ICO) website.

You must comply with the Public Sector Equality Duty and Equality Act 2010. This means you need to carefully consider and document the equality impact of processing personal data where it can be used to identify individuals as having a protected characteristic. This document is often referred to as an Equality Impact Assessment (EIA). For more information, consult the EHRC guidance on the Public Sector Equality Duty.

It’s good practice to publish your EIA. This will demonstrate that you’ve considered the potential equality implications (both positive and negative) of processing data that relates to any protected characteristics. Publishing your completed DPIA and EIA will build trust in your use of data and data-driven technologies.

Transparency in the context of AI and algorithmic tools

Ensuring transparency and explainability can be complex in the context of AI, algorithmic and automated systems. Transparency can be limited by ‘black box’ algorithms or proprietary commercial tools. Explainability may not be possible for certain forms of machine learning, or may only be achievable at the cost of performance.

To be transparent in the context of AI and automated systems, you should consider the following concepts.

Technical transparency

This is information about the technical operation of the AI system, such as the code used to create the algorithms and the underlying data sets used to train the model.

Process transparency

This is information about the design, development and deployment decisions and practices behind your AI solutions, and the mechanisms used to demonstrate that the solution is responsible and trustworthy. This may also include an audit log of the prompts and outputs generated by AI systems.

To enable process-based transparency, you should have the following in place:

robust reporting mechanisms
process-centred governance frameworks
AI assurance techniques

Outcome-based transparency and explainability

This means the ability to clarify to any person who is using or impacted by a service that uses AI, how the solution works, and which factors influence its decision making and outputs. This includes individual-level explanations of decisions, where this is requested. It can also include aggregate statistics about outcomes that can be used to monitor fairness.

Explainability in practice

For certain types of machine learning models, such as deep learning or large language models (LLMs), it can be impossible to fully explain the logic behind an output or decision due to their complex, ‘black box’ nature. For example, LLMs may hallucinate (where LLMs generate plausible but factually incorrect information) for unexpected reasons. The size and complexity of these models may make it difficult or impossible to identify a determinate causal chain for why this has happened.

You should assess the degree of explainability that’s necessary for your project given its specific domain context and use case. Do this before making technical or architectural decisions about the type of algorithmic techniques you’ll employ in model development. For instance, in high-impact contexts such as social care or criminal justice, you might need a higher level of explainability to ensure fairness and comply with applicable regulation. Considering explainability needs will help you avoid investing in inappropriate technology.

In other cases, where it’s impossible to understand or interpret the rationale behind the outputs of the model, consider alternative methods of explainability such as visualisation techniques or counterfactual explanations. The ICO has guidance on explaining decisions made with AI.

Using the Algorithmic Transparency Recording Standard

The Algorithmic Transparency Recording Standard (ATRS) establishes a standardised way for public sector organisations to publish information about how and why they’re using algorithmic tools.

All central government departments and arm’s length bodies that provide public or frontline services, or routinely interact with the general public, must use the ATRS. This involves documenting information about any algorithmic tools used in decision-making processes, and making this clearly accessible to the public.

We also recommend the ATRS to other public sector bodies, such as local government, even if they’re not currently mandated to use it.

Even where a tool does not strictly fall within scope of the ATRS, we recommend publishing an ATRS record.

If you’re using a third-party supplier

Vendors of AI systems must be able to clearly explain:

the steps they take to build tools
the logic and assumptions built into their tools
how their tools generate outputs

This explanation should include information about:

the data used to train the AI model
the function of the AI models
the system’s intended purpose
the steps they’ve taken to mitigate risks

Before entering into a contract, consider asking your supplier:

how they engaged with users and affected communities across the development life cycle
how they identified, evaluated and mitigated risks
what the source of the data is
how the data sets were created
what data the model has been trained on
if training data sets are sufficiently representative to facilitate fair performance
where third-party data has been used to train the AI, if the organisation has a valid and lawful basis for processing
if there are any known biases in the data or the trained model
how accurate the system is
if the system has been tested for differential performance on relevant demographic groups
if the system is based on an LLM, what the observed rate of hallucinations is, and under what conditions they normally occur
what benchmarks or other modes of model assessment and evaluation have been used to measure safety, reliability, robustness, performance, privacy preservation, toxicity and bias
if there are any issues likely to affect quality

You should update the answers to these questions when significant changes are made to data sets, models, APIs or other relevant components that you’re procuring.

For further guidance on procuring AI systems, refer to the AI Playbook and the Responsibly buying AI guide for councils in England.

You should consider any tools you’re procuring from a third party which are intended to be used in decision-making processes. These tools might fall within the scope of the mandatory requirement to use the ATRS. At the stage, it’s helpful to ensure that the supplier is providing you with all the required transparency information requested in the ATRS.

You can also check the Responsible Handover Framework to identify the information needed for a successful AI system handover from supplier to buyer.

Recommended actions

Your team should:

explain the what, how and why of your project in plain language to a non-expert audience; if you cannot articulate how the use of data or AI solves a defined problem, you should consider alternative solutions
maintain detailed records of the project – include the decisions that were made in its design, for example:
how the project delivers positive social outcomes for the public
what the user need is that it tries to address
who the senior responsible owner of the project is
how a relationship with a third-party supplier was set up
your EIA and DPIA
make information publicly available – consider where you can make information about a project or tool publicly accessible, such as on GitHub, in blogs, on GOV.UK or on your organisation’s webpage
use the ATRS if your organisation is within mandatory scope, and consider publishing an ATRS record even if your organisation is not in mandatory scope
give appropriate disclosure when AI has been used to produce an output – this concerns both internal and external uses of AI
maintain detailed records of all data sources used, including collection methods, licenses and pre-processing steps
publish DPIAs and EIAs where possible – refer to the Fairness and Privacy sections for more information
maintain detailed records of all the design decisions and assumptions that went into the development of a solution
create and regularly update assurance documentation that explains the model’s purpose, performance limitations and intended use cases – this might include model cards or data set cards
if possible, publish your source code openly – for example, on GitHub
build algorithmic tools with explainability in mind – consider how important explainability is for the intended use case, and the degree of explainability that would meet domain-specific expectations and regulation
use technical methods for assessing the interpretability of machine learning models – you should have a good working understanding of their limitations:
common explainability techniques such as SHAP (SHapley Additive exPlanations) and LIME (Local Interpretable Model-Agnostic Explanations) may be helpful in some cases, but may be ineffective for, and inapplicable to, LLMs and generative AI systems
consult technical experts and state-of-the-art research when choosing appropriate explainability methods

Question 3

Accountability

Accepted Answer

Accountability involves setting clear roles and responsibilities and being transparent about decisions. It means that people and organisations are responsible for the data they collect, use and share, and for the technology they build. It also means that if something goes wrong and a system or process harms someone, this can be raised and fixed.

Being accountable helps keep people safe, and ensures that data and AI is used fairly and responsibly. A lack of accountability or failure to uphold ethical standards can result in serious consequences, including: - unaddressed harm to people, such as unfair decisions - harm to public services - legal risks and repercussions - regulatory action - reputational damage and a loss of trust in organisations

What accountability means in practice

You should take the following steps to establish and support accountability across a data or AI project.

Set clear roles and responsibilities

You need strong oversight and clear responsibilities to ensure accountability across the life cycle of your data or AI project. You must set out who is responsible at each stage of the project.

This includes naming:

senior responsible owners (SROs) – as the primary risk owners for the project, they ensure a real policy or business need is being addressed, engage with stakeholders and assure ongoing viability
data owners – data sets must have named data owners accountable for quality, access controls and life cycle governance
data stewards – they should be responsible for metadata and day-to-day quality controls
data custodians – they should be responsible for capturing, storing and disposing of data in line with the data owner’s requirements
AI asset owners – they should be responsible for AI outputs, including model predictions and any generated data

For project governance to be effective, it’s essential that the people accountable have appropriate technical expertise and knowledge, and authority, to implement necessary changes. This is especially important for complex projects or those involving multiple suppliers, which is often the case with data and AI projects.

For more information, refer to the:

Service Manual – which explains how to build a digital and data team
Government Digital and Data Profession Capability Framework – which outlines the skills needed for any digital and data role
guidance on the role of the SRO
Government Project Delivery Functional Standard – which sets expectations for the management of government projects
guidance on Data Ownership in Government – which sets out a data ownership model that formalises the roles of the people in government responsible for managing data throughout its life cycle

Traceability and auditability

Your project team should keep clear records to show how your data or AI system was designed, developed and used, who made a decision and the evidence they used. This will make your project accessible for audit, oversight and review by appropriate parties. It will help others check that your work is trustworthy, and that it meets legal and ethical standards.

For any project that uses data, your team must:

name an SRO and data owner for the data
establish who is responsible for each component of the project, including third-party vendors
document lawful use and purpose of processing, including legal basis and a recording of processing activities
register new data assets – either via organisational data catalogues, or cross-departmentally by designating Essential Shared Data Assets (ESDAs)
record and retain audit trails of who has access to the data and why
record all important decisions and maintain a process log with evidence of responsible data practices
provide routes for people to raise concerns, complaints or request corrections

To ensure AI and algorithmic systems can be audited or reviewed, your team must: - record who is responsible for each part of the system, including suppliers - keep information about where the data came from and how it was used – from data collection and preparation, to training, testing and use - record decisions made during project design, model deployment and system deployment processes - keep clear records that show how the system was designed, developed, tested and implemented

This information should be easy to find and understandable to a range of stakeholders. For example, a data set factsheet can help explain how data was handled. The Algorithmic Transparency Recording Standard (ATRS) also provides a template for public sector organisations to document and share information about how and why they’re using algorithmic tools.

If you’re building an AI system, you should take steps to ensure that the system is explainable. For example, you can explain why a model or algorithmic tool has produced a particular outcome. There are some cases where it can be difficult to trace the inputs and outputs of AI systems, such as large language models (LLMs). The Transparency section has more information on explainability.

Establish feedback mechanisms

Upholding accountability means ensuring that people affected by the use of a data or AI system can:

understand how decisions were made – this may include data sources, decision-making criteria and logic
provide feedback
contest outcomes that are incorrect
seek redress if they‘ve been affected by erroneous or unjust outcomes – for example, by asking for the decision to be reviewed or corrected, especially in the case of automated decision making

You should create a clear mechanism for monitoring the ongoing use of your data or AI project. This involves evaluating how it’s used, and providing a feedback loop where users and operators can identify issues and send them back to the development team.

You should also set up a clear way of handling and responding to feedback, complaints and appeals.

Examples of feedback mechanisms might include:

surveys and feedback forms
reporting mechanisms integrated into the tool or service
stakeholder workshops

Where appropriate, you should engage with civil society organisations for guidance on how to design and integrate feedback and redress mechanisms for your AI project or system.

Maintain human oversight

People must be able to monitor and influence how systems work, even when the system is partially or fully automated. People should remain responsible for the decisions they make when these are supported or informed by an AI system.

If your system includes automated processing, make sure there’s appropriate human oversight. This includes:

recording how much of the process is automated and how much is controlled by people
naming the roles and people responsible for oversight and for managing risks
avoiding fully automated decisions where the outcome could significantly affect individuals or groups, and making sure a person makes the final decision in these cases
if using LLM tools, ensuring someone is verifying outputs for incorrect information and potential biases

In some AI use cases, automated decision making can be difficult to avoid. For example, it would typically be impossible for humans to review every conversational output of an LLM-based chatbot. Here you should weigh the risks of incorrect or harmful outputs and resulting negative impacts against the benefits of using the system.

In some cases, where the risks of potential harm and incorrect outputs are high, the right choice might be to not use the system at all. In other cases, you may want to put risk mitigations in place – for example, providing easy access to human agents if a chatbot malfunctions or is unable to help users.

If your project uses personal data, you must meet your legal duties under the UK General Data Protection Regulation (UK GDPR). This includes:

being responsible for how personal data is used
showing how you meet the data protection principles
completing and documenting Data Protection Impact Assessments (DPIAs)
fulfilling transparency obligations – including the publication of privacy notices or fair processing notices
enabling people to exercise their data protection rights

Your organisation’s data protection officer (DPO) or information assurance team should be able to advise on your data protection obligations.

Accountability if you’re using a third party supplier

If you buy a data-driven or AI system from a supplier, you must:

make sure the supplier understands their responsibilities
agree what risks they must manage
ask them to share information about how the system works

As a baseline, make sure that any supplier maintains the same standards as would be expected from you if the system was developed internally. This applies to data governance, performance evaluation, data minimisation and legal accountability.

You should include the following responsibilities in the contract:

product liability – suppliers may be liable if the system causes harm because it gives unsafe outcomes or fails
data liability – if the system breaks data protection or other applicable law or misuses data
algorithmic harm – if the system breaks equality law for discriminatory, biased or unfair outcomes, particularly in public services (such as social welfare or policing)
security liability – liability for damage caused by insufficient security measures or third-party manipulation
failure to meet contract terms – including penalties, clawback clauses and potential contract termination
insurance – you can ask suppliers to have insurance that covers harm caused by AI

For more guidance on procuring AI technologies, refer to the:

Ultimately, it’s important to keep in mind that the public body remains responsible for all data and algorithm-assisted decisions or outputs, and their impacts.

Recommended actions

Your team should:

nominate an SRO, typically a senior decision maker, to be accountable for the project
talk to your organisation’s DPO and ensure they have the necessary information, tools and authority to govern the project
establish mechanisms for independent reviews of your project to review and challenge decisions relating to data and AI systems, where appropriate
clarify legal responsibilities across the data and AI supply chain, ensuring that all parties involved in the data and AI project life cycle – from vendors and technical teams to system users – are acting lawfully and understand their respective legal obligations
- if you’re buying a system commercially, define detailed responsibilities and liability contractually
publish information that’s accessible to technical and non-technical stakeholders, and explains:
- how your AI system works (including its purpose, data sources and decision logic)
- how the actions and decisions you’ve taken across the entire AI project life cycle have ensured that it’s ethical, lawful and trustworthy
have mechanisms in place for users to provide feedback, such as reporting errors or contesting decisions that might be inaccurate or harmful
establish a human-in-the-loop process to oversee and validate outputs and decisions where AI is used in risky or high-impact situations – this person or team should be able to identify risks and intervene, where appropriate

identify who is responsible for the outputs and decisions made by the AI systems you use, and assume responsibility if this is not clear – this includes interactions with LLM-based tools such as ChatGPT, Gemini and Copilot

Question 4

Fairness

Accepted Answer

Fairness is a diverse concept. It concerns both the way people are treated and described, and how opportunities and resources are distributed in society. Fairness is deeply rooted in our cultural, moral and political practices, and you should regard it as a good in its own right.

Unfairness undermines social and economic opportunity and creates power imbalances. Inequality can undermine the legitimacy of government and erode trust.

Unfairness in your project may result in:

Fairness in data and AI involves a combination of social, technological and legal concerns to determine whether people and organisations are treated justly, without systemic advantage or disadvantage.

The government’s collection and use of data and implementation of AI systems have wide-reaching impacts on society. Each time a data point is collected or an algorithm deployed, it can have the potential to widen or close gaps in opportunity, representation and resources.

Equality and inclusion

Equality and inclusion are fundamental ethical principles for fairness. Equality is about:

ensuring that all people have equality of opportunity
exercising government functions in a way that’s designed to reduce inequitable outcomes stemming from socio-economic disadvantage

You must take care when ensuring equality in your project. Avoid any form of positive discrimination, as this is prohibited. Instead, take positive action such as targeted training and learning more about positive action in the workplace.

Inclusion is about ensuring that all groups can use and benefit from your project, accommodating for the needs of different physical and mental abilities and different contextual and cultural needs.

What fairness means in practice

Fairness considerations are critical across the life cycle of your project – from initial decisions about what data to collect and how, to the impacts of your project years after deployment.

To understand what fairness means in your project, first determine what aspects of fairness are most relevant to your project. For example:

if your project involves classifying events or objects – such as flagging fraudulent claims – the most relevant interpretation of fairness may be equal error rates between groups
if your system provides access to a public service or type of benefit – such as access to government grants for research – proportional access may be more important

You must understand and comply with your legal obligations and duties. Refer to the Public Sector Equality Duty (PSED) and the Equality Act 2010. The PSED is made of a general duty across Britain, with specific duties that vary between England, Wales and Scotland. There’s guidance for public sector bodies on the PSED and AI and a 10-step guide on how to deliver an Equality Impact Assessment.

Proxies

Proxies are data points that are strongly correlated to other characteristics. For example, postcodes might be strongly correlated with age and race. While use of proxies is, in many instances, lawful, it’s essential to review data sets and models for proxies that may be introducing bias.

This especially applies to bias against protected characteristics such as:

age
disability
gender reassignment
marriage and civil partnership
pregnancy and maternity
race
religion or belief
sex
sexual orientation

Some attributes can be used to accurately infer a protected characteristic. For example, someone’s age can be inferred by their date of birth. Other attributes may be used to deduce a protected characteristic with varying degrees of accuracy – for example, postcodes that indicate likely ethnicities.

Proxies are not inherently harmful. For example, an organisation might use a proxy to monitor where a policy is having positive impact, or use that data to improve outcomes for people with protected characteristics where direct data does not exist.

However, harmful outcomes can occur when a public body:

is not aware that the data it’s processing can be proxies for protected characteristics
does not consider the potential impact on people with protected characteristics
does not measure or remedy impacts, as this can lead to unlawful discrimination and reduced equality of opportunity

Understand impacted groups

You should understand which individuals or groups could potentially benefit or be treated unfairly by your project, and which are most vulnerable to harm as a result.

These may be groups that share a protected characteristic, or groups connected through different attributes such as their location, interests, the industry that they work in, and more.

Understanding and assessing the impact that a project may have on people, particularly in relation to the protected characteristics, helps public bodies demonstrate they comply with the PSED.

You must assess the equality impact of your project:

before making any decisions to initiate a project, including procuring and development
during implementation
after implementation

There’s no set way for you to record your equality considerations and your organisation may have its own Equality Impact Assessment (EIA) template for you to complete. We recommend that your EIA follows the 10-step guide published by the Equality and Human Rights Commission (EHRC). This will help you navigate what you must do to comply with the law, and what you should do to demonstrate best practice.

Representational and allocational harms

Another way to understand fairness in your project is by considering the harms that it may cause. Harms are often categorised as either representational or allocational. Both concepts existed long before AI or data-enabled services, but technology means they may manifest in different ways and may need to be addressed differently.

Representational harms concern how people are portrayed, including stereotyping, misrecognition and denigration. There are many cultural and institutional examples, such as in advertising, branding and entertainment. The impact of these harms can include reinforcing negative societal biases and social norms, psychological impacts and more.

Representational harms are linked with the third aim of the PSED – ‘to foster good relations between people who have protected characteristics and those who do not’. Avoiding these harms will help you uphold your duty.

Digital technologies, especially AI, are introducing new avenues for representational harms to occur. These harms are especially prevalent in generative AI models. The many different ways that AI may manifest in different types of content (text, video, image, audio) can make it particularly difficult to identify and mitigate.

For example, large language models (LLMs) are known to encode biases and stereotypes from the text that they’re trained on. This may result in displaying gender bias when describing certain job roles, or reproduce negative stereotypes associated with protected demographic groups. Meanwhile, AI models that generate images have been found to typically generate images of light-skinned men when prompted to produce images of CEOs and judges, perpetuating harmful biases.

Representational harms can also emerge in data sets. For instance, over-representation of minority groups in crime statistics could result in unfair and unlawful treatment.

Allocational harms are linked with the first and second aims of the PSED, and concern how resources or opportunities are distributed among society. In particular, they impact operational departments responsible for providing access to any kind of service or resource. Examples include citizen-facing services that give access to medical procedures or medicines, or structural systems like the allocation of funding to build new schools. The impact of allocational harms can include loss of opportunity, reinforcing and perpetuating inequalities, and more.

You should pay particular care to any algorithm employed where there are known historical inequalities, as the available training data is likely to encode these same biases.

Though representational and allocational harms are described as distinct areas, they often overlap in the real world. For instance, a hiring system that is biased towards hiring women into nursing roles displays both representational and allocational harms. It stereotypes nursing as a job conducted by women, and increases the likelihood that the employment opportunity is allocated to a woman.

Understand downstream impacts

The PSED and any other fairness and equality requirements that you’ve identified for your work are ongoing. It’s important for you to regularly review the impacts that your work is having on people and society.

This can help you to determine whether:

you’re delivering the intended benefits you set out to deliver through your project, and that your investment is delivering value for taxpayers’ money
your project is not leading to unintended consequences, including unlawful discrimination
the actions you’ve taken to mitigate negative impact and maximise positive impact are having the intended effect

Downstream impacts can be hard to predict. User research and journey mapping can help you understand the downstream impacts of your project. Civil society organisations may be able to provide important insights on potentially impacted groups. They may also be able to support the recruitment of user research participants in difficult-to-reach communities.

Building for fairness

If you determine that the risks and impacts identified in your EIA are acceptable, you’re likely to begin assembling a team to develop your project.

Having a diverse and multidisciplinary team can provide a strong foundation for building fairer tools and services. A diverse team is often able to spot issues early in the development process by bringing varied perspectives and lived experiences.

If it’s not possible to assemble a diverse team, you should note any potential gaps in understanding and proactively seek ways to engage with a variety of stakeholders. This might include through user testing, public engagement and, where possible, presenting ideas at internal or external community events.

You should note the potential blind spots and pay attention to these in testing and in conversation with civil society groups.

You should also apply these considerations when contracting out work to third-party organisations. For example, consider the diversity of their development teams and the methods they employ to mitigate potential gaps.

Fairness metrics are different ways to measure whether your data set is representative and free from bias, and whether AI systems treat people equitably. They can help to identify and quantify biases, and inform further development to ensure outcomes are not disproportionately favourable or harmful to certain demographic groups.

You should identify fairness metrics that are relevant to your project and address the potential harms you’ve identified. These can include:

demographic parity – where the proportion of positive or negative outcomes is the same for all groups
equalised odds – a measurement of the accuracy of results (true-positive rate and false-positive rate) across different groups

Fairness metrics are an evolving area. Each metric has relative strengths and weaknesses and research may uncover better methods. We recommend that you read current research on this topic to identify the most appropriate measurement for your project.

If you identify bias in your data or AI system, you must identify and implement mitigation techniques to reduce the risk and potential severity of harms. Apply these techniques anywhere in your project where bias occurs, including:

data collection and processing methods
bias in any systems, including AI systems
biased impacts felt downstream

Data on demographics, particularly related to protected characteristics, can support equality monitoring and meeting your obligations under the PSED. You should consider privacy here, referring to the guidance on the use of personal data to support the PSED.

Accessibility and fairness

Fairness is not just about the outputs of your data set or AI system, but also about how people interact with the project and who can access and use it.

It’s important to remember that a proportion of the population does not have access to the internet or devices such as smartphones and tablets. Some groups are more at risk of being digitally excluded, including older people and people with disabilities. Some users may prefer non-digital channels.

This means that certain, often vulnerable, groups may not have access to digital services or be represented in data collected through digital platforms. As a result, their needs may be entirely missed or underrepresented.

You should actively conduct research with groups that may be excluded to understand their needs. You should also provide non-digital alternatives where possible, such as phone support or paper forms.

All public sector websites and applications must adhere to the Web Content Accessibility Guidelines (WCAG) 2.2 requirements unless they qualify for a specific exemption. Even for exempt services, the WCAG 2.2 still provides useful guidance.

WCAG 2.2 provides information covering colour contrast, screen reader support, captions, and more. You should ensure that all content is written in plain language. Where appropriate, you should also make the content available in alternative languages and formats such as braille or British Sign Language. These and other similar changes may be considered reasonable adjustments. There are examples of reasonable adjustments applied to service delivery in the EHRC’s 10-step guide.

Data sets and labels

Fairness may be negatively impacted by bias in data sets, or bias emerging from the way a data set is transformed and used. Here are some of the ways in which your data sets may be biased.

Representation bias

Data may be biased if it’s not representative – for example, not including enough examples from a particular group, or not including the group at all. This could cause higher levels of inaccuracy for impacted groups, leading to inaccurate decision making. It could also cause lower levels of performance, which may result in frustrating user experiences or barriers to accessing a service.

Minority groups are inherently more likely to be underrepresented, which may lead to compounding negative effects. In these instances, oversampling (capturing more information about the concerned group) may be beneficial.

Certain locations have historically been better researched than others. On a global scale, the Global North (North America and Europe) tends to be more represented in data sets when compared to the Global South (Africa, South Asia and Latin America). Within countries, major cities tend to be more highly represented when compared to rural areas and informal settlements.

Societal bias

Bias may still be present in a data set even if it’s fully representative of the population, due to biases present in our current and past society. Societal bias occurs when the data we collect, generate and use reflects discriminatory patterns present in society.

Data aggregation bias

This occurs when data is grouped or aggregated in a way that distorts the results of the analysis. For example, over-aggregating ethnic groups in demographic data – such as by only using ‘White’, ‘Black’, ‘Asian’ or ‘Other’ – can mask differences between groups, such as ‘Gypsy, Roma and Traveller (GRT) communities’.

The act of grouping data can obscure important variations and nuances present within the subgroups or individual data points, leading to a misrepresentation of the underlying relationships. Note that aggregation bias in AI has a different meaning, referring to instances where trained models are applied in a one-size-fits-all manner.

Measurement bias

This arises from discrepancies between gathered data and real-world experiences or events. This can happen due to flawed measurement processes and inconsistent or inaccurate data labelling. Measurement bias may also be present in AI systems.

Data labelling practices (the process of assigning a tag, name or description to data to provide additional context) can also be a source of bias. Labels may be assigned inaccurately, and improperly designed labelling schemes may make it impossible to be accurate in all cases.

Ongoing fairness commitments

Your fairness commitments do not end once the project has gone live. Unforeseen problems may emerge in the real world. Shifts in the societal, cultural, economic and technological landscapes may mean that your project does not remain fair throughout its life cycle.

It’s vital that you continue to monitor the performance of your project post-implementation, including fairness and inclusion. You should include fairness and equality clauses in contractual arrangements with any third-party suppliers. Your monitoring should include information from suppliers if aspects of your project are procured or contracted out to third-party organisations. The Local Government Association has guidance on responsibly buying AI.

You should maintain user feedback and issue reporting systems throughout the system’s life, and make these visible to users. Consider what resources you should make available throughout the life cycle of the system, to fix newly identified fairness problems and to remediate harms. You should disaggregate your investigations into this feedback to understand the impact on different groups.

Disaggregated feedback

Aggregated feedback – the combined feedback from all the groups on your project – may help explain technical aspects of the project’s performance. However, it’s unlikely to tell the full story.

For instance, your project may involve a chatbot which receives 95% positive feedback from users. While this statistic may seem positive on the surface, it’s critical to understand which groups of people are satisfied, and which fall into the 5% that are dissatisfied. This may reveal that the dissatisfied group aligns with a particular protected characteristic. Identifying this provides an opportunity to mitigate impacts and build a fairer and more inclusive service.

The Algorithmic Transparency Recording Standard (ATRS) provides a way to publish information about:

fairness metrics – for example, those included in your EIA
activities such as testing and bias mitigations

If your project involves an algorithmic tool that’s in scope of the standard, your transparency record could provide a strong platform to communicate the fairness attributes of your project.

Recommended actions

Your team should:

define fairness – understanding what fairness and associated risks and opportunities mean in your project, and how you might measure that across its life cycle
understand your users – identify the groups most likely to be positively or negatively impacted by your project, and how this relates to the protected characteristics described in the Equality Act 2010
consider which proxies might relate to the protected characteristics in your data set and be mindful and transparent about their use and purpose
complete an EIA and consider publishing it alongside your DPIA and ATRS record (if relevant) for transparency and accountability
include diverse views – consider the diversity of your project team and any third parties you’re contracting
collect feedback and data throughout your project to ensure that it continues to be fair and delivers its intended benefits, as society and user groups change
regularly update your EIA

Question 5

Privacy

Accepted Answer

Privacy is a complex ethical, legal, technological and social principle. It protects individual autonomy, meaning people should have the freedom to live, think, choose, socialise and more without undue observation or interference.

In a digital and data-driven society, personal data is routinely collected, processed and used by public sector bodies to generate insights, develop AI tools and make decisions. This impacts peoples’ privacy and needs to be protected in line with ethical and legal obligations.

A failure to respect individuals’ privacy can lead to:

loss of public trust
harm to people, particularly vulnerable individuals
regulatory risk

Privacy is not just a legal compliance exercise, but a fundamental human right that fosters trust and empowers individuals in the digital age. This guidance explores ethical responsibilities that extend beyond legal compliance, but you must still comply with the law at all times by following:

What privacy means in practice

You must design and build privacy into your project from the start. This includes:

the very first decisions you make about what information you intend to record
every way that the data, or AI system built using this data, is used throughout your project
the safe decommissioning of the system and data set

This concept is referred to in the UK GDPR as privacy by design and default.

Privacy is about:

autonomy – citizens’ capacity to live their own lives and make decisions based on their own motives and reasons, free from external manipulation (provided actions are within the law)
confidentiality – data can be both sensitive and private, and confidentiality is the duty to ensure it’s only shared in appropriate and authorised ways

dignity – this is about the intrinsic value of all people, not as data points, but as people; it means ensuring that any methods of data collection, processing and use of AI do not exploit, humiliate or dehumanise people

Purpose and benefit in the context of data protection

Before you start collecting, sharing or using data, you should clearly understand:

why you’re processing the data (the purpose)
what public benefit this will bring

The scope of your project should be strictly limited to your identified purpose throughout its life cycle. You should:

engage with your organisation’s data protection officer (DPO)
consider whether your purpose could be easily understood by someone with no technical background in the domain – people must be able to understand your project so they can provide clear, informed and unambiguous consent
clearly record the purpose and benefit of your project – for example, who benefits, and how you’ll know if you achieve your aims
- examples of measures could include cost savings, reduced waiting times or the number of people who receive support
explain why AI is the right tool for this purpose, if your project uses AI

Articulating and recording your purpose and public benefit at this stage can help to avoid scope creep and may help you comply with regulation such as the UK GDPR.

Scoping and data acquisition

If your project involves the collection or use of any personal data, you should start by:

establishing precisely what information you need to meet your purpose
creating a high-level map of your data flow before this begins – for example, a document that lists each data element as a line item, and records why it’s necessary, proportionate and what the effect would be if it was not used

The data you request and acquire must be adequate, relevant and limited to what is necessary in relation to the purposes for which it is processed. This is known as data minimisation. The Information Commissioner’s Office has guidance on data minimisation, including how to determine if the use of data is adequate, relevant and limited.

Consider whether you could use anonymised or synthetic data to replace some or all of the personal data your project requires. If personally identifiable information is essential for your project, ensure that the risks are proportional to the benefits.

Your project might not itself gather or process personal data. For example, if you procure an AI model from a third party or use a pre-trained model. Here you must still ensure that any personal data used to train these models meets your privacy standards and complies with relevant legislation. The UK Statistics Authority (UKSA) provides an Ethics Checklist relating to the creation and use of synthetic data which may support these decisions.

The Local Government Association has guidance on responsibly buying AI which discusses data protection.

Privacy by design and default

There are 2 main principles to guide the development of your system or service:

privacy by design – making design and development decisions from the very start that enhance privacy objectives
privacy by default - ensuring that default settings for the product or service are the most privacy-preserving options

Privacy enhancing technologies (PETs) can help to achieve a privacy-by-design approach.

Transparency can also support privacy. Design methods and notices to explain:

what data you collect
why you need the data
how the data is used
how long the data will be stored for

A good principle to uphold is that nobody should be surprised about how their data is being used.

AI systems require greater attention and introduce new risks. For example, AI systems may re-identify individuals that were believed to be anonymised by linking information with other data sets, or information made available when it was trained. In addition, large language model (LLM) tools like ChatGPT, Claude, Gemini or Copilot may ‘memorise’ sensitive or personal information contained within their training data sets and leak this private information during their operation.

To mitigate this risk we recommend that you:

Test and validate

Before going live with a new service, you must thoroughly test to ensure that it works as intended and does not introduce new privacy risks.

You should conduct testing throughout the build phases, and repeat it after your service goes live. This is particularly important if any changes are made to the system, data source(s), or when there are significant societal changes that may impact the service.

Where possible, you should conduct tests with anonymised or synthetic data. If it’s necessary to test with personal data, make sure you do this in a secure environment with strict limitations on access.

Red teaming exercises may help to probe the service or AI system. Pay particular attention to any:

data that may be exposed which links the protected characteristics as defined in the Equality Act 2010
health and location data
data that may relate to vulnerabilities

The National Cyber Security Centre has guidance on protecting personal data, including the Cyber Assessment Framework.

Be transparent and give users control

Users who are impacted by your service should be able to understand what information you collect, including:

what it will be used for
how long you’ll retain it
how they can manage their privacy settings

This information should be clearly available at the point of use. Use clear and accessible language in privacy documentation, avoiding jargon and legalistic terms where possible. When interacting with the public, we recommend offering configurable privacy options. A good rule of thumb is to make it as easy to opt-in as it is to opt-out, give people free choices and respect users’ autonomy. You can also review previous government research into how different ways of collecting consent can impact users’ behaviour and enable ‘active’ online choices.

There are basic requirements for consent to be legally valid. It must be properly informed, freely given, specific and unambiguous. If you rely on the consent of data subjects, you also need to:

provide an effective and straightforward way for them to withdraw their consent
tell users what withdrawal means for the service they receive

The concept of ‘freely given consent’ implies providing a real choice for the data subject. Any element of inappropriate pressure or influence which could affect the outcome of that choice renders the consent invalid. The law takes a certain imbalance between the controller and the data subject into consideration.

For example, in an employer-employee relationship the employee may worry that refusal to consent may have negative consequences for their employment relationship. Similar issues of imbalance arise between people and the state, where there is little or no choice for people when it comes to many public services. For example, there’s no alternative when it comes to seeking welfare benefits, or applying for a passport. Here, consent can only be a valid lawful basis for processing in a few exceptional circumstances.

We also encourage a privacy-by-default approach, where the most privacy-preserving settings are enabled by default unless the user decides to choose otherwise.

Transparency extends beyond privacy notices. You should design processes to ensure users understand how their data is being used at every point. This builds trust and supports informed consent.

You should review your service at important points, including milestones and significant updates. This will ensure that you’re still achieving your intended goals and aligning users with your use of data. Even if your service does not change, societal expectations may.

Plan for and respond to data breaches

Even with strong safeguards, it’s impossible to guarantee that breaches will never happen. Prepare in advance to ensure you have answers to the following questions:

who investigates suspected breaches and how?
how can you contain potential breaches and secure data?
how will you inform impacted users of a breach?
what support will you offer to anyone harmed by a breach?

A well-handled breach can reduce or eliminate harm and preserve public trust, while a poorly handled one may cause lasting damage.

Ensure that your safeguards and processes evolve and adapt to a changing threat landscape.

Decommissioning

You must retire all platforms at an appropriate time.

This may be when:

the service is no longer needed
your purpose is better met, or greater public benefit is achieved, with a new service
the public benefit of the current system is no longer meeting social expectations

When decommissioning a service, make sure you honour data retention limits, including any data which has been stored or processed by third-party organisations.

Where there’s a legal duty or other obligation to retain certain data, ensure this is done securely and with strict access controls, and that confidentiality is protected.

We encourage teams to record, and publish in their internal knowledge bases, any lessons learned about:

how privacy risks were managed
public attitudes towards data sharing and AI – ensuring that these findings are aligned with any agreements that were in place with research participants

Recommended actions

Your team should:

involve information governance teams early in the project design process – this will help you to comply with privacy and data protection legislation, and plan resources effectively so that support is available for project teams when needed
record the purpose and benefit to understand why any data collection is necessary and whether it’s balanced against privacy risks, and use this record to ensure your project does not grow beyond its defined scope
embed privacy by design and default from the outset – undertake a Data Protection Impact Assessment (DPIA) to access and mitigate against data privacy risks
adopt data minimisation techniques – ensure that the data you collect and process is the lowest amount required to achieve your purpose
facilitate data subjects’ rights – build in mechanisms for data access and correction, and ensure consent is informed and meaningful
use PETs where possible, applying techniques such as anonymisation or the use of synthetic data
share data responsibly by having a robust, privacy-preserving governance regime that safeguards:
- security
- lawful basis
- explicit retention or deletion protocols,
- provisions that secure individuals’ privacy and data protection rights as part of sharing agreements and records
consider the impact of AI – explore additional privacy risks that AI systems may introduce, such as re-identification of individuals in data sets
- using public AI tools, such as ChatGPT, can increase the risk of data leakage
be transparent by providing clear privacy notices that tell users exactly how their data is being used and their options, and give users a free choice to configure settings
prepare for breaches – develop clear plans and assign responsibilities to investigate and tackle any suspected data breaches
plan for decommissioning – create plans for data archival or deletion at an appropriate time, considering how long the data will be needed for the service you’re providing and the life cycle of your project

Question 6

Environmental sustainability

Accepted Answer

Environmental sustainability means thinking about the wider impact of your project on people and the planet, and being mindful about the resources you use every day. It’s important to consider this from the start, as it can influence your choices about tools, suppliers and infrastructure.

For data and AI projects, acting sustainably means:

using only the necessary resources
protecting natural resources and using them responsibly
designing systems that last and are easy to maintain

Sustainability also includes social responsibility. This means supporting inclusion, fairness and human rights in how technologies are designed, manufactured, used and disposed of.

What environmental sustainability means in practice

Data and AI can support environmental goals, such as predicting weather patterns or monitoring biodiversity levels. However, developing and training AI models, especially large language models (LLMs), requires the use of critical minerals, energy, water and computing resources.

This means that you should consider environmental impact:

at every level in your organisation – from analysts to senior decision-makers
across the entire life cycle of your tool or project – including when it is decommissioned or handed over to another team

The complex supply chains of data-driven technologies, and the natural resources required to build them, mean they can have a long-lasting impact on people and the planet. For example, the mining of raw materials to build computing chips and other AI infrastructure. This has a significant environmental cost and impact on the health and wellbeing of workers and affected communities.

Setting clear sustainability objectives and using greener software practices will help make sustainability a core part of your project design.

You should also consider whether there’s a reasonable trade-off between the benefits of your tool or system and its environmental impact. There may be trade-offs between different ethical principles. For example, the use of a more energy-intensive algorithm that’s also more explainable, or that brings wider benefits and value to the public. Make sure you clearly document when you’ve made decisions about trade-offs, and the governance processes you used.

Measuring, monitoring and evaluating environmental impact

Sustainability reporting for cloud and wider digital services is an evolving area where research and development is ongoing. This means it may be challenging to identify appropriate metrics, and the reliability of reporting may differ across suppliers. The following tools and methods will provide you with a starting point.

Before you begin building a tool or system, think about:

whether AI is the best technology or tool for the task – there may be alternatives that provide as good, or better, results and require less resource-intensive training and development
project size and complexity – how much data will be processed, the number of end users, bandwidth and latency requirements, prompt efficiency
software requirements – including model type
hardware requirements – such as data centre usage or Internet of Things (IoT) devices

There are various environmental factors that you should monitor and manage across the life cycle of your project.

Electricity consumption
Data centres use large amounts of electricity for running servers and data processing. Some even use fossil fuel-based generators due to a lack of capacity in local grids. Using electricity results in the release of greenhouse gases such as carbon dioxide (CO2), methane and nitrous oxide, which contribute to climate change.

Water consumption
For example, data centres require significant amounts of fresh water for cooling systems and to avoid hardware failure.

Consumption of raw materials
The mining of raw materials, for example to build computing chips and other AI infrastructure, has a significant environmental cost and impact on the health and wellbeing of workers and affected communities.

E-waste
Electronic waste (e-waste) from discarded electrical devices contains rare earth materials such as neodymium; if not disposed of properly, these materials can contaminate soil and water, and release noxious gases into the atmosphere, posing extremely harmful risks to biodiversity, human health and the environment.

In your project, consider how the following may impact resource consumption:

data set size – large data sets require more energy to store and process, especially when used to train AI systems, and you should also consider the impact of the resolution or bit rate of the data sets
data type – generating or handling high-volume text (for example, training LLMs), video or image data is typically more energy-intensive than other data types because it requires more intensive computing, memory and bandwidth workloads
data velocity – high-velocity data (such as real-time streaming) increases compute load
architecture – using technologies like virtualisation can reduce energy consumption, hardware needs and associated resource demands by allowing data to be accessed and queried without the need for transfer or duplication; adopting modular design can help promote reuse and reduce future resource consumption
storage – data storage has an ongoing carbon footprint, primarily in data centres, due to the continuous generation of heat and use of water and electricity to run them; establishing and running data centres also involves biodiversity loss from land clearing, droughts caused by fresh water extraction, e-waste and pollutants
model type – multi-purpose, generative architectures typically use more resources than task-specific systems; you should always explore different options, including not using AI at all, before choosing a technical approach
model size – LLMs and other generative AI systems generally require more energy and emit more carbon compared to running a single prediction on a trained model
location – try to run algorithms in locations with higher uses of renewable energy to reduce your carbon footprint
- you can use electricity maps to understand the electricity usage and emissions of different regions
- you should adhere to any relevant data residency rules if selecting a server location based on energy mix
time of day – the source of energy on the grid changes during the day; where it’s viable, you can make carrying out computationally complex processing tasks more environmentally friendly by shifting the time they take place, even by a few hours, to when energy generation is cleaner

Selecting metrics

You should select appropriate metrics and key performance indicators (KPIs) that can be tracked over time. There are several ways of quantifying environmental impact across the development and deployment phases of a project.

Common metrics include:

carbon footprint (CO2-equivalent emissions)
energy consumption (measured kilowatt-hour or kWh)
compute efficiency (for example, FLOPs per watt – how many calculations a system can perform for every watt of power it uses)
water consumption (measured in litres)

There are some open-source tools that can help to analyse and measure the environmental impact of a data or AI project:

It’s important to note that there are often trade-offs when selecting particular metrics over others, and some impacts will be harder to quantify. For example, carbon emissions may be more easily tracked but neglect a tool’s broader impact, like water usage or impact on local communities.

Minimising environmental impact

There’s no one-size-fits-all approach to minimising your environmental impact, but even small actions can make a big difference. Depending on your project, there are many ways to make a positive change.

If you’re handling or analysing data, you should:

minimise the data you’re collecting and using – collect only the data necessary to meet the research or project objectives
use data compression and aggregation to reduce storage and bandwidth requirements – for example, compressing data before transmission (using formats like GZIP or Parquet) can help to save bandwidth
regularly review whether the data you hold to help reduce its volume – consider whether there’s any redundant, obsolete or trivial (ROT) data that can be safely deleted
avoid re-running entire pipelines – use incremental workflows which only re-process new or changed data
use cloud storage tiers, scaling and right-sizing instances – this can help to optimise data storage allocation by, for example, automatically moving infrequently accessed (‘cold’) data to lower-energy storage
use narrow prompts for generative AI tools – shorter or more focused prompts typically result in less processing time and improved accuracy of responses
use techniques such as virtualisation to reduce duplication
design modular components for potential reuse in future projects

If you’re building AI or algorithmic tools (including using pre-trained models), you should:
- consider reuse – where you can meet the requirements of your project lawfully and safely, consider reusing existing models or services (for example, open source, or models trained for previous projects) over building and retraining from scratch

If you’re using a third party supplier, you should:

include sustainability as a criteria during the supplier selection process – ask about the energy consumption and carbon footprint of the AI models and infrastructure before engaging a service
find information about the broader ethical practices of the organisation, such as labour standards and sustainability commitments – be aware of potential greenwashing terms and practices
ask suppliers what happens to hardware at the end of life – for example, if hardware is refurbished, donated or responsibly recycled

The Service Manual also has guidance on how to build more environmentally sustainable digital services in government.

Recommended actions

Your team should:

explore and evaluate alternative solutions – the latest technology or innovation isn’t always best for the job, so explore different options, including simple and low-energy options, before selecting an approach
consider reuse of existing models – advocate for the use of open-source models where source code, data and model weights are available for community reuse, rather than using energy to retrain or build similar models
select suitable metrics for measuring and assessing environmental impact – this can be a complex task but is vital for ensuring proper stewardship of data and AI projects
evaluate the sustainability credentials and practices of third-party providers, including their use of renewable energy
raise awareness about the environmental and social impact of digital technologies – share knowledge with colleagues and look for opportunities to bring sustainability into training and discussions

Question 7

Societal impact

Accepted Answer

The use of data and AI is already changing how people live and work. For example, the accessibility of tools such as ChatGPT has started to reshape how we undertake daily tasks. These rapid changes raise important questions about how digital technologies are used, and how they affect society.

Societal impact refers to the effects that data and AI can have on individuals, communities, social structures and economies. This includes both positive and negative impacts.

For example, adopting chatbots in public sector call centres might reduce costs, boost productivity and allow for faster handling of cases. However, it can also lead to job losses, or create a less personal relationship between government and the public that leaves citizens feeling disempowered.

Taking care to respect other principles such as privacy and fairness can have wider societal impacts, building trust and improving education and health outcomes.

Even relatively simple technical solutions can have significant societal impact. For example, an automated form that checks eligibility for a service might unintentionally exclude people if it does not account for different living situations or types of identification. This could stop people from getting the support they need and reduce trust in public services.

What societal impact means in practice

To understand how data and AI can impact society, you need to understand the broader social and technical context in which they are used and built.

For example, the context and mechanisms of data collection can have a significant impact on who is, and is not, represented in a data set. As a result, an entire group of people might be excluded or underrepresented. In the context of government, this could lead to poor or even discriminatory policy decisions.

Both complex supply chains and resource usage of data and AI have a significant societal impact. This includes the displacement of local communities, and food and water scarcity in already vulnerable parts of the world. You must consider what knock-on effects your project might have on communities.

Proactive public and stakeholder engagement is critical to understand the potential societal impact of a tool or system. Engaging with stakeholders can help to build trust and improve adoption of AI tools and systems.

You should consider:

how data was collected and processed
who technology affects and how
how and where data or a particular technology is used
how a data-driven system is designed and built

Data and AI tools can help improve public services and benefit society. However, they also raise societal risks that must be managed, such as:

unfair treatment – the use of data or AI systems can lead to unequal outcomes, such as people being treated differently based on race, gender or other protected characteristics
misinformation and disinformation – AI tools can be used to create misleading or deliberately false content, including text and images, which is then circulated online at scale; this can contribute to the spread of hate speech, in some cases causing widespread distrust and political violence
changes to jobs – automation may reduce demand for some roles and increase the need for new skills; the increased adoption of AI tools has the potential to reshape the world of work, including work becoming less fulfilling
changes to relationships – using technology can change the way people relate to each other, removing opportunities for care, connection and community that can combat loneliness and build trust and community cohesion
ownership of creative work – the use of generative AI, typically trained on web-scraped data, such as original images and academic articles, raises legal questions about intellectual property and the ownership of content
power imbalances – the information about many AI tools is not transparent or accessible; this can give more control to those who build or use them, and less to those affected, leading to distrust, disaffection and disempowerment
adverse impacts on human rights and fundamental freedoms – irresponsibly designed or misused AI technologies can, among other things:
- harm human dignity
- damage physical, psychological and moral integrity
- undermine freedoms of expression and assembly
- infringe on the right to private and family life

Understanding stakeholders

To understand your project’s societal impact, identify who could be affected and how. The act of public engagement can help to build trust and improve adoption of new tools and services.

At the start of your project, you should make sure that:

your project has a clear purpose
there’s a clear public benefit
you’re designing with humans in mind and in a human-centred way

Ethical use of data and AI means listening to different perspectives and checking that your tools work as intended. Otherwise, you could unintentionally exclude people or cause harm.

You should ask:

who could be negatively or positively affected?
do the risks outweigh the expected benefits?
how are the risks and benefits distributed across different groups?
what might be the long-term consequences?

To better understand the lived experiences of users, it can be helpful to use qualitative and ethnographic methods such as observations, informal conversations or interviews. Think about how different people experience services, including colleagues who will be using the tools.

Consider:

age
disability
gender reassignment
marriage and civil partnership
pregnancy and maternity
race
religion or belief
sex and sexual orientation
income
access to technology
other attributes relevant to your project

To better understand the views of, or potential impacts on, wider stakeholders and the public, consider proactive stakeholder or public engagement. For a helpful overview, refer to the Alan Turing Institute’s stakeholder engagement overview or resources from Connected by Data.

You should factor in ways to empower the public and workers who may be using data and AI tools. To adopt AI safely and effectively, teams should have:

suitable training and resources
a working environment that considers and prioritises broader ‘human’ factors – including cultural shifts, organisational change and skill development

To learn more, refer to the guidance on a human-centred approach to scaling AI tools.

You should continually revisit user needs throughout your project. The Service Manual explains more about user needs.

Identifying harms and benefits

A practical way to identify the harms and benefits of a data or AI project is to bring teams and affected people and communities together to discuss:

the intended and unintended consequences of the system
what could go wrong
what can be done to mitigate risks

Not all harms and potential impacts are obvious. You should factor in time during the project planning stage to run workshops, following practices such as The Model for Responsible Innovation or Consequence Scanning. The Model for Responsible Innovation is a practical tool created by the Department for Science, Innovation and Technology (DSIT) to help teams across the public sector to innovate responsibly with data and AI. You can use the model to run red teaming workshops with teams developing data-driven technology.

Horizon or risk scanning workshops can help to create an open dialogue, so team members, colleagues and external stakeholders can raise concerns and ideas. As part of this scanning, include structured reflection on the team’s perspectival limitations and stakeholder diversity to mitigate blind spots. Often, project teams do not sufficiently reflect the social and demographic characteristics of affected groups. This can limit inclusion of impacted viewpoints and reduce the ability to identify, understand and weigh risks, benefits and trade‑offs.

Positionality‑aware analysis, coupled with meaningful engagement, can surface concerns and opportunities more effectively and equitably. Adopting this reflective and inclusive approach can help to:

identify risks before they happen
make more informed decisions about design and use
align projects more closely with organisational and societal values
build systems that are safer, more trustworthy and more contextually responsive to user needs and interests

Continuous monitoring and evaluation

Usually, the full effect of a project or tool is not fully known during design and development. This means teams may miss risks or unintended consequences until the system is used in the real world.

You should continuously monitor for potentially harmful societal impacts, and actively address them. This includes:

providing ways for users and other stakeholders to give feedback – such as adding a feedback form to a service or webpage, or running follow-up user research sessions
making adjustments based on user engagement and testing – such as adapting your design or approach when you identify risks
being transparent about decisions – keep a clear record of what was changed, why, and who was involved; refer to the Transparency section for more on this
monitoring the system post-deployment for any unintended impacts of the system, such as factors relating to workforce engagement or public trust
- evaluate outcomes based on demographic segments, for example by checking if the tool is affecting groups in unequal ways
- think about how the introduction of a data-driven tool or approach could inadvertently embed a particular worldview
reviewing and evaluating on a continuous basis, and when real-world conditions change – pay attention to the dynamic and changing character of the technology (for example, AI models are regularly updated) and to the shifting conditions of the real-world environments in which systems will be embedded
considering independent evaluation – commissioning external evaluation, such as audits, can provide impartial perspectives that can help with building trust

For more guidance, refer to:

Recommended actions

Your team should:

write down project objectives in non-technical terms, so that you understand the objectives before jumping to a technical solution
- for example, ‘the project seeks to [enhance public safety, reduce waiting times, improve customer experience]’
involve diverse stakeholders in the design and testing process – undertake thorough user research and public engagement, ensuring that you include the views of those likely to be most affected by the project
- this might include running co-design workshops or focus groups
complete an Equality Impact Assessment (EIA) – this can help to ensure that the project and its outcomes are fair and do not disadvantage protected groups
use feedback mechanisms, such as surveys and advisory boards, to enable stakeholders to continuously shape project outcomes – this can help to reduce any negative impacts across the project life cycle
make sure you have meaningful human control at the right stages – having human oversight of AI systems can help to maintain accountability and correct any erroneous or potentially harmful impacts
continuously evaluate and improve the project to ensure it’s meeting user needs and its intended use, by asking:
- at the beginning of the project, are we doing the right thing?
- during the project, have we designed it well and how can we improve?
- after the project, is it still doing the thing we designed it for, and if it’s no longer serving a purpose, is there a process or decommissioning the system with minimal impact on end users?

Question 8

Safety

Accepted Answer

In the context of data and data-driven technologies, safety refers to themes such as accuracy, security, reliability and robustness. It also covers topics such as the safety of researchers and research participants, and safe personal data collection and management.

In the context of AI, safety means ensuring that a system performs as intended across its life cycle, even in unexpected circumstances. This is particularly important in high-risk areas such as healthcare, policing and justice.

The scale and criticality of government services means they have the potential to cause widespread harm if they’re not designed with safety in mind.

If data-driven systems are not built safely, they can:

perform poorly and produce incorrect results
cause harm to individuals and society
reduce trust in public services and organisations

Important concepts for data and AI safety are:

accuracy – making sure that data is accurate and high quality, and that AI systems perform accurately
security – protecting data and AI systems against unauthorised access, tampering and leakage
reliability – ensuring that data pipelines and AI systems consistently deliver acceptable and expected results in all expected scenarios across the duration of use and deployment
robustness – designing data infrastructure and AI systems to withstand unexpected inputs, adversarial attacks and failures across the duration of its deployment
protection from harm – data use and AI systems must not cause harm to individuals, society or the planet
controllability – maintaining human oversight over data and AI systems, including the ability to intervene and align outcomes with societal values

Find out more about AI safety principles in the Alan Turing Institute’s guidance on AI Safety in Practice and Understanding artificial intelligence ethics and safety.

What safety means in practice

Safety considerations can emerge across a project life cycle – from initial research and data collection, to the downstream impacts of a system.

In the context of data collection, sharing and use, safety means:

ensuring data is only used for lawful purposes
handling data in a way that protects individuals’ privacy and minimises harm
ensuring data is of high quality
using secure systems for storing and processing data
applying appropriate anonymisation techniques

In the context of data-driven tools and AI systems, safety means:

designing data-driven systems that work as expected and do not cause harm
testing systems to make sure they perform accurately and reliably under unexpected conditions
keeping systems secure and protecting them from misuse

Start with safe research

Research is vital to design a project responsibly and robustly, and make sure that systems and products meet user needs. Research can help you understand fairness issues and build trust.

Research and data collection are often essential for improving government services and shaping new projects. But both quantitative and qualitative research can carry risks for participants.

You must consider the safety of both researchers and participants before starting a project. You should have safeguards in place and maintain them for as long as needed.

Qualitative research can carry risks if not planned carefully. This type of research involves collecting non-numerical data, such as:

interviews
focus groups
direct observations

The safety of research participants in qualitative research is critical. Participants may be at risk of harm or distress, especially if the research involves sensitive topics. There may also be risks to their physical safety – for example, if taking part causes tension within their community or social group.

Reduce risks for participants

You must take steps to protect research participants. This applies to your team and to any third-party researchers you work with.

You should:

make sure that participants understand the research and any risks involved
only include participants who give clear, informed consent without pressure or coercion
allow participants to withdraw at any time (with exceptions for situations like medical research where participant information is anonymised and aggregated and cannot be removed from the research data set)
anonymise all data used in reports or analysis

Anonymising qualitative data can be difficult. Even if names, places or dates are removed, the detail in participants’ stories may still make them identifiable. You should take extra steps to protect their identity. The Service Manual has guidance on managing user research data and participant privacy.

Reduce risks for researchers

Some research projects involve sensitive topics, such as child protection or adult safeguarding. These can be distressing for both researchers and participants. To reduce risk, you should:

highlight any sensitive topics in workshop or interview plans
set up a ‘safety check-in’ protocol to support staff after sessions, particularly for one-to-one or lone working
use existing ethical guidelines and processes, such as the Service Manual’s guidance on researching emotionally sensitive subjects

In rare cases, researchers may be targeted by participants or others. If this happens, or if the risk is known in advance, put safeguards in place to protect staff.

The Government Analysis Function provides a list of resources related to conducting research and analysis in government.

Safe data collection and management

There are several safety risks related to the collection and management of data, including:

poor data quality – including incomplete data, duplicated data, and insufficient or poor-quality metadata
data leakage – the unauthorised exposure of personal or sensitive information
data drift – the changes in the statistical properties of real-world data distributions over time, which can affect the performance of machine learning models trained on historical data
data poisoning – a malicious act where a hacker compromises data sources during collection and pre-processing by introducing corrupted data

Data security principles and processes are required the moment you begin collecting data. Limiting the amount and sensitivity of data that you collect reduces the potential for harm if data is exposed. Principles like data minimisation, as covered in the Privacy section, are relevant to safety and security.

Loss or theft of personal data can have wide-reaching legal and financial consequences. It can also cause significant psychological harm to people affected. Working directly or indirectly with data in the public sector comes with a duty to protect the personal data you hold about individuals. Refer to the government’s personal data security principles when processing and sharing personal data.

If you’re working with any third-party suppliers to store, collect or process data, make sure they’re vetted and that they comply with the security requirements of your project. There’s guidance on security measures for AI systems in the AI Playbook for the UK Government.

Ensure data is fit for purpose

All civil servants have a responsibility to ensure that the data they manage and use is of high quality and fit for purpose. Data quality should be checked and validated across the data life cycle. This includes ensuring data is fit for purpose before, during and after data analysis or model development and deployment.

Refer to the Government Data Quality Framework for guidance on understanding, documenting and improving data quality, and for information about real public sector case studies. The framework sets out the main dimensions of data quality, including:

completeness
uniqueness
consistency
timeliness
validity
accuracy

Maintaining data quality is a continuous process. You should maintain clear documentation of any known limitations, biases and quality concerns. If using data for analysis and model development, you should conduct ongoing validation of outputs, especially where models are reused or retrained.

Tools such as data sheets or data set cards can be helpful in documenting information about data quality. The Transparency section has more information on this.

Data and model poisoning

Data poisoning, otherwise known as AI or model poisoning, is a type of cyberattack that involves hackers modifying or introducing new data points into a training data set.

Data poisoning attacks can be direct or indirect. Direct attacks involve deliberately manipulating a machine learning model to behave in a particular way. Indirect attacks aim to degrade the performance of models more generally. This could include inserting irrelevant or incorrect data into a training set, reducing model accuracy.

It can be difficult to determine if and how data poisoning has taken place, particularly when using an AI model from a third-party source. For example, poisoned models may function as expected until triggered by a specific prompt or scenario.

Data poisoning can occur in different ways across the data and AI life cycle, such as:

collecting and using data from uncontrollable or unreliable sources, like social media or Internet of Things (IoT) devices
weak access controls
malicious actors modifying training data labels to reflect wrong information

A data poisoning example

A company builds a voice assistant that helps users control smart devices. To improve the system, the company invites users to help by transcribing voice recordings.

A competitor finds out about this and deliberately mislabels thousands of recordings. For example, they label recordings of users saying “turn the lights off” as “turn the lights on”.

Over time, the system learns the wrong behaviour. After an update, the AI assistant starts turning lights on when users ask for them to be turned off, and vice versa. This leads to a poor user experience and damages the company’s reputation.

The impact of data poisoning

The impacts of an AI model trained or fine-tuned with poisoned data are wide-ranging, including:

security threats to the organisation deploying the model
biased or harmful outputs being sent to users
poor system performance and accuracy, such as misclassification

Reducing the risk of data poisoning should include technical safeguards and clear policies for data management.

Example steps include:

Building safe AI systems

AI systems can behave in unexpected ways, especially if they’re designed poorly or deployed without adequate testing. Many AI systems work in complex and unpredictable environments. This makes it harder to build systems that are safe and reliable.

Elements to consider in the context of AI safety include:

safety of data – ensuring that data used for AI model training and AI system operation is accurate, high quality and legally compliant
- this should include testing data for anomalies, identifying and addressing harmful biases, and documenting information about data sets
safety of the AI model – ensuring that the model is technically robust and reliable, and performs accurately even in unexpected conditions
- consider testing and measuring model performance on validation data sets against metrics such as accuracy and precision, and fairness metrics
safety of the AI system – ensuring that the AI system functions safely and as intended within its wider context, including with users and other systems
- continually monitor for unusual behaviour post-deployment, including any potentially harmful outputs

You should prioritise the technical objectives of accuracy, reliability, security and robustness from the outset. Building an AI system that meets these objectives requires testing, validation and re-assessment throughout its entire life cycle. It also requires maintaining human oversight and control.

You should run rigorous testing to identify potential failure modes for AI systems. This may include using semi-automated techniques, such as using adversarial models to develop prompts, or red teaming exercises, to record how models behave under unexpected scenarios.

If it’s not possible to make the system sufficiently safe for the intended use, even with available mitigations, because of the potential risks or failure modes, you should not use the system to address the problem.

If using off-the-shelf generative AI tools, you will likely need to put additional guardrails in place, such as pre- and post-prompt filters. While no generative AI system can be guaranteed to be entirely safe, these measures, combined with ongoing testing and monitoring, can help reduce risks to an acceptable level.

Consider adopting the following practices to make AI systems safer and more secure:

be transparent – you should document information about models, data sets and system prompts, and make sure operators understand how the system works in different situations
design for resilience – use backup components for critical functions so that the system can continue operating safely even if one element fails
apply least privilege principles – distribute responsibility and prevent any single person from having too much control or access
build in risk controls – build in ways for the system to slow down or stop if it starts to behave in unexpected or unusual ways
establish multiple system defences – apply multiple safety measures rather than relying on just one, including a human-in-the-loop to review outputs
continually evaluate security factors – you should continually reassess factors that might impact security, including training data sets, model complexity and supply chains

Find out more about AI safety and security in the Code of Practice for the Cyber Security of AI and the National Cyber Security Centre’s (NCSC) guidelines for secure AI system development.

There’s also guidance in the:

Introduction to AI Assurance which explores AI assurance techniques
Responsible AI Toolkit which lists various resources that support the safe and responsible development and deployment of AI systems
AI Testing and Assurance Framework for Public Sector which sets out a practical approach for testing, evaluation and assurance of AI systems
AI Playbook for the UK Government for information about the security and safety of AI systems

Recommended actions

Your team should:

make sure researchers and research participants are physically and emotionally safe – consider how a research topic could cause distress or discomfort, and reduce these risks by providing clear information, gaining informed consent, and allowing participants to withdraw at any time
comply with security requirements and standards – new services, including AI systems, must comply with the government’s Secure by Design principles which help ensure that systems are secure throughout their life cycle
apply data minimisation to limit the potential harms of a data breach – this means processing only the minimum amount of personal data necessary to achieve your purpose
keep all data secure throughout the project, referring to the personal data security principles
check for issues that could affect data quality, accuracy or reliability, such as data drift or model poisoning, by referring to the Government Data Quality Framework
ensure human oversight and have strategies to intervene where necessary, especially in high-risk use cases of AI, so that humans can validate decisions and outputs
run rigorous testing to identify potential failure modes for AI systems
- this may include using semi-automated techniques, such as using adversarial models to develop prompts, or red teaming exercises
- if it’s not possible to make the system sufficiently safe for the intended use, you should not use it to address the problem
use available resources, including the courses on risks and ethics on Civil Service Learning and the AI Playbook

Question 9

Further resources

Accepted Answer

There’s a range of data ethics guidance, standards and frameworks developed by and for government and public sector bodies. These resources support the responsible use and development of data and data-driven technologies, including AI.

The Responsible AI Toolkit also lists guidance related to deploying AI systems.

The UK Statistics Authority provides guidance related to applied data ethics in research and statistics.

Here are some other public sector resources.

This framework is one of a growing suite of tools which enable better use, reuse and sharing of data across government.

Government Data Quality Framework

This guidance sets out principles and practices to assess, communicate and improve the quality of data in government. Linked with this, there’s guidance on implementing a data quality action plan.

Dstl biscuit book – Assurance of AI and Autonomous Systems

This is a guide to understanding and describing the trustworthiness of autonomous systems and associated AI algorithms. It was written by the Defence Science and Technology Laboratory (Dstl).

Explaining decisions made with AI

This guidance is from the Information Commissioner’s Office and the Alan Turing Institute. It gives organisations practical advice to help explain the processes, services and decisions delivered or assisted by AI, to the individuals affected by them.

Five Safes framework

The Five Safes framework is a set of principles which enable data services to provide safe research access to data.

Guidelines for AI Procurement

Use these guidelines when procuring AI-powered solutions.

Model for Responsible Innovation

The Model for Responsible Innovation is a practical tool created by the Department for Science, Innovation and Technology (DSIT). It helps teams across the public sector and beyond to innovate responsibly with data and AI. You can use the model to run red teaming workshops with teams developing data-driven technology.

PETs Adoption Guide

The PETs Adoption Guide provides an interactive flowchart to help decision makers determine which privacy enhancing technologies (PETs) might be useful for their project. The guide is primarily aimed at technical architects and product owners working on projects that involve the sharing or processing of sensitive information.

Portfolio of AI assurance techniques

The portfolio of AI assurance techniques provides a searchable repository of cross-industry AI assurance techniques.

Technology Code of Practice

The Technology Code of Practice is a set of criteria to help the government design, build and buy technology.

UKSA Ethics Self-Assessment Tool

The Ethics Self-Assessment Tool developed by the UK Statistics Authority gives researchers an easy-to-use framework to review the ethics of their projects throughout the research cycle.

Cookies on GOV.UK

Introduction

Who this guidance is for

How to use this guidance

Data and AI Ethics Self-Assessment Tool

Why this guidance is important

Principles

Transparency

Accountability

Fairness

Privacy

Safety

Societal impact

Environmental sustainability

Trade-offs

Transparency

What transparency means in practice

Transparency in the context of data

Open data

Documentation about the provenance (collection or generation) of data

Documentation about any pre-processing to address limitations of the data

Documentation about how the data is used in a specific project

Documentation about data sharing and access

Data protection-related transparency

Fairness-related transparency

Transparency in the context of AI and algorithmic tools

Technical transparency

Process transparency

Outcome-based transparency and explainability

Explainability in practice

Using the Algorithmic Transparency Recording Standard

If you’re using a third-party supplier

Recommended actions

Accountability

What accountability means in practice

Set clear roles and responsibilities

Traceability and auditability

Establish feedback mechanisms

Maintain human oversight

Accountability if you’re using a third party supplier

Recommended actions

Fairness

Equality and inclusion

What fairness means in practice

Proxies

Understand impacted groups

Representational and allocational harms

Understand downstream impacts

Building for fairness

Accessibility and fairness

Data sets and labels

Representation bias

Societal bias

Data aggregation bias

Measurement bias

Ongoing fairness commitments

Disaggregated feedback

Recommended actions

Privacy

What privacy means in practice

Purpose and benefit in the context of data protection

Scoping and data acquisition

Privacy by design and default

Test and validate

Be transparent and give users control

Plan for and respond to data breaches

Decommissioning

Recommended actions

Environmental sustainability

What environmental sustainability means in practice

Measuring, monitoring and evaluating environmental impact

Selecting metrics

Minimising environmental impact

Recommended actions

Societal impact

What societal impact means in practice

Understanding stakeholders

Identifying harms and benefits

Continuous monitoring and evaluation

Recommended actions