Policy paper

Government response to the report of the equity in medical devices: independent review

Published 11 March 2024

Ministerial foreword

In a world-leading institution like the National Health Service, the right tool in the right hand saves lives.

Doctors, nurses and other health professionals should be confident that they have appropriate medical technology for individual patient needs.

As with many matters, the COVID-19 pandemic made us assess whether this is true in today’s healthcare system. Given the emergent evidence of potential bias in pulse oximeters, the government decided we needed to act - and the equity in medical devices: independent review was established.

Commissioned to explore equity in medical technologies, the review has made a valuable contribution to our understanding of health disparities and how to safeguard against future inequity.

My sincere thanks go to Professor Dame Margaret Whitehead and the review panel, Professor Raghib Ali, Professor Enitan Carrol, Professor Chris Holmes and Professor Frank Kee, for their time and expertise in producing this insightful report.

The report and recommendations prepared by the chair and review panel contribute to how we will ensure that Britain’s future is fair and healthy. This aligns with the Health and Social Care Secretary’s commitment to make the health and care system faster, simpler and fairer for patients. In turn, these actions will help build a longer-lasting, more robust NHS that is fit for purpose, with patient safety and high-quality care at the heart of all healthcare in our country. 

We wholeheartedly agree with the principles of the report: that medical technology should be unbiased and equitable. Each individual recommendation and sub-recommendation has been carefully considered by the relevant teams across the government and beyond, and I would like to thank all those who have contributed to the government response.

Following the publication of the inaugural medical technology strategy in February 2023, this report comes at an opportune moment. We are determined to harness the transformative potential that medical technology (or ‘medtech’) has to improve patient care and efficiency. Our commitment extends beyond solely supporting effective medtech - we are dedicated to fostering its equitable adoption. By doing so, we aim to ensure that every individual across the country not only gains access to the health benefits offered by medtech, but that the tech itself plays a pivotal role in addressing health disparities.

The cornerstone of our health system, the NHS, is committed to providing a comprehensive service, available to all irrespective of sex, race, disability, age, sexual orientation, religion, belief, gender reassignment, pregnancy and maternity, or marital or civil partnership status. As we rightly prioritise a transformation of medtech in the NHS, we must ensure that tackling disparities is considered as a priority.

We look forward to working with colleagues across government, the health system, the devolved administrations and crown dependencies, and patients to proactively tackle bias in medical devices, preventing bias from occurring in the future and improving the fairness of our health and care system.  

Rt Hon Andrew Stephenson CBE MP
Minister of State for Health and Secondary Care

Executive summary

The final report of the equity in medical devices: independent review advised that bias can be introduced at any stage in the design and use of a device, and makes recommendations for more equitable solutions. The following report is the government’s full response to the equity in medical devices: independent review. This government report provides a direct and specific response to each of the 18 recommendations made by the review panel.

The response presents a summary of the government’s view on each recommendation, and the calls to action, followed by a detailed discussion of the work involved in fulfilling the core sentiment of the recommendations and sub-recommendations.

Introduction

Background to the equity in medical devices: independent review

In February 2022, the Secretary of State for Health and Social Care, Sajid Javid, appointed Professor Dame Margaret Whitehead to lead an independent review into the extent and impact of potential ethnic and other unfair biases in the design and use of medical devices, and make recommendations for more equitable solutions.  The terms of reference were confirmed and a panel of experts appointed in April 2022.  

Professor Dame Margaret Whitehead and a select panel of experts were appointed to review the evidence, and provide recommendations on how to make technology fairer and improve the health of all.

Over the course of 15 months, the panel engaged in a wide range of evidence-gathering activities, including:

• engaging with stakeholders across the medical device life cycle
• reviewing and commissioning academic research
• running a public call for evidence
• conducting a series of roundtables

The final report on equity in medical devices: independent review (‘the report’) was submitted to government in June 2023.  

The report examines 3 types of medical devices that may be particularly prone to bias:  

• optical medical devices, including pulse oximeters
• artificial intelligence (AI)-assisted medical devices
• polygenic risk scores (PRS)

The report sets out how the risk for bias in medical devices exists across the device life cycle: from conception and development all the way through to deployment and use of devices in the NHS and the home. Highlighting many of the ongoing strands of work in these fields, it recommends a system-wide approach to the topic and a move away from conventional or developmental silos.

In total, the report makes 18 recommendations and 51 sub-recommendations, and includes 3 further calls to action.

Background to the government’s response to the report 

To be truly effective, healthcare must be equitable. In acknowledgement of this, the government has made tackling health disparities a key priority.

In recent years alone, we have created broad schemes providing targeted support for the cost of living and the COVID-19 pandemic, and established the Office for Health Improvement and Disparities, a government unit dedicated to reducing health disparities across the country. We have commissioned work like Core20PLUS5, established the Maternity Disparities Taskforce, and invested £50 million in health inequalities research for local authorities. 

This government committed to providing patients with the right product at the right price and in the right place, as set out in its first ever medical technology strategy in February 2023. Ensuring that medical devices are safe and clinically effective for all, regardless of ethnicity, sex or any other attribute is critical to achieving this ambition.

The report provides a useful analysis and synthesis of these issues as they relate to medical devices specifically, and how they can manifest across the life cycle of innovation. As part of our response, we discuss ongoing work in this area and reflect on further potential actions to help us fulfil one of the NHS’s key aims: to provide a comprehensive and safe service to all. 

We welcome the broad scope and the system-wide approach that the independent panel has taken to investigate the topic of equity in medical devices. The review panel conducted a comprehensive and systemic review of all available scientific evidence and literature applicable to NHS patients, to identify opportunities for improved action across different areas. In acknowledging the comprehensive nature of this review, it is important to note there are some areas which may warrant further investigation. For example, there could be benefit from further research into the use of medical devices in sub-Saharan Africa. Given our recognition of the limitations and biases of some technologies in individuals with darker skin tones, further population-wide studies would be of benefit in this area.

Furthermore, we also recognise that other sources of bias across the innovation life cycle may exist that were out of scope for the review, and which will need to be addressed in the future. We therefore see the equity in medical devices: independent review as an important milestone contributing to the way we should shape our collective thinking of equity in medical devices, but also innovation and healthcare more broadly.

The government’s response to the report

The government fully accepts the report’s main argument that, unless appropriate actions are taken, bias can occur throughout the medical device life cycle, from research, development and testing through to approval, deployment and post-market monitoring, as well as in the use of devices once deployed.

With respect to the initial stimulus for the review, it should be noted that the review did not identify clear evidence of worse clinical outcomes in the NHS for different patient groups that could be attributed to existing biases. The government accepts that lack of evidence may be due to insufficient data, and hence strengthening data collection and monitoring of medical devices is an important area that we are addressing.

The panel concluded that bias in the life cycle of medical devices often arose from misguided but well intentioned reasons, rather than being intentionally discriminatory. Throughout the report, the panel also commended much of the existing work to tackle bias arising throughout the medical device life cycle. 

To better understand the ongoing and upcoming work in this area, the feasibility of the recommendations made in the report and actions that could be taken to address any gaps, the government has engaged with relevant stakeholders across the health system, government and in the devolved administrations and crown dependencies. Those recommendations that impact operational policy in Scotland, Wales and Northern Ireland fall to the devolved administrations. While we have tried to capture all relevant activities and initiatives in this report, we recognise that other organisations may be engaging in other relevant projects that are not listed in this document.

Although considerable work is already being undertaken by multiple stakeholders and across different strands of work, we cannot stop here. Work to resolve and prevent health disparities is and will continue to be an ongoing priority across government and the health system as new technologies and issues emerge. We fully expect that more work will need to be done in this area and as part of global efforts with our international partners, as well as with our clinical, patient, research, industry and innovation communities in the UK.

Many of the report’s recommendations align with or build upon ongoing or planned work across government, including improving data in the NHS through the Data saves lives: reshaping health and social care with data policy, and improving participation of ethnic minorities in clinical research studies as part of the Inclusive Britain action plan. The report also aligns with recommendations made in the ‘First do no harm’ report of the Independent Medicines and Medical Devices Safety  (IMMDS) Review and the resulting government work. The medical technology strategy sets out actions including improving device evaluation, incorporating the patient voice and developing best-in-class regulations. The government will continue to build upon this work, and the report’s recommendations serve as a valuable reminder to consider our work on medtech through an equity lens. 

The report makes recommendations to the Medicines and Healthcare products Regulatory Agency (MHRA) to better account for potential bias in medical devices. Updates to the regulations for medical devices in the UK are currently under development. The new regulatory framework will ensure that medical devices are safe for patients, while allowing more innovative products to be placed onto the UK market. To ensure global alignment, we will continue to work with international partners. This includes MHRA’s active role in the International Medical Device Regulators Forum (IMDRF), a group of regulators from around the world who come together to accelerate international medical device regulatory harmonisation.

As set out below in response to specific recommendations, many of these regulatory updates will allow MHRA to achieve the aims of the report’s recommendations. The government is aiming for core aspects of the future regime for medical devices to apply from 2025.

Identifying and implementing solutions for any ongoing areas of concern will be a broad task and require system-wide collaboration. We welcome future relevant work and initiatives in this complex and multidisciplinary field - be that ongoing work on polygenic risk scores, the development of appropriate integrated data systems, or other new and innovative technologies.

In this response, we highlight the government action backing the overarching principal of the report: all parties - manufacturers, regulators and healthcare systems across the world - will need to work together to tackle and prevent unfair biases occurring in the design and use of medical devices.

Summary of the government response to each of the recommendations

Recommendation 1

Regulators, developers, manufacturers and healthcare professionals should take immediate mitigation actions to ensure existing pulse oximeter devices in the NHS can be used safely and equitably for all patient groups across the range of skin tones.

Government response

The government is committed to ensuring that pulse oximeters are safe and effective for all patients. Work is already underway to mitigate any inaccuracy in these devices, fulfilling many of this recommendation’s sub-recommendations.

Recommendation 2

MHRA and approved bodies for medical devices should strengthen the standards for approval of new pulse oximeter devices to include sufficient clinical data to demonstrate accuracy overall and in groups with darker skin tones. Greater population representativeness in testing and calibration of devices should be stipulated.

Government response

MHRA has a new validation process for clinical investigations in the UK that requests applicants demonstrate how they intend to address bias, in response to the commissioning of the equity in medical devices: independent review.

Other strands of work by MHRA in this area include creating joint diversity and inclusion guidance with the Health Research Authority (HRA) and contributing internationally to updating the International Organization for Standardization (ISO) 80601-2-61:2017 standard.

Recommendation 3

Innovators, researchers and manufacturers should co-operate with public and patient participants to design better, smarter oximeters using innovative technologies to produce devices that are not biased by skin tone.

Government response

The National Institute for Health and Care Research (NIHR) ensures patients and the public are involved in all aspects of the research process. NIHR welcomes funding applications for research into smarter oximeters. The government encourages researchers and innovators to consider these issues and use existing funding routes to carry out this research.

Recommendation 4

The professional practice bodies in the UK, such as the Royal Colleges, should convene a task group of clinicians from relevant disciplines - including medical physicists, public and patient participants, developers and evaluators - to carry out an equity audit of optical devices in common use in the NHS, starting with dermatological devices, to identify those at particular risk of racial bias with potential for harm, which should be given priority for further investigation and action.

Government response

The government agrees that healthcare providers and manufacturers must recognise the limitations and biases of these tools in individuals with darker skin tones.

Following its review of pulse oximeters, MHRA intends to investigate whether there is evidence of inaccuracy in different skin tones for other optical devices, using a range of data sources, and propose relevant regulatory action based on this, working closely with global partners.

Recommendation 5

Renewed efforts should be made to:

• increase skin tone diversity in medical imaging databanks used for developing and testing optical devices for dermatology, including in clinical trials
• improve the tools for measuring skin tone incorporated into optical devices

This will require a concerted effort on several fronts.

Government response

The government is aware of the risk of racial bias in these data sets if not collated from a representative population, and efforts should be made to ensure that diverse skin tones are included in these data imaging banks. 

HRA, NIHR and MHRA are all undertaking work in this area, and NIHR’s Randomised controlled trial participants: data diversity report showed that progress is already being made in this area. The government supports the continuation of this work across the health system.

Recommendation 6

Once in use, optical devices should be monitored and audited in real-world conditions to evaluate safety performance overall and by skin diversity. This will ensure any adverse outcomes in certain populations are identified early and mitigations implemented.

This requires a whole-system approach.

Government response

There is a rationale for monitoring optical devices in real-world application to avoid potential harms, especially for ethnic minority patients. Monitoring of these devices should align with post-market surveillance (PMS) approaches currently being developed by MHRA.

Recommendation 7

A review should be conducted by the relevant academic bodies of how medical education and continuing professional development requirements for health professionals currently cover equity issues arising in the use of medical devices generally and skin diversity issues in particular, with appropriate training materials developed in response.

Government response

We agree that education and training in the healthcare field should include a focus on ethical considerations and equity issues related to the use of medical devices, as well as more widely. We believe it is important to ensure that new materials published consider equity and bias.

Succeeding at this will require a collaborative and concerted effort across education providers, professional and educational membership bodies, and NHS England’s Workforce, Training and Education Directorate (NHS WT&E).

Such work could build on existing initiatives such as the NHS England Enhancing Generalist Skills programme, which upskills clinical professionals on issues including health equity, or the Artificial Intelligence (AI) and Digital Healthcare Technologies Capability Framework.

Recommendation 8

AI-enabled device developers and stakeholders, including the NHS organisations that deploy the devices, should engage with diverse groups of patients, patient organisations and the public, and ensure they are supported to contribute to a co-design process for AI-enabled devices that takes account of the goals of equity, fairness and transparency throughout the product’s life cycle.

Engagement frameworks from organisations such as NHS England can help hold developers and healthcare teams to account for ensuring that existing health inequities affecting racial, ethnic and socio-economic subgroups are mitigated in the care pathways in which the devices are used.

Government response

We know that involving a diverse group of people helps us make better decisions and will contribute to addressing existing health disparities.

HRA and NIHR, along with a host of organisations across the UK, have signed up to a shared commitment to public involvement to bring about changes that will drive up standards in health and social care research.

Recommendation 9

The government should commission an online and offline academy to improve the understanding among all stakeholders of equity in AI-assisted medical devices.

This academy could be established through the appropriate NHS agencies, and should develop material for lay and professional stakeholders to promote better ways for developers and users of AI devices to address equity issues.

Government response

While we recognise the value of some of the sub-recommendations made, the government believes the aim of this recommendation can be achieved through alternative approaches, including some work that is already ongoing and in development.

Recommendation 10

Researchers, developers and those deploying AI devices should ensure they are transparent about the diversity, completeness and accuracy of data through all stages of research and development. This includes the sociodemographic, racial and ethnic characteristics of the people participating in development, validation and monitoring of product performance.

Government response

We support the drive for transparency of information to improve the safety of AI medical devices including work to improve the trustworthiness of AI products that influence clinical decisions.

Ongoing and planned work in this area by MHRA, NIHR and HRA will continue to improve transparency surrounding data quality in AI-enabled devices.

Recommendation 11

Stakeholders across the device life cycle should work together to ensure that best practice guidance, assurance and governance processes are co-ordinated and followed in support of a clear focus on reducing bias, with end-to-end accountability.

Government response

End-to-end accountability is a whole-system responsibility that involves ‘hard’ and ‘soft’ governance, the support and development of responsible working cultures, and appropriate independent oversight.

We continue to support the ongoing work of NHS England’s Transformation Directorate and MHRA in this area, including initiatives that aim to co-ordinate resources across the life cycle.  

Recommendation 12

UK regulatory bodies should be provided with the long-term resources to develop agile and evolving guidance, including governance and assurance mechanisms, to assist innovators, businesses and data scientists to collaboratively integrate processes in the medical device life cycle that reduce unfair biases and their detection, without being cumbersome or blocking progress.

Government response

We agree that an agile approach is required to deal with the ever-changing needs of software and AI, with regulatory bodies working collaboratively to create a sustainable guidance development approach.

The government continues to support the appropriate funding required for the operation of MHRA, taking into account all of its functions.

Recommendation 13

The NHS should lead by example, drawing on its equity principles, influence and purchasing power, to influence the deployment of equitable AI-enabled medical devices in the health service.

Government response

The government agrees that there are opportunities for equity to be considered within NHS procurement and deployment processes. A proportionate approach that does not excessively limit the number of devices available in the NHS is required.

Implementation of this recommendation, and the related sub-recommendations, would require careful consideration and planning to ensure the best approach is taken to achieve the recommendation’s aim.

Recommendation 14

Research commissioners should prioritise diversity and inclusion. The pursuit of equity should be a key driver of investment decisions and project prioritisation. This should incorporate the access of underrepresented groups to research funding and support, and inclusion of underrepresented groups in all stages of research development and appraisal.

Government response

We agree that the inclusion of under-represented groups is an important factor in research project prioritisation decisions. NIHR has developed sessions for researchers on inclusion in the research cycle, along with a range of guidance and toolkits.

Alongside this, Inclusive Britain commits the government to increasing ethnic minority participation in trials, through measures such as the government-funded INCLUDE framework and this will satisfy any recommendations to factor in equity.

HRA supports research ethics committees to consider health equity impacts. The government supports the continuation of this work to fulfil this recommendation and ensure inclusion in research.

Recommendation 15

Regulators should be properly resourced by the government to prepare and plan for the disruption that foundation models and generative AI will bring to medical devices, and the potential impact on equity.

A government-appointed expert panel should be convened - made up of clinical, technology and healthcare leaders, patient and public involvement representatives, industry, third sector, scientists and researchers who collectively understand the technical details of emerging AI and the context of medical devices - with the aim of assessing and monitoring the potential impact on AI quality and equity of LLM [large language] and foundation models.

Government response

Ensuring that regulators are prepared for the inevitable disruption from new AI technologies is a key priority for government, and we believe that encouraging cross-sector collaboration is the best approach to achieve the aim of this recommendation.

As set out in the 2023 AI regulation: a pro-innovation approach white paper, the government is putting in place a range of measures that are designed to support regulators in addressing the risks and challenges posed by new AI technologies. These measures include:

• guidance on regulatory principles for AI
• access to central risk and horizon-scanning functions
• central support for regulators seeking to develop their AI capabilities and skills

Recommendation 16

The focus of PRS studies should be widened beyond genetic diversity to include:

• the contribution of the social determinants of health - including lifestyle, living and working conditions, and environmental factors such as air pollution - to overall disease risk
• how these affect the predictive potential of PRS among different ethnicities and socio-economic groups

Developments with this wider research focus should aid the refinement of overall risk assessments so that they better reflect the role that PRS play alongside non-genetic risk factors.

Government response

Use of PRS in predicting overall disease risk is dependent upon diverse, reliable and consistent data being readily available to researchers at the initial planning phases of research.

The government will continue to support initiatives to ensure diversity in genomic research, such as Our Future Health and Genomics England’s Diverse Data initiative.

Recommendation 17

National research funders should commission a broad programme of research and consultation with the public, patients and health professionals to fill the gaps in knowledge and understanding concerning PRS. The programme should cover:

• the public’s understanding of the nature of genetic risk and the meaning of the PRS they are presented with
• explorations of how health professionals interpret these risks, and can best communicate and support people in understanding the results of their PRS

Government response

Public understanding of genomics and the communication of risk, particularly among diverse groups, are important considerations as genetic testing is developed and introduced.

Recently published research jointly funded and carried out by Genomics England is an example of work in this area to begin to understand these issues.

Recommendation 18

UK professional bodies - such as the Royal Colleges and health education bodies across the UK - should develop guidance for healthcare professionals on the equity and ethical challenges and limitations of applying PRS testing in patient care and population health programmes. This guidance should:

• include the interpretation of risk scores, communicating risk to patients and the public, and counselling and support
• be informed by extensive public and patient engagement

Government response

PRS are an additional tool to potentially improve outcomes - however, they can also increase health inequalities. Most genomic studies have analysed European ancestry - therefore PRS may not be as accurate on populations from other ancestries.

Calls to action

Call to action 1

These recommendations need to be implemented as a matter of priority with full government support.

Government response

The government is grateful to the panel for making these recommendations and values their views. We agree with the vast majority of the recommendations and, as set out in this response, are taking action across government and our arm’s length bodies (ALBs) to fulfil these.

We have also indicated where we determine there are alternative means to achieve the aims of the recommendations.

Call to action 2

Addressing inequities in access is therefore an essential task for the government and leadership of the NHS.

Government response

Addressing inequities in access to medical devices and technology is important to the government and the NHS, and dedicated work in the space is already underway. 

Call to action 3

A review should be carried out of equity in the medical devices encountered during pregnancy and the neonatal period, as part of the wider investigations of health outcomes for ethnic minority and poorer women and their babies.

Government response

The safety of maternity care is a priority for the government, and we have commissioned Donna Ockenden to lead an independent review of cases of concern in maternity services.

However, we recognise the need to go further, and that research is needed to understand whether there is equity in the use of medical devices encountered during pregnancy and the neonatal period.

The Department of Health and Social Care (DHSC), through the NIHR Policy Research Programme, is launching a call to understand disparities in the use of medical devices in pregnancy and the neonatal period, and expects to commission this research, subject to receiving high-quality applications.

Detailed response to recommendations

Recommendation 1

Recommendation 1

Regulators, developers, manufacturers and healthcare professionals should take immediate mitigation actions to ensure existing pulse oximeter devices in the NHS can be used safely and equitably for all patient groups across the range of skin tones.

Government response

The government is committed to ensuring that pulse oximeters are safe and effective for all patients. Work is already underway to mitigate any inaccuracy in these devices, fulfilling many of recommendation 1’s sub-recommendations as follows.

Recommendation 1.1

MHRA should strengthen its guidance for patients and caregivers using oximeters at home, and for healthcare professionals, on the accuracy and performance of pulse oximeters. This should include guidance on taking and interpreting readings from patients with different skin tones. Renewed efforts should be made to promote this guidance to health professionals throughout the NHS, patients and the public

Government response

MHRA’s ongoing work fully supports this recommendation. MHRA uses a risk minimisation approach to address the risk of inaccuracy with pulse oximeters on different skin pigmentation, which includes raising awareness of the correct use and limitations of pulse oximeters (particularly including the risk of inaccuracy with darker skin tone) through communication and education. This was supported by the Interim Devices Working Group at its meeting on 4 July 2023 following an internal review of the evidence.

MHRA will take a phased approach on developing and implementing communication packages, including:

• reviewing current guidelines
• developing a broader communication strategy for healthcare professionals, patients and caregivers
• producing devices safety information to disseminate the messaging further

More proactive, public-facing communication may be considered based on feedback from engagement with relevant groups including patients.

MHRA will also work across the healthcare system to ensure consistency and harmonisation, cross-linking relevant guidance where appropriate.

Recommendation 1.2

Health professionals should advise patients who have been provided with a pulse oximeter to use at home to look at changes in readings, rather than just a single reading, to identify when oxygen levels are going down and they need to call for assistance. Patients should also be advised to look out for other worrying symptoms such as shortness of breath, cold hands and feet, chest pain and fast heart rate.

Recommendation 1.4

Health Education England (part of NHS England) and the respective agencies in the devolved nations should educate clinicians about how the technology of pulse oximeters works, and advise that treatment should not be withheld or given on the basis of absolute thresholds alone. Clinicians should be trained to monitor trends rather than absolute thresholds for action.

Government response

It is vital to ensure our multidisciplinary workforce is trained to recognise and address these potential biases to provide equitable healthcare for all patients, as well as to support patients in understanding how to interpret both their pulse oximetry readings and the wider symptoms, which provide a greater context for their health. 

Following recommendations from the NHS Race and Health Observatory in April 2021, NHS England updated its existing COVID Oximetry @home guidance to enhance clarification on the potential limitations of these devices for people with darker skin tones. 

NHS WT&E, through its elearning for healthcare service, makes available a range of learning resources that include the use of pulse oximeters in at-home services, and the management of deterioration and use of early warning score appropriately.

These are currently accessed via the elearning for healthcare hub but will be migrated to the NHS Learning Hub during 2023 to 2024. This includes the virtual wards enabled by technology e-learning programme.

These e-learning resources are reviewed regularly, and will be updated to ensure they include specific information on the limitation and biases of pulse oximetry in individuals with darker skin tones. 

Recommendation 1.3

Clinical guideline developers and health technology assessment (HTA) agencies such as the National Institute for Health and Care Excellence (NICE) should produce guidance on the use of pulse oximeters, emphasising the variable accuracy of readings in patients with darker skin tones, and recommend the monitoring of trends rather than setting absolute thresholds for action.

Government response

As outlined in our response to recommendation 1.2 above, updated guidelines have been published.

With respect to monitoring trends, this may not be a solution on its own as there would still need to be a point defined at which clinical action is taken. Monitoring trends may also not allow a timely response in acute scenarios.

Pulse oximeter readings should also be interpreted in conjunction with other symptoms and adjustment made in people where the readings may be less accurate, as acknowledged in recommendation 1.2.

Recommendation 1.5

Manufacturers of pulse oximeters must update their instructions for use to inform patients and clinicians about whether the device is ISO compliant, the limitations of their model of pulse oximetry and any contra-indications, and its differential accuracy in patients with different skin pigmentation.

Government response

We recognise that the product information for pulse oximeters should be strengthened regarding any limitations of their accuracy in different populations.

MHRA’s ongoing work should partially support this recommendation in the long term. We believe it would be most effective to work towards this recommendation through changes to the ISO 80601-2-61:2017 standard, which would create a wider reach and help to achieve international harmony. MHRA is an active participant on an international committee that is examining the evidence and will be updating this standard as per the conclusions of these proceedings in due course.

Additionally, MHRA proposes in parallel to work with the approved bodies, who are able to use the conformity assessment process to appropriately address the risk of inaccuracy of pulse oximeters due to different skin tones. MHRA, in addition to undertaking regular audits of their activity, will liaise with approved bodies in the third and fourth quarters of 2023 to strengthen the work on this for pulse oximeters. This is an effective route to ensure risk mitigation is undertaken for both current and future manufacturers of pulse oximeters.

As part of the conformity assessment process, an approved body reviews a sample of technical files from a manufacturer. Reviewing the instructions for use forms part of this assessment to ensure they meet the requirements of the relevant regulations (such as the UK Medical Devices Regulations (statutory instrument 2002 No. 618 as amended). For example, the label must include any “warnings and/or precautions to take” as well as details allowing medical staff to brief patients on any contra-indications and precautions to be taken. 

MHRA is working to implement a strengthened regulatory framework for medical devices, with the core aspects expected to be in place from 2025. This is an extensive reform that will:

• enhance patient safety
• foster innovation
• ensure the UK aligns with global partners

This will include enhancing MHRA’s powers to:

• monitor the safety and effectiveness of medical devices in clinical use
• require vigilance activities from manufacturers such as the surveillance of particular concerns

Part of the regulator actions in response to MHRA’s review of the performance of pulse oximeters will be to require manufacturers to monitor the impact of skin tone on the accuracy of their devices. Any new safety concerns identified with medical devices, including any biases in different patient groups, would result in regulatory action, such as relevant updates to the instructions for use to inform users.

Recommendation 1.6

MHRA should issue updated guidance to developers and manufacturers on the need to make the performance of their device across subgroups with different skin tones transparent.

Government response

Guidance on the performance of medical devices in various subgroups will be issued and MHRA will be actioning this through relevant organisations to have the widest impact.

In addition to working to achieve this through changes to the ISO 80601-2-61:2017 standard, MHRA will look to develop best practice for approved bodies on the appropriate interpretation of the regulatory requirements. In the case of pulse oximeters, this would include ensuring an approved body is looking at how the manufacturer has addressed the impact of skin tone on the accuracy of its devices.

IMDRF, a voluntary group of medical device regulators from around the world, of which MHRA is a member, has issued Guiding Principles to Support Medical Device Equity (N79). These guiding principles have been developed to advance health equity discussions for underrepresented populations in the development and regulation of medical devices. MHRA will publicly support and publicise these principles to encourage uptake within the UK medtech industry. 

Recommendation 2

Recommendation 2

MHRA and approved bodies for medical devices should strengthen the standards for approval of new pulse oximeter devices to include sufficient clinical data to demonstrate accuracy overall and in groups with darker skin tones. Greater population representativeness in testing and calibration of devices should be stipulated.

Government response

MHRA has a new validation process for clinical investigations in the UK, which requests that applicants demonstrate how they intend to address bias in response to the commissioning of the equity in medical devices: independent review.

Other strands of work by MHRA in this area include creating joint diversity and inclusion guidance with HRA and contributing internationally to updating the ISO 80601-2-61:2017 standard.  

Recommendation 2.1

MHRA and UK-approved bodies following the US Food and Drug Administration (FDA) in requiring manufacturers to obtain validity data from a diverse subject pool with a:

• large number of participants
• diverse range of skin tones
• clinically relevant range of oxygenation levels

Government response

We fully support the view that manufacturers should gather data from a range of participants to identify any difference in safety and efficacy in subgroups.

A number of ongoing and planned strands of work should fulfil this recommendation as follows:

• in response to the commissioning of the equity in medical devices: independent review, MHRA now requests that all clinical investigation applicants who are conducting a clinical investigation with a device that does not yet have a CE or UK Conformity Assessed (UKCA) marking describe at the point of application how they intend to address bias, and this will be incorporated into the guidance for medical device clinical investigations
• UK-approved bodies are required to assess clinical evaluation data as part of the conformity assessment process for devices that are UKCA marked
• we believe this recommendation can also be achieved through the update of the ISO 80601-2-61:2017 standard, again ensuring wider reach beyond the UK and harmonisation
• as referenced in our response to recommendation 1.6 above, the IMDRF, of which MHRA is a member, has also issued guiding principles to support medical device equity, which will encourage greater representation in development of medical devices
• HRA and MHRA are working together to help researchers increase the diversity of people taking part in research in the UK for both clinical investigations for medical devices and clinical trials for medicines
• there is also work underway to develop new UK legislation in line with global best practice. This will include a requirement for performance studies to be designed and conducted in such a way as to minimise potential bias and ensure that the data generated is scientifically valid, reliable and robust
• under incoming UK regulations, there will be enhanced requirements for post-market clinical follow up and PMS. This will also be reviewed by UK-approved bodies and will require information about user experience, including through patient and public engagement, as part of the PMS plan

Recommendation 2.2

Manufacturers and research-funding bodies commissioning studies that include the population upon which the device will be used, subjects with a diverse range of skin pigmentations and critically unwell subjects with poor perfusion. Validation of devices should be conducted in the intended use population and setting, such as at home or in an intensive care unit.

Government response

NIHR has already commissioned research in this area through the HTA 21/608 Diagnostic accuracy of pulse oximeters in home settings funding call, providing more than £580,000 in 2022 focusing on pulse-oximetry devices specifically.

NIHR’s Efficacy and Mechanism Evaluation (EME) programme as well as the HTA researcher-led open call specification document funding streams provide additional scope for focus on diverse utility and efficacy of optical devices, and we encourage researchers to make use of these funding streams. 

Recommendation 2.3

Manufacturers of medical-grade pulse oximeters being required to comply with BS EN ISO 80601-2-61:2019 (medical electrical equipment - particular requirements for basic safety and performance of pulse oximeter equipment) to gain market approval.

Government response

We welcome the recommendation to use the updated ISO standard to achieve a better representation of the limitations of pulse oximeters in different skin tones.  

A manufacturer must ensure its medical device meets the requirements of the regulations set out in the UK Medical Devices Regulations 2002 in order to place a device on the market in Great Britain. 

In Great Britain, a manufacturer may claim compliance with a relevant designated standard to demonstrate their device is safe, effective and state of the art. However, the UK Medical Devices Regulations 2002 do not mandate that a manufacturer must use technical standards to demonstrate compliance with the regulations, as the report recommends.

Once the above standard has been updated, which we expect to happen in early 2024, MHRA will consider including it as a designated standard. This would ensure there is a more robust and consistent approach for assessing pulse oximeters. In the meantime, MHRA will continue to closely monitor pulse oximeters, and the impact of its updated guidance for healthcare professionals and patients on the accuracy of pulse oximeters on different skin tones. 

MHRA has developed a common vigilance platform for reporting of safety concerns identified by patients and healthcare professionals, which will make it easier for reports to be made and updated as new information is available. This will enhance MHRA’s surveillance capabilities and make it easier to identify where medical devices are not meeting the requirements of the regulations and relevant standards. 

Recommendation 2.4

Healthcare equity impact assessments being essential requirements for developing or supplying pulse oximeters in the UK in order to identify whether mitigating actions are needed to ensure they are fit for purpose for all racial and ethnic groups and people of varying skin tones. Making these assessments an essential requirement is in line with technological progress and international best practice.

Government response

We support the aim of employing healthcare equity impact assessments to drive the development of these devices in the UK. 

MHRA’s ongoing work partially addresses this recommendation. As noted in our response to recommendation 2.1, MHRA has a new validation process for clinical investigations taking place in the UK, which now requests that applicants investigating newly developed and non-CE or UKCA-marked medical devices demonstrate how they intend to address bias.

There is no specific provision in the regulations that requires equity impact assessment for medical devices, and it would not be appropriate to mandate one in the essential requirements for all medical devices. However, should a safety or effectiveness concern be identified that is related to a specific population or subgroup then this would be addressed through the vigilance process, as has the pulse oximeter accuracy issue.

Furthermore, it is intended that new vigilance requirements will be implemented with updated medical device regulations in line with international best practice, such as manufacturers having to monitor any specific safety concerns identified with a product in a periodic report. MHRA intends to use these powers to ensure pulse oximeter manufacturers continuously monitor issues related to inaccuracies in different skin tones.

In relation to the supply of pulse oximeters, there is a considerable risk that, if NHS Supply Chain is the sole procurement body requiring healthcare equity impact assessments in its specifications, this could limit the pool of bidders at the detriment of the NHS. Therefore, we believe that this is best achieved through a globally led standard, which MHRA is already involved in updating.

NHS Supply Chain is in the process of re-tendering the current Pulse Oximetry, Capnography and Related Monitoring Technologies framework for a launch in March 2024. The specification for this framework sets an evidence requirement around device accuracy in relation to skin pigmentation and low perfusion. We believe that this work fulfils the spirit of this recommendation, in addition to MHRA’s ongoing work detailed above.

Recommendation 3

Recommendation 3

Innovators, researchers and manufacturers should co-operate with public and patient participants to design better, smarter oximeters using innovative technologies to produce devices that are not biased by skin tone.

Government response

NIHR ensures patients and the public are involved in all aspects of the research process. NIHR welcomes funding applications for research into smarter oximeters. The government encourages researchers and innovators to consider these issues and use existing funding routes to carry out this research.

Ongoing work referenced in our response to recommendation 5.3 below will improve diversity in clinical trials and research so that research results are more generalisable to the population groups upon which the product is used.

Recommendation 3.1

Developing enhanced algorithms for oximeter device software to address measurement bias.

Recommendation 3.2

Exploring the use of multi-wavelength systems, which measure and correct for skin pigmentation, to replace conventional 2-wavelength oximeters.

Government response

We welcome funding applications into any aspect of human health, including smarter oximeters.

Government funders of health research, including NIHR, do not allocate funding for specific disease areas. The level of research spending in a particular area is driven by factors including scientific potential, and the number and scale of successful funding applications.​

NIHR ensures patients and the public are involved in all aspects of the research process including commissioning, designing, awarding and delivering the research. This includes the research NIHR has supported investigating the accuracy of oximeters by levels of skin pigmentation.

The government encourages researchers and innovators to consider recommendations 3.1 and 3.2, and use existing funding routes to carry out this research.

Recommendation 4

Recommendation 4

The professional practice bodies in the UK, such as the Royal Colleges, should convene a task group of clinicians from relevant disciplines - including medical physicists, public and patient participants, developers and evaluators - to carry out an equity audit of optical devices in common use in the NHS, starting with dermatological devices, to identify those at particular risk of racial bias with potential for harm, which should be given priority for further investigation and action.

Government response

The government agrees that healthcare providers and manufacturers must recognise the limitations and biases of these tools in individuals with darker skin tones, particularly with the expansion of community diagnostics. Work that supports the objective of this recommendation is underway elsewhere in the system.

Following its review of pulse oximeters, MHRA intends to:

• investigate whether there is evidence of inaccuracy in different skin tones for other optical devices using a range of data sources
• propose relevant regulatory action based on this

Recommendation 5

Recommendation 5

Renewed efforts should be made to:

• increase skin tone diversity in medical imaging databanks used for developing and testing optical devices for dermatology, including in clinical trials
• improve the tools for measuring skin tone incorporated into optical devices

This will require a concerted effort on several fronts.

Government response

The government is aware of the risk of racial bias in these data sets if not collated from a representative population, and efforts should be made to ensure that diverse skin tones are included in these data imaging banks. 

HRA, NIHR and MHRA are all undertaking work in this area, and NIHR’s Randomised controlled trial participants: diversity data report shows that progress is already being made.

The government supports the continuation of this work across the health system. 

Recommendation 5.1

Encouraging links between imaging databank compilers, professional bodies, optical device developers and clinicians to develop and improve accessibility of imaging data resources that reflect skin tone diversity within the population, such as in databanks for skin cancer diagnosis.

Government response

With growing evidence of the role of data imaging banks in research, innovation and development, we welcome and support this recommendation. As highlighted by the recommendation, this is a collaborative concerted effort requiring individual approaches by healthcare providers, manufacturers, regulatory bodies and educational institutions.

Ongoing work by the NHS AI Lab’s AI Ethics Initiative aims to address issues of racial and other bias in imaging data sets. The NHS AI Lab’s NHS AI Virtual Hub brings together AI developers, data scientists, NHS clinicians and others, and encourages co-operation and sharing of expertise and best practice.

We support the continuation of this work by the NHS AI Lab to connect stakeholders across the device life cycle to improve the representativeness of databanks.

Recommendation 5.2

MHRA providing strengthened guidance to developers and manufacturers on improving skin tone diversity in testing and development of prioritised optical devices. MHRA is already working towards such guidance as part of its programme on pulse oximeters.

Government response

We welcome the widening of MHRA’s review to investigate any similar impact with other optical devices based on different skin tones. 

Following on from the review of pulse oximeters, MHRA intends to investigate whether there is evidence of inaccuracy in different skin tones for other optical devices, using a range of data sources to review any evidence identified, and will propose relevant regulatory action based on this, which would include vigilance activities by the manufacturers for any concerns identified.  

With regards to improving the testing and development of optical devices, MHRA’s new process for validation of clinical investigation applications, as described in our response to recommendation 2, applies to all medical devices going through this process.

Recommendation 5.3

Research funders supporting additional incentives and patient-centred approaches to address logistical, financial and cultural barriers that limit participation of minority ethnic groups in clinical studies of optical devices.

Government response

It is crucial that funders and regulators ask for and support sponsors to ensure that they plan for appropriate diversity of participants in clinical trials so that trial results are generalisable to groups who would potentially benefit from the findings.

Public involvement is another critical factor to ensure patient-centred approaches, and funders should make sure that this is embedded throughout the clinical study process.

NIHR’s Randomised controlled trial participants: data diversity report showed that, while participation of individuals from ethnic minority backgrounds was roughly proportionate with the Office for National Statistics’ census data available at the time, more is being done to encourage greater diversity, including increased or improved guidance to researchers relating to:

• inclusive recruitment of research participants
• costing for inclusion
• diversity data monitoring

A significant amount of activity has been undertaken on action 24 of Inclusive Britain to promote, disseminate and adopt the Innovations in Clinical Trial Design and Delivery for the Under-served (INCLUDE) framework through NIHR’s Under-served communities programme. 

From 2023 to 2024, NIHR will also support researchers by delivering quarterly sessions relating to inclusion in the research cycle. NIHR also recently published a blog post on tackling health inequalities including a statement of intent to partner with the NHS Race and Health Observatory. 

HRA is working with MHRA to develop a diversity and inclusion plan that will be submitted by sponsors as part of the clinical trial and clinical investigation authorisation process. This will ask sponsors to outline how they plan to ensure inclusion of appropriate population groups, including ethnic minority groups where this is appropriate, which should include addressing logistical, financial and cultural barriers to participation. This is being co-developed with individuals from academic institutions, industry, research ethics committee (REC) members, research funders and members of the public, and a draft is scheduled for public consultation towards the end of this year.

The Medical Research Council (MRC) recently published a policy commitment to embed consideration of relevant diversity characteristics into the design and conduct of all MRC-funded research and innovation, and to ensure that the benefits and impact of these activities extend across all communities and populations.

There is further ongoing work across the devolved administrations, such as the establishment of an involvement inclusivity advisory group in Wales to help co-create initiatives to promote and embed diversity in public involvement community and researcher practices. This includes taking a leading role on the development of guidance for the reasonable reimbursement of public contributors, which is often a barrier to some groups of society being able to be involved in research. 

This work connects to the UK Recovery, Resilience and Growth programme’s Future of UK Clinical Research Delivery: 2021 to 2022 implementation plan activities focused on ‘people-centred research’, which are being led by HRA with contributions from key organisations across the UK research landscape.

We support the continuation of this wide range of work to continue to fulfil recommendation 5.3.

Recommendation 5.4

Researchers and dermatologists developing more accurate methods for measuring and classifying skin tone, which are objective, reproducible, affordable and user-friendly. Current practice of using uncertain descriptors of ancestry, ethnicity or race to define patients with dark skin tones is ambiguous and problematic. In its discussions on updating standards, MHRA is examining which measures would be most appropriate, with the aim of agreeing a consensus. This work is to be commended.

Government response

We endorse the development of accurate methods of measuring and classifying skin tone and their implementation as more accurate methods to establish efficacy on different skin tones.  

As part of its involvement in the review and update of the ISO 80601-2-61:2017 standard, MHRA will continue to engage with wider international stakeholders, and support and influence this process to ensure appropriate methods for measuring skin tone are adopted where possible. 

However, it is important to note that MHRA is not leading on this and cannot determine which measures of skin tone will be applied in this standard.

Recommendation 6

Recommendation 6

Once in use, optical devices should be monitored and audited in real-world conditions to evaluate safety performance overall and by skin diversity. This will ensure any adverse outcomes in certain populations are identified early and mitigations implemented.

This requires a whole-system approach.

Government response

There is a rationale for monitoring optical devices in real-world application to avoid potential harms, particularly for ethnic minority patients. Monitoring of these devices should align with PMS approaches currently being developed by MHRA.

MHRA is in the process of updating the regulatory requirements for PMS, which will come into force in 2024. These changes will strengthen the ability of manufacturers and MHRA to identify issues with a medical device - for example, through provision of periodic safety update reports to the approved body. 

Recommendation 6.1

Commitment from manufacturers at the pre-qualification stage to fund and facilitate the establishment of registries for collecting data across all population groups on patient demographic characteristics, use and patient outcomes, following deployment of the technology.

Government response

In response to the Cumberlege review and Paterson inquiry, we have established the Outcomes and Registries Programme, which is a patient-centred, clinically led programme aiming to improve patient safety and outcomes by collecting, linking and analysing clinical and patient outcome data more effectively to support direct care and clinical practice improvement, and enable research and innovation.

As part of this programme, in May 2023, we launched:

• a national, mandatory medical device outcome registry across all medical device procedures
• consolidation of existing device-level registries
• onboarding of several high-priority non-device procedure registries

This will record key details of the procedure, the clinicians involved and high-risk devices used, as well as demographic data of patients, and link these details to clinical observational and patient outcome data to improve patient safety and outcomes.

The Outcomes and Registries Programme is centrally funded and managed by NHS England, rather than being funded through an industry levy, due to the need to have a consistent approach across clinical areas, device types and manufacturers. Medtech companies are involved in the programme, working within steering groups that are led by patient representatives, clinical experts and digital data professionals. Patients, clinicians and manufacturers can propose Outcomes and Registries Programme coverage in new device treatment areas if required.

Recommendation 6.2

HTA agencies (such as NICE, the Scottish Health Technologies Group and Health Technology Wales) being provided with access to post-deployment monitoring and adverse effects data as part of their assessments of optical devices. This data should be considered alongside the wider evidence when determining the value of the optical device for NHS use.

Government response

There are processes in place across the UK to capture data on adverse events, and we recognise the importance of joined-up data to ensure this is considered at various stages of the life cycle.

MHRA can further explore sharing adverse effect data with HTA agencies, and NICE would welcome greater access to post-deployment monitoring and adverse effects data to inform its guidance and updates. This work would require further consideration and planning.

Recommendation 6.3

NHS Supply Chain, National Services Scotland, NHS Wales Shared Services Partnership, Northern Ireland Procurement and Logistics Service and other contracting authorities including a minimum standard of device performance across subgroups of the target population, which will make transparent any equity impacts as part of the pre-qualification stage when establishing national framework agreements. Manufacturers need to declare whether they have considered minimum standards for equity.

Government response

The specification for this is currently being developed in England and will incorporate wording in relation to skin pigmentation and device performance.

NHS Supply Chain would welcome further engagement with clinical teams prior to the publication of this document to the marketplace to ensure this wording is appropriate.

Recommendation 6.4

DHSC and the devolved administrations updating the national pre-acquisition questionnaire used by NHS trust electrical biomedical engineering teams when buying medical equipment to include a minimum designated standard for equity as part of the pre-purchase validation checks.

Government response

The pre-acquisition questionnaire is due to transfer from DHSC to NHS England, at which point it will be reviewed for any required updates, including issues of equity.

This work will require further consideration and wider engagement with the health system, users at trust level and the devolved administrations.

Recommendation 6.5

The approved body conducting regular surveillance audits of prioritised optical devices. The audits should include data submissions from the manufacturer and the Medical Device Safety Officers or Incidents and Alerts Safety Officers networks (representatives from NHS trusts in charge of reporting on safety), and should include data from the MHRA Yellow Card scheme for reporting adverse incidents and the Learn from patient safety events service. These audits should include an evaluation of differential safety by ethnic group.

Government response

We support the aim of using post-market data to support manufacturers in earlier detection of trends or signals that may have an impact on the safety of a medical device. 

Approved bodies are required to undertake periodic surveillance audits of manufacturers and review a sample of technical documentation as part of the ongoing monitoring of compliance. Manufacturers are already required to report vigilance data to their approved body (where their medical device’s certification requires approved body involvement). The approved body will assess the impact of this on their certification. When MHRA undertakes direct audits of manufacturers, MHRA reviews the manufacturers’ own data to see if there are any relevant reports that need to be considered. However, this is all reliant on reports being made by users, such as Medical Device Safety Officers and relevant devolved organisations, or patients that are using these devices in their own home.  

MHRA is in the process of updating the regulatory requirements for PMS. A statutory instrument that will place enhanced requirements on manufacturers will come into force in 2024, which will strengthen the ability of manufacturers and MHRA to identify issues with a medical device - for example, through provision of periodic safety update reports to the approved body. Once the statutory instrument has been laid before Parliament, MHRA will publish guidance to clarify the requirements in relation to PMS plans and how the new requirements should be implemented.

Recommendation 6.6

The continued strengthening of MHRA’s vigilance role, as specified in the Cumberlege report’s recommendation 6, which called for substantial improvements in adverse event reporting and medical device regulation with an emphasis on patient engagement and outcomes.

Government response

MHRA and the government remain committed to delivering on the recommendations from the IMMDS review and have already made significant progress since the review was published.

Over the last 2 years, MHRA has delivered an ambitious organisation-wide transformation to ensure it becomes a progressive and responsive patient-focused regulator of medical products. It has established a new organisational structure to improve how it listens and responds to patients and the public, and has developed a more responsive system for reporting adverse incidents, which will strengthen the evidence to support timely and robust decisions that protect patient safety. 

MHRA’s patient involvement strategy, published in October 2021, sets out how it will engage and involve the public and patients at every step of the regulatory journey. Its 5 strategic objectives include introducing clear processes for public and patient engagement and involvement, as well as embedding the public and patient voice when designing and delivering its services.

With these objectives in mind, since 2021, MHRA: 

• has been involving patients in the early stages of medical product development and encouraging the wider research landscape to do the same
• is incorporating patient views and personal experience in more of its benefit-risk reviews of medical products
• has developed a more consistent and effective approach to public consultations by introducing an enhanced and user-friendly online platform
• has launched a training programme on patient involvement specially designed for its staff and built a network of staff ‘Patient Involvement Champions’

We recognise that there is more to do to ensure MHRA delivers on its commitment to put patients first - and this is just the start of a journey. MHRA is committed to being open and transparent regarding its benefit-risk evaluation decisions, and ensuring that patients are involved in benefit-risk evaluation assessments. There have been recent examples of patients presenting on their perspectives at meetings of the Commission for Human Medicines. We are continuing to reflect on the lessons learned by these examples and further develop our plans to be more patient centred.  

MHRA is working to ensure that the future regulatory framework for medical devices will:

• improve and safeguard public health
• better assure the safety and quality of devices placed on the UK market
• deliver on the need for improved regulation of implantable devices 

MHRA is also undertaking a major investment programme to upgrade its Yellow Card safety reporting systems and has delivered several improvements to the patient journey, including using AI to support the more rapid identification of product quality defects and safety signals, and improving the user experience of the Yellow Card website.

Recommendation 6.7

Better routine capturing of ethnicity data in electronic healthcare records, alongside better collection and collation of data on medical devices in use. This would enable MHRA to conduct more rapid studies to build the evidence when a hypothesis about potential inequity in an optical device is made.

Government response

We recognise that the quality of routinely collected data can also affect quality of research and know that it will take a concerted effort across multiple stakeholders to develop appropriate data-capturing systems. This issue has been considered by different teams, including as part of nationwide Health Data Research UK work to understand data issues related to ethnicity within routinely collected data.

Building on learning from the COVID-19 pandemic, NHS England is undertaking work to improve the recording of patient characteristics in frontline services, including development of resources for frontline staff.

NHS England’s 2023 to 2024 priorities and operational planning guidance recognises the importance of improving the completeness of data on patient characteristics. This is one of the 5 strategic priorities in NHS England’s drive to reduce healthcare inequalities. We have therefore asked systems to continue to improve the collection and recording of ethnicity data across:

• primary care
• outpatients
• A&E
• mental health
• community services
• specialised commissioning

NHS England will also work to update the classification used for collection of ethnicity data to include more granular information.

Recommendation 7

Recommendation 7

A review should be conducted by the relevant academic bodies of how medical education and continuing professional development requirements for health professionals currently cover equity issues arising in the use of medical devices generally and skin diversity issues in particular, with appropriate training materials developed in response.

Government response

We agree that education and training in the healthcare field should include a focus on ethical considerations and equity issues related to the use of medical devices, as well as more widely. We believe it is important to ensure that new materials published consider equity and bias.

Succeeding at this will require a collaborative and concerted effort across education providers, professional and educational membership bodies, and NHS WT&E. Such work could build on existing work such as NHS England’s Enhancing Generalist Skills programme, which upskills clinical professionals on issues including health equity, or the AI and Digital Healthcare Technologies Capability Framework.

By integrating ethical principles and promoting health equity into medical education and continuing professional development, we can ensure that healthcare professionals are equipped with the necessary knowledge and skills to address biases in medical devices and provide equitable care to all patients. We welcome and encourage consideration of equity by organisations independent of government, including professional bodies, Royal Colleges, universities, and professional, statutory and regulatory bodies that work with universities.

Recommendation 7.1

Undergraduate and postgraduate medical and allied health professions training, including teaching clinicians about clinically relevant conditions where disease presentation differs between white and ethnic minority patients.

Government response

NHS England is incorporating equity in its plans for education and training, such as through the Enhancing Generalist Skills programme, which aims to broaden the expertise of all clinical professionals. It includes a domain focused on social justice and health equity. One learning outcome is to contribute to designing and delivering care pathways that:

• reduce inequalities
• promote inclusion
• engage and protect the most vulnerable.

We welcome and encourage collaboration on these issues with professional bodies and independent educational bodies who set curricula to ensure that issues of equity are addressed comprehensively throughout the health system. To further support these efforts, the findings and recommendations of the independent review should be disseminated among the relevant bodies that can influence training and education in the field.

Recommendation 7.2

Clinicians being made aware that, when using dermoscopy or other medical devices to examine skin lesions, clinical signs may differ according to skin tone, and their training should include images of skin lesions in all skin tones.

Government response

An audit of the NHS website found that very few pages described various skin tones and few included images of non-white skin. The NHS UK website team has developed guidance on making NHS content about skin symptoms more inclusive, and pages detailing symptoms have begun to be updated to include descriptions and images of symptoms on darker skin tones. We support the continuation of this work raising awareness of differing presentation of symptoms across different skin tones for healthcare professionals and patients.

We encourage professional bodies and independent educational bodies who set curricula to continue to improve inclusion in the education and training of healthcare professionals, ensuring that imagery used and training on skin symptom presentation covers all skin tones. 

Manufacturers should, in line with global best practice, ensure that they are explicit about the limitations of their devices on various skin tones in their instructions for use. They should ensure that, in any training they provide to clinical staff, these limitations are explained thoroughly and, where appropriate, how the limitations can be mitigated against in practice.

Recommendation 7.3

Clinicians receiving training in identifying potential sources of bias in medical devices and how to report adverse events to MHRA.

Government response

Healthcare professionals are encouraged and supported to develop relevant skills and capabilities required to work with digital healthcare. In February 2023, former Health Education England (HEE) and now NHS WT&E published the AI and Digital Healthcare Technologies Capability Framework to allow for the health and care workforce to build up skills in AI and digital healthcare technologies, which would partially help to address this recommendation. The relevant capability statement in the framework states:

I champion a culture of ethical responsibility around the use of Artificial Intelligence (AI) and digital technology to ensure that systems and processes are fair, transparent, equitable and non-discriminatory to patients, staff and the wider public; espousing the principles of beneficence, non-maleficence and autonomy.

NHS WT&E, through its NHS Digital Academy and National Patient Safety Improvement Programmes, has developed the digital clinical safety curricula, introduced a train the trainer programme and produced e-learning called Essentials of Digital Clinical Safety. This material is available through the elearning for healthcare hub and will migrate to the NHS Learning Hub during 2023 to 2024.

The curricula and subsequent learning content will be regularly reviewed and updated as appropriate, including on the risk of bias in medical devices and the process of reporting to MHRA. This training is primarily targeted at those in digital roles including digital clinical safety officers, but it is also used in levels 3 and 4 patient safety training for patient safety specialists as defined in the 2019 NHS Patient Safety Strategy.

MHRA is working to raise awareness of the Yellow Card scheme, which should be used by clinicians and healthcare professionals to report potential issues with medical devices. Yellow Card resources are available online to help healthcare professionals understand and use the system, including specific information on medical device adverse incidents and video guidance on submitting reports. MHRA also runs an annual #MedSafetyWeek to encourage engagement with the Yellow Card system.

The Yellow Card website has also been made easier to use with new search and help functions as proposed by patients. Furthermore, functionality to improve the user experience and interactive nature of the site is due to be released in the coming months.  

Recommendation 7.4

Where new devices are introduced into clinical practice, organisations and clinicians using the new devices, ensuring there is sufficient training to acquire skills and competencies before the device is used.

Government response

Individual employers have a responsibility to ensure that their staff are trained and competent to carry out their roles - it is particularly important, when introducing new devices, that staff who use the devices are appropriately trained and supervised. 

This recommendation is partially supported by the AI and Digital Healthcare Technologies Capability Framework (see our response to recommendation 7.3 above). An example capability statement is:

I recognise the need for protected time and space for professionals to access appropriate learning resources related to digital technologies. I support a learning culture and emphasise the importance of continuing professional development in digital skills and capabilities.

Recommendation 8

Recommendation 8

AI-enabled device developers and stakeholders, including the NHS organisations that deploy the devices, should engage with diverse groups of patients, patient organisations and the public, and ensure they are supported to contribute to a co-design process for AI-enabled devices that takes account of the goals of equity, fairness and transparency throughout the product’s life cycle.

Engagement frameworks from organisations such as NHS England can help hold developers and healthcare teams to account for ensuring that existing health inequities affecting racial, ethnic and socio-economic subgroups are mitigated in the care pathways in which the devices are used.

Government response

We know that involving a diverse group of people helps us make better decisions and will contribute to addressing existing health disparities.

HRA and NIHR, along with a host of organisations across the UK, have signed up to a shared commitment to public involvement to bring about changes that will drive up standards in health and social care research. 

The patient and public voice is integral to magnify the impact of the initiatives, reporting guidelines and standards mentioned in the report that function to improve the transparency and rigour of AI research in healthcare and promote diversity, inclusivity and generalisability. Such an inclusive approach ensures that the development, deployment and use of AI-enabled devices address the needs and concerns of a wide range of individuals, leading to more effective and equitable healthcare solutions.

It is important that different patient populations are considered and engaged in the development of AI-enabled devices to minimise the potential impact of non-representative training data and the misalignment of training goals with those of users. This will promote:

• efficacy - irrespective of protected characteristics
• better understanding of specific health disparities, which can be designed into the training of AI-enabled devices

Indeed, engaging with the public and patients will help inform and respond to the ethical issues (data privacy, consent and algorithmic biases) associated with AI-enabled device development. To support researchers to make excellent patient and public involvement a normal part of their research, HRA makes sure researchers are aware of their best practice principles and what they expect to see in an application for approval.

NHS AI Lab has published, in collaboration with the Ada Lovelace Institute, an algorithmic impact assessment tool that enables the involvement of patients and healthcare professionals to assess benefits and risks at an early stage of AI development, when there is greater flexibility to adjust and respond to their concerns.

Recommendation 9

Recommendation 9

The government should commission an online and offline academy to improve the understanding among all stakeholders of equity in AI-assisted medical devices.

This academy could be established through the appropriate NHS agencies, and should develop material for lay and professional stakeholders to promote better ways for developers and users of AI devices to address equity issues.

Government response

While we recognise the value of some of the sub-recommendations made, the government believes the aim of this recommendation can be achieved through alternative approaches, including some work that is already ongoing and in development.

We agree with the importance of stakeholders across the life cycle understanding issues of equity in AI-enabled devices, including manufacturers, developers and healthcare professionals. There is risk of perpetuating further bias with the development and deployment of AI technologies that are classified as software as a medical device.

We continue to support ongoing and planned work by the NHS AI Lab and NHS WT&E, among other organisations, to help ensure that healthcare professionals are equipped to navigate the ethical challenges posed by AI and digital technology in healthcare. As a first step, the NHS Digital Academy will be conducting a gap analysis of the existing Digital, Artificial Intelligence and Robotics Technologies in Education (DART-Ed) programme, which will allow us to determine whether further resources and work are required in this area.

The NHS AI Lab works in collaboration with other entities to support AI-related workforce development and transformation, including partnering with NHS WT&E (formerly HEE) to publish 2 reports relevant to this recommendation:

• a framework for understanding healthcare workers’ confidence in AI (published May 2022)
• the baseline and advanced educational requirements needed to establish confidence in AI (published November 2022)

The reports, which include considerations for algorithmic bias and health inequalities, can guide how educational and training providers, and educators of healthcare workers plan, resource, develop and deliver educational offerings to equip the workforce with the necessary knowledge, skills and capabilities. The reports have also informed NHS WT&E’s AI and Digital Healthcare Technologies Capability Framework (published February 2023), which will guide future related educational offerings. 

The continuation of this work will be finalised following the merger of HEE with NHS England, the current re-structure of teams, resources and responsibilities, and the work to be completed as part of the NHS Long Term Workforce Plan. 

The 2023 NHS Long Term Workforce Plan outlines workforce education and training initiatives including the ongoing investment in the Fellowship in Clinical Artificial Intelligence programme. The plan includes a commitment for NHS England to convene an ‘expert group’ to work through in more detail where AI can best be used and what steps need to be taken so that it supports NHS staff in the coming years. This recommendation can be considered for action by this expert group. 

The NHS Digital Academy has begun to collate educational materials for AI as part of continuing the existing DART-Ed programme and will be conducting a gap analysis based on the capabilities outlined in the AI and Digital Healthcare Technologies Capability Framework. This will allow for an accurate measure of whether this education is covered in existing material and what further resources should be made available or developed in this area, in addition to the ongoing work described here. 

Recommendation 9.1

Ensuring undergraduate and postgraduate health professional training includes the potential for AI to undermine heath equity, and how to identify and mitigate or remove unfair biases.

Recommendation 9.2

Producing materials to help train computer scientists, AI experts and design specialists involved in developing medical devices about equity, and systemic and social determinants of racism and discrimination in health.

Government response

We recognise the value of upskilling and educating all professionals who work across the life cycle of medical devices about the issues concerning bias in medical devices and potential mitigations and solutions to these biases.

As stated in response to recommendation 7 above, we encourage independent educational bodies who set curricula to ensure health equity is incorporated into undergraduate and postgraduate education on AI.

Concerning recommendation 9.2, the NHS AI Lab has produced resources for AI developers and convened a Community for Racial and Ethnic Equity in AI on the AI Virtual Hub, bringing together stakeholders across the life cycle to consider these issues.

Recommendation 9.3

Ensuring that clinical guideline bodies identify how health professionals can collaborate with other stakeholders to identify and mitigate unfair biases that may arise in the development and deployment of AI-assisted devices.

Government response

Clinical guidelines should highlight potential inequity from the use of AI, but equity within the technology itself and in its algorithms should be demonstrated by the manufacturer before it enters clinical practice. The primary purpose of guidelines is to illustrate the best practice in the care of patients - identifying how health professionals collaborate with others is beyond the remit.

We do not agree that clinical guidelines are an appropriate space to resolve these issues. Collaboration to identify and mitigate unfair biases that may arise in the development and deployment of AI-assisted devices should be done before procurement and deployment, rather than during the care of a patient. However, we are supportive of ensuring clinicians are well informed of and feel empowered to report issues through the Yellow Card system.

Therefore, this recommendation would be best achieved by upskilling health professionals through the work programmes described above.

We provide more information on different initiatives and resources for supporting healthcare professionals to ethically deploy and use AI-assisted and other digital healthcare technologies in our response to recommendation 11 - in particular 11.3 - below.

Recommendation 9.4

Encompassing an appreciation of AI within a whole-system and life cycle perspective, and understanding the end-to-end deployment and potential for inequity.

Government response

We agree with this recommendation and highlight the ongoing work referenced in response to recommendation 11, below, which supports this aim.

In our response to recommendation 11, we outline initiatives that support a whole-system approach to developing medical devices, such as the AI and Digital Regulations Service, and the work of the NHS AI Lab to provide guidance to stakeholders across the life cycle. 

Recommendation 10

Recommendation 10

Researchers, developers and those deploying AI devices should ensure they are transparent about the diversity, completeness and accuracy of data through all stages of research and development. This includes the sociodemographic, racial and ethnic characteristics of the people participating in development, validation and monitoring of product performance.

Government response

We support the drive for transparency of information to improve safety of AI medical devices, including work to improve the trustworthiness of AI products that influence clinical decisions. Ongoing and planned work in this area by MHRA, NIHR and HRA will continue to improve transparency surrounding data quality in AI-enabled devices.

Recommendation 10.1

The government resourcing MHRA to provide guidance on the assessment of biases that may have an impact on health equity in its evaluation of AI-assisted devices, and the appropriate level of population detail needed to ensure adequate performance across subgroups.

MHRA has outlined guidance focused on identifying, managing and mitigating bias in development, evaluation and maintenance of AI medical devices within their Software and AI as a Medical Device Change Programme, specifically items WP9-05, WP9-06 and WP9-07. We consider that the guidance specified here fully addresses this recommendation.

The government continues to support the appropriate funding required for the operation of MHRA, taking into account all of its functions.

Recommendation 10.2

Encouraging the custodians of data sets to build trust with minoritised groups and take steps with them to make their demographic data as complete and accurate as possible, subject to confidentiality and privacy.

Recommendation 10.3

Developers, research funders, regulators and users of AI devices recognising the limitations of many commonly used data sets, and seeking ones that are more diverse and complete. This may require a concerted effort to recruit and sample underrepresented individuals. We commend initiatives internationally and in the UK (such as the NIHR-led INCLUDE guidance) to encourage the development and use of more inclusive data sets. Data collection by public bodies must be properly resourced so that data sets are accurate and inclusive.

Government response

We support the aim to tackle data representativeness and quality issues at the source, as this will improve safety and strengthen international alignment.

The November 2023 AI Safety Summit further highlighted that AI models can amplify biases present in their training data, which can pose challenges to achieving equitable and ethical AI usage. More robust and consistent diversity monitoring of participants would enable demographic data to be used more readily to highlight areas of underrepresentation in research.

There are a number of initiatives across the UK that support diversity and inclusion in research data, including work to fulfil action 24 of the Inclusive Britain report. For example, ongoing collaborative work between HRA, MHRA, the UK government, and devolved administrations and crown dependencies aims to develop guidance for researchers on considering diversity in their trial development. This includes how ethics committees review diversity elements of proposed trials.

Several strands of NIHR work focus on achieving better diversity data regarding populations involved in research, including programmes and initiatives to improve the participation of under-served groups by age, sex, disability, location, socio-economic status, racial and ethnic diversity.

NIHR published the results of a pilot project in 2022, and committed to continuing work to understand and improve the diversity of those taking part in NIHR-funded research. A diversity monitoring question and answer set has been developed by NIHR, with the aim of improving data quality across all NIHR-funded projects, and it is developing guidance for researchers who are collecting this data.

Though in the early stages, these commitments are examples of positive work underway to improve data set diversity and representativeness. 

NHS AI Lab conducted a public dialogue in 2022 to understand how the public feel decisions should be made about access to their personal health data for AI research. This work is now informing a discovery project to design and assess the feasibility of data stewardship models that could increase:

• visibility over health data
• transparency over its use
• empowerment of patients and the public in decisions about granting access to it for AI purposes

Other example initiatives include those by Health and Care Research Wales (HCRW) who are updating the UK Policy Framework for Health and Social Care Research to provide clearer expectations on how diversity should be considered in trial design, development and delivery. Diversity considerations are already included as part of the application and assessment stages of HCRW funding calls.

HCRW is also working to build improved links with historically underserved communities to engage them in our public involvement networks, and building support processes to ‘connect’ researchers with specific communities as early as possible in the research process.

Finally, HCRW also collaborates with the NIHR INCLUDE team to run training workshops and events with the Welsh research community to provide support on engaging underserved groups in a meaningful way.

Recommendation 10.4

Data set curators, developers and regulators using consensus-driven tools, such as those by STANDING Together, to describe the data sets that are used in developing, testing and monitoring.

Government response

Consensus-generated standards will improve data quality and safety, and reduce bias within medical devices and their development. MHRA supports the STANDING Together project and will consider the outcomes of the project in its policy work for managing bias (such as Software and AI as a Medical Device Change Programme WP9-06). We consider that the guidance specified here fully addresses this recommendation. 

Recommendation 10.5

Regulators requiring manufacturers to report the diversity of data used to train algorithms.

Government response

MHRA supports the recommendation and intends to capture this within planned guidance (such as Software and AI as a Medical Device Change Programme WP9-04), but there are no current plans to implement this as a legal requirement.

The feasibility of manufacturers meeting legal requirements must be considered and the requirement must be enforced if not met. International alignment with other regulatory jurisdictions is key within the medical device space, and the impact of deviation from other jurisdictions must also be considered when modifying the legal framework.

Alongside these factors, the AI and software space is rapidly changing and, therefore, regulatory policy often benefits from a more agile approach. As such, MHRA intends to address most AI and software-related requirements in regulatory guidance, including guidance highlighting the need to consider transparency, and how the data diversity and quality must align with the intended purpose.

Recommendation 10.6

Regulators providing guidance that helps manufacturers enhance the curation and labelling of data sets by assessing bias, being transparent about limitations of the data, the device and the device evaluation, and how to mitigate or avoid performance biases.

Government response

MHRA intends to deliver these points across planned guidance documents, mainly within Software and AI as a Medical Device Change Programme WP9-04 and WP9-05.

Recommendation 10.7

Regulators enforcing requirements for manufacturers to document and publicise differential limitations of device performance and, where necessary, place reasonable restrictions on intended use.

Government response

The current regulatory requirements cover aspects of this recommendation. Manufacturers are required to document and provide relevant information to intended users in order to safely use the device. Specific aspects relating to AI will be picked up within guidance for reasons outlined in response to recommendation 10.5, above, but would not currently fall within items outlined within the Software and AI as a Medical Device Change Programme.

Additionally, there are no current plans to require manufacturers to publicise such information to the general public.

Recommendation 10.8

Making sure that the Health Research Authority and medical ethics committees approving AI-enabled device research do not impose data minimisation constraints that could undermine data set diversity or the evaluation of equity in the outcomes of research.

Government response

Careful consideration is necessary to facilitate the right balance between data minimisation and data diversity. A key covenant in ethics reviews is to ensure that individual privacy rights are respected, hence the disclosure of sociodemographic, racial and ethnic characteristics of the people participating in development, validation and monitoring of product performance must be weighted appropriately with the need to limit the exposure of sensitive information and reduce the potential misuse of personal information.

We suggest it would be more appropriate to ensure that HRA and RECs make sure they do not impose unnecessary data minimisation constraints that could undermine data set diversity or the evaluation of equity in the outcomes of research.

HRA is already undertaking work in this area. In 2022, HRA, in partnership with the devolved administrations, conducted a shared ethical debate on reviewing health and social care research involving data-driven technologies (including AI). Shared ethical debate is a quality assurance exercise designed with the purpose of reviewing consistency of ethical review across RECs. A key purpose of this exercise was to establish the current views of REC members, and identify what learning resources and guidance would be of most benefit so that RECs are able to confidently review the increasing number of research applications in this area.

This has led to the development of bespoke training units for RECs so that they can consider the following principles when reviewing AI research projects:

• transparency and explainability
• fairness
• accountability and governance
• compatibility with public interest

Recommendation 11

Recommendation 11

Stakeholders across the device life cycle should work together to ensure that best practice guidance, assurance and governance processes are co-ordinated and followed in support of a clear focus on reducing bias, with end-to-end accountability.

Government response

End-to-end accountability is a whole-system responsibility involving ‘hard’ and ‘soft’ governance, supporting and developing responsible working cultures and appropriate independent oversight. We continue to support the ongoing work of NHS England’s Transformation Directorate and MHRA in this area, including initiatives that aim to co-ordinate resources across the life cycle.  

MHRA works closely with other safety regulators and evaluation bodies within the healthcare sector to ensure co-ordination wherever possible. The recently launched AI and Digital Regulations Service brings together regulatory requirements and good practice guidance in one place for developers and adopters of AI technology, supporting a ‘whole-system approach’ and end-to-end accountability.

NHS England Transformation Directorate’s ongoing work involves developing and publishing guidance for the safe, effective and ethical deployment of AI technologies in healthcare settings, and supports cross-organisational initiatives and collaborations in the space.

Regarding end-to-end accountability, NHS England Transformation Directorate is collaborating with the Wellcome Trust and Sloan Foundation to support the Oxford Internet Institute with developing tools to assess and enhance the efficacy of AI accountability toolkits used in health and care. AI accountability toolkits enable users to confront and address potential risks like algorithmic bias. The Oxford Internet Institute will deliver a meta-toolkit for trustworthy and accountable AI consisting of technical methods, best practice standards, and guidelines designed to encourage sustainable development, usage and governance of trustworthy and accountable AI systems. 

Broader tools and initiatives that support assessment of the risks and impacts of AI technologies prior to their use can also address aspects of this recommendation. These include algorithmic and stakeholder impact assessments as designed by entities like the Ada Lovelace Institute (commissioned by the NHS AI Lab) and Alan Turing Institute.  

DHSC will also work with the Department for Science, Innovation and Technology (DSIT) to inform the development of the new central functions as outlined in the AI regulation: a pro-innovation approach white paper. This will ensure our work on devices can inform and benefit from cross-sector assessments of risks and horizon scanning in an end-to-end way.

Recommendation 11.1

MHRA adjusting its risk assessment of AI-assisted devices so that all but the simplest and lowest-risk technologies are categorised under Class IIa or higher, including a requirement for their algorithms to be suitable for independent evaluation, the use of a test of overall patient benefit that covers the risks of biased performance, and a requirement for manufacturers to publish performance audits with appropriate regularity that include an assessment of bias.

Government response

The current medical device classification requirements cover this recommendation, specifically that there are 4 risk tiers and only the lowest-risk products are below Class IIa - that is, Class I. AI medical device products fall within this risk framework and would meet this recommendation currently. 

The public consultation on changes to the medical device regulatory framework outlined their intention to align more closely with IMDRF guidelines, which include a Class I risk category. Such changes would continue to meet this recommendation.

Products of risk Class IIa or higher are subjected to independent evaluation by a recognised approved body. Therefore, suitability to undertake such assessment would be covered by the current regulations, and this would include an assessment of product performance within its intended use population. Currently, MHRA’s guidance on Crafting an intended purpose for Software as a Medical Device (SaMD) specifies consideration of performance across subgroups of the intended use population. Additional guidance is planned within the Software and AI as a Medical Device Change Programme to further clarify the need to minimise risks to the intended population, including those resulting from unwanted bias. 

Additionally, the current regulatory and quality systems used under the medical device framework require audits to be performed regularly. While the publication of this information is not enforced, MHRA intends to encourage this behaviour by highlighting the benefits within planned guidance for manufacturers and approved bodies.

We recognise that our approach to risk assessments can feed into wider cross-government AI-related risk assessment activity. DHSC will support and input into the development of the central AI risk monitoring function being developed by DSIT to ensure the specific challenges of equity in medical devices inform the government’s systematic assessment of risks.

Recommendation 11.2

Supporting health professionals’ involvement early in the development and deployment of AI devices. We commend the use of ethical design checklists, which may assist in the quality assurance of these processes.

Government response

We agree with this recommendation, and it is fully supported by ongoing work.

Involving health and care professionals at early stages in the development of AI and data-driven technologies is essential to ensure that they are designed with the needs of patients and healthcare providers in mind, and is best practice.

The NHS AI Lab is supporting, through published guidance, healthcare professionals’ ability to engage and decide on related issues when deploying AI and, by late 2024, will publish a meta-toolkit for trustworthy and accountable AI. 

To date, NHS England’s Transformation Directorate has published 2 resources, which provide advice to developers and adopters of AI:

• firstly, to support developers to understand the needs of the NHS when thinking about building AI solutions, the AI Lab developed a guide to good practice for digital and data-driven health technologies, which includes ethical considerations
• secondly, the AI Lab assisted the creation of principles to support the development and deployment of AI or machine learning-enabled medical devices across jurisdictions (see Deliverable 2)

The recently launched AI and Digital Regulations Service provides adopters of AI with guidance to make well informed decisions about buying or adopting digital technologies in health and social care.

The Developing healthcare workers’ confidence in AI report highlights the need for good user design of these technologies to support confidence and the Fellowship in Clinical Artificial Intelligence programme provides a good example of an approach that allows for clinicians to be involved with AI development and implementation from the start.

By involving healthcare professionals in the development and implementation of AI technologies from the start, we can ensure that these tools are designed with the needs of patients, and that best practice guidance, assurance and governance processes are co-ordinated and followed, with a clear focus on reducing bias, as well as end-to-end accountability.

The learning developed both through the educational programme within the fellowship and the experiential learning will allow for a clinical workforce that is confident and competent in deploying AI technologies for the benefit of patients. We continue to support programmes and initiatives such as this to fulfil this recommendation.

Recommendation 11.3

Manufacturers adopting MHRA’s Good Machine Learning Practice for Medical Device Development: Guiding Principles.

Recommendation 11.4

All stakeholders supporting MHRA’s Software and AI as a Medical Device Change Programme Roadmap, such as promoting the development of methodologies for the identification and elimination of bias, and testing the robustness of algorithms to changing clinical inputs, populations and conditions.

Government response

We agree with recommendations 11.3 and 11.4, and welcome calls for additional support for delivered and planned guidance within MHRA’s Software and AI as a Medical Device Change Programme by stakeholders across the device life cycle. 

As part of this, we encourage the adoption of Good Machine Learning Practice for Medical Device Development: Guiding Principles, which was jointly created and published by MHRA with international regulatory partners the US FDA and Health Canada. 

Recommendation 11.5

Placing a duty on developers and manufacturers to participate in auditing of AI model performance to identify specific harms. These should be examined across subgroups of the population, monitoring for equity impacts rather than just unequal performance.

Government response

MHRA recognises and supports the importance of ensuring products perform fairly across their intended population. There are existing requirements within medical device regulations for manufacturers to review their products within quality system requirements, risk management requirements and PMS.

Where a conformity assessment is required, manufacturers are also subject to periodic surveillance auditing by their approved body.

Examination of performance across relevant subgroups will be addressed in Software and AI as a Medical Device Change Programme guidance (WP9-05) by MHRA.

There is already a mechanism for monitoring equity, under existing PMS systems, within the scope of the intended use of the device.

Recommendation 12

Recommendation 12

UK regulatory bodies should be provided with the long-term resources to develop agile and evolving guidance, including governance and assurance mechanisms, to assist innovators, businesses and data scientists to collaboratively integrate processes in the medical device life cycle that reduce unfair biases and their detection, without being cumbersome or blocking progress.

Government response

We agree that an agile approach is required to deal with the ever-changing needs of software and AI, with regulatory bodies working collaboratively to create a sustainable development approach. The government continues to support the appropriate funding required for the operation of MHRA, taking into account all of its functions.

As set out in the AI regulation: a pro-innovation approach white paper in March 2023, the government is taking an agile, flexible, pragmatic and proportionate approach to AI regulation across all sectors, empowering sector-specific regulators to implement principles including safety, fairness and transparency. 

We will continue to support MHRA’s work in this area, building on the Software and AI as a Medical Device Change Programme Roadmap. 

Recommendation 13

Recommendation 13

The NHS should lead by example, drawing on its equity principles, influence and purchasing power, to influence the deployment of equitable AI-enabled medical devices in the health service.

Government response

The government agrees that there are opportunities for equity to be considered within NHS procurement and deployment processes in line with regulations. A proportionate approach that does not excessively limit the number of devices available in the NHS is required.

Implementation of this recommendation, and the related sub-recommendations, would require careful consideration and planning to ensure the best approach is taken to achieve the recommendation’s aim.

Recommendation 13.1

NHS England and the NHS in the devolved administrations including a minimum standard for equity as part of the pre-qualification stage when establishing national framework agreements for digital technology.

Government response

There are some examples of suppliers investing more time and resource into developing data sets and algorithms to combat risks of differential analysis than others but, without the support of a national standard, they can struggle to persuade customers of the added value.

We recognise the importance of ensuring equitable access to medical devices and digital health technologies. DHSC will work with NHS England to understand whether developing a minimum standard for equity as an entry requisite for national framework agreements will be practicable for digital health technology. 

The NHS Central Commercial Function is developing a strategy and playbook to improve the procurement standards across the NHS, including with national framework providers, and will include an agreed minimum standard for equity for the NHS in England as part of this work.

Beyond digital and AI-enabled technology, we believe it is important to consider this issue more broadly for procurement of other medical technologies. Where a minimum standard for equity is developed and agreed as the national standard by the national bodies, NHS Supply Chain will incorporate this standard into its tender processes for procuring medical devices for the NHS.

When developing category strategies, NHS Supply Chain ensures that, for a specific category, all relevant national standards, guidance, policies and regulations are incorporated into strategies, and this informs the criteria for the evaluation of tenders. Suppliers responding to NHS Supply Chain tenders looking to be awarded onto frameworks must meet the criteria set in the tender documentation.

NHS Supply Chain works in collaboration with the DHSC, NHS England, and the devolved administrations and crown dependencies on its supplier engagement activities. It conducts regular engagement with trade, and thus can highlight the issues and encourage suppliers to ensure that existing health inequalities affecting racial ethnic and socio-economic subgroups are considered in the design process and, when in use, in the NHS. 

It should be noted that many public bodies use the open procedure when conducting procurement exercises to set up frameworks. The pre-qualification stage does not exist when using the open procedure as this is a one-stage procurement process.

Recommendation 13.2

NHS England updating the digital technology assessment criteria used by health and social care teams when buying digital technology to recommend equity as part of the pre-purchase validation checks.

Government response

This would require a bespoke update to the digital technology assessment criteria, and NHS England will need to consider whether the digital technology assessment criteria is the best vehicle for this purpose.

Recommendation 13.3

Working with manufacturers and regulators to promote joint responsibility for safety monitoring and algorithm audits to ensure outcome fairness in the deployment of AI-assisted devices. This will require support for the creation of the right data infrastructure and governance.

Government response

As the UK statutory regulator for healthcare products, MHRA’s remit extends to all AI-enabled medical devices placed on the UK market and their legal manufacturers. However, many AI-enabled medical devices are deployed via the health services of England and the devolved administrations - particularly devices intended to support clinical decisions, and requiring data generated and housed within NHS infrastructure.

The current regulatory framework for medical devices covers requirements for manufacturers to conduct safety monitoring activities under PMS. Additionally, the secondary legislation to strengthen this component of the UK Medical Devices Regulations 2002 (as amended) is underway. Both manufacturers and their approved bodies must conduct process and product audits over the lifetime of the product in order to maintain compliance with quality and regulatory requirements.

While MHRA’s remit does not extend to health and social care providers or members of staff (unless they also qualify as manufacturers), it supports the promotion of education and activities that improve the safety of medical devices in use. MHRA is a partner member of the NHS England-funded AI and Digital Regulations Service, which provides advice and signposting to adopters of AI technology. The AI and Digital Regulations Service project website contains information on safety monitoring and this is a priority focus area of its pathway co-ordination forum.

MHRA’s Software and AI as a Medical Device Change Programme has items under WP 9 AI RIG (AI Rigour) that highlight the importance of safety monitoring and algorithmic auditing for AI as a medical device. More widely, MHRA is supporting NHS AI Lab in its AI Deployment Platform project, which will provide some of the essential infrastructure and standardisation for testing AI as a medical device product in radiology at scale.

NICE provides guidance and best practice on AI as a medical device, assessing cost and clinical effectiveness of health technologies for national-scale adoption. Most AI products do not yet have the body of evidence required for NICE to be confident in their benefits and to be considered for national-scale adoption. However, there are a growing number of products that have received MHRA classification and are considered technically safe to use, and are therefore being adopted at smaller scale within the NHS.

To address the issue of a poor evidence base, in 2022 NICE launched an early evaluation approach (called early value assessment), which identifies what additional evidence these technologies should gather, as they are being used in local pilots, to ensure they could be recommended for national adoption by NICE in the future.

Recommendation 14

Recommendation 14

Research commissioners should prioritise diversity and inclusion. The pursuit of equity should be a key driver of investment decisions and project prioritisation. This should incorporate the access of underrepresented groups to research funding and support, and inclusion of underrepresented groups in all stages of research development and appraisal.

Government response

We agree that the inclusion of under-represented groups is an important factor in research project prioritisation decisions. NIHR has developed sessions for researchers on inclusion in the research cycle, along with a range of guidance and toolkits.

Alongside this, Inclusive Britain commits the government to increasing ethnic minority participation in trials, through measures such as the government-funded INCLUDE framework and this will satisfy any recommendations to factor in equity.

HRA supports research ethics committees to consider health equity impacts. The government supports the continuation of this work to fulfil this recommendation and ensure inclusion in research.

Recommendation 14.1

Requiring that AI-related research proposals demonstrate consideration of equity in all aspects of the research cycle.

Government response

We agree with this recommendation, but note that there are a range of drivers for research investment decisions, such as need and quality, that are also important.

AI-related research proposals should demonstrate consideration of equity in all aspects of the research cycle. From 2023 to 2024, NIHR will deliver quarterly sessions to researchers relating to inclusion in the research cycle.

There are several tools available that support this, such as the INCLUDE guidance, the Equality, Diversity and Inclusion (EDI) Toolkit and Focus on Research and Equity (FOR-EQUITY).

Recommendation 14.2

Ensuring that independent research ethics committees consider social, economic and health equity impacts of AI-related research.

Government response

The consideration of social, economic and health equity impacts of AI-related research by independent RECs is essential for ensuring responsible and ethical development and deployment of AI technologies.

To minimise the exacerbation of inequity, independent RECs must evaluate whether the benefits of AI-related research are distributed equitably and that any negative consequences are addressed responsibly. Public involvement in the design and conduct of research plays a key role in ensuring that these issues are considered throughout the research process. Independent RECs should ensure that researchers and developers have embedded public involvement into their projects.

NIHR and HRA both support RECs to consider social, economic and health equity impacts of all healthcare research activities, including for AI-related research. The Governance arrangements for Research Ethics Committees for HRA and the devolved administrations and crown dependencies state that any proposed research must be ethical and worthwhile, while any risks, burdens or intrusion will be minimised for the people taking part in research and are justified by the expected benefits for the participants or for science and society.

The RECs aim to protect people who take part in research, earning public confidence in the conduct of researchers, and safeguarding the dignity, rights, safety and wellbeing of research participants.

Recommendation 15

Recommendation 15

Regulators should be properly resourced by the government to prepare and plan for the disruption that foundation models and generative AI will bring to medical devices, and the potential impact on equity.

A government-appointed expert panel should be convened - made up of clinical, technology and healthcare leaders, patient and public involvement representatives, industry, third sector, scientists and researchers who collectively understand the technical details of emerging AI and the context of medical devices - with the aim of assessing and monitoring the potential impact on AI quality and equity of LLM and foundation models.

Government response

Ensuring that regulators are prepared for the inevitable disruption from new AI technologies is a key priority for government, and we believe that encouraging cross-sector collaboration is the best approach to achieve the aim of this recommendation.

As set out in the 2023 AI regulation: a pro-innovation approach white paper, the government is putting in place a range of measures that are designed to support regulators in addressing the risks and challenges posed by new AI technologies. These measures include:

• guidance on regulatory principles for AI
• access to central risk and horizon-scanning functions
• central support for regulators seeking to develop their AI capabilities and skills

It is essential to have mechanisms in place to ensure that AI technologies (in particular generative AI and LLMs) are developed and deployed responsibly, ethically and with consideration for their impact on society, especially because such technologies continue to advance very rapidly. Government’s agile approach to regulation, as set out in the AI regulation: a pro-innovation approach white paper in March 2023, will allow rapid assessment of and changes to regulation as needed. Our response aligns with the aims of MHRA’s Software and AI as a Medical Device Change Programme to ensure clear and protective regulatory requirements for software and AI in healthcare.

Horizon scanning is central for monitoring the emergence of new AI model disruptions and capabilities. DHSC will support and inform the work of DSIT to establish a central AI risk monitoring function as outlined in the AI regulation: a pro-innovation approach white paper. Where relevant, DHSC will also engage with the AI Safety Institute as it continues to build the technical foundations for AI research. This will ensure DHSC can access strategic assessments of emerging issues to ensure regulators are supported to respond proportionally in relation to the implications for AI-enabled medical devices.  

Within the healthcare context, HRA provides expert advice to the Office of AI on the privacy, legal and ethical considerations of generative AI models in a research context for healthcare.

The AI and Digital Regulations Service advocates for transparency in the development of such technologies and suggests guidelines, regulations or best practices to ensure maximum societal benefits.

Recommendation 16

Recommendation 16

The focus of PRS studies should be widened beyond genetic diversity to include:

• the contribution of the social determinants of health - including lifestyle, living and working conditions, and environmental factors such as air pollution - to overall disease risk
• how these affect the predictive potential of PRS among different ethnicities and socio-economic groups

Developments with this wider research focus should aid the refinement of overall risk assessments so that they better reflect the role that PRS play alongside non-genetic risk factors.

Government response

Use of PRS in predicting overall disease risk is dependent upon diverse, reliable and consistent data being readily available to researchers at the initial planning phases of research. The government will continue to support initiatives to ensure diversity in genomic research, such as Our Future Health and Genomics England’s Diverse Data initiative.

We recognise the importance of taking a ‘beyond genetics’ view of disease risk, as exemplified by the work of UK Research and Innovation on its Securing better health, ageing and wellbeing strategic theme. This work is taking a holistic, 360-degree perspective of the biological, social, cultural and environmental influences on the physical and mental health and wellbeing across the lifespan, to strengthen prevention and interventions, reduce health disparities and improve human health.

NIHR welcomes funding applications for research into any aspect of human health, and several of NIHR’s funding programmes may be suited to applications from researchers looking to undertake PRS-focused studies, including researcher-led HTA funding, the EME programme, and the Invention for Innovation (i4i) programme.

As with other government funders of health research, NIHR does not allocate funding for specific disease areas, except in the case of commissioned calls. The level of research spending in any particular area is driven by multiple factors, including scientific potential, and the number and scale of successful funding applications.

Recommendation 17

Recommendation 17

National research funders should commission a broad programme of research and consultation with the public, patients and health professionals to fill the gaps in knowledge and understanding concerning PRS. The programme should cover:

• the public’s understanding of the nature of genetic risk and the meaning of the PRS they are presented with
• explorations of how health professionals interpret these risks, and can best communicate and support people in understanding the results of their PRS

The research programme should cover impacts on diverse population subgroups, and be informed by extensive engagement with the public and patients to gain their perspectives.

Results from this research programme, together with actions on recommendation 16, should feed into the development of clinical applications for PRS medical devices covered in recommendation 18.

Government response

Public understanding of genomics and the communication of risk, particularly among diverse groups, are important considerations as genetic testing is developed and introduced. Recently published research jointly funded and carried out by Genomics England into The legacy of language: what we say, and what people hear, when we talk about genomics is an example of work in this area to begin to understand these issues.

We value the review’s emphasis on highlighting key issues concerning equity in PRS, many of which are relevant to genomic research and clinical applications more broadly. As part of the Diverse Data initiative at Genomics England, several of the major programmes will further our understanding and take action based on the review’s recommendations. These programmes will include a significant public engagement effort that focuses on underrepresented groups in genomics to assess attitudes and understanding of new and emerging methods in genomics research.

Additionally, in collaboration with the NHS Race and Health Observatory and clinical colleagues across the UK, Genomics England will aim to develop a common framework to identify and address potential sources of bias in care, and raise awareness of the equity challenges in genomics.

Recommendation 18

Recommendation 18

UK professional bodies - such as the Royal Colleges and health education bodies across the UK - should develop guidance for healthcare professionals on the equity and ethical challenges and limitations of applying PRS testing in patient care and population health programmes.

This guidance should:

Recommendation 18.1

Include the interpretation of risk scores, communicating risk to patients and the public, and counselling and support.

Recommendation 18.2

Be informed by extensive public and patient engagement.

Government response

PRS are an additional tool to potentially improve outcomes - however, they can also increase health disparities. Most genomic studies have analysed European ancestry -therefore, PRS may not be as accurate on populations from other ancestries.

PRS are not yet used in the NHS - however, if they are implemented, appropriate guidance should be developed and made available to health professionals.   

Calls to action

Call to action 1

These recommendations need to be implemented as a matter of priority with full government support.

Government response

The government is grateful to the review for making these recommendations and values their views. We agree with the vast majority of the recommendations and, as set out in this response, are taking action across government and our ALBs to fulfil these.

We have also indicated where we determine there are alternative means to achieve the aims of the recommendations.  

Work to tackle health inequalities will need to be an ongoing priority across government and the health system as new technologies and issues emerge. Any potential further interventions would need to be considered at the next Spending Review.

In the next section, we set out some initial steps we will be taking to support the implementation of the recommendations and ensure equity is considered across the life cycle of medical devices.  

Call to action 2

Addressing inequities in access is therefore an essential task for the government and leadership of the NHS.

Government response

Addressing inequities in access to medical devices and technology is important to the government and the NHS, and dedicated work in the space is already underway.  

Our recently published medical technology strategy aims to ensure people across the UK’s diverse population have access to effective and safe medical technology, regardless of ethnicity, sex or any other characteristic. This strategy is for England, and there is strong collaboration and link-up with the devolved administrations and crown dependencies on this work, which the government will continue to support. We will publish an implementation plan to deliver on this strategy in 2024.

The government continues to support NHS initiatives, such as the Innovation for Healthcare Inequalities Programme, which aims to address local healthcare inequalities by improving access to the latest health technologies and medicines.

We are also actively tackling digital exclusion, such as through the Widening Digital Participation programme and digital skills training in local communities, with anticipated benefits to equitable access of digital health technology.

Call to action 3

A review should be carried out of equity in the medical devices encountered during pregnancy and the neonatal period, as part of the wider investigations of health outcomes for ethnic minority and poorer women and their babies.

Government response

The safety of maternity care is a priority for the government, and we have commissioned Donna Ockenden to lead an independent review of cases of concern in maternity services.

However, we recognise the need to go further, and that research is needed to understand whether there is equity in the use of medical devices encountered during pregnancy and the neonatal period. We will duly consider the proposed call to action with a view to commissioning research through NIHR.

Conclusion

As set out in this report, there is a considerable amount of work already underway, initiated by this government, which address many of the essential elements of the independent review’s recommendations.

Monitoring and evaluation are key cornerstones of success and understanding how to improve. The programmes and initiatives cited in this report all engage in monitoring and evaluation, and so their contribution is two-fold - as well as generating direct change and contributing to increased equity and a fairer healthcare experience, the learnings from these programmes will be invaluable to government.

The medtech strategy sets out that, as part of the ambition to deliver the right product, for the right price, in the right place, medical devices must be safe and clinically effective for all. We will consider equity issues throughout the medical device life cycle in the implementation of the strategy.

To support the progression of the recommendations, DHSC will work alongside counterparts from the devolved administrations and crown dependencies, and meet regularly to monitor the advancement of this work and look to future developments.

These are all positive steps towards making medical devices equitable in design and use, but government cannot do this alone. It is important that we are able to work with partners in trade, industry, education and across healthcare to embed best practice and support the NHS in its mission to find the best and most appropriate products so that it can deliver the best possible care.