Guidance

​​Guidance on accessing and using comparable datasets for UK government officials​

Published 11 April 2024

Foreword

Good data and evidence are the bedrock of policy making. Analysts and policy professionals across government know that to deliver the best outcomes for the citizens of the UK, policy decisions must be rooted in robust and dependable data and evidence. Improving this evidence base and deepening our understanding of how to deliver the best outcomes is an agenda shared by decision makers at all levels of government across the UK. 

Across the UK we have a particular opportunity to strengthen our evidence base. Devolution allows different levels of government to develop creative and often innovative approaches to addressing shared challenges.

This offers great potential to draw learning from these different approaches and better understand what works in different contexts. However, this potential is not always fully realised without active efforts to assemble and analyse data. Without this, policymakers cannot draw lessons from innovative approaches taken across Scotland, Wales, Northern Ireland, and England. 

This, of course, is not a new challenge, nor one that only impacts policymakers within government. The National Statistician previously commissioned work to consider how best to maintain comparability and coherence between the official statistics published across the UK, resulting in a report, Comparing Official Statistics across the UK (PDF, 635 KB), produced by the Government Statistical Service.

However, there is refreshed focus on responding to this challenge. In 2021, the UK government and the Devolved Administrations signed a refreshed Concordat on Statistics in 2021 committing to a set of principles underpinning the production of coherent, comparable statistics at the UK and devolved levels; recognising that the policy context within administrations will not always be identical.

With that in mind, UK government has produced this guidance to support colleagues to maximise the value of the vast data and analysis produced across the UK. We hope this guidance will serve as a valuable tool for analytical colleagues across the whole of the government, providing teams with straightforward, targeted actions to improve their access to UK-wide evidence, and help colleagues communicate the importance of UK-wide evidence to other professions and colleagues more widely.

Stephen Aldridge
Director for Analysis and Data
Department for Levelling Up Housing and Communities

1. Introduction

Good data is essential to the design of good policy, and to its monitoring and evaluation through delivery. Devolution across the UK allows policymakers to take different approaches to shared public policy challenges in devolved areas. Having data from across the UK, to analyse the impact of these different approaches on public sector outcomes, offers crucial insight. It allows us to better design and deliver policies that benefit people across the UK, harness the full benefits of policy experimentation across all parts of the UK, and maximise the value of our datasets.

The UK government, alongside the devolved administrations, are committed to improving access to comparable UK-wide data for the production of high-quality statistics that serve the public good. Colleagues are working in close collaboration across the UK government and devolved administrations and beyond to ensure data is coherent, comparable, and fully utilised by analysts and policy professionals alike to effectively inform and evaluate public policy.

In recognition of this, the UK and devolved administrations signed a refreshed Concordat on Statistics in October 2021. Whilst recognising different policy contexts and local needs, this agreement commits administrations to working together on shared data priorities. It is owned primarily by the UK Statistics Authority (UKSA), a non-ministerial department. The Government Statistical Service (GSS) Coherence Work Programme is one element of the work already underway which delivers on the spirit of the Concordat. 

This guidance is designed to be used primarily by analytical professionals working in the UK government, to understand and deliver against their responsibilities under the Concordat on Statistics and the GSS Coherence Work Programme. In particular, the guidance is to help colleagues to (i) better understand the policy rationale for comparable UK-wide data, and (ii) consider potential routes to improve the comparability of data across the UK in certain policy areas. Reflections from research conducted by Government Statistical Service colleagues are highlighted in case studies throughout this guidance.

We recognise the resourcing challenges faced by analytical teams, which limit UK government departments’ capacity to address data coherency issues in the short term. However, by not investing in short term actions to improve data comparability across the UK, we risk a weaker evidence base in the long term. Ironically, this means administrations use up more analytical capability to interrogate administration-specific data or international evidence, duplicating efforts rather than benefiting from the learning that can come from interrogating comparable data from across the UK to identify lessons learnt. With that in mind, this guidance is designed to provide teams with straightforward, targeted actions to improve their access to UK-wide evidence, and to help colleagues communicate the importance of this to policy staff.

Engagement between the United Kingdom Government, the Scottish Government, the Welsh Government, and the Northern Ireland Executive rests on mutual respect and collaborative relationships. As analytical professionals, we all share a responsibility for building and maintaining trust through effective communication; sharing information, research and evidence; respecting confidentiality; and resolving disputes according to a clear and agreed process accepting that each administration will be different by different priorities that also impact resourcing. This guidance provides specific suggestions to improve the quality of data, respecting these collaborative inter-governmental relationships.

Going further: National Data Strategy

Improving access to and use of comparable UK-wide data is one of the UK government’s key data priorities. At the highest level, the National Data Strategy sets out the UK government’s broader approach to the governments collection, storage and use of data. Mission Three of the strategy covers the UK government’s use of data in public policy making it  clear that it is  only by collaborating across borders (both institutional and geographical) to harmonise data concepts, definitions and variables that we can  truly harness the power of data available to government.

 The Department for Science, Innovation and Technology maintains overall responsibility for the delivery of the National Data Strategy while Mission Three of the strategy is owned primarily by the Central Digital and Data Office (CDDO) – part of the Cabinet Office.

The National Data Strategy sits alongside the 2025 Roadmap for Digital and Data, also produced by the CDDO, which proposes a cross-government vision of our use of data in 2025. Like the National Data Strategy, it also commits to improving data in decision making.

2. What do we mean by ‘comparable UK-wide’ data?

By comparable UK-wide data we mean statistics, evidence, and analysis which can be meaningfully compared at different levels of government across England, Scotland, Wales, and Northern Ireland – data that lends itself to better analysis and can be used to draw policy lessons on what works. Comparable ‘UK-wide’ data does not mean data that is only produced at a UK-wide level; collecting and analysing data at granular levels for use by policy makers at every level of government is equally valuable. Examples of comparable UK-wide data might include:

a. Data that is already collected and analysed on a UK-wide basis – for example if the policy is reserved and the UK government retains the responsibility for collecting data and evidence for example employment rates. For this data, it is often useful to disaggregate the data and look at it through a country, subnational, or interregional lens; for example, by using the ONS’ subnational indicator explorer to compare Local Authorities’ employment rates.

b. Data that is collected and published separately by UK government, the devolved administrations, local government or wider delivery partners, but can be put together to provide a UK-wide insight. This can be:

• Data that is collected separately but where the definition and methodology are closely enough aligned that direct comparison is appropriate for example, court case timeliness is published separately by HM Courts & Tribunals Service, Scottish Government and the Northern Ireland Department of Justice.

• Data where direct comparison is not immediately possible because significant definitional or methodological differences exist but where some effort to achieve statistical standardisation allows for meaningful comparison for example pupil attainment. We know that there can be methodological, technical, legislative, or data privacy challenges in assembling comparable UK-wide data, particularly where data is collected in different ways across the UK. Close cooperation and collaboration across the UK will enable us to address these challenges in the most effective way.

3. Why is comparable UK-wide data in devolved policy areas important?

Collaborating across the UK to create coherent UK-wide data is critical to informing public policy decisions, building the evidence base on what works and improving outcomes for citizens. While the data needs of each administration may not always be identical, comparable UK-wide data allows us to learn important lessons from different policy approaches across the UK. The Concordat on Statistics commits us to working together to identify such opportunities. There are several key reasons why comparable UK-wide data offers such valuable insight:

a. Comparable UK-wide data enables us to bring together evidence and learning from across the UK, identifying what works and allowing policy makers at all levels of government to tailor interventions accordingly. Even for colleagues working exclusively in areas that are entirely devolved or reserved, improved access to comparable UK-wide data enhances practitioners’ understanding of what works, allowing them to evaluate policy outcomes against the outcomes of different policy approaches elsewhere in the UK. This additional layer allows colleagues, where appropriate, to apply successful policy approaches for the benefit of citizens within their own administration.

b. In the course of implementing policy, it allows us to monitor the effectiveness of our interventions in real time, ensuring they deliver maximum benefit for citizens across the United Kingdom. The UK’s COVID-19 response demonstrates the importance of comparable UK-wide data in directing government resources – particularly in times of national emergency.

c. It underpins and supports the principles of intergovernmental relations (IGR) from official-level engagement to ministerial-level intergovernmental discussions. It enables ministers from across the UK to conduct more impactful conversations, learning from the different approaches taken to tackling common challenges and drawing informed conclusions.

d. It maximises the value of datasets we collect and hold in the course of routine policy delivery. By contextualising data alongside comparable statistics from elsewhere in the UK, analysts are able to extract insights that otherwise would not be possible.

e. By improving access to comparable UK-wide data, we can meet the increasing demand for evidence in policy design and public spending. Without developing the data required for evaluating and evidencing what works across the UK, we run the risk of assuming that impacts of interventions in e.g., the South West of England will work the same way in the North East or in coastal Northern Ireland.

f. Comparable UK-wide data enables us to transparently demonstrate to the public the value for money of interventions and improvements in outcomes.

g. By looking at data across the UK, analysts may be better able to identify datasets (e.g., longitudinal data) that offer insights into specific areas that they did not know otherwise existed.

Case study: Homes for Ukraine

Following the Russian invasion of Ukraine in February 2022, comparable UK-wide data was critical to DLUHC’s Homes for Ukraine (HfU) Sponsorship Scheme to enable the operation of the scheme to be monitored in its entirety and on a consistent basis across the UK.

By collaborating early with counterparts across the UK, analytical colleagues working on the programme were able to generate local authority-level management information quantifying the geographical distribution of visa applications, visas issued and arrivals in the UK by sponsor locations. This provided transparency for interested citizens, as well as enabling policymakers to monitor the operation of the scheme within each of England, Scotland, Wales, and Northern Ireland, taking account of different sponsorship arrangements and other circumstances within the different parts of the UK.

For example, data published on Homes for Ukraine visa data by location.

Summary table: data as received from the Home Office as at 18 April 2023

Location	Sponsor	Number of visa applications	Number of visas issued	Number of arrivals in the UK by sponsor location
England	Sponsored by individuals	129,866	107,361	88,814
Northern Ireland	Sponsored by individuals	2,807	2,165	942
Scotland	Sponsored by individuals	7,061	5,753	4,661
	Sponsored by Scottish Government	38,304	32,588	19,602
	Total	45,365	38,341	24,263
Wales	Sponsored by individuals	5,070	4,272	3,536
	Sponsored by Welsh Government	5,349	4,613	3,126
	Total	10,419	8,885	6,662

The lack of comparable UK-wide data does not impact the UK government and devolved administrations alone. Open data is critical to transparency – an absence of comparable data limits accountability, diminishing public understanding of the evidence behind policy decisions, and trust in decision making. Moreover, we know that many external stakeholders such as academics and think tanks have a need for more comparable UK-wide data, particularly on public services.

Case study: Exploring the user need for coherent data across the UK

Independent think-tank, the Institute for Government (IfG), is clear on the need for coherent data. In their 2016 report, Devolution as a policy laboratory, they explain:

At the most basic level, comparable data is a prerequisite for useful evidence exchange. It is important that data on policy outcomes is collected in a comparable way across the UK and at the right spatial level to enable useful comparisons.

The IfG is one of many external organisations who stress the importance of comparable UK-wide data to their work, and the impact when that data is incomplete or missing.

The UK Collaborative Centre for Housing Evidence emphasised the importance of collaboration for devolved policy areas:

…because housing’s devolved, it’s making sure that the different data controllers in the different nations talk to each other about housing data…

The Social Mobility Commission also highlighted difficulties when data is not coherent across the UK:

Our job is to monitor and to make recommendations and without the first piece of that, we can’t really effectively do the second piece. Monitoring across the UK just becomes a lot harder when we don’t have that consistent data, or we’ve got things missing.

Homelessness charity Crisis spoke of their need to understand how homelessness data can be joined up across Great Britain, emphasising the communication challenges posed by a lack of comparable data.

We are a GB organization, one of the things I’m always very aware of is we’re often asked about the state of homelessness in the UK and at GB level, when actually homelessness is a devolved policy issue, it’s about trying to look at to what extent we can draw those data sets together.

4. Assembling comparable UK-wide data: new data

Provided the time and departmental resources, the most direct route to assembling a comparable UK-wide dataset may be through a new data collection. When considering new UK-wide data collections in devolved policy areas, you must engage with chief statisticians in devolved administrations from the outset to ensure that new statistics are mutually beneficial, and that your work is not duplicative.

Before embarking on an altogether new data collection, first consider the alternative means to achieving your data needs. In many cases, you will find that existing data, either within government or beyond, can provide an equally useful insight. A non-exhaustive list is provided in the section, ‘Assembling comparable UK-wide data: existing data’ below.

As set out in paragraph 30 of the UK Concordat on Statistics: “Permission for the ONS to publish devolved statistics is subject to section 20 of the Statistics and Registration Service Act 2007. Although not enshrined in legislation, in the spirit of continued co-operation all administrations will adhere to the principle that they do not publish any statistics in relation to the functions of another administration without seeking consent and ensuring sufficient time is given for quality assurance and, if appropriate, pre-release access.”

While consent is not a legal requirement for most departments, it is strongly recommended that you obtain the agreement of the respective devolved administrations. You should engage with the Chief Statisticians across the UK if you want to produce and publish statistics on devolved matters. To do so, we recommend that you provide information outlining: the publication, planned dates, what the publication will cover, and ask for comments or questions.

Engaging with devolved administrations – engaging with your counterparts

Establishing and maintaining a close, trusting, and productive relationship with your devolved administration counterparts is critical to building your understanding of the barriers to data comparability. It is the first step to agreeing joint work to improve the availability of coherent data in your work area.

If you are not already personally familiar with your devolved administration counterparts, you should reach-out proactively to introduce yourself and your work. If you do not know who your counterparts are, or have no immediate way of identifying them, the territorial offices, the ONS, your policy colleagues, or your department’s devolution team will be able to help. Use them to identify a first point of contact in the respective administration and work from there. Bear in mind that each administration is different – the portfolio responsibilities that you hold in your role may not be precisely replicated in the devolved administrations, and your counterpart will not necessarily be one person.

For gaining devolved administration consent, your department’s GSS Head of Profession should be able to help you as they can lead on engaging with the Chief Statisticians of the devolved administrations.

Here are some of the things to consider when looking to collect new data:

a. Where there are shared policy ambitions across the UK, you could explore setting up a new data collection to be UK-wide, or comparable across the UK government and the devolved administrations, that is mutually beneficial. Joint data collections could be more efficient, as well as supporting comparability.

b. Where UK government departments are considering collecting new datasets, you should consider the merits and value-add of collecting data UK-wide, unless there is a clear reason why this isn’t appropriate. You should explore how similar data is collected and presented elsewhere in the UK and consider aligning from the outset or collecting additional data points to allow comparison.

c. When collecting new data, you should ensure that sample sizes are large enough to make meaningful comparisons across the UK. Additionally, the Welsh language legislation requirements should be considered when collecting data via survey work, by providing questions and allowing responses in Welsh.

d. There are several funds across government that can help with demonstrating the value of  new data sources. This includes HM Treasury’s Economic Data Innovation Fund.

5. Assembling comparable UK-wide data: existing data

Identify the data

The first step in thinking UK-wide is to identify what data is available from different sources. Depending on whether the policy area is devolved or reserved, this may mean looking for data for different areas from different sources. Common sources of data include:

• the Office for National Statistics
• UK government department websites
• the websites of devolved administrations, devolved administration bodies, and statistics bodies
• local government
• think-tanks, academics, and international standards
• private sector organisations; such as company information, mobile phone travel data or till scanner data

This is not an exhaustive list of data sources but is designed to prompt thinking about potential routes to improving use of comparable UK-wide data.

Assess the comparability of the data

You may find that there are similar datasets available from a range of different sources, for example from the UK government and the devolved administrations. Your next step is to consider the comparability and coherence of these data sources. Some questions you may want to use to consider the coherence and comparability of existing data include:

• is the policy area reserved to the UK government, or is it devolved?  Are there different policy targets across the UK in your area? Are there differences in definitions/legislation that mean that similarly named statistics are not comparable?
• are there standardisation guidelines/processes in place in your policy area? (e.g., ONS harmonisation standards)
• how frequently are the statistics published and what period do they cover? (annually, monthly, financial/calendar year, etc)
• are calculations made in similar ways?
• what is the lowest disaggregated data that is published?
• where you can’t find the exact statistic you want, do other statistics exist that might provide an equivalent insight?

Consider routes to assemble comparable UK-wide data

You may find that the existing data sources are not directly comparable. There are a number of routes you could consider in order to access comparable UK-wide data:

• it might be possible to aggregate data in different ways for each area to produce a similar metric (e.g., combining monthly data to get a yearly average or changing absolute numbers to percentages)
• data modelling can be used to align data as much as possible
• consider plotting trends for each region on separate plots to limit direct comparison
• policy and analytical teams should work together with the devolution lead in each department to test the analysis and understand the data
• include clear caveats with the limitations of comparison

Going further: Subnational Data Strategy

The Subnational Data Strategy explains the UK government’s vision for the future production of increasingly timely, granular, and coherent subnational statistics (i.e. those that measure within regions, rather than between regions).

Whilst not specifically dedicated to the production of coherent UK-wide data, the subnational data strategy is nonetheless a highly relevant enabler of this work, setting out a framework for addressing many of the same policy, legislative, and methodological barriers facing the availability of comparable UK-wide data.

The subnational data strategy is produced by the Government Statistical Service (GSS), a cross-government network for all civil servants working in the production and communication of official statistics. It is based at the Office for National Statistics (ONS).

Where the existing published data does not meet the policy need, you may want to consider other solutions to get a better UK-wide picture:

a. Engage with your counterparts in the devolved administrations or local governments on whether there is additional data that could be shared between you. Seek a clear, mutually beneficial agreement, with a specific policy-based purpose, reflecting the scarce analytical resources available to some departments. Consider involving and cooperating with counterparts at early stages of your policy and analytical development, rather than only considering this approach at the point you are looking for additional data. Note the Statistics and Registration Service Act 2007 which prohibits the sharing of ONS data pre-release.

• departments vary significantly in size, and devolved administration departments are generally far smaller than their UK government counterparts. As you engage with colleagues in devolved administrations, you must be aware of this and make requests that can realistically be met
• further, colleagues in devolved administrations are answerable to different ministers, and different policy priorities. Colleagues’ ability to support your work may be driven by the extent to which it aligns with those priorities

b. Alternatively, where data is held by devolved administrations but not publicly available, you can use either the intergovernmental relations (IGR) structures or engage with existing inter-governmental governance, such as the Data Standards Authority (DSA) Peer Review Group and Chief Data Officers (CDO) Council.

It is important to consider the policy context when discussing comparability. There are some instances where, even if the data is statistically comparable, differing policies do not allow for meaningful direct comparison. Analysts should work closely with policy colleagues to explore any constraints in comparability created by the policy context.

Going further: Devolution guidance

The Government Statistical Service has produced separate statistical best practice guidance for those colleagues working with counterparts in devolved administrations: Working across the devolved administrations.

There are also a number of valuable resources via the Devolution Capability Programme:

• Visit Civil Service Learning to access the Devolution and intergovernmental working course, a foundation level introduction to devolution.
• Sign up for Devolution Demystified sessions – exploring why knowledge of devolution and intergovernmental working is important to all civil servants.
• Access devolution guidance and resources on GOV.UK.

Consider external data sources

In the event that these avenues are not available to you, there may still be alternative means by which to gather comparable UK-wide data:

• research if any international comparators exist that look at the same policy area and cover England, Scotland, Wales, and Northern Ireland separately; such as PISA scores for education comparisons

• research if there are any academic/think tank papers on your policy area. There can be papers with experimental analysis to compare data across the UK or those that explore the limitations of datasets for comparison; such as ESCoE’s discussion paper on UK Interregional Trade Estimation.

• explore whether there are other creative solutions to gathering data, for example accessing data that isn’t initially collected by government; such as the work of the Geospatial Commission to use anonymised mobile phone data to track mobility and transport needs.

Case studies: External data sources

Comparable UK-wide data does not necessarily need to originate from the UK. When Department for Education colleagues determined GCSE results could not be comparable across borders, they turned to PISA scores as a possible proxy metric. This is an international testing programme on 15-year-olds’ ability in reading, science and maths.

Likewise, collaboration with external third parties such as academics, think tanks, charities, and indeed offices of the devolved administrations themselves can be a critical tool in building comparable UK-wide datasets. Collaboration of this type creates opportunities to build on other data producers’ analysis and facilitates data sharing for the public good. The Scottish Fiscal Commission (a non-ministerial office of the Scottish Government) highlighted the importance of UK-wide collaboration:

We now have a regular meeting with them [HMRC] and there’s a working group involving ourselves, the Scottish Government, the Welsh Government and HMRC, where we help shape some of their publications around income tax. And, you know, the analysts are very responsive and that’s been really good.

Third party data offers opportunities to strengthen the UK-wide data landscape. When looking at labour demand changes, data from third parties on the number of online job adverts can be used to see a local level comparison. For example, using data from Text kernel published by the ONS.

Work towards greater coherence in the long term:

a. The National Data Strategy is clear that we must work with data owners across government to harmonise concepts, definitions, geographies, and variables Early engagement between officials across administration is critical to sharing plans, discussing difference, and identifying opportunities to align. This in turn produces more opportunities for data users to create a coherent picture across datasets. By working collaboratively with DA counterparts, build your collective understanding of why data cannot be compared and consider whether there are areas where data collection could be changed to create consistency for the benefit for all.

b. Take interim measures to avoid misuse or misinterpretation of incomparable data, making it clear if or how data differs – and why it can or can’t be compared with other seemingly equivalent datasets.

Going further: OSR, CDDO and ONS information

For further information, the Office for Statistics Regulation (OSR), the independent regulatory arm of the UK Statistics Authority, has produced an insight report to share accessible lessons on coherence from the OSR’s regulatory work.

Stay up to date, by following the work of the Central Digital and Data Office (CDDO) and access their latest guidance, news, and communications on the Central Digital and Data Office website.

Keep in touch with the ONS’s work to drive UK-wide data on their UK-wide statistics page.

6. Conclusion: What next

Harnessing the power of comparable UK-wide evidence and analysis delivers many benefits, from better policymaking to more productive inter-governmental discussions. Having followed the steps described in the guidance above, you will now be in a position to utilise your comparable UK-wide evidence for the benefit of public policy outcomes across the UK:

a. Apply comparable data to policy development by comparatively evaluating the different policy approaches across UKG and devolved administrations applying the lessons to your own decision making. Where you recognise positive anomalies, examine the causal factors, considering the extent to which public policy decisions may have driven that positive outcome.

b. Harness the analytical potential of comparable UK-wide data and the enhanced sample size it offers.

c. Don’t keep it to yourself. Share your analysis with your departmental devolution teams to allow them to better inform intergovernmental relations, in particular inter-ministerial discussions. You may also consider sharing your analysis with counterparts across administrations where there is a data need, in line with the spirit of mutual respect and open collaboration.