Government has been in the data business for a very long time.
The earliest known examples are over 5,000 years old. You can see them at the British Museum: clay tablets from ancient Mesopotamia, about the size – interestingly enough – of a modern computer tablet. They record the daily beer ration paid out to workers on big public projects like temples and irrigation systems.
We’ve come a long way since then. Yet in thousands of years the essential property of government data has remained the same.
Government data is a record of what’s happened, of what is. From ancient times it’s often recorded how much you’ve spent and on what.
But while the data may have stayed the same, what it’s recorded on has changed. We’ve evolved from clay, to parchment, to paper and printing press, and now pixel.
Each time the recording technology has changed the cost of storing and replicating the data has fallen. This always has profound implications.
Parchment records meant secure property rights.
The printing press brought down feudalism.
Now, in the switch from paper to pixel, the cost of what it’s stored on has fallen dramatically.
We’re in the foothills of a data revolution. Data is no longer just a record. Over the last few years it’s acquired a very new set of characteristics:
As a mineable commodity, from which value can be extracted.
As the unseen infrastructure of the digital economy.
And as a new form of property. It matters hugely who owns what data: and the more decentralised the ownership of data, the more democratised the power.
Crucially, big data has a network effect: its value increases the further it spreads. And it’s now almost infinitely replicable at almost no cost.
We must be clear-eyed about these changes: secure against the risks, alive to the opportunities.
For if the role of data is changing so too is the nature of government.
When information was low, top-down decisions were inevitable. The minister in Whitehall always had more data than anyone else, so ministerial decisions carried disproportionate weight.
And with scarce real-time performance data, the easiest way to manage a service was by setting targets. Government agencies then had to periodically check in with Whitehall to show that they’d hit their targets.
Yet it’s a bit like judging a worker’s productivity solely on whether they’ve punched their timecard.
By contrast, in an open data world we’re no longer constrained by data scarcity or centralisation. Instead we can use data to drive continuous improvement.
We now have performance metrics for all 800 transactions between citizen and government, which anyone can see on the GOV.UK performance platform.
For digital services these are incredibly useful tools. We can see how long each transaction takes to complete, at what cost, and whether the user was happy with the service.
This isn’t so ministers can micromanage from on high, but so the service managers, who deal directly with the users, can look for constant incremental improvements.
It’s all part of a broader shift within government. Instead of telling people what to do, we see our job as unleashing human ingenuity, in the public sector and beyond.
Because the truth is that civil servants want to be entrepreneurial, they want to create; they want to find better ways of doing things. And here’s the thing: data means they can.
From openness to usability
It’s why, in the last Parliament, we deliberately focused on getting as much data as possible out in the open.
We began by putting serious power behind data.gov.uk as an inventory of datasets owned by government – published and unpublished. So far we’ve published over 20,000 datasets, covering almost £200 billion of public spending.
This approach has won us plenty of plaudits. For the second year running, we’ve topped the World Wide Web Foundation’s Open Data Barometer. Last year we were number one in the Global Open Data Index.
But I want us to go further. It’s not enough just to be open. Openness is a means to an end. The end is to make government work better for the people of this country.
That means better decision-making within government: policy based on data and evidence not dogma and theory.
It means public services that continually improve, not because someone in Whitehall says ‘something must be done’, but because we’re capturing constant marginal gains.
It means supercharging the UK’s nascent data economy, putting high quality data in the same category as faster broadband or better roads.
And it means enhancing citizen trust, with a clear approach about what should and should not be done with these powerful tools.
To see what data-driven government can do, let’s look at some recent examples.
Take the LiDAR data recently published by Defra. LiDAR is elevation data collected by the Environment Agency. They map the height of things by flying round the country, firing lasers at the ground.
That’s not the plot from Spectre, it’s just an arm’s-length body going about its business. LiDAR scanning allows us to create detailed 3D maps of the landscape and now we’re opening them up.
It means that all flood risk assessments from insurers or planners can now be based on this data, cutting down validation time.
And since the Agency is also a buyer of LiDAR-derived products, it has major cost-cutting potential.
In the past only a few players could afford the licensing costs, now anyone can break in.
So there you have it: efficiency and savings, not pushed through by central edict, but carried along with the free flow of information.
And it goes further than that. The most exciting thing about open data is the innovation you can’t foresee.
One local authority is using this data to make the case for new flood defences. Council staff 3D printed the local area and fashioned blocks to show where the flood defences might go. Then they poured water on the model, to show local residents exactly which areas would flood, depending on where they put the defences.
Nor is it just local engagement. Precision farming, archaeological digs, urban planning, even uploading England to the game Minecraft: these are just some of the applications we’ve heard about since the data was published.
Let’s take another example. Two years ago Land Registry released the Price Paid Dataset (PPD), tracking residential property sales in England and Wales. The PPD is used by sites like RightMove and Zoopla to bring up-to-date sales data to an audience of millions.
Now we’re enriching it. As of last week, this dataset will also include sales through repossession, those purchased by companies and by-to-lets. It will also allow users to see the sales of non-residential property for the first time.
The applications include developing valuation software, improving planning policy, building apps that analyse market trends, and for academic research.
And the point is this. No minister, even armed with the best policy advice, could possibly conceive of all the things that government data can do.
The only way to find out is to open it up.
But opening up is only the first step. It’s no good just putting up a link to a PDF, the data has to be usable.
Think back to the printing press. It took off, not just because printing was faster than scribing, but because books were printed in English rather than Latin.
In our time, data that isn’t usable, mashable and open to all isn’t really open at all.
So I now want to set out what I see as the next steps.
Next steps on government data programme
First we have to modernise our data infrastructure. We need to get better at standardising and maintaining our data. We need to move away from government’s reliance on bulk data sharing and create an economy of APIs.
And as with every other aspect of government, we need data services built around the needs of users, not the internal logic of Whitehall.
This starts with the dog-fooding principle. Dog-fooding means using your own product. It’s named after the legendary pet food CEO who was so committed to the quality of his product that he ate a tin of it in front of his shareholders.
In short, one of the best ways to make sure our open data is of high quality is if we use it in our day-to-day operations. So we’ll be looking much more closely at how data flows into government: how it’s collected, how it links together, who uses it and how it’s made available for wider use.
The National Information Infrastructure – developed in collaboration with the ODI – gives us a good base on which to build, and I’m grateful for your help in developing that.
Getting this right is critical to the future of digital government in the UK. The digital platforms we’re building, led by the brilliant GDS, will depend on strong data foundations.
Our core datasets must talk to each other, built on high quality registers instead of lists of data replicated in each government department.
Second, we must continue to build capability across the Civil Service. In the last Parliament we set up a cross-government Data Science Accelerator Programme, to train analysts in cutting-edge tools and techniques. Now in its fourth cohort, it’s helping to embed expertise and drive innovation across Whitehall.
But to be truly data-driven, we also need policymakers and operational managers to be alive to the transformational power of data. So we’re going to set up a programme of lunch-time code clubs to develop opportunities for civil servants to roll up their sleeves and get stuck into data.
This isn’t about turning everyone into a data-scientist; it’s about making sure that departments are intelligent consumers of their own data.
Third, we need to put trust at the heart of this process. It’s our duty to keep your data safe and secure and we will always treat personal and sensitive data with the utmost care.
But securing our data assets doesn’t just mean making them harder to steal, it also means safeguarding their integrity. Inaccurate or corrupted data means citizens being lost between the cracks, so it’s vital that we ensure our data is well-managed and of high quality.
Fourth, we will strengthen our collaborative approach to data policy and governance. It’s why we’re putting data at the heart of our third Open Government Partnership National Action Plan, itself an exercise in open, collaborative policymaking.
Paul Maltby has been instrumental in this process, and last week he was in Mexico helping confirm the UK as a Lead Steward for the new International Open Data Charter.
We’re also supporting a new Data Leaders Network in Whitehall. This group will continue to review the legislation on data-sharing, making sure it supports our goal of open, effective, data-driven government.
And to make sure that our work is genuinely pushing boundaries there will be a Steering Group of digital and data visionaries driving this agenda forward. This includes top names from the UK’s data-driven economy:
- Sir Nigel Shadbolt from the ODI
- Mustafa Suleyman from Google DeepMind
- Fran Bennett from Mastadon C
- Xavier Rolet from the London Stock Exchange
- Mark Thompson from Judge Business School
- Dame Fiona Caldicott, former Chair of the National Information Governance Board for Health and Social Care
I’m delighted they have agreed to serve their nation and help drive the revolution.
I want to end on the role of the ODI.
The ODI is a crucial, critical friend to the Cabinet Office, and it will continue to play a valuable role in holding us to account for delivering this programme. We need you to keep challenging us. But we also need you to connect us to the many minds outside of government who are striving for the same thing.
As a data-driven government, we want to engage with the UK’s data economy. So from today, I’ve asked the ODI to help us connect with the businesses, start-ups and innovators progressing this field. Those who are at the leading edge, not just in open data but across the whole data spectrum.
Like data, knowledge and expertise on this subject is dispersed and decentralised. And like data, it will only realise its full potential when it’s networked and interlinked.
This will be a wide-ranging, on-going conversation. And I’m grateful to the ODI’s brilliant Technical Director and Deputy CEO, Jeni Tennison, for helping us launch that conversation.
I’ll finish on a busy road opposite Westminster Abbey, where an enterprising merchant called William Caxton opened a bookshop, back in 1476.
Many of the oldest printed books in English come from Caxton’s press. Those books were the opening shots in an earlier data revolution, one that changed the world.
I think it’s fitting that the site of his shop is now the Department for Work and Pensions – one of the most data intensive government departments.
Government data is no longer a forgotten filing cabinet, locked away in some dusty corner of Whitehall. It’s raw material, infinite possibility, waiting to be unleashed. No longer just a record of what’s happened, but a map of what might be.
I shouldn’t call it government data, because there is no such thing as government data. It belongs to all of us. Open data is a way of giving it back. I’m looking forward to working with you as we make it happen.