Guidance

API technical and data standards

Design, build and operate APIs in a consistent way

From:
Central Digital and Data Office
Published
7 February 2018
Last updated
11 July 2022 — See all updates

These standards are for people building APIs in government who want to:

  • save time
  • save resources
  • reassure users that their service meets minimum standards
  • use agile methods to improve products and services through testing
  • take an API-first approach to development

These standards will provide you with best practice guidance about how to design, build and operate your APIs for use in government and public services.

By following these standards, you’ll work in the same way as other people and produce APIs that work better across different platforms and services.

Design your API

Before you build an API, you should carefully plan what it will do and how it will work. Your team should agree on a set of user needs, including who will use your API, and what they will need to do with it.

Start by:

  • gathering user and business requirements to help define what the API needs to do
  • identifying the key business entities users need to interact with via the API
  • developing a specification before you start to code
  • testing your assumptions with users
  • iterating the design based on feedback

This will allow you to focus on simplifying the interface, removing any features that are not useful to users.

Consistency improves the developer experience by making it easier and more intuitive to integrate with the API. You need to make your API as easy to predict as possible as users may not read all of your documentation.

You should avoid introducing breaking changes. Once an API is integrated with a service, people may be less willing or unable to update their code, because they will often want to make as few changes as possible after it goes live. You should think about services which depend on your API and how changes will impact them. If possible, do not make changes that will change how services have to interact with the API.

Implementing user-centred design makes it more likely an API will meet user needs and need fewer changes over its lifetime. If you do need to introduce breaking changes, follow our guidance on versioning and taking your API out of service.

Check for existing APIs

Establish whether there are any existing APIs which could be used instead of building a new one. These might be internal or external, or commercially available. You should look at internal API catalogues and the cross-government UK API Catalogue.

It’s faster and simpler to reuse an existing API than build one from scratch. Building this discovery step into the design process prioritises reuse and means APIs are built only when necessary. You should also make sure any API you choose to use will work for your use case, in terms of licensing and functionality.

You should follow the Technology Code of Practice when designing any technology in government, including APIs. Point 10 - Make better use of data - may be particularly relevant when you’re working with APIs.

You should design your API to follow all appropriate government data standards. You can find a list of endorsed standards in the Data Standards Catalogue. Certain data standards endorsed by the government are particularly useful in API design. They are the:

You should follow all guidance and regulations around the General Data Protection Regulation (GDPR) and using personal data in your organisation.

Practice API-first design

API first is the practice of designing software with an API as the first interface to your data - and then having further interfaces such as web or mobile use that API as a way to access the data.

Developing the API before the rest of the service means a platform or service can be built around the API. This would reduce the need to repeat work later if an external API is required.

APIs are often an afterthought, built when a service already exists as a way for other services to access its data. If you’re building an API to a legacy system, that may be your only choice, but you should still think about the user needs for your API.

Developing your API before any other interfaces has other advantages. These include:

  • other services being able to use your API
  • stress-testing of your API by your own internal services, allowing you to make continuous improvements to it and improve it for external users - this will improve developer experience by exposing the complexities of the API and making sure, for example, that documentation is fit for purpose
  • improved modularity and reuse of code, as the API will not have to be customised to fit an existing service - this leads to more consistent interfaces, meaning developers can be more comfortable with your API and can speed up integration
  • the underlying data structures of your API will be fit for purpose - starting with the API means that the design of the underlying data store must be considered from the beginning, and separated from the business logic of the service (see below ‘Abstract away your processes’)

Use REST to build your API

Representational state transfer (REST, which is sometimes known as RESTful) means an API follows the REST architectural style, and works with RESTful web services. This can make the API easier to use and faster to integrate with. REST principles are industry standard, which means developers will understand them more quickly. It also means you can use industry standard ways of documenting your API, and off the shelf testing software.

Using REST is a good way to build an API, but other approaches such as GraphQL may still be a better choice for specific projects. For example, GraphQL is useful for prototyping services, when you’re not sure what views of the data other developers will need.

You should always choose the architectural style for your API based on your individual project or service’s needs.

Produce an OpenAPI document for your API

The OpenAPI Specification is a standardised way of describing RESTful web APIs - it allows you to produce a file (the OpenAPI document) which is both machine and human readable and which describes the format and responses of your API.

A good principle is to produce an OpenAPI document as the first output of your design process, so the document is developed along with the API design.

This can be used to:

  • show the API has been developed consistently with others from your organisation
  • test it against rules and for security issues using a software linter
  • generate reference documentation automatically - documentation generated in this way should be supported by further resources.

OpenAPI is recommended by the government Open Standards Board.

Plan your API securely

Design securely

You should think about security from the very beginning of the design process for your API. The National Cyber Security Centre provides advice about designing services securely.

API security is complicated, and involves:

  • data level security - making sure users only have access to the data provided by the API that they are authorised to see
  • application level security - making sure only authorised users can access the API
  • auditing - making sure the usage of the API is monitored

The OWASP foundation has a list of top 10 security risks for APIs. You should make sure to avoid these when designing your API.

Host your API securely

When you design an API, it’s important to think about where you will host it and how it will continue to work during its lifecycle.

When you name and host your API, as well as its documentation, you should follow guidance on choosing an API domain name.

Follow the Service Manual guidance on using HTTPS when serving your API over the web, to make it as secure as possible. The National Cyber Security Centre also has guidance on using TLS to securely deliver web services.

Build your API

Use the UTF-8 standard to encode your API

Unicode is the world standard for consistently encoding, representing and handling text in most global writing systems. You should use the Unicode Transformation Format (UTF-8) standard when encoding unicode character sets, to help you read, write, store and exchange text that will remain stable over time and across different technologies.

Use JSON for response formats

Use the JSON standard where possible when structuring your REST API’s response formats.

You should use the official specification for JSON - ECMA-404 The JSON Data Exchange Standard. There are multiple JSON formats in use today, and if your organisation has not already specified one, we recommend JSON:API which is specifically designed for API responses, and has the benefits of making some design choices for you by specifying conventions.

Sometimes you’ll need an alternative to REST to match a data structure to a specific use case. For example, if you’re updating an API which currently provides responses in XML, it may be better for your users if you keep the same format. You should make sure this is well documented, because XML is increasingly uncommon.

Choosing a broadly-adopted standard whenever possible gives you advantages, such as:

  • saving time by avoiding debate about what format to use
  • allowing you to follow industry best practices
  • making it easier for external developers to integrate with your API

Use consistent names for resources

Developers should be able to assume the names of the resources in your API from context. Use similar terms for similar resources, for example user_id and address_id.

Your users should not have to reread documentation to be able to know what the name of a particular resource is.

Make your resources persistent

The resources your API provides (answers to requests) should not change, as this could break integrations.

You should make sure your API design provides a level of abstraction from the underlying data sources. It should not matter if the columns in your database change name, for example, because the API should provide a map from the resource name to the underlying data.

Use standard HTTP responses

You should make sure you match error codes with HTTP responses. Standard HTTP response codes are defined here.

Consistent and easy to read error codes are crucial to understanding where an error has occurred. There are often cases where the same endpoint could return the same HTTP status code for different conditions, so descriptive error messages will help users understand what’s gone wrong. You should document all error codes and make sure they’re easy to find.

Make sure any custom error codes only contain information needed to diagnose the problem, and do not include any non-essential information, which could help an attacker target the service. For example, technical details of the system the API is running on.

Control levels of user authorisation

When you’re building your API, you need to decide the best way to authorise people to access its data. Read the API management guidance.

User-level authentication is good for audit and access control. Use it if you want to control who can access your API. This is essential when dealing with personal or sensitive data.

Use application-level authorisation if you want to control which applications can access your API without limiting who can access them. This is suitable if you want to do rate limiting, auditing, or billing functionality. Application-level authorisation may not be suitable for APIs holding personal or sensitive data.

Consider performance and scalability

You can measure your API’s performance by how fast it can deal with a single request and make a response. Its scalability is the amount of requests it can deal with at the same time while maintaining an acceptable performance level.

You can improve the performance and scalability of your API response by making it cacheable. This means the API response can store copies of frequently-accessed data along its request and response path. This reduces bandwidth, latency and server load, as well as making network failures less of a problem for your users.

A cacheable API response has other advantages, for example letting you use a Content Delivery Network (CDN). A CDN can make your API faster and more reliable by caching data for responses in locations that are closer to the user.

Provide an API test service

You should try to offer a testing service (also known as a sandbox) for your users. They may find an API harder to integrate without a test service, so providing one is a good use of your project’s time and budget. Sandboxes can also be generated much more easily with an OpenAPI document - tools are available which will convert these documents into test endpoints.

You should make your test services available without authentication and make sure the test service never contains real data.

Having a test service available also means your developers can start to get comfortable with an API in a sandbox environment very early in its development. It means your developers can work while other parts of the project are being completed. For example, work can begin in a test service, with test data, while a data sharing agreement is still being drawn up to access the real data for the finished API.

However, providing a sandbox of good quality can be complicated, so you should carefully consider your options if you are going to do it. A good sandbox should include:

  • useful data that reflects the real API
  • implementation and dependencies behaviour that is properly simulated
  • the ability to save, protect and restore data

Test your API’s compliance

You need to make sure your API meets the standards your organisation or team’s legal requirements need.

You must comply with UK GDPR when you do this.

Put your API in the cross-government API Catalogue

After you’ve tested your API’s compliance, you should add your API to the cross-government API Catalogue. Making your API discoverable means it’s more likely to get used.

Document your APIs

Your team’s developer, user researcher and technical writer should work together on your API’s documentation. You should follow guidance on:

Operate your API

Support older versions of your API

When you make new versions of your API, you should try to make sure you do not make changes that will stop older versions of your API working properly. If you cannot keep older versions working, you should tell users these older versions are no longer supported.

You can think about retiring older versions of an API if you can tell from the logs that few people are using it anymore.

Use an API management system or gateway

An API management platform provides services for your API that it rarely makes sense for you to build yourself. For example, access control and authorisation, audit and logging, and network management. In production all of these services are important, and can be more reliably and easily provided by an API management tool or gateway - of which there are several open source options.

When to authenticate your API

Authentication allows you to monitor who is using your API, and for what purpose. You need to authenticate your API to use:

  • rate limiting or throttling - making sure that users don’t overuse your resources
  • auditing - checking what data was accessed by which users
  • billing - if your API is chargeable
  • authorisation - enabling different levels of access based on users or roles

We recommend using a third party tool for authentication, either as part of or in conjunction with an API gateway. Several open source options are available.

It’s unlikely you’ll ever need to provide completely open access to your API, and for security reasons it’s never a good idea to do so. If you do want to provide open data in this way, you should consider publishing it as a CSV file to the internet, for example on data.gov.uk.

Log your API’s use

If your API provides personal or sensitive data, you must log (ideally using the functionality of an API gateway, as above) when the data is provided and who you provide it to. This will help you follow UK GDPR, respond to data subject access requests, and detect fraud or misuse.

Monitor APIs for unusual activity

You must make sure all technology in your team or organisation is secure. APIs are no exception. Read guidance from the Technology Code of Practice to help you make things secure. Using the logging and audit tools from your API gateway will help you recognise when use of the API changes over time.

Published 7 February 2018
Last updated 11 July 2022 + show all updates

  1. Updated to improve the structure of the guidance, and clarified a few points.

  2. Updated to remove references to 'whitelists' in line with the GDS style guide

  3. Version 2 of the API standards includes sections on linked data, namespaces, sub-resources and query arguments and providing a test service. We've also added to sections on reusing and managing personal data, responding to data requests and how to design data fields.

  4. First published.

Contents

Related content