Guidance

Assessing if artificial intelligence is the right solution

Guidance to help you assess if artificial intelligence (AI) is the right technology for your challenge.

From:
Department for Science, Innovation and Technology, Office for Artificial Intelligence, and Centre for Data Ethics and Innovation
Published
10 June 2019

This guidance is part of a wider collection about using artificial intelligence (AI) in the public sector.

This guidance will help you assess if artificial intelligence (AI) is the right technology to help you meet user needs. As with all technology projects, you should make sure you can change your mind at a later stage and you can adapt the technology as your understanding of user needs changes.

This guidance is relevant for anyone responsible for choosing technology in a public sector organisation.

Assessing if AI is the right solution for your users’ needs

AI is just another tool to help deliver services. Designing any service starts with identifying user needs. If you think AI may be an appropriate technology choice to help you meet user needs, you will need to consider your data and the specific technology you want to use. Your data scientists will then use your data to build and train an AI model.

When assessing if AI could help you meet users’ needs, consider if:

  • there’s data containing the information you need, even if disguised or buried
  • it’s ethical and safe to use the data - refer to the Data Ethics Framework
  • you have a large quantity of data for the model to learn from
  • the task is large scale and repetitive enough that a human would struggle to carry it out
  • it would provide information a team could use to achieve outcomes in the real world

It’s important to remember that AI is not an all-purpose solution. Unlike a human, AI cannot infer, and can only produce an output based on the data a team inputs to the model.

Working with the right skills to assess AI

When identifying whether AI is the right solution, it’s important that you work with:

  • specialists who have a good knowledge of your data and the problem you’re trying to solve, such as data scientists
  • at least one domain knowledge expert who knows the environment where you will be deploying the AI model results

Consider your current data state

For your AI model to work, it needs access to a large quantity of data. Work with specialists who have the knowledge of your data, such as data scientists, to assess your data state. You can assess whether your data is high enough quality for AI using a combination of:

  • accuracy
  • completeness
  • uniqueness
  • timeliness
  • validity
  • sufficiency
  • relevancy
  • representativeness
  • consistency

If your problem involves supporting an ongoing business decision process, you will need to plan to establish ongoing, up-to-date access to data. Remember to follow data protection laws.

Choosing AI technology for your challenge

There is no one ‘AI technology’. Currently, widely-available AI technologies are mostly either supervised, unsupervised or reinforcement machine learning. The machine learning techniques that can provide you with the best insight depends on the problem you’re trying to solve.

Machine learning technique Description Examples of machine learning technique
Classification Learns the characteristics of a given category, allowing the model to classify unknown data points into existing categories - deciding if a consignment of goods undergoes border inspection

- deciding if an email is spam or not
Regression Predicts a value for an unknown data point - predicting the market value of a house from information such as its size, location, or age

- forecasting the concentrations of air pollutants in cities
Clustering Identifies groups of similar data points in a dataset - grouping retail customers to find subgroups with specific spending habits

- clustering smart-meter data to identify groups of electrical appliances, and generate itemised electricity bills
Dimensionality Reduction or Manifold Learning Narrows down the data to the most relevant variables to make models more accurate, or make it possible to visualise the data - used by data scientists when evaluating and developing other types of machine learning algorithms
Ranking Trains a model to rank new data based on previously-seen lists - returning pages by order of relevance when a user searches a website

Common applications of machine learning

There are certain types of problems for which machine learning is commonly used. For some of these you will be able to buy or adapt commercially available products.

Machine learning application Description Examples of machine learning application
Natural language processing (NLP) Processes and analyses natural language, recognising words, their meaning, context and the narrative - converting speech into text for automatic subtitles generation

- automatically generating a reply to a customer’s email
Computer vision The ability of a machine or program to emulate human vision - identification of road signs for self-driving vehicles

- face recognition for automated passport controls
Anomaly detection Finds anomalous data points within a data set - identifying fraudulent activity in a user’s bank account
Time-series analysis Understanding how data varies over time to conduct forecasting and monitoring - conducting budget analyses

- forecasting economic indicators
Recommender systems Predicts how a user will rate a given item to make new recommendations - suggesting relevant pages on a website, given the articles a user has previously viewed

Getting approval to spend

Because of its experimental and iterative nature, it can be difficult to specify the precise benefits which could come from an AI project. To explore this uncertainty and provide the right level of information around the potential benefits, you can:

  • carry out some initial analysis on your data to help you understand how hard the problem is and how likely the project’s success would be
  • build your business case around a small-scale proof of concept (PoC) and use its results to prove your hypothesis

Once you have secured budget, you’ll need to allow enough time and resources to conduct a substantial discovery to show feasibility. Discovery for projects using AI can often takes longer for similar projects that do not use AI.

If your organisation is a central government department, you may have to get approval from the Government Digital Service (GDS) to spend money on AI. At this point most AI projects are classified as ‘novel’, which requires a high level of scrutiny. You should contact the GDS Standards Assurance team for help on the spend controls process.

Deciding whether to build or buy

When assessing if AI could help you meet user needs, consider how you will procure the technology. You should define your purchasing strategy in the same way as you would for any other technology. Whether you build, buy or reuse (or combine these approaches) will depend on a number of considerations, including:

  • whether the needs you’re trying to meet are unique to your organisation or you could fulfil users’ needs with generic components
  • the maturity of commercially available products that meet those needs
  • how your product needs to integrate with your existing infrastructure

It’s also important to address ethical concerns about the use of AI from the start of the procurement process.

The Office for AI and the World Economic Forum are developing further guidance on AI procurement.

Build your AI solution

Your team can build or adapt off-the-shelf AI models or open source algorithms in-house.

When making this decision, you should work with data scientists to consider whether:

  • your team has the skills to build an AI project in-house
  • your operations team can run and maintain an in-house AI solution

Buy your AI solution

You may be able to buy your AI technology as an off-the-shelf product. This is most suitable if you are looking for a common application of AI, for example optical character recognition. However, buying your AI technology may not always be suitable as the specifics of your data and needs could mean the supplier would have to build from scratch or significantly customise an existing model.

Your AI solution will still need to be integrated into an end-to-end service for your users, even if you are able to buy significant components off the shelf.

Allocating responsibility and governance for AI projects

When using AI it’s important to understand who is responsible if the system fails, as the problem may lie in a number of areas. For example, failures with the data chosen to train the model, design of the model, coding of the software, or deployment.

You should establish a responsibility record which sets out who is responsible for different areas of the AI. It would be useful to consider whether:

  • the models are achieving their purpose and business objectives
  • there is a clear accountability framework for models in production
  • there is a clear testing and monitoring framework in place
  • your team has reviewed and validated the code
  • the algorithms are robust, unbiased, fair and explainable
  • the project fits with how citizens and users expect their data to be used

Depending on your organisation’s maturity, it may be useful to set up a dedicated board, committee or forum to handle AI data and model governance.

Recording accountability

It can be useful to keep a central record of all AI technologies you use, listing:

  • where AI is in use
  • what the AI is used for
  • who’s involved
  • how it’s assessed or checked
  • what other teams rely on the technology
Published 10 June 2019
Contents

Related content