How to set performance metrics for your service
As a service owner, you should know how well your service is working for users. To do this, you’ll need to design metrics that have a clear meaning and collect data that tells you how your service is performing against them.
You must collect and publish data for the 4 mandatory key performance indicators (KPIs). You’ll also need your own KPIs to fully understand whether your service is working for users and to communicate its performance to your organisation.
Have a performance analyst on your team from the start of discovery and make them part of the process of building your service. Work out your measurements as you build the service, not afterwards.
You should always combine your metrics with user research and iterate your service based on insight from both to make sure it’s always improving.
Base your metrics on a sound understanding of your service’s purpose
Well designed, meaningful metrics tell you whether your service is doing what it should. Start designing your metrics by clearly defining the purpose of your service.
You should consider:
- the user needs your service is designed to meet
- any policy outcomes your service is intended to produce, for example a reduction in fraud or costs
Aim to sum up why your service exists in a single line. For example:
GOV.UK Pay will make it easier and more efficient for government to process payments, saving time and effort for service teams across government.
You might find it useful to write a ‘value proposition statement’ with your team to achieve this.
Define goals, or ‘benefits’, for your service
Derive a small number of ‘benefits’ for your service. A benefit should be a short description of a certain outcome that will contribute to your service fulfilling its purpose. Base your benefits on your understanding of your users’ needs.
Example
Need: “as a UK citizen abroad, I need quick access to money for healthcare so that I can get medical treatment on time.”
Service benefit: “Make it easier to get timely medical treatment abroad by speeding up the application process for funding.”
Develop hypotheses based on your benefits
Articulate how something you are going to do will help you achieve one of your service’s benefits and why.
Use the format:
Doing/changing [something]…
Will [benefit]…
Because [assumption/logic/strategy]…
Aim to get to the core assumptions about how your service will add value. This will help you make your measurements as relevant as possible.
For example:
If we provide an online form, it will speed up the application process because digital applications are quicker than paper ones.
Hypotheses will help you make better decisions with data, because they let you measure the effect of a specific course of action you’ve chosen to follow.
Decide what to measure based on your hypotheses
Your hypotheses should take you to a point where you have a set of things to measure that are easily measurable. At the start of developing your service you do not have to think about specifically how you’ll measure them, but you should have a clear understanding of what each measurement means.
For example:
We will measure how long it takes a user to make an application using the online form we build.
As your service evolves, you’ll need to identify which metrics will be the KPIs you share to show how your service is performing. Choose 3 or 4 metrics that would answer the question: ‘is this service working?’
You’ll still probably need other metrics within your team to understand your service’s performance in more detail.
Find data sources for your measurements
After you’ve chosen what to measure, you’ll need to get access to all the data sources you’ll need to make your measurements. Data sources could include:
- digital (web) analytics
- user feedback
- site performance
- call centre data
- financial information (for example, how much your hosting costs)
Digital analytics will tell you how people are interacting with the online parts of your service. You should make sure:
- you choose a digital analytics tool that will give you what your service needs
- your performance analyst is involved in building your service - it’s much easier to implement data collection and solve problems as you build than to go back and do it afterwards
Do not just use digital analytics. Combining data from a range of sources will give you a richer picture of how your service is performing.
There are different ways to access each data source and you’ll be able to use different tools to do so. Your performance analyst can help you consider practical questions such as:
- how you’ll gather user feedback - for example, you might need to run a user satisfaction survey or collect data from your call centres
- what software you’ll need to install if you’re going to monitor service uptime and performance
Collect and analyse data
Once you’ve identified all your data sources, your team’s performance analyst will:
- make sure your data is collected accurately
- check for statistical significance
- test for errors or inconsistencies in the data
- do appropriate analysis
Analysis needs to be timely and help you take action. Effective analysis should tell you:
- what happened
- why it happened
- what you need to do about it
Give context to your measurements
To be useful, metrics need to have context. You should provide context for your metrics by:
- using segments in your analysis
- measuring your service’s performance over time
- comparing your service’s performance with that of similar services
Segmenting data
Make sure you use segments in your analysis. Segments group your users, for example by:
- device type - whether they used a phone or a desktop computer to access your service
- new or repeat user - whether they had used your service before
Using segments correctly will help you identify and understand trends, because they help you understand how your users are different. For example, new users may behave very differently to returning users, making an overall average of user behaviour inaccurate.
Measuring performance over time
Establish a ‘baseline’ of how your service performs currently, across all channels. Judge changes to the service’s performance against this.
If you’re building a new service, you can use the performance of existing or legacy systems to inform your baseline.
Measure performance continuously as much as possible, rather than taking ‘snapshots’. This will give you a ‘trend line’ against which peaks and dips in performance can be measured, making it easier to see the effects of:
- changes to the service
- communications initiatives
- seasonal variation
Set targets for what you expect each KPI to be at a specific time in the future if your plans to improve your service are successful. For example, in 6 months’ time you might expect to see a 10% reduction in the time it takes for a user to make an application because of your work to simplify the process.
Comparing performance with similar services
There are likely to be services similar to yours in terms of:
- number of users
- complexity
Comparing your service’s performance with other, similar services will let you know whether your service is performing significantly better or worse than expected.
You can get performance data for government digital services on data.gov.uk.
Using dashboards, reports, alerts and visualisations to share findings
Find a way to share findings that suits what you’re trying to do.
Presenting data visually in a prominent place or message is a good way to maximise the likelihood that the information will be acted upon and improvements made to your service.
Dashboards are objective-focused and will help inform decisions, often with the help of real-time data. Read about building a data dashboard for a service delivery team
Alerts are used to inform the users about a change or an event, often using attention-grabbing delivery mechanisms.
Reports provide regular, scheduled snapshots of data and tend to require extra context and time to digest.
You should try to automate the creation of standard reports, for example automating email alerts for ‘trending’ content so your analyst can spend their time on more valuable work.
Monitor, iterate and improve
Iterate your metrics as your service changes - you’ll need to know different things as your service changes over time.
Further reading
You might find the following blog posts useful:
- Last update:
-
Guidance first published