Using evidence to scope test ideas (phase 2)
Use research and data to understand local needs, scope ideas for testing and build a theory of change with stakeholders.
In this phase you use research to explore opportunities for testing. The aim is to use evidence to develop a shortlist of test ideas that relate to the problem statement.
Main aims of this phase
The main aims of this phase are to:
- use evidence to further scope the challenge area and problem statement
- identify where and how doing things differently might improve outcomes
- explore and understand how people with lived experience are affected by the issue
- develop and prioritise test ideas to design
Outcomes of this phase
At the end of this phase you should have:
- a deeper understanding of issues and opportunities relating to the problem statement
- a shortlist of test ideas
- a theory of change
- agreement from partners or decision makers on test ideas to design
Related guidance
Before starting this phase you can:
How you approach scoping depends on your situation. You may be exploring the idea of a new service or intervention, or considering how to adapt something that already exists.
Scoping takes time. Consider:
- local context
- availability of existing data and findings
- access to local people and staff
Balance ambition with pragmatism. That means setting a bold, transformative vision while defining a focused, deliverable starting point. Prioritising helps you manage this balance without choosing a solution too early.
Work with local partners to turn opportunities into manageable test ideas. Involve your team and stakeholders in the research of your ideas. This helps people new to test and learn see how user insight and evidence guide each phase.
What you need to find out
The amount of research you do in this phase depends on the evidence you already have.
If you’re starting with a clearly defined local problem, you may need less initial research. Before you move to design, make sure you validate the problem with community groups and people with lived experience.
If you’re working with a broad or less defined opportunity, use research in this phase to understand:
- how existing services are working for users
- the context of the challenge area and the wider system
- local context, such as cultural practices in the community
- what frontline staff think are the main problems in the service and system
- what is already happening locally to address the problem
- assumptions people have about the problem and opportunities to solve it
Example: building an understanding of local context
The Test, Learn and Grow team and Northumberland County Council wanted to find out how Family Hub services could work more closely with schools and early years providers. This work aimed to help more reception-age children meet early learning goals.
Because effective early intervention relies on strong local networks, the team started by researching existing connections in Northumberland.
They wanted to understand:
- how schools, early years providers and Family Hubs worked together
- which families were using local services and which missed out
- what it felt like for families and staff to use these services
- how to measure the effectiveness of building stronger connections
The team ran 38 interviews with staff and families. They spoke to local authority staff, Hub managers and staff from schools and early years providers, both those working with the Hubs and not. And they spent time listening to parents, speaking to regular Family Hubs users and those who had never visited.
They then combined interview insights with quantitative analysis and desk research.
These activities helped the team:
- understand local services, relationships and parents’ experiences
- narrow their scope to 5 opportunity areas
- build trust and ongoing local partnerships
Involve local people and communities from the start of your project. Use research and participatory design methods to understand the opportunity from a local perspective.
Policymakers and people with lived experience often see issues differently.
Ways to gather insights
Start by reviewing existing local research.
If you need to gather more insights you could use:
- focus groups to discuss local problems
- workshops with participatory activities, for example systems mapping or design probes
- shadowing and ethnographic research to observe and understand how things work in real life
- interviews to hear the experiences of local people
Identifying who to speak to
You should write research questions and make a research plan. Use these to help you identify who you need to speak to, including:
- frontline staff
- people working locally or across the challenge area, for example in voluntary organisations or community groups
- people with lived experience
Ethics and safety
You must make sure you conduct your research safely and sensitively, especially if you’re working with people in vulnerable situations.
Before you start, read about:
- conducting research safely
- inclusive design in the Service Manual
- participatory design
- ethical principles in the Ministry of Justice Design System
Systems mapping
Creating a systems map is a good way to start exploring an opportunity. It looks beyond a service to include everything that can influence or change a system.
Mapping helps you identify:
- work already underway in the challenge area
- people and organisations you need to speak to and where they connect
- areas of friction and things that are working well
- opportunities for further research
Use the process to develop a shared understanding of the system in your team and with partners. Share the map with subject matter experts and stakeholders to check your findings and add to it.
Update the map as you learn and the system becomes clearer, or changes.
Get started with the systems thinking toolkit for civil servants.
Example: mapping Children’s Services to improve support
The Test, Learn and Grow team worked with Barnsley Council to map processes in Children’s Services and speak to social workers about their day-to-day experiences. This work supported efforts to see how AI and data tools could help improve outcomes for children in care.
The team began by looking at the Council’s case management system, gathering detailed information about technology, workflows and legal requirements.
As trust grew, staff shared experiences of working in Children’s Services, including stories of families they tried to help and the obstacles they encountered. These conversations gave the team insight into social workers’ daily activities and the challenges families and staff face.
This work revealed a gap. While the map showed formal processes, it did not show where there were opportunities for earlier intervention. The team decided to test whether council data could track how children move through services, so they could provide targeted help earlier.
Design probes
Use design probes, also called cultural probes, to engage local people in your research and design. Participants interact with carefully chosen objects, documents or tasks to generate insights and new ideas. You can run design probes individually or in workshops.
Example: using design probes to identify what problem to focus on
The Test, Learn and Grow team worked with Islington Council to develop test ideas that would help tackle violence against women and girls.
They held a workshop with local survivors, using a tree metaphor as a design probe to break down each idea and assess how it would work in practice.
The group used the metaphor to consider:
- roots – the foundations and conditions needed
- trunk – who is involved and responsible
- leaves – the benefits and outcomes
- bird’s-eye view – how the idea fits into the wider system
This exercise helped the team identify which parts of each idea they could explore further in testing.
Combine data and user research to understand the problem and validate findings.
The data you use will depend on what is available and the level of detail you need. You could look at:
- national data, to compare local trends with national averages
- local demographic or deprivation data, to identify where you want to focus your work
- service data, such as agency usage statistics, user journeys through the service, helpdesk logs, website analytics or annual reports
Example: using data to see the journeys people take through services
The Test, Learn and Grow team worked with data analysts from Islington Council Children’s Services to understand:
- how domestic abuse cases move through Children’s Services
- how many cases are turned away because they do not meet the services’ eligibility criteria
The team used this data to develop their understanding of the problem and identify opportunities for interventions.
Document and review your findings as you work to narrow your focus. Create a theory of change as you plan for the next phase.
Documenting research
Record your research while it’s fresh in your mind to help you capture accurate insight and share evidence with stakeholders.
You could create:
- a systems map
- a user journey map
- a user needs log
- an insights log
- an evidence review
- an assumptions log
Analysing and synthesising research
You should analyse your research continually. Read the Service Manual guidance on how to analyse a research session.
You have enough information and evidence to move to the next phase when you have:
- summarised clear themes
- defined the problem within your local context
- narrowed down the test ideas for design in the next phase
- created a theory of change to map how your ideas will work
Use your findings to turn insights into actions for the next phase. You could create:
- opportunity areas by using idea cards or storyboards to define test ideas and show how research insights could help solve parts of the problem
- problem statements by iterating your initial challenge using “How might we” questions, such as “How might we increase the positive influence of parents on children’s employment outcomes?”
Once you’ve explored your test ideas, you need to evaluate them and focus your effort.
Setting criteria to refine your scope
Set your criteria based on:
- research and evidence
- outcomes
- strategic priorities
Next, check your options against desirability, feasibility and viability.
Desirability
To check if an idea is the right thing to work on, consider whether it:
- delivers measurable benefits for local users, the community and staff
- supports change across the wider system
- aligns with your strategic priorities
Feasibility
To check if you can realistically do the design work now, consider whether you:
- have the skills, experience and time in your team
- have the technology and data you need
- can gather evidence to track change and measure success
- can involve local people in designing solutions, including groups that are less engaged
Viability
To check if the designed idea will be supported and sustained, consider:
- senior buy-in, to make sure local and national leaders understand and back the work
- team commitment to testing new approaches
- scalability, so the work can expand over time, influence the wider system or be adapted by other teams
Running a prioritisation workshop
A prioritisation workshop will help you shortlist your test ideas. Involve stakeholders and the community or service users, if relevant.
- Assess each test idea against basic requirements. Rule out any that do not fit.
- Review the prioritisation criteria as a team. Add or remove criteria to fit your context.
- Check the remaining test ideas against the criteria, and rank them low, medium or high.
- Use dot voting to choose the idea you’ll start with, then agree your next steps.
The outcome should be a clear, agreed scope. Everyone in the team should understand what you’re working on and why.
Example: using research and participatory design to prioritise ideas for testing
In Wakefield, the Test, Learn and Grow team used participatory design and user research to narrow down test ideas for helping young people into employment.
The team worked with local partners following a 5-step process.
- The team identified 24 opportunity areas through desk research, systems mapping and problem-framing workshops with council and voluntary, community and social enterprise (VCSE) staff.
- Council and VCSE staff voted on the most important challenges, narrowing these 24 opportunity areas down to 5.
- The team combined their own research with an AI-assisted review of academic and policy research across the 5 opportunity areas. They created a map showing how young people move between education and employment.
- The team ran user research with young people and employment staff, using questions from the mapping to guide discussions. The research produced 28 problem statements, which the team narrowed to 9 based on feasibility and desirability.
- Council and VCSE staff voted to decide the top 3 opportunity areas, and together with the team developed test ideas for each one.