Guidance

The What Works Trial Advice Panel

The What Works Trial Advice Panel brings together top trialling and experimentation experts from across Government and academia to provide a free-to-use service for all civil servants.

Documents

Details

Overview of the Panel

The What Works Trial Advice Panel brings together top trialling and experimentation experts from across Government and academia to provide a free-to-use service for all civil servants. The Panel provide advice and support to help civil servants design and implement effective impact evaluations that will help us understand whether programmes and policies are really delivering desired outcomes. The Panel offer advice and support on all kinds of impact evaluation methods and approaches.

Examples of the Panel’s work

The Panel helped the Ministry of Housing, Communities and Local Government design their first ever randomised controlled trial showing that providing English language classes to women in segregated communities made these women more likely to use local amenities and develop friendships with people from different backgrounds. The trial fed into the government’s 2018 integrated communities strategy green paper and a new £6 million Integrated Communities English Language Programme in 2019/20.

The Panel supported Ofgem’s first large-scale randomised controlled trial, which found that switching rates tripled when electricity and gas customers were sent letters with information on the cheaper tariffs offered by other providers.

See more examples in our report here.

Reflections about the Panel

Joseph Sherlock, Principal Behavioural Science Advisor, HMRC

The Trial Advice Panel supported the largest RCT that HMRC had ever run, which tested the effectiveness of different messaging to self-assessment customers. We met with advisers several times over a 6-month period to discuss and solve a range of technical and implementation challenges. The benefit of using the panel was felt beyond the trial, extending to other work and helping build the competence of our team. Our experience has encouraged us to put further trials forward to the panel.

Raymond Duch, Director, Centre for Experimental Social Science, University of Oxford

I have very much both enjoyed and benefited from my participation in the Trial Advice Panel. Most importantly, as a result of the interaction with government officials engaged in policy planning and evaluation, I have developed a much better understanding of how to design feasible experiments that can actually answer important policy questions. I also developed an appreciation for the impressive talent that makes up the UK Civil Service. The officials I interacted with were bright but also quite demanding in a positive sense, forcing me and my colleagues here at Oxford to give serious thought to challenging design and data analytics questions.

Riika Hofmann, a Senior Lecturer at the University of Cambridge reflects on her experience as being part of the Panel on our What Works blog

What the Panel does not do

We are very keen that this Panel meets a current gap in provision rather than duplicating or undermining any existing processes or practices. As a result the Panel will not:

  • Take on the full running or ownership of an evaluation – evaluations will be wholly owned by the Department that is running the trial, this panel will just provide advice and support.
  • Narrowly focus on any one method (e.g. RCTs) – the panel will advise on the best sort of impact evaluation to generate the most useful results.Replace analytical teams – this panel will not replace any of the functions of a Department’s own analysis or trialling teams. We will work with existing analytical teams and link – where appropriate – to Department’s analytical experts.
  • Provide a quality assurance service - this panel will provide advice and support but will not provide a robust quality assurance service or take any responsibility for the results of the evaluation.

All external members of the Panel have signed a Memorandum of Understanding regarding confidentiality and data protection.

More information

If you would like to contact the Panel for advice or support please email Trialadvicepanel@cabinetoffice.gov.uk and we will do our best to help!

Published 6 August 2015
Last updated 30 October 2019 + show all updates
  1. Whole page updated.

  2. First published.