Corporate report

UK Vaccine Network Project annual review 2018 to 2019

Published 28 January 2021

Introduction

Outline of programme

In the 2015 spending review the Global Health Security (GHS) team was given £477 million of UK Official Development Assistance (ODA) funding to develop projects in and for low and middle income countries (LMICs), with the aim of contributing to a ‘world safe and secure from infectious disease threats and promotion of Global Health as an international security priority.’ This accounts for 34% of total Department of Health and Social Care (DHSC) ODA funding.

The programme is made up of 5 projects:

• Fleming Fund
• Global Antimicrobial Resistance Innovation Fund (GAMRIF)
• UK Public Health Rapid Support Team
• International Health Regulations Strengthening project
• Vaccines Project

Through delivery of each of these projects the programme aims to support ODA eligible countries to:

• prevent and reduce the likelihood of public emergencies such as disease outbreaks and antimicrobial resistance (AMR)

• detect health threats early to save lives

• provide rapid and effective response to health threats

Outline of project in relation to the programme

The UK Vaccine Network project is a £110 million ODA-funded project that contributes to the Global Health Security programme’s aim to ‘prevent and reduce the likelihood of public health emergencies’. In particular, the project aims to support the development of new vaccines and vaccine technologies for emergent threats. This will allow outbreaks of diseases with epidemic potential to either be prevented through proactive vaccination campaigns or controlled through quick development of new vaccines and/or responsive vaccination campaigns upon outbreak detection.

An investment strategy for the project was developed using advice from the UK Vaccine Network (UKVN), a group of experts from academia, industry, government and philanthropic organisations and chaired by the DHSC Chief Scientific Adviser (CSA). The UKVN was established in 2015 in response to the Ebola outbreak and met 6 times across 2015 and 2016.

The group identified 12 priority pathogens with epidemic potential in LMICs on which efforts should be focused. The UKVN also advised that investment should be centred on:

• late-stage pre-clinical development and early-stage clinical development of vaccine candidates for the priority pathogens 

• development of novel vaccine platforms and manufacturing techniques to:

  • enable vaccines against unknown pathogens to be developed faster

  • improve accessibility and delivery of vaccines in LMICs

• associated technologies and epidemiological work that would support effective vaccine deployment in an outbreak

The project team used this advice to design 7 research competitions. Delivery partners with the required experience and expertise were chosen to deliver these competitions and to manage the resulting research projects. Our delivery partners are:

• the Biotechnology and Biological Sciences Research Council (BBSRC)

• the Engineering and Physical Sciences Research Council (EPSRC)

• Innovate UK

• two commissioning facilities of the National institute for Health Research (NIHR): the NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC) and the Central Commissioning Facility (CCF)

Alongside the delivery of the UK Vaccine Network project, the project team are responsible for the continuation of the UKVN, which now holds annual meetings. In addition to the advice given to the UK Vaccine Network project, the UKVN also has working groups to:

1. maintain the priority pathogen list, updating the list as necessary

2. determine the criteria for when investment in the development of a vaccine is the most appropriate response to an outbreak

3. understand the vaccine development process from discovery to licensure, and associated bottlenecks

4. assess existing UK vaccine manufacturing capacity and opportunities to enhance this 

The project team facilitates both the UKVN and its working groups to ensure actions are completed and the group has maximum impact.

Outline summary of project’s last year annual review

1. Project management: Amber Green (Medium Low)
2. Finance: Amber Green (Medium Low)
3. Theory of change: not applicable
4. External engagement: Amber Green (Medium Low)

Overall delivery confidence RAG rating from last annual review: Amber Green (Medium Low)

Summarised key recommendations from the previous review

The following recommendations were made by Vaccines Project and accepted by the programme board at the last annual review:

Project management

	Recommendation	Current status
1	Invite feedback from delivery partners on our relationship management to identify further improvements.	Complete
2	Proactively increase engagement with Department for International Development (DFID) teams and the World Health Organisation (WHO) R&D blueprint teams to inform project strategy and prevent duplication of work.	In progress
3	Produce a clear milestone delivery plan and log frame for Apr 2018-Mar 2019 to support project progression and aid delivery assessment.	Complete

Finance

	Recommendation	Current status
4	Work closely with delivery partners and, in particular, with the partners for who expenditure is in the early stages, to ensure the ODA reporting requirements are well understood.	Complete
5	Ensure delivery partners provide updates on project progress at agreed intervals, including actual expenditure and revised forecasts, to increase DHSC confidence in the in-year monitoring returns submitted to DFID/HMT.	Complete
6	Publish documents for the UK Vaccine Network on GOV.UK or an alternative accessible URL to increase the project transparency score. DFID have a list of 12 recommended documents to publish (if available), which include: Business Case; Log-Frame; MoU/MoU amendments; Annual Review; and Evaluation Report.	In progress
7	Encourage delivery partners to attend the DHSC Transparency workshop, which is scheduled to be hosted in early summer 2018, to ensure that they are aware of their Transparency requirements.	Complete

Theory of change

	Recommendation	Current status
8	The developing evaluation strategy should include review of all assumptions. For example, using analysis of application numbers to each competition and interviews/surveys of the research community to assess assumption 2. This will also help inform future strategy.	Complete

External engagement

	Recommendation	Current status
9	Finalise and implement project communications strategy to improve proactive communications and extend reach, including building on relationships with stakeholders to collaborate where beneficial.	Complete
10	Continue to support UKVN academics to publish the outputs of working groups 1, 2 and 3 in academic journals to raise awareness of the website tools within academic audiences.	In progress
11	Linked to recommendation 8, develop the evaluation process so that it considers the reach of the competitions to understand how to improve this for future competitions.	In progress

Key successes

Key achievements include:

Delivery of the final second-stage Innovate UK competition, meaning that all planned research competitions under the Vaccines Project are now complete (7 unique competitions, 2 of which had second stage competitions). All the projects successful in these competitions have been contracted and the portfolio now consists of 78 research projects, covering all 12 priority pathogens and including pre-clinical and clinical vaccine development and research on associated vaccine technologies and methodologies. This diverse portfolio has also enabled full use of the UK Vaccine Network project budget in financial year 2018 to 2019.

Continued development of project management processes including:

• production of a project logframe
• development of delivery partner scorecards
• continued evolution of the Project Delivery Board

These tools have strengthened relationships with delivery partners, allowed shared learning and improved the accountability of project delivery.

Development of the first UK Vaccine Network project Communications Strategy and a corresponding increase in communications from the project, including tweets, press releases and short films. This work provides a base for further engagement of the public, the scientific community and wider government.

Delivery of a successful UKVN meeting in November 2018, which included detailed discussion on future priorities of the network, challenges of experimental Ebola vaccine use and analysis of the vaccine development funding landscape. These discussions will inform our thinking for possible future iterations of the UK Vaccine Network project.

Joining the Coalition for Epidemic Preparedness Innovations (CEPI) with a £10 million investment to the organisation. CEPI is a key international stakeholder in the development of vaccines against diseases which cause epidemics in LMICs, especially beyond the early clinical stages of development. The UK investment in this organisation demonstrates our commitment to tackling these diseases and complements the UK Vaccine Network project, ensuring it is connected to international work.

Project management

Delivery confidence assessment RAG rating for reporting year: Amber Green (Medium Low)

Changed since last year: No

1. Evidence of managing the delivery of project

Quarter	RAG rating
1	Green
2	Amber Green
3	Amber Green
4	Amber Green

Overall delivery RAG rating over the reporting period: Amber Green

Key points

Overall, the project is delivering well, including the delivery of all planned competitions by delivery partners as scheduled.

The key changes to internal project processes in this reporting period are:

1. Development and introduction of delivery partner scorecards to support relationships between DHSC and delivery partners and performance management.

2. Further development and embedding of the Project Delivery Board. The board met for the first time in February 2018 to improve co-ordination between DHSC and delivery partners, to discuss common issues across delivery partners and to share lessons learned both by the project team and by the delivery partners. The recent independent process evaluation of the UK Vaccine Network project found that these meetings had evolved from ‘resolving basic cash flow projection and reporting issues, to decision-making on key strategic issues’ and that all delivery partners ‘expressed support to the Project Delivery Board mechanism,’ with some commenting that these meetings allow the building of ‘a wider team’.

3. Introduction of quarterly finance meetings with Innovate UK to allow a more in-depth understanding of their financial processes. These meetings were instigated in response to repeated unforeseen changes in financial forecasts from research projects.

4. Project progress reporting processes have been agreed with all delivery partners. These have been developed in partnership with each delivery partner to ensure alignment with their existing processes, as far as possible.

During the year, the RAG rating fell from Green in Q1 to Amber Green in remaining quarters based on 2 factors:

1. Delays in the contracting and initiation of new projects from competitions held in Q4 of 2017 to 2018. For a number of projects this was only a one-month delay. However, for some projects within the Epidemiology for Vaccinology competition this delay was up to 4 months, largely due to external factors (including project leads being engaged with work to support the response to the ongoing Ebola outbreak #10 in eastern Democratic Republic of the Congo). All projects are now contracted and have begun work against agreed milestones. Projects have now reprofiled work to account for delayed start dates.

2. Ongoing uncertainty around the accuracy of financial forecasts and delays in the attainment of other financial information, including finalisation of the 2018 to 2019 accruals. See finance section for further details.

2. Evidence that the project is meeting the agreed milestones and deliverables (delivering against the stated project objectives)

The following milestones (deliverables) were made by the Vaccines Project for this reporting year.

Output 1: high-quality research

High-quality research that aims to:

• feed the vaccine development pipeline for 12 priority pathogens
• test new platforms and technologies to accelerate vaccine development
• produce processes and products to support vaccine manufacture and delivery in LMICs

Produce and test epidemiological models for optimal vaccine deployment for UKVN priority pathogens.

Output indicator	Milestones (deliverables)	Current status
1.1	Competitions that cover these 4 research areas are run successfully. Year 2 milestone: Epidemiology (CCF) and One Health (BBSRC) competitions completed.	Complete
1.2	Number of projects active and completed. Year 2 milestone: >70 projects contracted (cumulative), around 20 of which are complete.	Complete
1.3	Number of externally published, peer-reviewed papers made available in open access format. Year 2 milestone: ≥2.	Complete
1.4	Proportion of completed projects receiving follow-on funding (either from further UKVN competitions or other sources). Year 2 milestone: 10%.	Under review
1.5	Number of non-academic impacts, for example, patents filed, recorded engagement of research with policy. Year 2 milestone: ≥2.	Complete
1.6	UK research from across academia and small and medium-sized enterprises (SMEs) is accessed to support project aims. Year 2 milestone: ≥25% active projects are from SMEs.	Under review

Output 2: clear UK vaccine investment strategy contributes to global leadership in this space

Milestone/deliverable: use of UKVN policy tools by international stakeholders

Output indicator	Year 2 milestones	Current status
2.1	Publication of academic papers to raise profile/awareness of UKVN working group findings in community. Continued development of outcomes, where appropriate.	In progress
2.1	Statistics from website show 10% increase in visits from 2017 level.	Complete

Milestone/deliverable: UK Vaccine Network project funded research supports the development of collaborations between LMIC and UK researchers and organisations.

Output indicator	Year 2 milestone	Current status
2.2	≥10% of entire portfolio has active collaborations with LMIC-based researchers or organisations.	Complete

Milestone/deliverable: findings of Vaccines Project funded research are disseminated to non-academic audiences, including public health practitioners and the public.

Output indicator	Year 2 milestones	Current status
2.3	Communications strategy developed	Complete
2.3	≥ 3 press releases announcing projects funded	Complete
2.3	≥ 12 vaccine relevant tweets/retweets from GHS account	Complete
2.3	≥ 2 case studies on website	In progress

Milestone/deliverable: UKVN strategy clear and communicated to research community and other stakeholders.

Output indicator	Year 2 milestone	Current status
2.4	Presentation of UKVN portfolio by policy team at 2 or more academic or policy conferences to demonstrate scope.	Complete

Output 3: effective management, governance and oversight of the Vaccines Project

Milestone/deliverable: project budget is fully committed and investments represent Value for Money (VfM).

Output indicator	Year 2 milestones	Current status
3.1	£25 million 2017 to 2018 budget spent	Complete
3.1	≥2 case studies of projects produced to demonstrate impact and VfM of spend	In progress

Milestone/deliverable: project delivery approach allows competitions to be delivered to timelines and delivery risks are identified and managed.

Output indicator	Year 2 milestones	Current status
3.2	3 competitions delivered by 3 delivery partners	Complete
3.2	Quarterly Project Delivery Boards and delivery partner performance management process show effective risk management	Complete
3.2	Interim process evaluation finds approach was effective	Complete

Key points

All competitions have now been successfully delivered and all successful projects have been contracted and are progressing against agreed milestones. We have a portfolio that includes 78 research projects; 39 of which are ongoing and 39 of which have now completed.

The completed projects all sit under the Innovate UK 2-stage competitions where a larger number of smaller grants were given for a one-year project period, then these projects were all eligible to apply for second stage funding and the most promising projects were selected for continued funding.

As a result of the successful competitions and completion of all contracting processes, the UK Vaccine Network project budget in financial year 2018 to 2019 (£25 million) was fully used (indicators 1.1 and 1.2).

A process evaluation was also commissioned and completed during the reporting period to assess whether the multi-delivery partner model was an appropriate approach for the UK Vaccine Network project.

The review found that this was an ‘efficient and appropriate arrangement’ to: cover diverse thematic areas, reach out to respective communities and to receive relatively good quality response despite the time pressure (indicator 4.2).

To support the project management across delivery partners and the project portfolio, the Project Delivery Board (established in February 2018) continue to evolve and the process evaluation provides evidence that these meetings are useful both for project management and from the wider perspective of delivery partners.

In line with recommendations from the last annual review, the team have also developed a quarterly scorecard performance management process with delivery partners and this has now been used for 2 quarters. This new process will be reviewed in Q2 of 2019 to 2020 to assess its value in supporting project delivery and to identify potential improvements.

The indicators under output 1 of the logframe are designed to measure whether high quality research has been identified and funded. All but one of these indicators have been met.

Notably, the numbers of externally published, peer-reviewed papers and of non-academic impacts (indicators 1.3 and 1.5 respectively) exceeded the targets with 8 papers and 8 non-academic outputs recorded across the portfolio (as opposed to the target of 2 for each).

This provides a strong indication that high-quality research is being funded. Future targets should also be reviewed and adapted in light of these higher numbers.

Analysis against indicator 1.4 (which was unmet) identified a complication when calculating the percentage of projects that attained follow-on funding (target was set at 10%). This challenge was whether to include the projects that attained follow on funding from the 2-stage Innovate UK competitions within the calculation.

Successful second-stage projects have new objectives and a new research contract is signed, so are counted as new projects in the portfolio analysis (within the total portfolio of 78 projects, 11 are second-stage projects), with the original first-stage project then counted as closed.

However, including the projects that were successful in the second-stage competitions is arguably not comparable to those projects that have obtained follow-on funding from an alternative funding source (stage 2 competitions are competitive but only projects funded in stage 1 of the competition are eligible to apply).

This will be clarified in the logframe for future years. However, for this year, if second-stage projects are:

• included,16 of 78 projects (20.5%) have received follow-on funding for continued development of the vaccine or platform
• not included, 5 of 67 unique projects (7.5%) have received follow-on funding for continued development of the vaccine or platform

Analysis against indicator 1.6 (25% of active projects from SMEs) also identified a complication. When this target was set, the proportion of SMEs of the whole portfolio was intended to be reflected.

However, the majority of SME-led projects are within Innovate’s preclinical competitions which have now closed. Therefore, whereas 29% of funded projects overall are SME-led, only 18% of active projects are.

The majority of communications targets have been met, apart from publication of case studies. The development of these case studies was delayed due to resource constraints.

However, both studies are in draft form and the team plan to publish these by summer 2019. Further detail on the results of communications work can be found in the external engagement section.

3. Evidence of risk management

The following risks were identified and managed by the UK Vaccine Network project for this reporting year.

Risk 1

Failure of individual projects funded by the programme.

Mitigation actions

Funding is spread across projects working on the full range of pathogens and using diverse technologies and scientific techniques. Working closely with delivery partners to identify and support projects with difficulties to prevent failure due to poor project management. Use of project delivery boards to highlight common issues and for delivery partners to connect projects to share best practice and learning from both positive and negative results, where appropriate.

RAG rating: Green

Current status

Likelihood of this risk at the individual project level remains high, however overall impact is low due to spread of funding across projects. The project team, working with delivery partners, are also considering how to encourage the sharing of negative results and to also capture the reasons for, and share lessons learned around, scientific failure of projects, where appropriate.

Risk 2

Continuing changes in project forecasts results in increased funding pressure in future years and underspending against this year’s HMT ODA target.

Mitigation actions

1. Continue close working with delivery partners to have latest information and to understand likely upcoming changes in finances. The team will also now include a budget update at each Project Delivery Board meeting.
2. Quarterly finance meetings with Innovate UK finance business partner (largest spend through Innovate UK).
3. Work closely with the GHS finance lead, to monitor forecasts and produce adjusted profiles to cover different scenarios.

RAG rating: Red

Current status

Full use of the budgets in 2017 to 2018 and 2018 to 2019 helps to manage the spending pressure in future years and we are not overcommitted against the remaining programme budget of £50 million. However, forecasting accuracy remains a challenge to ensuring the spend is evenly spread across the remaining 2 years of the programme.

Risk 3

Capacity of partners to engage with the UK Vaccine Network project team is reduced as a result of wider changes as UKRI is formed.

Mitigation actions

Continue to monitor UKRI transition process through discussions at the Project Delivery Board and in regular meetings with relevant delivery partners.

RAG rating: Amber Green

Current status

This risk has not materialised and likelihood remains low based on strong relationships with delivery partners.

Risk 4

GMP (Good Manufacturing Practice) manufacture issues with clinical projects results in failure or need for increased spend/time to meet project aims.

Mitigation actions

1. Continue to facilitate lesson learning between delivery partners so that identified mitigating actions from one project team can be shared with other project teams.
2. Scientific scrutiny by delivery partner on causes of any failures, to identify which are due to human error vs. scientific failure and therefore where financial extensions should be considered.

RAG rating: Amber

Current status

This risk first materialised as an issue toward the end of the previous reporting period and has continued to present this year. Mitigating actions are being used to reduce the impact.

Key points

The accuracy of financial forecasting by the research projects within the portfolio remains a risk to the UK Vaccine Network project meeting both its budget and the HMT targets in future years. To help manage this issue, the team have established quarterly financial meetings with Innovate UK (who manage the majority of the project portfolio).

Through these meetings we are ensuring we obtain financial information as promptly as possible and will continue to work with all delivery partners to support projects to provide financial information that is as accurate and timely as possible.

Working within the financial processes of delivery partners limits our ability to change how financial data is collected from projects. To help address this the GHS financial lead has been working to apply risk-adjustments to our forecasts using financial patterns from previous years.

Risk 4 above has materialised this year, with manufacturing issues arising in a number of projects. Scientific issues with projects are to be expected in an innovative research portfolio and through the Project Delivery Board we have shared the identified issues and have produced a standardised process to consider contract variation requests, where this is deemed both necessary for the desired impact and possible from the management perspective.

Summary of overall project delivery issues and recommendations for improvement

1. The biggest ongoing risk to project delivery is the accuracy of financial forecasting. Recommendations to improve and mitigate this are covered in the finance section.
2. The use of scorecards to assist performance management and relationships with delivery partners were instigated this year. Recommendation: this process should be reviewed by the end of Q2 to understand value and identify potential improvements.
3. The project logframe was developed in response to a recommendation in the 2017 to 2018 annual review and this forms the basis for the indicators in this section of the review. Through the process of completing this review and from the process evaluation findings, it is clear that further work is needed to align DHSC reporting requirements and project output reporting (to delivery partners). This information is crucial for monitoring and for impact evaluation of the UK Vaccine Network project. Recommendation: develop clearer mechanisms for output/impact reporting from projects to DHSC, through alignment with delivery partner existing mechanisms, where possible.
4. Review and update the logframe, including updating targets set for upcoming years, and update the indicator for follow on funding to clarify what is being measured.

Finance

Delivery confidence assessment RAG rating for reporting year: Amber Green (Medium Low)

Changed since last year: No

4. How the is funding being used

Annual summary

Total annual budget for this reporting year: £25 million

Total annual spend for reporting year: £26.23 million

	Budget allocation per activity	Committed spend	Actual spend per activity (including fees)
Innovate UK (£10m)	£0.42m	£0.41m	£0.63m
Innovate UK (£25m)	£1.10m	£1.08m	£3.52m
Innovate UK (£35m)	£12.66m	£12.41m	£9.86m
NETSCC)	£4.25m	£4.17m	£3.59m
EPSRC	£3.15m	£3.09m	£5.4m
BBSRC	£1.71m	£1.67m	£1.98m
NIHR CCF	£1.70m	£1.67m	£1.02m

All figures rounded to 2 decimal places.

5. Evidence for effective budget management and forecasting

	Date	Quarter 1	Quarter 2	Quarter 3	Quarter 4
Planned budget	13 Jun 2018	£5.01m	£8.11m	£4.93m	£6.94m
Forecast	Mar to Jun 2018	£4.98m	£5.98m	£7.59m	£5.69m
Actual spend	10 Apr 2019	£4.65m	£4.49m	£5.25m	£11.84m

Provide summary-level explanation for deviations between forecast and actual spend

Key points

Deviations between forecast and actual spend in Q1 and Q2 were the result of slightly slower expenditure than expected across a range of projects, for example in one competition the start dates of 2 of the 5 projects were delayed from Q2 to Q3.

A significant amount of expenditure planned for Q3 was delayed until Q4. There was also additional, unforecasted expenditure from projects managed by Innovate UK in Q4. This additional expenditure increased the total project spend for 2018 to 2019 to £26.23 million (over the £25 million budget by £1.23 million).

We worked closely with Innovate UK during Q4 to increase accuracy of the forecasts as Q4 progressed so that we had time to discuss how to manage this increase with other DHSC projects. As a result, this overspend was manageable through underspend across the wider DHSC ODA portfolio in this financial year.

We will continue to work with Innovate UK and all delivery partners to increase forecast accuracy. The UK Vaccine Network project is not overcommitted against the remaining programme budget of £50 million. However, forecasting accuracy remains critical to ensure the spend is evenly spread across the remaining 2 years of the programme.

While not part of the original investment strategy, in 2018 to 2019 there was the opportunity to complement the UK Vaccine Network project portfolio by supporting the Coalition for Epidemic Preparedness Innovations (CEPI) with an investment of £10 million, funding that became available due to underspend in the wider Global Health Security programme budget.

As this was not part of the original UK Vaccine Network project budget, this is not reflected in the tables above to simplify analysis of the budget management.

6. Evidence of ability to administer ODA funding

Outline any process changes to finance reporting and monitoring

Key points

Further work is needed to allow confidence in financial forecasting across the UK Vaccine Network project.

The project team has set up quarterly meetings with Innovate UK finance and project staff (as the holder of the majority of the budget), with the aim of proactively aiding communication between both parties’ finance and policy colleagues and ensuring that, where forecasts change, all are made aware as early as possible and able to consider if any action is required across the project to mitigate changes.

These regular finance meetings have helped to us to improve the understanding of Innovate UK and DHSC’s financial processes within the 2 organisations.

As a result, the team has identified that Innovate UK’s standard forecasting process does not provide an accruals-based picture of expenditure, which would support the production of accurate quarterly forecasts by the DHSC finance team.

We are now working with Innovate UK to address this by the end of Q1 2019 to 2020. To address the limitations in our ability to change processes we have introduced a mechanism to risk adjust the forecasts (see below).

At the level of individual research projects, the DHSC team have worked with delivery partners to develop suitable financial reporting processes.

Quarterly reports have been received from Innovate UK since September 2018 and these include a financial update on each project, allowing the UK Vaccine Network project team to see and understand any emerging issues that may affect spend before this appears in the invoices (which can be up to 3 months after spending).

These actions aim to help the UK Vaccine Network project team understand when expenditure is moving into later quarters, discuss the issues early and incorporate this into DHSC forecasts. DHSC receives financial information on a quarterly basis from NETSCC and NIHR-CCF and 6-monthly financial reports from EPSRC on the manufacturing hubs which allow the team to understand spend progression by these projects, even though the funding is provided on a flat-profile. A similar report is received by BBSRC on an annual basis.

The GHS team has recently introduced risk-adjusting to forecasts, whereby the forecasting and expenditure of our delivery partners is modelled to predict over and under-spends based on patterns from previous years and quarters.

The DHSC finance team will be working with the UK Vaccine Network project team in Q1 2019 to 2020 to ensure that risk adjustments are appropriate and will engage delivery partners in this process, as appropriate.

7. Evidence of activities undertaken to meet International Aid Transparency Initiative (IATI) transparency standards

Self-assessed score against the IATI transparency standards: Very Poor, Poor, Fair, Good, Excellent

Project score: Good

Summary of what steps UK Vaccine Network project have taken to ensure transparency of activities

Key points

To ensure transparency, the UK Vaccine Network project team has worked with the GHS Finance Manager to ensure financial information for funded projects is uploaded on to dev-tracker and d-portal. We are undergoing a process currently to ensure this information is up to date following subsequent project extension requests.

The following information for each of the competitions is also available:

• general description
• objectives
• target group and related activities

The team is also preparing the following documents for publication:

• 5 MoUs that are in place with delivery partners
• the UK Vaccine Network project Business Case
• Log Frame and Theory of Change documents

The 2017 to 2018 Annual Review is also published. It is intended that these documents will be published ahead of the first Publish What You Fund assessment of GHS later in 2019.

Summary of finance issues and recommendations for improvement

This year has highlighted the importance of understanding the underlying financial processes of delivery partners to ensure accurate forecasts.

This enhanced understanding of the limitations of both DHSC and partner financial systems has resulted in a need to ensure accurate risk adjustment is carried out by DHSC as well as continuing ongoing actions to improve forecast accuracy with partners.

Theory of Change

8. Evidence to show if the theory of change assumptions remain accurate

The Theory of Change project identifies 9 underlying assumptions

1) Budget available to meet project proposal requirements.

The UK Vaccine Network project budget has remained fixed since inception of the project, despite a reduction in Gross National Income in 2017 to 2018. We do not anticipate the budget will change in the course of this Spending Review period, particularly given that the funding is now fully committed.

2) Research market capable and responsive to funding calls.

Some competitions received lower than expected application numbers. However, the recent process evaluation did not find an explanation for this and anecdotal evidence suggests that this may have been due to low awareness of the competitions concerned, rather than saturation of the research field. All funding calls are now complete and the full UK Vaccine Network project budget has been fully committed. All projects were selected by independent review panels to ensure quality was maintained, even when application numbers were lower than expected.

3) Competition process attracts and selects high-quality research proposals.

4) Researchers identify key questions and issues in vaccine development, manufacturing and deployment.

5) Research projects deliver positive results that enable further product development.

6) Findings from epidemiological models can be turned into realistic protocols.

As with the review last year, assumptions 3, 4, 5 and 6 remain reasonable based on the rigorous competition process that our delivery partners apply to ensure the best projects are identified.

Furthermore, emerging early outcomes from the project portfolio, such as projects receiving follow on funding for vaccine development and presenting research findings at conferences and in journal publications, provide evidence to support these assumptions.

7) Other funders provide follow-on investment for the most promising vaccine candidates and technologies developed.

There is developing evidence in the portfolio that this assumption is accurate as we have seen 5 projects acquire external follow-on investment (alongside the 11 projects that obtained UKVN follow-on funding in stage 2 competitions:

• 2 projects had received follow on funding at the time of the last Annual Review (projects led by Prokarium Ltd. and Themis Bioscience)
• 3 further projects have received follow on funding:
  • CEPI has provided further funding to support development of a MERS vaccine (led by Jenner Institute and Janssen) and a self-amplifying RNA platform (led by Imperial College) which are currently being supported in the portfolio
  • Innovate UK has awarded funding to support the adaptation and further development of a vaccine platform from our portfolio so that it can be used to develop an influenza vaccine

From this evidence and analysis of the funding landscape, CEPI is an important stakeholder for the further development of promising vaccines. Consequently, and of relevance to this assumption, the DHSC has now committed £10 million of funding to CEPI to support their mission to develop promising vaccine candidates against epidemic diseases through phase 2 clinical trials and to enable stockpiling of these vaccines for use in outbreak situations.

If the UK Vaccine Network project funded research portfolio is very successful, there will be a large increase in the number of new vaccine candidates requiring next stage testing. There is a risk that the global system will not be able to accommodate this increased demand. The team should therefore seek to engage more fully with CEPI and other late-stage R&D funders who are able to support phase 2 trials to assess this risk and to ensure alignment of future funding strategies.

8) Phase 2-ready vaccines are effective when deployed.

The efficacy of a promising phase-2 ready vaccine before deployment cannot be known, therefore there is some level of assumption required here. However, a paper published in the Lancet this year provides an analysis of the probability of vaccine candidates proving successful at any given stage of testing.

This research suggests that 4 to 5 successful phase 1 vaccine candidates would be required to ensure that at least one vaccine candidate successfully completes phase 2a of clinical trials (which show the vaccine is effective in protecting against the pathogen in a natural environment).

This analysis suggests that our assumption should be revisited to ensure it is consistent with this evidence. For example, it should not be assumed that any given candidate will be effective, but rather that if there are a number of phase 2 ready candidates (from this analysis 4 to 5) then one of these will be effective when deployed. The UK Vaccine Network project supports this new assumption by increasing the likelihood of a greater number of vaccine candidates reaching phase-2 testing.

Review of these assumptions with DFID colleagues highlighted that there is a further assumption that follows assumption 8 regarding how candidates that are successful at phase 2 will be selected, approved by regulators and made available for use in the countries where vaccines are needed. This process will differ for the different priority pathogens. DHSC should work with stakeholders such as the WHO R&D blueprint and CEPI to understand these routes to impact and contribute to the development of a system that will allow effective vaccines from this portfolio to have impact where they are needed.

9) Country and political acceptance of vaccine deployment.

WHO highlighted vaccine hesitancy as one of the top 10 threats to global health in 2019 and therefore this assumption should be regularly revisited. The role of vaccine hesitancy during emergency use of an experimental vaccine in an outbreak setting (the context in which vaccines in this portfolio would likely be first used) is complicated.

However, the WHO has guidance on community engagement and risk communication which is being used alongside the experimental vaccine in the current Ebola outbreak in the DRC. The DRC trial suggests that when vaccine deployment is informed by community engagement, experimental vaccines can be acceptable to communities.

Within the UK Vaccine Network project portfolio there is a project investigating vaccine acceptance, amongst other factors, in a range of LMIC case studies. Emerging findings from this work will also provide evidence towards this assumption.

To support vaccine acceptance in the LMICs affected by these pathogens it is important for vaccine development to be informed by, and in partnership with, researchers and organisations in these countries. Because of UK Vaccine Network project implementation being through UK research organisations, all projects in the current portfolio are led by UK-based researchers, with ≥10% of projects having LMIC collaborations.

The design of future funding strategies for the UK Vaccine Network project should look to increase LMIC involvement, both through encouraging the development of UK-LMIC collaborations in the competition assessment criteria and through global funding calls.

10) There is global/country capacity to deliver vaccines in emergencies.

11) Protocols for vaccine deployment/trialling are successful in context.

Assumptions 10 and 11 are much broader, dependent on many other stakeholders and are also context specific to any given outbreak.

However, the current deployment of the experimental Ebola vaccine in the DRC demonstrates that this is possible, although extremely challenging.

Summary of any changes recommended to the Theory of Change

1. Relevant to assumption 7, the UK Vaccine Network project team should increase coordination with CEPI and other downstream vaccine R&D funders to proactively support the transition of projects between funders, in the context of open competition, and to ensure future funding strategies are aligned.
2. Assumption 8 (phase 2 ready vaccines are effective when deployed) should be updated to reflect the need for multiple phase 2 ready candidates before this assumption is reasonable.
3. An additional assumption should be added relating to the selection, regulation and availability of the most successful and suitable vaccine candidates following phase 2 trials. The UK Vaccine Network project team should work with relevant stakeholders, including LMIC stakeholders, to assess these routes to impact for the vaccine candidates in the portfolio and identify if/how corresponding bottlenecks and challenges can be addressed globally.
4. For any future iteration of the UKVN and associated funding, the UK Vaccine Network project team should:

• consider and incorporate learning from the large-scale use of the experimental Ebola vaccine in a complex outbreak environment. Learnings from this experience, will inform assumptions 9 to 11, providing evidence that should be utilised in future programme design

• look to increase links to LMICs that are affected by the relevant diseases, engagement should be both at the project design stage and within R&D collaborations from the outset of projects

External engagement

Delivery confidence assessment RAG rating for reporting year: Amber (Medium)

Changed since last year: Yes

9. Evidence of use and success of the communication strategy

Key points

The objectives of the 2018 to 2019 communications strategy were to raise awareness of funding that DHSC is providing to support development of vaccines against diseases of epidemic potential in LMICs, to boost the reputation of the department and to raise the profile of the UK Vaccine Network.

There were 3 target audiences for these objectives:

• the UK general public
• the UK scientific community
• internally across DHSC and government

The communications strategy combined the use of social media, formal press releases, video production and informal news stories.

The strategy was ambitious and aimed to exceed the key targets.

Over the 2018 to 2019 period, in line with the strategy, we have delivered the following communications activities:

• 4 press releases or news stories about the work being funded, including ministerial quotes
• an internal blog to raise awareness of the work of the GHS team within DHSC
• a GOV.UK news story to announce funding to CEPI
• 12 Twitter posts from the GHS Twitter page and one from the DHSC Twitter page and 8 retweets which are directly connected to the UK Vaccine Network project from either GHS or DHSC pages
• 11 videos showcasing elements of the 2 vaccines hub projects have been developed. These have been shared on social media and internally within the GHS team, and these will be further used in 2019 to 2020

These activities cover a range of content, including announcements about specific projects (for example, announcement of an Ebola and Lassa project by University of Cambridge), direct partnerships (for example, funding to CEPI) and announcements of competition outcomes.

The key content from the strategy was largely delivered, however not all of the planned or supporting activities were, due to resource constraints. Much of the activity was also delivered as small bursts of activity with subsequent long periods with little communications activity.

These limitations mean that not all of the completed communications activity has had the full desired impact. As an example, the strategy planned to engage with external organisations in the run up to European Immunisation Week and deliver co-ordinated communications.

In practice, though substantial DHSC communications were undertaken, including tweets about the UK Vaccine Network project portfolio (including one from DHSC’s Twitter page) with a quote from Chris Whitty (Chief Scientific Adviser) and some initial engagement with the Wellcome Trust was undertaken, we were not able to drive engagement with external organisations to the extent planned.

This meant the activities were less co-ordinated with others than desired. Similarly, 11 videos were successfully produced and shared on social media, internally and with delivery partners as part of communications work surrounding the Vaccine Manufacturing Hubs from the portfolio. However, not all of the videos have been shared and there is potential to utilise these further to reach a wider audience.

The future communications strategy will need to use learning from this year to deliver maximum communications impact within current resource.

The communications strategy in this reporting period has focused on showcasing individual projects and/or competitions under the Vaccines Project. The Vaccines Process Evaluation that was completed in February 2019 found that public visibility of the Vaccines Project and the UK Vaccine Network is low and recommends communication of the complete portfolio to an external audience as a priority.

The revised strategy is currently under development and will have a particular focus on communications about the whole portfolio and the rationale for the investment strategy. This will aim to improve awareness of the UK Vaccine Network project as a strategic, coherent portfolio of investments rather than individual competitions, as anecdotal evidence suggests the portfolio was previously viewed.

Additionally, the learning of the importance of this will be taken forward in any future iterations of the UK Vaccine Network project and shared with other teams.

10. Evidence of external engagement (other)

Key points

The research community

The routine engagement of the UK Vaccine Network project team with the UKVN ensures that the team regularly interact with members of the research community. The UKVN is composed of 33 experts, attending meetings as individuals but with involvement in 29 different organisations (including research institutions, industry and government departments).

As a consequence, activities from this reporting period including the UKVN meeting and secretariat engagement or group meetings with working groups 1, 2 and 3 has naturally engaged a range of key stakeholders with the project.

Prior to the 2018 to 2019 reporting period, outputs of working groups 2 and 3 were published online through a website hosted by the Medical Research Council (MRC). These outputs include a guide to support decisions on whether supporting vaccine development is a recommended response to an epidemic (WG2) and a process map of the stages required for the end-to-end development of a new vaccine which can be used by researchers and decisions makers alike (WG3).

To increase awareness and engagement of the community, we planned to publish papers in the 2018 to 2019 period to highlight both these tools and the work of Working Group 1 (around prioritisation of pathogens). Publication of academic papers by the UKVN working groups is in progress, with one journal article accepted and one ready for submission.

Following further discussions, the final working group has chosen to continue development of the decision tool before publication to make the tool as useful to the community as possible. The UKVN secretariat will continue to support the working group to refine the tool before publication.

Despite the delay in publication we have seen an increase in website users, with 862 users viewing 3,695 pages in 9 different languages in this 12 month reporting period, compared to the 428 users in the first 7 months of the website (September 2017 to March 2018). This equates to an approximate 16% increase in users when averaged per month.

The external engagement and communications activity to date has focused on the UK research community. These activities in 2019 to 2020 should consider how to engage internationally, particularly with the research community in LMICs, within the existing resource restraints. This will be increasingly important if there are future funding calls.

Delivery partners

Responding to the feedback we sought as part of the 2017 to 2018 UK Vaccine Network project annual review process, we have increased the routine engagement of our delivery partners through quarterly performance monitoring conversations.

Two rounds of these conversations have now been held. Engagement with delivery partners has also continued to develop through the Project Delivery Board.

The view from the secretariat, anecdotal feedback from partners and the process evaluation all reflect the Project Delivery Board as having a positive impact on DHSC’s relationship with partners.

Wider stakeholders

In addition to those described above, multiple internal and external partners have been engaged during the reporting period to increase awareness and collaboration to maximise the impact of these investments, and to inform strategic thinking ahead of the next spending review period. These include:

• the CSA for DHSC
• relevant DFID teams
• relevant WHO stakeholders
• the Wellcome Trust vaccines team
• specific individuals representing UK Academic Institutions

The UK Vaccine Network Project has also engaged with CEPI, a key stakeholder in the area of epidemic vaccines development, both through CEPI representation at the UKVN meeting in November 2018 and through DHSC’s new position as an investor in CEPI.

The UKVN project portfolio has been presented by the policy team at an event at the London School of Hygiene and Tropical Medicine launching all of the NIHR-CCF Epidemiology for Vaccinology projects. The portfolio was also presented at an event hosted by the MRC for the vaccine networks set up through the Global Challenges Research Fund.

This audience included representatives of all 5 of these vaccine networks as well as attendees from the MRC. Both of these events have engaged relevant members of the scientific community.

The Vaccines Project has also engaged with the Independent Commission for Aid Impact (ICAI) to support their review of ‘How UK Aid Learns’.

Continuing and increasing engagement with DFID teams and the WHO R&D blueprint teams to inform project strategy and prevent duplication of work will be a priority for 2019 to 2020.

Summary of engagement issues and recommendations improve the effectiveness of stakeholder and delivery partner engagement

1) Focus 2019 to 2020 communications strategy on delivering communications which provide a narrative of the whole Vaccines Project portfolio, to complement ongoing work highlighting individual projects.

2) Maintain momentum of external communications by considering timing of communications activities in the 2019 to 2020 communications strategy and balancing the types of communications based on resource-intensity (for example, social media vs. press releases) to ensure a steady stream of engagement.

3) Collaborate with relevant external organisations to:

• enhance the reach and impact of communications activities, making use of existing content already in place, for example, the manufacturing hub videos
• increase our communication activities to engage the LMIC research community ahead of future competitions

4) Increase engagement with the WHO R&D blueprint teams to inform project strategy and prevent duplication of work will be a priority for 2019 to 2020.

Lessons identified

Key points:

1. The benefit of proactive delivery partner engagement was identified as a lesson in the previous annual review. The continued evolution of the Project Delivery Board has demonstrated not only the importance of this level of engagement as an exercise in expectation setting but also the added value this can bring for engagement in the programme, shared problem solving and lesson sharing amongst delivery partners. The process evaluation findings further highlight the value of this mechanism.

2. This year has also highlighted the importance of understanding the underlying financial processes of delivery partners. This has been vital for us to provide constructive challenge to forecasts and financial information, while understanding what is not possible or practicable for partners.

3. The communication of the overall strategy of the Vaccines Project investment was highlighted by the process evaluation as a weakness in the project to date. The focus for 2018 to 2019 has been on highlighting individual projects and successes and prior to this, communications had not been a priority due to resource constraints. In response to this finding the team will be working to strengthen communications about the investment strategy and rationale for the existing programme of spend in the coming year. The importance of communicating an overall strategy (as a necessity rather than a luxury) will be taken forward in future iterations of the Vaccines Project and will also be shared with other projects within the DHSC GHS Programme.

4. The process of producing a logframe for the project this year has also raised the importance of clearly connecting the outputs of the Vaccines Project to the desired impact and proactively seeking ways to support the necessary assumptions made in this process. This exercise identified the relevance and importance of CEPI for a number of our priority pathogens and as a consequence has resulted in the Vaccines Project, through wider DHSC ODA budget availability, making a £10 million commitment to this organisation as a key mechanism for ensuring impact of our larger Vaccines Project investment.

Overall project delivery and recommendations

Overall assessment RAG rating

Activity areas	RAG rating	RAG rating change since last annual review?
Project management	Amber Green	No
Finance	Amber Green	No
Theory of change	Not applicable	Not applicable
External engagement	Amber Amber	Yes

Overall delivery confidence rating: Amber Green (Medium Low)

List of recommendations

Project management

1) The biggest ongoing risk to project delivery is the accuracy of financial forecasting.

Recommendations to improve this are covered in the finance section.

2) The use of scorecards to assist performance management and relationships with delivery partners was instigated this year.

Recommendation: this process should be reviewed by the end of quarter 2 to understand value and identify potential improvements.

3) The project logframe was developed in response to a recommendation in the 2017 to 18 annual review and this forms the basis for the indicators in this section of the review. Through the process of completing this review and from the process evaluation findings, it is clear that further work is needed to align DHSC reporting requirements and project output reporting (to delivery partners).

This information is crucial for monitoring and for impact evaluation of the Vaccines Project.

Recommendation: develop clearer mechanisms for output/impact reporting from projects to DHSC, through alignment with delivery partner existing mechanisms, where possible.

4) Review and update the logframe, including reviewing targets set in subsequent years considering this year’s findings and update the indicator for follow on funding to clarify what is being measured.

Finance

5) This year has highlighted the importance of understanding the underlying financial processes of delivery partners to ensure accurate forecasts.

This enhanced understanding of the limitations of both DHSC and partner financial systems has resulted in a need to ensure accurate risk adjustment is carried out by DHSC as well as continuing ongoing actions to improve forecast accuracy with partners.

Theory of change

6) Relevant to assumption 7, the vaccines team should increase coordination with CEPI and other downstream vaccine R&D funders to proactively support the transition of projects between funders, in the context of open competition, and to ensure future funding strategies are aligned.

7) Assumption 8 (phase 2 ready vaccines are effective when deployed) should be updated to reflect the need for multiple phase-2 ready candidates before this assumption is reasonable.

8) An additional assumption should be added relating to the selection, regulation and availability of the most successful and suitable vaccine candidates following phase 2 trials.

The Vaccines Project team should work with relevant stakeholders, including LMIC stakeholders, to assess these routes to impact for the vaccine candidates in the portfolio and identify if/how corresponding bottlenecks and challenges can be addressed globally.

9) For any future iteration of the UKVN and associated funding, the project team should:

• consider and incorporate learning from the large-scale use of the experimental Ebola vaccine in a complex outbreak environment. Learnings from this experience, will inform assumptions 9 to 11, providing evidence that should be utilised in future programme design
• look to increase links to LMICs that are affected by the relevant diseases, engagement should be both at the project design stage and within R&D collaborations from the outset of projects

External engagement

10) Focus 2019 to 2020 communications strategy on delivering communications which provide a narrative of the whole Vaccines Project portfolio, to complement ongoing work highlighting individual projects.

11) Maintain momentum of external communications by considering timing of communications activities in the 2019 to 2020 communications strategy and balancing the types of communications based on resource-intensity (for example, social media versus press releases) to ensure a steady stream of engagement.

12) Collaborate with relevant external organisations to:

• enhance the reach and impact of communications activities, making use of existing content already in place, for example the manufacturing hub videos
• increase our communication activities to engage the LMIC research community ahead of future competitions

13) Increase engagement with the WHO R&D blueprint teams to inform project strategy and prevent duplication of work will be a priority for 2019 to 2020.