Guidance

UK NSC disease, clinical effectiveness and cost effectiveness modelling

Published 29 September 2022

The UK National Screening Committee (UK NSC) is tasked with making a range of recommendations about the use of new and established screening programmes in the UK. These include:

• advice as to whether the benefits of a screening programme outweigh the harms
• whether the overall benefits of a screening programme justify the costs
• how a screening programme should be implemented
• what further evidence should be gathered

The UK NSC can request that a decision analytic modelling approach be applied to generate quantitative information to support these recommendations. Currently decision modelling is requested on a case by case basis. This document outlines a systematic approach to determine when the UK NSC would request the use of a model. It addresses the capacity and resource available to undertake modelling within the UK NSC processes, and the usefulness of models built under different resource and time constraints.

1. Introduction

The goal of the UK NSC is to provide evidence-based recommendations about the use of health screening. Reviews of the best available international evidence are central to the evidence review process used in the development of recommendations. This includes evidence addressing internationally recognised screening criteria on the epidemiology and natural history of a condition, test accuracy, effectiveness of treatment, and the clinical and cost-effectiveness of the screening programme as a whole. Using this evidence, the UK NSC assesses whether benefits of a screening programme would outweigh harms at a reasonable cost to the NHS. Consequently, these decisions also involve assessing benefit, harm and cost, certainty of the evidence, which populations are benefited and harmed, and whether different sub-populations should be screened differently.

Decision analytic models offer a framework for synthesising all available, relevant evidence on pathways of care, the current standard of care and the effectiveness of interventions, such as screening programmes. The impact of any effect is then linked to outcomes and costs to determine cost-effectiveness. Specifically, these models (including decision tree, Markov, and discrete event simulation) assess the incremental benefits, harms, and associated costs of screening programmes by examining the effect of a specific screening strategy versus a comparator (such as current symptomatic care and/or alternative screening strategies).

Models can also explore the clinical effectiveness (benefits and harms) of screening without including costs or they can simulate the natural history of a condition in a disease mode. The final section of this document includes examples of models that have been completed by the UK NSC to inform recommendations. Such models can also highlight uncertainties in the evidence used to determine effectiveness. This can be useful in prioritising further research.

This document provides a statement on the way in which modelling can be applied in the work of the UK NSC and proposes ways of improving and integrating modelling work into the UK NSC’s processes.

2. Rationale for the use of decision analytic models for the UK NSC

2.1 Why the UK NSC uses modelling

The UK NSC uses models to help decide:

• whether to recommend that a screening programme be introduced in the UK and what specific strategy to recommend (for example eligibility or frequency)
• whether and how to modify a screening programme from its current or proposed form
• whether the current evidence is sufficient to support recommendations or to discontinue screening programmes
• how to direct future research to address the most important evidence gaps

Sometimes, the UK NSC uses modelling to explore potential screening pathways and their impact or what they would achieve, to stimulate discussion about how the policy area could progress. More recently, the UK NSC has begun using modelling to assess the feasibility and resource impact of implementing screening programmes that have already been recommended. In the short term, this means assessing if the programme could be implemented now within the current system along with current constraints. In the long term, this means assessing what changes are needed and how long these would take to enable the screening programme to be implemented as recommended.

The UK NSC bases its recommendations to introduce, modify or cease a screening programme on population health outcomes such as length of life, quality of life and freedom from disease and possible or likely harms. These are balanced against the resources required to achieve these outcomes via a screening programme by comparing the value of the outcomes with the value of using those resources in other ways. If the health outcomes and costs that result from the use of a screening programme have been measured directly within studies or in published health economic evaluations, the UK NSC can base recommendations on this existing evidence.

However, if this information is not available, a model may be able to estimate the required information. An example would be the introduction of a screening programme before evidence on its effectiveness had been produced, or if a screening programme is altered in some way from that implemented in the existing studies. In these cases, a model may be able to estimate the required information if there are good data from primary studies on different parts of the screening pathway that can be linked together.

A model may also be able to produce the required information in a reasonable timeframe for decision making rather than commissioning and waiting for a prospective study. Some of the diseases are very rare so the research would be enormous or the timeframes extremely long. In those cases, research might not be feasible. Therefore, modelling can be used to estimate clinical effectiveness, especially in the absence of primary evidence on the clinical effectiveness of a screening programme as a whole and to assess the cost-effectiveness of a screening programme.

In these situations, modelling is useful to:

• assess the benefits, harms, and cost-effectiveness of screening in new or established screening programmes, including different strategies (for example alternative screening tests, appropriate starting or stopping ages, alternative intervals of screening, or trade-offs between different levels of sensitivity and specificity of screening tests)
• translate or summarise the potential benefits, harms, and costs of screening programmes in ways that are more suited to making recommendations
• extrapolate the current evidence base to a longer time horizon or from a different geographical setting to the UK
• estimate the benefits, harms, and costs in different populations

Importantly, models can also assess the uncertainty in the estimates of benefits, harms, and costs, and highlight where in the evidence base the uncertainty lies. As a result, they can be useful to gauge if the evidence base is sufficiently robust to support decision making and focus attention on important evidence gaps to inform discussion on future research priorities. In particular, value of information analyses can estimate the expected gain from reducing uncertainty through primary data collection (for example a trial or epidemiological study) for specific model inputs ^{[footnote 1]}.

Finally, the process of modelling the screening pathways and estimating the health benefits and harms of each step in the pathway can help to define future research studies. For example, modelling can help to define the minimally acceptable criteria for sensitivity and specificity based on the modelled downstream consequences of the pathway, which in turn can inform the hypothesis and sample size for a test accuracy study ^{[footnote 2]}.

Below is a simple illustration of these uses for models in relation to the UK NSC screening criteria and evidence requirements. This is only to demonstrate where models can be useful in different phases of evidence gathering and is not necessarily a depiction of the linear timeline modelling would follow for each screening topic.

Decision analytic modelling for a screening recommendation, programme modification or programme cessation

A recommendation for a screening programme not currently implemented may go through the entire process and every phase (using models and/or other study designs). Once implemented, a modification request may be received which requires some or every phase to start again with modelling.

2.2 When the UK NSC uses modelling

Whether the UK NSC uses models depends on the need and urgency, the state of existing empirical evidence, and the suitability of models for specific purposes. The UK NSC has commissioned models on a case-by-case basis depending on need, appropriateness, and urgency. Previously, the need for models has generally originated from the following scenarios.

1. Changes in the wider context for a screening programme (for example a model was required to assess the benefits, harms, cost-effectiveness, and evidence gaps of antenatal screening for asymptomatic bacteriuria as screening was previously implemented based on National Institute for Health and Care Excellence (NICE) guidance, which was being removed).
2. Where there is trial evidence on the clinical effectiveness of screening but there is the need to determine the most cost-effective and optimal screening strategy for a screening programme as it is not available from the evidence base. This might be because there is no evidence on cost-effectiveness or the evidence available did not include appropriate evidence, outcomes, time horizons or screening strategies. For example, for targeted screening of lung cancer in smokers, a UK NSC review concluded that a clinical and cost-effectiveness model was required to inform the optimal screening strategy in this programme that would result in the benefits outweighing the harms at a reasonable cost.
3. Needs of rare disease setting. For example, a model was commissioned to determine the benefits, harms, and costs of newborn screening for tyrosinaemia type 1 to help the UK NSC understand what could be achieved by screening and whether the cost would be reasonable. Direct evidence to assess the impact of screening would not be feasible due to the rarity of the condition. Modelling was therefore the best method to estimate the impact of screening.
4. Programme modification proposals. For example, a programme modification proposal to use cfDNA in the sickle cell and thalassemia screening programme led to a model to define minimally accepted criteria for sensitivity and specificity to determine sample size calculations for diagnostic accuracy studies.
5. Persistent evidence gaps in the literature. For example, a model was commissioned to explore the potential impact of antenatal screening for vasa praevia in the UK as previous reviews continuously found no evidence around this condition in the UK.
6. Stakeholder engagement. For example, a model was commissioned to develop consensus around specific aspects of antenatal screening for group B Streptococcus.

2.3 Stakeholder engagement

The UK NSC uses an iterative, inclusive and transparent approach to develop a model. Over and above the standardised 3-month consultation for UK NSC policy reviews, important stakeholders are involved throughout the process of developing and interpreting a model. Stakeholders such as clinicians, laboratory experts, methodologists, screening programme managers, and patient and public voice members (PPVs) input into the specification of the model, including the model structure and data inputs. They also check the results and interpretation in the final report. This engagement has been crucial as it allows a shared and transparent set of assumptions to make sure models represent disease processes appropriately and adequately address the decision problem.

3. Proposals to improve and integrate models in the UK NSC process

Now that the UK NSC has had some experience with commissioning decision analytic modelling, the next step is to improve and integrate modelling better into the UK NSC evidence process to ensure long-term sustainability of modelling efforts and consistency and fairness across topics.

This will include a programme of work to develop a topic selection process, improvements in methodology, and infrastructure development for screening models. The UK NSC will work with expert modellers and stakeholders in the field as part of this process. This section highlights our proposal for topic selection. The UK NSC will begin applying this process and refine it over time.

3.1 Selection of topics for modelling

Decision analytic modelling is time and resource intensive. It requires appropriate planning and expertise to ensure clear and consistent results. Modelling is used selectively by the UK NSC for highest priority topics, but the number of topics for which modelling could be considered is expanding for a number of reasons, including:

• an increase in programme modifications for established screening which require quicker decisions than long-term studies allow
• the speed of technological advances
• the expansion of the remit of the UK NSC from population only to population and targeted screening

However, resources for modelling are limited. Therefore, the UK NSC is working to develop a systematic approach for selecting and prioritising topics for modelling, by combining scientific considerations, process issues, and value judgments. Adapting the US Preventive Services Task Force’s framework for topic selection for models ^{[footnote 3]} we propose the following framework (see flowchart below). The framework has 3 questions that should be answered to decide whether to commission a model or not.

Process for determining whether to use decision analytic modelling for a topic

1. What is the purpose of the potential model?

The first objective of the framework is to determine the primary reason for developing a decision analytic model. Where the benefit of screening has been established, modelling may be useful to determine the cost-effectiveness, the optimal strategy, or the feasibility of a screening programme in the UK population. Where the benefit of a screening programme has not been established, modelling may be used to develop an exploratory analysis of screening’s potential impact, characterise the strengths and limitations of an evidence base, highlight the value of future research in response to this, specify the details of future studies, or to support decision making on a candidate screening programme where trial data may not be feasible (for example newborn blood spot screening).

Consideration should also be given to whether the model is likely to be used for purposes other than those that prompted the initial project. For example, in addition to assessing the cost effectiveness of candidate screening programme, one may want to explore the use of the model for subsequent changes to the programme. It might be useful here to consider the additional reasons for a model as this could influence the design of the model.

2. Is the information gained from modelling likely to be worth the opportunity cost of modelling?

After the purpose of modelling has been established, the expected advantages and limitations of using a model must be articulated. The advantages of the information gained from a model would include the value of the recommendation itself and the added value of information that the model would bring to the recommendation (for example, potentially reducing risk of decision error). The opportunity costs of modelling include the resources expended on the model and any extension in the process that delays a recommendation.

Considerations on the advantages and limitations here should include whether a recommendation could be made without a model as well as the consequences of pursuing and not pursuing the model. Consequences could include, for example, potential benefit or harm in terms of population health, and cost savings or wasted resources to the population. To help understand the consequences of a modelling project, thought must be given to the burden of a condition, the urgency of the problem, and whether existing models on the topic address the question of interest.

3. Can the desired scope for the model be clearly outlined?

If the advantages of a potential model are deemed to be worth the costs, it must be determined whether the scope of the model can be clearly outlined or whether additional information is needed from the literature or from key stakeholders. This involves defining the decision problem/objective, including the rationale for decision modelling, population, the decision-important modelling outputs, strategies to be compared, and time horizons in the model. Using the PICOS (population, intervention, comparisons, outcomes and study) framework to specify these modelling constructs could be useful. Consideration must also be given to whether there is sufficient data for model input parameters to develop a useful model. If the main constructs of relevance cannot be defined, it would be helpful to deliberate whether the modelling-relevant information will likely become clearer as the model progresses, or whether a model is premature at this point and a scoping exercise would be more appropriate to start with.

These questions frame the considerations around whether to commission a model for a specific topic, but do not make that decision. This decision will be based on the findings from these questions and judgement about the relative needs and priorities of the UK NSC, taking the overall UK NSC work plan into account. Given multiple competing priorities and resource constraints, this process should help to focus efforts on the most important topics for potential modelling and enhance transparency.

4. Conclusions and future directions

Decision analytic modelling has been useful to inform and explore UK NSC recommendations. It has helped the UK NSC integrate evidence across the screening pathway for topics where clinical trials are not feasible, compare alternative screening methods and strategies, weigh harms, benefits, and costs of screenings, and direct future research.

The next step is to integrate modelling more systematically into the UK NSC’s processes. The selection process outlined here can be used to guide the committee’s decision making on commissioning modelling studies. In the interests of longer term sustainability, the UK NSC aims to establish a workstream to consider the procedural, methodological, and infrastructural considerations to improve the use of models for screening.

5. Examples of models previously completed by the UK NSC

5.1 Asymptomatic bacteriuria (ASB)

Origin: regular review update and a UK NSC proactive response to NICE change in recommendation highlighted in a regular meeting.

Aim: to evaluate whether the current clinical practice of offering women screening by midstream urine culture in pregnancy for ASB is effective in preventing pyelonephritis.

Outputs:

• report of initial scoping exercise
• full cost-effectiveness model report (if viable)
• model manuscript to be published in a peer-reviewed journal (if viable)

Stakeholder involvement: workshops with health professional and researchers with interests in screening and/or in ASB in pregnancy and experts in antenatal screening programmes.

5.2 Bowel cancer: enhanced risk-based screening

Origin: Horizon scanning coming from a research advisory committee (RAC) submission.

Aim: Provide evidence to support the evaluation and possible implementation of an enhanced risk-based bowel cancer screening programme (BCSP) (with the aim of a future programme modification).

Outputs:

• model technical report
• a model manuscript to be published in a peer-reviewed journal

Stakeholder involvement: Collaboration between UK NSC evidence team and group of experts in the field. This project is also seeking an academic collaboration with the Dutch BCSP.

5.3 Cervical cancer: human papillomavirus (HPV) screening in vaccinated populations

Origin: prompted by the introduction of the HPV vaccination programme and a programme modification.

Aim: to inform a decision of which screening strategies should be implemented in vaccinated populations and in unvaccinated women in settings where the HPV vaccination is offered to boys and girls and/or where there are high levels of herd immunity.

Outputs:

• model technical report
• a model manuscript to be published in a peer-reviewed journal

Stakeholder involvement: expert input throughout the project.

5.4 Fetal anomaly screening: diagnostic accuracy of the ‘triple ‘test’ to detect T18

Origin: programme modification prompted by quality assurance (QA) visit report.

Aim: to gather evidence to determine the diagnostic accuracy of the ‘triple ‘test’ to detect T18 using a risk threshold of 1:150 in the UK population, and to provide a context for discussion in relation to the programme workload required to modify this point in the NHS Fetal Anomaly Screening programme (FASP) pathway.

Outputs: basic model or flowchart to gauge outcomes compared to current practice.

Stakeholder involvement: collaboration between UK NSC evidence team and the Down’s Syndrome Screening Quality Assurance Support Service (DQASS).

5.5 Group B streptococcus (GBS)

Origin: modelling.

Aim: to fill gaps surrounding key evidence needed to decide on GBS screening policy in the UK.

Outputs:

• cost effectiveness model or an appraisal of a cost effectiveness model
• potential manuscripts of the above to be published in a peer-reviewed journal

Stakeholder involvement: collaboration between UK NSC evidence team, researchers, and the GBS-3 Trial team.

5.6 Hepatitis C virus (HCV) screening in pregnancy

Origin: regular review process and contacts within the IDPS Programme, and stakeholders’ interest in HCV antenatal screening to detect HCV infection in the community.

Aim: to evaluate which screening strategies could contribute to the NHSE HCV elimination programme and by how much.

Outputs:

• model technical report
• a model manuscript to be published in a peer-reviewed journal

Stakeholder involvement: collaboration between UK NSC evidence team and HTA colleagues.

5.7 Non-invasive prenatal testing (NIPT) and/or non-invasive prenatal diagnosis (NIPD) in the NHS Sickle Cell and Thalassaemia (SCT) Screening Programme

Origin: initial programme modification.

Aim: to determine how different accuracy parameters impact pre-defined consequences and costs of NIPT and NIPD compared to current practice thereby informing the definition of minimally accepted criteria for diagnostic accuracy studies and sample size calculations.

Outputs:

• final model report
• model manuscript to be published in a peer-reviewed journal

Stakeholder involvement: 2 workshops with a wide range of stakeholders, including but not limited to: experts from the SCT programme, researchers and public and patient voice representatives.

5.8 Pulse oximetry (PO) screening for hypoxaemia in newborn babies

Origin: programme modification, and prompted by past unsuccessful attempts to address the cost effectiveness of PO screening.

Aim: collect the evidence required for the UK NSC to make a future recommendation on PO screening for hypoxaemia in newborn babies.

Outputs:

• feasibility assessment report
• extension of the feasibility assessment looking at data sources in more detail
• model development
• model manuscript to be published in a peer-reviewed journal

Stakeholder involvement: 2 stakeholder workshops and discussion on consultations.

5.9 Repeat screening for syphilis in pregnancy

Origin: programme modification, and prompted by the detection in 2016 to 2017 of 4 isolated atypical cases of congenital syphilis in babies whose mothers had true negative screening results.

Aim: to explore the value of a repeat antenatal screening strategy for congenital syphilis.

Outputs:

• final model report
• model manuscript illustrating was published in the British Medical Journal (BMJ) Open in 2020

Stakeholder involvement: workshops with key stakeholders and ongoing dialogue throughout the project.

5.10 Tyrosinaemia type 1

Origin: regular update, stakeholder workshops and modelling.

Aim: to collect the evidence required for the UK NSC to make a future recommendation on newborn screening for tyrosinaemia type 1.

Outputs:

• model technical report
• a model manuscript to be published in a peer-reviewed journal

Stakeholder involvement: 3 workshops with a wide range of stakeholders, and consultation on the draft cost effectiveness model.

5.11 Vasa praevia (VP)

Origin: regular update.

Aim: to explore the effects of second-trimester, ultrasound-based antenatal detection strategies for VP in a hypothetical UK cohort.

Outputs:

• final model report
• a model manuscript to be published in a peer-reviewed journal

Stakeholder involvement: workshops with key stakeholders and ongoing dialogue throughout the project.

6. References

1. Wilson ECF. A Practical Guide to Value of Information Analysis. PharmacoEconomics 2015; 33(2): 105-21. ↩

2. Korevaar DA, Gopalakrishna G, Cohen JF, Bossuyt PM. Targeted test evaluation: a framework for designing diagnostic accuracy studies with clear study hypotheses. Diagnostic and Prognostic Research 2019; 3(1): 22. ↩

3. Use of Decision Models in the Development of Evidence-Based Clinical Preventive Services Recommendations. 2019. https://www.uspreventiveservicestaskforce.org/uspstf/about-uspstf/methods-and-processes/use-decision-models-development-evidence-based-clinical-preventive-services-recommendations (accessed 05 January 2022). ↩