Consultation outcome

NHSP standards consultation: data report

Published 15 September 2021

The newborn hearing screening programme (NHSP) aims to identify permanent moderate, severe and profound deafness and hearing impairment in newborn babies. Finding out early can give these babies a better chance of developing language, speech, and communication skills. The newborn hearing screening pathways define the route through the screening process for babies eligible for screening in England.

Public Health England (PHE) Screening supports health professionals and commissioners in providing a high quality screening programme. This involves developing and reviewing screening standards against which data is collected, reported and published annually. The standards provide a defined set of measures that providers must meet to make sure local screening services are safe and effective.

Screening standards provide:

• reliable and timely information about the quality of the screening programme

• data at local, regional and national level

• quality measures across the screening pathway without gaps or duplications

PHE Screening review standards on an annual basis where wording or definitions are clarified and thresholds assessed. A comprehensive review takes place every 3 years where new standards may be introduced or existing standards withdrawn.

In 2020, a task and finish group was established to carry out a major comprehensive review of the NHSP standards covering the screening journey up to and including the point of referral to audiology and entry into audiological assessment.

Data used in the review

Data for the NHSP was extracted from the NHSP national IT system for each screening year from 1 March 2017 to 30 April 2020. Data for each of the 109 newborn screening sites was compared between sites and against the current standard thresholds.

The minimum, maximum, median and inter-quartile range for each standard was calculated across each year. The 1st and 3rd quartiles of the range were compared to the current standard thresholds (see Appendix 1 for the rationale for using inter-quartile range during threshold setting).

Threshold changes are based on the performance of services during the 2018 to 2019 screening year as the COVID-19 pandemic has had an impact on performance data during Q4 2019 to 2020.

Setting thresholds assists NHS England and Improvement, commissioners and Screening Quality Assurance Service (SQAS) to drive improvements in the delivery of the programme. Thresholds are unlikely to be ‘reduced’ even if more services than expected are not currently meeting the relevant thresholds, unless there are clear reasons or potential clinical implications.

The report summarises decisions made by the task and finish group for each unchanged screening standard definition, and for the performance thresholds.

A detailed summary describing the data analysis underpinning the decision to propose a change to the timescale from 4 weeks to 3 weeks for NHSP-S04 and for setting the thresholds is provided.

Summary of standards data - thresholds review

NHSP-S01: Coverage

The thresholds for this standard that were in use during the review period were:

• acceptable threshold: ≥ 98.0%

• achievable threshold: ≥ 99.5%

Table 1: Summary statistics for NHSP-S01 over the 3 screening years

Year	Minimum	Lower Quartile	Median	Upper Quartile	Maximum
2017 to 2018	85.6%	98.3%	98.9%	99.4%	99.9%
2018 to 2019	93.9%	98.6%	99.1%	99.5%	99.9%
2019 to 2020	96.2%	97.9%	98.6%	99.1%	99.9%

The main findings are:

• improvement in the minimum value for the standard over the 3 years from 85.6% to 96.2%

• an improving trend above the current acceptable threshold for the lower quartile from 98.3% in 2017 to 2018 to 98.6% in 2018 to 2019

• the achievable threshold for the upper quartile was relatively static

The negative impact of COVID-19 on coverage, evident in the 2019 to 2020 data supports the decision not to make changes to the thresholds for this standard at present.

NHSP-S02: test: well babies who do not show a clear response in both ears at automated otoacoustic emission 1 (AOAE1)

Hospital model services

The thresholds for this standard that were in use during the review period were:

• for 2017 to 2018:
  • acceptable threshold: ≤ 30.0%
  • achievable threshold: ≤ 25.0%
• from 2018 to 2019 onwards:
  • acceptable threshold: ≤ 27.0%
  • achievable threshold: ≤ 22.0%

Table 2: Summary statistics for NHSP-S02 in hospital model sites over the 3 screening years

Year	Minimum	Lower Quartile	Median	Upper Quartile	Maximum
2017 to 2018	7.4%	20.7%	24.6%	26.7%	35.4%
2018 to 2019	7.3%	19.5%	24.5%	26.9%	39.6%
2019 to 2020	5.8%	18.8%	23.2%	25.4%	38.2%

The main finding is an improving trend for the minimum, lower quartile, median and upper quartile values.

Reviewing the performance against the acceptable threshold it was observed that:

• in 2018 to 2019, 78 out of 103 services (76%) met the acceptable threshold

• in 2019 to 2020, 89 out of 103 services (86%) met the acceptable threshold

• improvements in 2019 to 2020 were during a year affected by COVID-19

Reviewing the performance against the achievable threshold it was observed that:

• in 2018 to 2019, 33 out of 103 services (32%) met the achievable threshold

• in 2019 to 2020, 42 out of 103 services (41%) met the achievable threshold

The improvement in this measure is more obvious over a longer period. COVID-19 has not adversely impacted the data even with screening teams attempting to complete the screen on much younger babies prior to discharge from hospital. Improved knowledge and education about the effect of screener technique in achieving a clear response at AOAE1 may be a contributory factor.

The 2018 to 2019 data was used to set the thresholds with the acceptable threshold remaining unchanged at ≤ 27.0% and the achievable threshold changed from ≤ 22.0% to ≤ 20.0%. This is in line with service achievements during that period.

Community model services

The thresholds for this standard that were in use during the review period were:

• acceptable threshold: ≤ 15.0%
• achievable threshold: ≤ 13.5%

Table 3: Summary statistics for NHSP-S02 in community model sites over the 3 screening years

Year	Minimum	Lower Quartile	Median	Upper Quartile	Maximum
2017 to 2018	10.9%	12.5%	14.6%	16.0%	21.6%
2018 to 2019	10.8%	11.6%	12.6%	16.2%	18.2%
2019 to 2020	10.6%	11.4%	12.3%	14.3%	16.1%

The main findings are:

• a reducing number of services providing a community model (n=5)

• an improving trend for the lower quartile from 12.5% to 11.4%

• an improving trend for the median from 14.6% to 12.3%

• a drop in the upper quartile value in the 2019 to 2020 year to 14.3%

• an improving trend for the maximum value from 21.6% to 16.1%

There are fewer community model services, with only 5 remaining. In 2018 to 2019 and 2019 to 2020 4 of the 5 services meeting the acceptable threshold also met the achievable. To make a distinction between the services, and encourage further improvement, the thresholds were adjusted to ≤14% for the acceptable and ≤12% for the achievable. Attainment of this standard in community sites is likely to correlate with their practice of screening babies after 10 days of age.

Relationship between NHSP-S01 and NHSP-S02

Concerns were expressed during the public consultation that making it harder to meet the achievable threshold might have a negative effect on coverage. Data for NHSP-S01 (coverage) and NHSP-S02 (no clear response (NCR) at AOAE1) was compared to review this.

Figure 1 shows a scatter plot by services delivering a hospital model in 2018 to 2019. It shows the proportion of babies with a no clear response at AOAE1 on the y axis, and coverage on the x axis. The plot contains horizontal and vertical lines showing the acceptable and achievable thresholds for both standards. The chart shows that the majority of services met the acceptable thresholds for both standards. A linear trend line shows the relationship between the two measures and falls slightly from around 31% on the left to about 22% on the right of the chart.

Figure 1: Hospital model sites comparison of coverage (NHSP-S01) and NCR at AOAE1 (NHSP-S02) in the 2018 to 2019 screen year

The correlation coefficient (R) for the trend line is -0.2512. The most likely interpretation of this is that there is no correlation (no link) between a service’s coverage and the proportion of babies with a no clear response. At best, there is a weak relationship indicating that services with a high coverage are slightly more likely to have a low proportion of no clear responses at AOAE1.

This data provides evidence that making changes to the thresholds for NHSP-S02 is unlikely to have any impact on services ability to meet the coverage thresholds.

NHSP-S03: test: referral rate to audiological assessment

Hospital model services

The thresholds for this standard that were in use during the review period were:

• for 2017 to 2018:
  • acceptable threshold: ≤ 3.0%
  • achievable threshold: ≤ 2.5%
• from 2018 to 2019 onwards:
  • acceptable threshold: ≤ 3.0%
  • achievable threshold: ≤ 2.0%

Table 4: Summary statistics for NHSP-S03 in hospital model sites over the 3 screening years

Year	Minimum	Lower Quartile	Median	Upper Quartile	Maximum
2017 to 2018	0.5%	1.5%	2.0%	2.9%	5.6%
2018 to 2019	0.5%	1.3%	2.0%	2.8%	6.7%
2019 to 2020	0.6%	1.4%	1.9%	2.8%	7.1%

The main findings are:

• the data is static over the review period

• the upper quartile is below the acceptable standard

• the lower quartile is below the achievable threshold

• the achievable threshold was changed at the 2018 review

Improvements to the acceptable threshold support changing this from 3.0% to 2.8%. Data does not support a change to the achievable threshold.

Community model services

The thresholds for this standard that were in use during the review period were:

• for 2017 to 2018:
  • acceptable threshold: ≤ 2.0%
  • achievable threshold: ≤ 1.0%
• from 2019 to 2020 onwards:
  • acceptable threshold: ≤ 1.6%
  • achievable threshold: ≤ 1.3%

Table 5: Summary statistics for NHSP-S03 in community model sites over the 3 screening years

Year	Minimum	Lower Quartile	Median	Upper Quartile	Maximum
2017 to 2018	1.3%	1.3%	1.7%	2.1%	2.2%
2018 to 2019	0.7%	1.2%	1.5%	1.7%	2.3%
2019 to 2020	1.2%	1.6%	1.6%	2.1%	2.3%

The main findings are:

• a reducing number of services providing a community model (n=5)

• the lower quartile performance was around the achievable threshold

• the upper quartile was above the acceptable threshold

There is no benefit to, or justification for, changing the thresholds for this standard.

Converting thresholds to a range

There is a value below which a service may no longer refer enough babies to safely identify permanent childhood hearing impairment (PCHI) in the screened population. Historical data shows some services with a minimum referral rate near 0.5%. The PCHI rate in the screened population is 0.1%. In recognition of the need for a minimum threshold to monitor if services are under referring, a minimum achievable level of 0.5% has been introduced in addition to the current thresholds for both hospital and community models.

Relationship between NHSP-S01 and NHSP-S03

Comments were received during the public consultation expressing concern that making it harder to meet the achievable threshold for S03 could have a negative effect on coverage. Data for NHSP-S01 (coverage) and NHSP-S03 (referral rate to audiology) was compared.

Figure 2 shows a scatter plot for services delivering a hospital model in 2018 to 2019. It shows the referral rate to audiology on the y axis, and coverage on the x axis. The plot contains horizontal and vertical lines showing the acceptable and achievable thresholds for both standards. The chart shows that the majority of services met the acceptable thresholds for both standards. A linear trend line shows the relationship between the two measures and falls slightly from around 3.5% on the left to just below 2% on the right of the chart.

Figure 2: Hospital model sites comparison of coverage (NHSP-S01) and referral rate to audiology (NHSP-S03) in the 2018 to 2019 screening year

The correlation coefficient (R) for the trend line is -0.22. The most likely interpretation of this is that there is no correlation (no link) between a service’s coverage and the rate of referral to audiology from screen. At best, there is a weak relationship indicating that services with a high coverage are slightly more likely to have a low rate of referral.

This data provides evidence that making changes to the thresholds for NHSP-S03 is unlikely to have any impact on services ability to meet the coverage thresholds.

NHSP-S05: diagnosis/intervention: time from screening outcome to attendance at an audiological assessment appointment

The thresholds for this standard that were in use during the review period were:

• acceptable threshold: ≥ 90.0%

• achievable threshold: ≥ 95.0%

Table 6: Summary statistics for NHSP-S05 over the 3 screening years

Year	Minimum	Lower Quartile	Median	Upper Quartile	Maximum
2017 to 2018	62.1%	86.5%	90.6%	93.5%	100.0%
2018 to 2019	70.7%	87.1%	90.9%	93.6%	100.0%
2019 to 2020	49.0%	84.4%	88.5%	92.5%	100.0%

The main findings are:

• the lower quartile is about 3% below the acceptable threshold

• the upper quartile is about 2% below the achievable threshold

• the median is about the same as the acceptable threshold

• the median in 2019 to 2020 was lower than previous years which may be due to the impact of COVID-19 on audiology services

There is no benefit to, or justification for, changing the thresholds for this standard. There should be no impact for audiology services with sthe timing of the audiological assessment remaining unchanged.

Summary of standards data - with changed definition

NHSP-S04: referral: time from screening outcome to first offered appointment for audiological assessment

Considerations for altering the standard:

• the standard supports achievement of NHSP-S05 (key performance indicator NH2) - attendance at audiology

• if a service offers an appointment at 4 weeks and parents delay or do not attend (baby was not brought) it is unlikely that they will be able to reschedule the appointment in line with the NHSP-S05 timescales, resulting in a lower performance against this standard

• a 2018 report aimed at driving improvements by sharing learning from the best performing sites for NHSP-S05 had limited impact

Data for the 4-week offer and attendance over the 2018 to 2019 screening year was reviewed. The main findings are:

• most services offered more babies an appointment by 4 weeks than attended by 4 weeks

• there was a moderate correlation between offering an appointment by 4 weeks and seeing babies by 4 weeks (R = 0.6446)

• when breaking down the data by weeks from screen complete to offer of appointment

  • 17.9% of babies are offered an appointment within 1 week of screen completion

  • 41.9% are offered an appointment within 2 weeks and

  • 69.9% are offered an appointment within 3 weeks

• for most services, the average time from the first offered appointment to actual attendance is less than 1 week, indicating that reducing the time to first offered appointment by one week would give sufficient time for rescheduling appointments allowing the baby to be seen by 4 weeks

Data for offering an appointment by 3 weeks was reviewed alongside the 4-week data:

• 15 services saw more babies by 4 weeks than were offered an appointment by 3 weeks - demonstrating that it is possible to ‘catch up’ appointments in the fourth week

• the correlation between 3-week offer and 4-week attendance was better (R = 0.7589)

The overall aim of the NHSP-S04 definition of helping to increase attendance in audiology by 4 weeks (NHSP-S05) has not been achieved.

In summary:

• 69.9% of babies are offered an appointment within 3 weeks
• 75% of services offered at least 90% of their babies an appointment within 3 weeks during 2018 to 2019
• 25% of services offered at least 97% of their babies an appointment within 3 weeks during 2018 to 2019
• at least 75% of services had an average time from offer to attendance of less than 1 week

NHSP proposed changing the timescale for offering an audiology appointment by 4 weeks to an offer by 3 weeks. Acknowledging the impact of COVID-19 on audiology services, thresholds were proposed close to the inter-quartile range of values from the 2018 to 2019 screening year, which were:

• 90% for the acceptable threshold

• 97% for the achievable threshold

Feedback received during the public consultation expressed concern that changing the timescale would potentially increase the number of babies seen earlier following screen completion. This could have an adverse impact on audiology capacity if a larger proportion of babies were identified with a temporary conductive hearing loss or were not able to have their hearing type confirmed.

Data from 2018 to 2019 showed:

• 28.4% of babies seen in audiology between 2 and 3 weeks following screen completion had a ‘conductive temporary’ or ‘not yet determined’ hearing type

• 24.0% of babies seen in audiology between 3 and 4 weeks following screen completion had a ‘conductive temporary’ or ‘not yet determined’ hearing type

The data provides sufficient evidence that changing standard 4 could have an unintended clinical impact. The timescale for NHSP-S04 will remain unchanged at 4 weeks, as in the current definition (after updating the wording in line with other standards), as will the thresholds, which are:

• ≥ 97% for the acceptable threshold

• ≥ 99% for the achievable threshold

The programme team will work together with the professional bodies and local NHSP services to better understand why standard NHSP-S05 is not met. The programme team will also work alongside the screening quality assurance service (SQAS) and commissioners to support local services and determine how improvements can be made.

Table 7: Babies born between 01 April 2018 and 31 March 2019 who required an immediate referral by time from screen to first attended appointment

Time from screen outcome to first attended appointment	Babies requiring immediate referral to audiology	Babies with a better or worse ear type of either ‘conductive temporary’ or ‘not yet determined’	Proportion of babies with a better or worse ear type of either ‘conductive temporary’ or ‘not yet determined’
<1 week	1253	574	45.8%
>= 1 to <2 weeks	4062	1318	32.4%
>= 2 to <3 weeks	3986	1133	28.4%
>= 3 to <4 weeks	2535	609	24.0%
>= 4 to <6 weeks	1420	171	12.0%
>= 6 weeks to < 3 months	671	15	2.2%
>= 3 months	529	5	0.9%
All babies	14456	3825	26.5%

The inter-quartile range used in threshold setting

Figure 3: The spread of data from the inter-quartile range, illustrating that the middle 50% of values is covered by the inter-quartile range

Figure 3 shows a horizontal box plot with the median line in the centre of the box. The box is labelled Q1 at the left (lower) boundary, and Q3 at the right (upper) boundary. Beneath the box plot is a distribution curve centred on the median, with vertical lines descending from the Q1 and Q3 boundaries. The area under the curve is shaded between these lines to show that the middle 50% of the values in the distribution are between Q1 and Q3, with 25% of the values each above and below the inter-quartile range.

As the inter-quartile range identifies the middle 50% of values around the median, we can be sure that if an acceptable threshold is set to the lower quartile, then at least 75% of the services are meeting the threshold as they are to the right of this value. If 75% of services can meet this value it is reasonable, in most cases, to anticipate that the remaining 25% can also reach this threshold.

Similarly, if the achievable threshold is set to the upper quartile (Q3 on Figure 3), then around 25% of services are meeting this ‘aspirational’ threshold as they are to the right of this point. It is generally safe to assume that if 25% are already achieving it, then it is an achievable threshold that is not out of reach for other services to aim for.