Official Statistics

Indicators of species abundance in England

Updated 3 May 2024

Applies to England

Last updated: 2024

Latest data available: 2022

Contact

Enquires on this publication to: enviro.statistics@defra.gov.uk

Tel: 03459 335577 (Defra enquiries) Find out more about call charges at – GOV.UK

Lead statistician: Clare Betts

Environmental Statistics and Reporting team,
Department for Environment, Food and Rural Affairs,
Mallard House,
Kings Pool,
3 Peasholme Green,
York,
YO1 7PX

Website: Biodiversity and wildlife statistics – Gov.UK

Introduction

As this is an official statistic in development, the Environmental Statistics and Reporting team welcomes feedback on the novel methods used in the development of this indicator. For example, feedback on whether this new indicator measures something users feel should be measured, and on how well it measures species abundance in England.

Official statistics in development are official statistics that are undergoing a development; they may be new or existing statistics, and will be tested with users, in line with the standards of trustworthiness, quality, and value in the Code of Practice for Statistics.

To give feedback, email the Environmental Statistics and Reporting team at enviro.statistics@defra.gov.uk

This release covers two measures of species abundance in England; the first which contains species for which we have suitable data, the second for only those species which are deemed priority species. Monitoring the abundance of species is important for our understanding of the state of the wider environment, particularly as measures of species abundance are more sensitive to change than other aspects of species populations. It should be noted that for a more comprehensive indication of the state of the wider environment, indicators of species abundance should be reviewed alongside species distribution and extinction risk indicators.

When fully developed, the all-species abundance measure will be used to track the government’s progress towards meeting the statutory target of halting the decline in species abundance by 2030, and then reversing these declines by 2042. Currently this measure includes data for 1,177 species, plans for developing the indicator further are detailed in the Development Plan.

The all-species indicator mainly represents species found in terrestrial and freshwater environments. The all-species indicator was developed with the aim of producing an index to summarise trends in abundance for the broadest possible set of organisms that are representative of English biodiversity, although the species coverage is limited by data availability. Two earlier versions of a measure of species abundance were published in the Biodiversity Targets Consultation detailed evidence report in 2022. The new all-species indicator builds upon these earlier versions (see Version History), following feedback from users and additional work to expand the species coverage and improve the indicator methodology. Development of the all-species indicator was also informed by extensive independent expert input (see Expert input to indicator development). Given the complexity of producing a combined species indicator on this scale and its importance to the target, we will continue to take a transparent approach and seek advice from experts, stakeholders and users, the outcomes of which will be detailed in the future releases.

Priority species are defined as those appearing on the priority species list for England (Natural Environmental and Rural Communities Act 2006 - Section 41). Currently this measure includes data on 149 of the 940 priority species in England.

An indicator for priority species abundance in the UK was first published in 2014, and since then the methods have been continually improved and used to create an indicator for priority species abundance in England (first published in 2021). These existing priority species indicators were used as a starting point for development of the all-species abundance indicator.

For both the all-species and priority species indicators two possible versions of the indicator are presented, one with a greater degree of smoothing applied and one with a lesser degree of smoothing. Smoothing is applied to the species abundance indicators to reveal long-term trends in otherwise noisy data. A greater degree of smoothing may provide a clearer view of the underlying long-term trend, while a lesser degree of smoothing preserves the shorter-term patterns in the data. The results given in the commentary are based on the values of both trends, and are intended to portray the extent to which these trends are dependent on methodological decisions. We are interested in users’ feedback on both options as part of the ongoing development of the indicator.

This release also breaks these two measures down by taxonomic group. Defra already publish species abundance indices for birds and butterflies in their Wild bird populations in England and Butterflies in England publications, so whilst these species are included in the indicators presented here, these taxonomic breakdowns are not included in this publication.

Presented in this publication are indicators of abundance relative to the starting year (set to a value of 100), rather than absolute abundance. Changes to this value reflect the average change in species abundance; if on average species experienced a doubling in abundance, the indicator would rise to 200, if they halved it would fall to a value of 50.

Assessment of change

Table 1: Results from the assessment of change

Measure Assessment Time period Result
All-species Long term 1970 to 2022 Deteriorating
All-species Medium term 2012 to 2022 Little or no overall change
All-species Short term 2017 to 2022 Little or no overall change
Priority species Long term 1970 to 2022 Deteriorating
Priority species Medium term 2012 to 2022 Little or no overall change
Priority species Short term 2017 to 2022 Little or no overall change

Formal assessment of change is made on the basis of credible intervals for the time period; if the indicator value for the first year falls outside of the credible intervals for the final year then the indicator is deemed to have changed over that time period. The assessment process will be reviewed as part of the ongoing development of these statistics (see Development Plan for more details).

All species

The all-species indicator draws on data for 1,177 species for which we have suitable data. See the Technical Annex for more information about the standards applied for data inclusion.

By 2022 the index of change in relative abundance of species in England had declined to around 69% of its 1970 value (Figure 1). Over this long-term period 37% of species showed a strong or weak decline, while 34% showed a strong or weak increase (Figure 2).

More recently, between 2017 and 2022, the relative abundance index did not change significantly (Figure 1). Over this short-term period, 48% of species showed a strong or weak increase and 37% showed a strong or weak decline (Figure 2).

Figure 1: Change in relative abundance of species in England 1970 to 2022, shown using two smoothing options.

Notes about Figure 1

  • Figure 1 shows the two options for the smoothed trend (solid line) with their 95% credible intervals (shaded area).
  • Index values represent change from the baseline value in 1970, the credible interval widens as the index gets further from the 1970 value and confidence in the estimate of change relative to the baseline falls.
  • The credible intervals capture uncertainty in the trends of individual species that contribute to the index. They do not capture uncertainty associated with the spatial locations of sample points, nor about the degree to which the species represent wider biodiversity. The credible intervals partially capture uncertainty in the species abundance estimates.
  • Several different levels of smoothing were considered, with Option 1 being smoothed on a ten-year timescale and Option 2 being smoothed on a three-year timescale (see discussion of smoothing in Caveats and limitations).

Notes about Figure 2

  • Figure 2 shows the percentage of species within the indicator that have increased (weakly or strongly), decreased (weakly or strongly) or shown little change in abundance based on set thresholds of change (see Background and Methodology for more detail).
  • Due to rounding, the data labels may not sum exactly to 100%.
  • There are seven species for which a short term assessment of change isn’t available.

The headline indicator (Figure 1) does not show the variation between the taxonomic groups. Figure 3 shows the index for each taxonomic group separately, generated using the same methods as the headline indicator. The relative abundance measure comprises 168 bird species, 16 mammals, 37 freshwater and estuarine fish, 55 butterflies, 446 moths, 11 bumblebees, 235 freshwater invertebrates and 209 vascular plants. The moths have undergone the biggest decline with an index value in the final year (2022) that was only around 56% of its value in 1970, although most of the decline occurred prior to 2000. Fish, mammals and vascular plants have all increased compared to their baseline year (2000, 1995 and 2015 respectively). Bumblebees and freshwater invertebrates have shown little change compared to their baseline years (2010 and 2013, respectively). Defra already publish species abundance indices for birds and butterflies in its Wild bird populations in England and Butterflies in England publications, so those taxonomic breakdowns are not included in this publication. Data collection for each taxonomic group spans different time periods and so the baseline year for each differs (see Table 4, Background and Methodology).

Table 2: All-species indicator values broken down by taxonomic group

Taxon Number of species Baseline year Option 1 index value in 2022 Option 2 index value in 2022
Birds - - - -
Bumblebees 11 2010 108.6 (79.3-148.9) 108.7 (77.6-150.7)
Butterflies - - - -
Fish 37 2000 200.2 (119.0-326.0) 200.5 (120.9-329.2)
Freshwater invertebrates 235 2013 105.9 (102.6-109.5) 105.8 (102.4-109.2)
Mammals 16 1995 115.5 (101.3-130.5) 115.7 (101.4-131.6)
Moths 446 1970 56.1 (49.5-63.3) 55.8 (49.4-62.5)
Vascular plants 209 2015 105.9 (97.9-114.4) 106.6 (98.3-115.0)

Notes about Table 2:

  • Defra already publish species abundance indices for birds and butterflies in its Wild bird populations in England and Butterflies in England publications, so those taxonomic breakdowns are not included in this publication.
  • Credible intervals for each value are shown in brackets. Where these include the value 100, we conclude that the index for the taxonomic group has shown little or no change since its baseline year.

The width of the credible intervals in Figure 3 (shaded area) is determined by several factors. The most important determinant is the number of species: groups with many species (freshwater invertebrates, moths, vascular plants) have much narrower credible intervals than those with few species (bumblebees, fish, mammals). A second factor is the degree to which the multispecies trend varies over time: it is wider for groups where the direction of the trend changes over time (fish) than groups for which it is relatively stable (mammals). A third factor is the number of years since the baseline: because the uncertainty is measured relative to the baseline year, the width of the credible interval grows steadily over time: for this reason, the credible interval for moths is wider than the credible interval for freshwater invertebrates.

Figure 3: Change in relative species abundance by taxonomic group, 2000 to 2022, shown using two smoothing options

Notes about Figure 3

  • Figure 3 shows the two options for the smoothed trend (solid line) with their 95% credible intervals (shaded area).
  • Included in Figure 3 are 168 bird species, 11 bumblebee species, 55 butterfly species, 37 freshwater and estuarine fish, 235 freshwater invertebrates, 16 mammals, 446 moths, and 209 vascular plants.
  • Indices for the years prior to the year 2000 are not shown as data for many individual groups are not available prior to that year.
  • Index values represent change from the baseline value for each group; in 2010 for bumblebees, 2000 for fish, 2013 for freshwater invertebrates, 1995 for mammals, 1970 for moths and 2015 for vascular plants. The credible interval widens as the index gets further from the baseline value and confidence in the estimate of change relative to the baseline falls.
  • The credible intervals capture uncertainty in the trends of individual species that contribute to the index. They do not capture uncertainty associated with the spatial locations of sample points, nor about the degree to which the species represent wider biodiversity. The credible intervals partially capture uncertainty in the species abundance estimates.
  • Several different levels of smoothing were considered, with Option 1 being smoothed on a ten-year timescale and Option 2 being smoothed on a three-year timescale (see discussion of smoothing in Caveats and limitations). For Option 1, there were several groups where the timeseries was too short to allow for smoothing over ten years: these have been smoothed to the maximum number of years possible (4 years for vascular plants, 5 years for freshwater invertebrates and 6.5 years for bumblebees).
  • Defra already publish species abundance indices for birds and butterflies in their Wild bird populations in England and Butterflies in England publications, so those taxonomic breakdowns are not included in this publication.

Priority species

The priority species abundance indicator draws on species observation data for species which are defined as priority species. Priority species are defined as those appearing on the priority species list for England (Natural Environmental and Rural Communities Act 2006 - Section 41). The priority species were highlighted as being of conservation concern for a variety of reasons, including rapid decline in some of their populations. The indicator therefore includes a substantial number of species that, by definition, are becoming less abundant. In England there are 940 species on the priority species list, and this indicator shows the average change in the 149 species for which abundance trends are available in England.

The method for calculating this indicator has been updated in line with work done to develop the all-species indicator. As a result, the method is not yet finalised and two smoothing options are presented equivalent to the two options presented for the all-species indicator. Note that the number of species is fewer (149 versus 153) than in the previous publication (last updated in November 2023) of the priority species indicator: this is because the modelled trend for four moth species did not meet the quality assurance thresholds for inclusion (see Background and Methodology).

By 2022, the index of change in relative abundance of priority species in England had declined to around 21% of its baseline value in 1970 (Figure 4). Over this long-term period, 13% of species showed a strong or weak increase and 68% showed a strong or weak decline.

More recently, between 2017 and 2022, the relative abundance index of priority species did not change significantly. Over this short-term period, 51% of species showed a strong or weak increase and 38% showed a strong or weak decline.

Figure 4: Change in the relative abundance of 149 priority species in England, 1970 to 2022, shown using two smoothing options

Notes about Figure 4

  • Figure 4 shows the two options for the smoothed trend (solid line) with their 95% credible intervals (shaded area).
  • The width of the credible interval is in part determined by the proportion of species in the indicator for which data are available.
  • Index values represent change from the baseline value in 1970, the credible interval widens as the index gets further from the 1970 value and confidence in the estimate of change relative to the baseline falls.
  • The credible intervals capture uncertainty in the trends of individual species that contribute to the index. They do not capture uncertainty associated with the spatial locations of sample points, nor about the degree to which the species represent wider biodiversity. The credible intervals partially capture uncertainty in the species abundance estimates.
  • Several different levels of smoothing were considered, with Option 1 being smoothed on a ten-year timescale and Option 2 being smoothed on a three-year timescale (see discussion of smoothing in Caveats and limitations).

Notes about Figure 5

  • Figure 5 shows the percentage of species within the indicator that have increased (weakly or strongly), decreased (weakly or strongly) or shown little change in abundance based on set thresholds of change (see Background and Methodology for more detail).
  • Due to rounding, the data labels may not sum exactly to 100%.
  • There are five species for which a short term assessment of change isn’t available.

The headline indicator (Figure 4) masks variation between the taxonomic groups. Figure 6 shows the index for each taxonomic group separately, generated using the same methods as the headline indicator. The relative abundance measure comprises 44 bird species, 21 butterflies, 7 mammals and 77 moths. The moths have undergone the biggest decline with an index value in the final year (2022) that was only around 15% of its value in 1970. Butterflies and birds have also experienced strong declines in 2022, with butterflies having an index value that was roughly 42% of its value in 1976, and birds have an index value of around 28% relative to its value in 1970. The mammals index is the only taxonomic group out of the four to have not changed significantly from its baseline value in 1998.

Table 3: Priority species indicator values broken down by taxonomic group

Taxon Number of species Baseline year Option 1 index value in 2022 Option 2 index value in 2022
Birds 44 1970 27.6 (21.7-35.3) 27.6 (21.6-35.2)
Butterflies 21 1976 42.9 (25.3-72.7) 42.2 (24.6-71.2)
Mammals 7 1998 117.7 (99.1-139.3) 116.5 (98.4-137.0)
Moths 77 1970 14.8 (10.9-20.0) 14.6 (10.9-19.9)

Notes about Table 3:

  • Credible intervals for each value are shown in brackets. Where these include the value 100, we conclude that the index for the taxonomic group has shown little or no change since its baseline year.

There are twelve Section 41 species included in the all-species index that have not yet been included in this version of the priority species index (8 fish, 1 bumblebee, 1 vascular plant and 2 freshwater invertebrates). To retain consistency and comparability with the previous release of this indicator, we have not added them to this release. The effect of adding these twelve species on the overall index would be small. It is planned that these species will be included in future versions of the indicator.

Figure 6: Change in relative species abundance by taxonomic group, 1970 to 2022, shown using two smoothing options

Notes about Figure 6

  • Figure 6 shows the two options for the smoothed trend (solid line) together with their 95% credible intervals (shaded area) for each of the four taxonomic groups included in the composite indicator.
  • Index values represent change from the baseline value for each group; in 1970 for birds, 1976 for butterflies, 1998 for mammals and 1970 for moths. The credible interval widens as the index gets further from the baseline value and confidence in the estimate of change relative to the baseline falls.
  • The width of the credible interval is in part determined by the number of species in the indicator for which data are available.
  • Included in Figure 6 are 44 bird species, 21 butterfly species, 7 mammal species and 77 moth species.
  • The credible intervals capture uncertainty in the trends of individual species that contribute to the index. They do not capture uncertainty associated with the spatial locations of sample points, nor about the degree to which the species represent wider biodiversity. The credible intervals partially capture uncertainty in the species abundance estimates.

Discussion

All-species and priority species indicators

Both indicators capture a decline in abundance across species in England since 1970. For the all-species indicator, this trend appears to level around the year 2000 to just under 70% of the 1970 value. Over the past five years, fluctuations in the all-species indicator have been within the 95% credible intervals and therefore are not considered to represent meaningful change. The priority species indicator has declined much further than the all-species, to just over 20% of the 1970 value, but with a similar levelling off period from 2000.

There are several contributing factors to the differences between the two indicators. Firstly, their taxonomic composition is very different, with the addition of several entire groups (fish, freshwater invertebrates, bumblebees, vascular plants) and many individual species to the all-species index compared to the priority species index. The criteria for selection of species were also different. For the all-species indicator, the list includes the broadest range of relevant species for which we have suitable abundance data available. In comparison, the priority species list is based upon species which have been identified as being of conservation concern and are very likely to be those which have already experienced significant declines.

Change in relative species abundance by taxon

As shown in Figure 3 and Figure 6, different patterns of change were observed between taxonomic groups. Further work is needed to fully explore these changes, but existing publications provide an indication of the drivers of change for specific taxonomic groups. Defra already publish species abundance indices for birds and butterflies in its Wild bird populations in England and Butterflies in England publications, so those taxonomic breakdowns are not discussed here.

Bumblebees

The bumblebee species included in the all-species indicator have shown little or no change in abundance since 2010. The BeeWalk 10-year report (Comont & Dickinson, 2022) describes the impact of changes in weather on bumblebee abundance. There was a drop in abundance in 2018, likely due to the summer heatwave that year, and numbers fell again in 2021 as the cold spring hindered colony establishment. In 2022, the weather was warmer than average and changes in abundance were mixed (Comont & Dickinson, 2023). Many of the rarer bumblebees showed an increase in abundance compared with the long-term trends, while several other species (including some of our most widespread and abundant bumblebees) had a poor year compared to the long-term averages. More generally, the drivers of bumblebee declines in England include habitat loss and degradation, use of pesticides, and climate change (Whitehorn et al., 2022).

Fish

The freshwater and estuarine fish species included in the all-species indicator have shown an overall increase in abundance since 2000. According to the World Wide Fund for Nature (WWF) Living Planet Index, populations of freshwater fish declined globally by 83% between 1970 and 2012. A recent study assessing freshwater fish in England found that 7 out of the 34 species assessed were classified as threatened with extinction according to IUCN Red List criteria (Nunn et al., 2023). Those under particular threat included the European eel and Atlantic salmon. Common threats to freshwater fish include habitat loss and fragmentation, pollution, overexploitation, invasive species and climate change. In England, the total number of serious pollution incidents to water fell by almost two-thirds (61%) between 2001 and 2021 (Outcome Indicator Framework B2).

Freshwater invertebrates

The freshwater invertebrates included in the all-species indicator have shown an overall increase in abundance since 2013. Freshwater invertebrates have seen significant improvements in diversity since 1989 (Qu et al., 2023), with the number of families recorded in England’s rivers increasing by 66% on average. For some groups, this increase began to slow from 2003 onwards. The published findings show that overall, there was no trend found in abundance between 2003 and 2018, although this differed between groups, with some, like decomposers, showing a relative decline. Despite the relatively low proportion of water bodies receiving good or high ecological status in England, the increase in freshwater invertebrate diversity may reflect that some of the chemicals of greatest concern with respect to water quality have declined. In addition to the chemical status of the water, there are also physical characteristics such as flow, temperature and morphology, which can be impacted by pressures from climate change and human activity. Sites which saw declines in family richness tended to be rural catchments, connected to poor farming practices and habitat damage.

Mammals

The mammal species included in the all-species indicator have shown an increase in abundance since 1995, but little to no change in the priority species index since 1998. Of the 17 mammal species in the all-species indicator, 10 are species of bat. On average bat species in England have increased since 1999 (Widespread bats in England). However, these trends reflect relatively recent changes in bat populations (since 1999 for most species). It is generally considered that prior to this there were significant historical declines in bat populations dating back to at least the start of the 20th century. This suggests that current legislation and conservation actions to protect and conserve bats are having a positive impact. Bats and their roosts have had legal protection since 1981, and all species of bat are protected in England.

Other mammals in the all-species indicator come from a wide variety of taxa, all of which are threatened by human activities, changing habitats and disease, consequently many have shown strong declines (for example hazel dormice and red fox). Conversely, some, such as brown hare, have shown strong increases.

Moths

The moth species included in both the all and priority species indicators have declined overall since 1970. A large proportion of the moth species in the all-species indicator are macromoths (a very small number are micromoths). As reported in The State of Britain’s Larger Moths 2021 (Fox et al., 2021), which also described declines in moth abundance over the long term, the causes of changes in moth abundance are not fully understood. Habitat destruction and deterioration remain pressing concerns for moths, driven by land-use change and chemical pollution. Artificial light at night has negative effects on moth development and behaviour, but links to population-level decline are yet to be proved. There is also growing evidence of negative impacts of climate change, particularly on moths that are adapted to cooler conditions in northern, western and upland Britain.

Vascular plants

The vascular plant species included in the all-species indicator have shown little to no change overall since 2015. Both native and naturalised species of vascular plants were included in the all-species relative abundance index. According to the recent report on Britain’s Changing Flora (Walker et al., 2023), distributions of native species and ancient introductions (pre-AD1500) have largely been declining since the 1950s, whereas distributions of modern introductions have been increasing in the same period. These declines reflect changing lowland landscapes due to intensive farming, urbanisation, and the associated impacts from pollution and land-use activities. In comparison to other areas of the UK, distributions were found to be stable in species associated with montane habitats and steadily increasing in those associated with coniferous woodland. Warming climates and milder winters have allowed some southern species to expand their ranges northwards, whereas some northerly species have faced higher competition and reduced their ranges to higher ground.

Official statistics in development designation

Our statistical practice is regulated by the Office for Statistics Regulation (OSR). OSR sets the standards of trustworthiness, quality and value in the Code of Practice for Statistics that all producers of official statistics should adhere to. You can read about how Official Statistics in Defra comply with these standards on the Defra Statistics website.

This publication is an official statistic in development, official statistics in development are official statistics that are undergoing a development; they may be new or existing statistics, and will be tested with users, in line with the standards of trustworthiness, quality, and value in the Code of Practice for Statistics.

Details of how we plan to develop these statistics are laid out in the Development Plan. We particularly welcome feedback from users on the methodology and presentation of the statistics set out in this release, and our future plans for development.

Background and methodology

Source data

Much of the data on species abundance is collected through well-established volunteer-based recording schemes, many of which are run through partnerships between government bodies, Non-governmental organisations (NGOs), and research organisations, or through statutory monitoring schemes. The species included in these indicators (Table 4) are intended to be as representative as possible of priority species and wider species found in England. However, the taxonomic and species coverage is limited by data availability and these measures are, therefore, not fully representative of species in the wider countryside. See the Technical Annex for more detail.

Table 4: Summary of information on the datasets included in the indicators.

Name of scheme Taxonomic coverage Number of species in all-species indicator Number of species in priority species indicator Timespan included in indicators
Breeding Bird Survey (BBS) Birds 95 25 1970-2022
Rare Breeding Birds Panel (RBBP) Birds 33 8 1970-2021
Seabird Monitoring Programme (SMP) Birds 12 1 1986-2019
Statutory Conservation Agency and RSPB Annual Breeding Bird Scheme (SCARRABS) Birds 8 6 1971-2016
Wetland Bird Survey (WeBS) Birds 20 4 1975-2021
BeeWalks Bumblebees 11 0 2010-2022
UK Butterfly Monitoring Scheme (UKBMS) Butterflies 55 21 1976-2022
National Fish Population Database (NFPD) and Transitional/Coastal waters Data (TRaC) Fish 37 0 2000-2022
Freshwater Invertebrates (BIOSYS) Freshwater invertebrates 235 0 2013-2022
Breeding Birds Survey (BBS) Mammals Mammals 5 1 1995-2022
National Bat Monitoring Programme (NBMP) Mammals 10 5 1998-2021
National Dormouse Monitoring Programme (NDMP) Mammals (single species) 1 1 1995-2022
Priority Moths Moths 11 11 1995-2021
Rothamsted Insect Survey Light Trap Moths 435 66 1970-2022
National Plant Monitoring Scheme (NPMS) Vascular plants 209 0 2015-2022

Notes about Table 4:

  • The Breeding Bird Survey began in 1994 and incorporates the Waterways Breeding Bird Survey and the Heronries Census. Prior to this, data came from the Common Bird Census (CBC).
  • Data is available in the freshwater invertebrates (BIOSYS) dataset from the mid-1990s to the present. Data prior to 2013 wasn’t considered to meet the criteria for taxonomic resolution to species level, so data from 2013 onwards is used in the indicator.

Three criteria were used to assess whether data were suitable for inclusion in the indicator:

  1. Scheme uses standardised approach delivering annual abundance indices based on survey protocols and analytical methods that are appropriate for the organisms being studied.
  2. Spatially replicated survey design with coverage across England (or, for very rare species, the data captured should cover the vast majority of populations that are known to exist)
  3. Taxonomic resolution ideally to species level. In some cases, it was considered desirable to include data at a higher level to improve taxonomic coverage (for example, aggregated groups of species, or genus-level).

The rationale for these dataset criteria is described in more detail in the Technical Annex.

Robust English population time-series were sought for as many species as possible to produce the indicator for species abundance in England. The measure is a composite indicator of 1,177 species from many taxonomic groups. See the Technical Annex for a detailed breakdown of the species and groups included. Much of the data in this indicator has previously been published and many of the datasets are currently used elsewhere within the England Biodiversity Indicators.

Regardless of advances in statistical techniques, it is known that there are many species for which little monitoring data are available. Reasons for this include rarity, difficulty of detection, or those for which monitoring methods are unreliable or unavailable. For the indicator to be representative of all-species in England, a robust method of assessing the changing status of these remaining data-poor species would need to be available.

Structured schemes where data are collected annually, following a strict pre-determined protocol, allow reliable conclusions to be derived from the data on the national status of species and how their populations are changing in the long term. The methods used vary by scheme to allow data collection to be appropriate for the target taxonomic group, but include repeat sampling in randomised stratified surveys, complete censuses and targeted surveys. The measure of abundance also varies by scheme depending on the focal taxa, for example, number of individuals, percentage cover of quadrats. Structured scheme sampling does involve bias, some of which can be accounted for and others that are more difficult to control (see Caveats and limitations for more discussion on biases).

There is ongoing research and development work into improving the evidence generated from volunteer based recording schemes and statutory monitoring schemes, including reducing bias in volunteer datasets, enhancing verification methods, and integrating different types of datasets to better understand species trends at finer spatial scales.

Species included

All-species indicator

The species in the all-species indicator are intended to be as representative as possible of wider biodiversity in England, although the coverage is limited by the availability of data from existing monitoring schemes. All native and naturalised species with suitable data were considered for inclusion in the indicator. Invasive non-native species were excluded. All species that were naturalised before 1500 were included, as well as those that colonised England from mainland Europe more recently (for example, the tree bumblebee Bombus hypnorum which arrived in England from Europe in 2001).

The taxonomic breakdown of species in the indicator can be found in the published datafile. The number of species included in each year of the index is shown in Figure 7.

The vast majority of the 1,177 taxa in the all-species indicator are individual species. There are 20 species groups and 66 genera – the majority of these species groups and genera are from the freshwater macroinvertebrate dataset. This reflects the fact that many invertebrates are difficult to identify to species level, especially in their larval stage. The decision to include these higher taxa reflects the desire for the indicator to be broadly representative.

The species in the all-species indicator align with those listed in Schedule 2 of The Environmental Targets (Biodiversity) (England) Regulations 2023, which sets out 1,195 species that should be monitored as part of the species abundance targets. Throughout the rest of this publication we will refer to it as Schedule 2. The indicator does not yet include data for all 1,195 these species, as data are not yet ready for inclusion for a small number of species (10 plants, 6 moths, 1 fish and 1 mammal). More detail on each of these is set out below.

The National Plant Monitoring Scheme (NPMS) and National Dormouse Monitoring Programme (NDMP) are both UK schemes that currently produce abundance indices for the whole of the UK, not for the English subset. In both cases, the majority of sites are in England (approximately 65% for NPMS, approximately 90% for NDMP). For dormouse, the UK model was considered sufficiently representative of the pattern of change in England, so data has been included in this indicator.

Another point of difference between NPMS and other schemes is that NPMS models trends of each species at the habitat scale, rather than nationally (Pescott et al., 2019a,b). We therefore evaluated whether the trend for each habitat could be said to be representative of each plant’s status in England. Of the eleven habitat types in the NPMS, seven have a majority (more than half) of sites in England. We therefore decided to include the data for these seven habitat types and exclude the four habitats where most sites are outside England. About 20% of NPMS species have models from more than one of the seven habitat types included. For these species, we calculated species’ national index of abundance as a weighted mean of the habitat-specific indices, in which the weights reflected the number of study plots represented by each habitat type.

Based on this procedure, we calculated trends for 209 plant species on Schedule 2. These have been included in the indicator. There are 10 species in Schedule 2 that only occur on the four habitat types that were excluded, so these have been withheld from the published indicator until new England-specific models have been developed. The affected species are maidenhair spleenwort (Asplenium trichomanes), star sedge (Carex echinata), common sedge (Carex nigra), crested dog’s-tail (Cynosurus cristatus), cross-leaved heath (Erica tetralix), common cottongrass (Eriophorum angustifolium), hare’s-tail cottongrass (Eriophorum vaginatum), heath wood-rush (Luzula multiflora), purple moor-grass (Molinia caerulea), and Rubus chamaemorus.

Following addition of more recent data, as well as updates to the modelling methodology, the data for six of the moth species in Schedule 2 no longer pass the quality assurance tests that are completed as part of producing the indicator. Further data will be needed to provide sufficient confidence in the trends for these species before they can be included. The affected moth species are dotted carpet (Alcis jubata), Haworth’s minor (Celaena haworthii), grey mountain carpet (Entephria caesiata), crescent (Helotropha leucostigma), emperor moth (Saturnia pavonia), and heath rustic (Xestia agathina).

After publication of Schedule 2, some questionable features of the data for crucian carp (Carassius carassius) were identified. Records of the crucian carp declined over time, whereas records of hybrids between crucian carp and either common carp (Cyprinus carpio) or brown goldfish (Carassius auratus) increased. The changeover appears to coincide with better understanding of hybridization and the publication of an improved guide to the identification of hybrids between these carp species (Hanfling & Harley, 2003). This raises the possibility that trends in the crucian carp were influenced by variation in identification rather than a change in abundance, and that the apparent decline is in fact a trend toward lower misidentification of hybrids. It was therefore decided to exclude the crucian carp from the indicator this year, pending further investigation.

Similarly, after publication of Schedule 2 questions were raised about the available data for water vole (Arvicola amphibius). Water vole was added to Schedule 2 based on abundance data from two national water vole surveys in 1989 to 1990 and 1996 to 1998, as well as more recent data from the National Water Vole Monitoring Programme (NWVMP). It has become clear that some of this data measures the distribution of water vole, rather than abundance, and therefore would not meet the criteria for use in the indicator. The decision was made to exclude the water vole from the indicator this year, to enable further investigation of the available data.

Figure 7: The number of species included in each year of the all-species indicator.

Notes about Figure 7:

  • The dashed line at 1,195 species indicates the target number of species for the index, based on the list published in Schedule 2
  • Jumps in the number of species included represent the start of data collection for certain datasets (see Table 4)
  • The two dips in species included in the indicator are due to the outbreak of foot and mouth disease in 2001 which impacted the collection of bird data, and the COVID-19 pandemic in 2020 which impacted the collection of bird and fish data. The fall in species in 2021 and 2022 is due to routine delays in data from various monitoring scheme becoming available.

Priority species indicator

The species considered for inclusion in the England Priority Species Indicator are those on the S41 list and is based upon species which have been identified as being of conservation concern and are very likely to be those which have already experience significant declines. Species on the S41 list are those on the 2007 UK Biodiversity Action Plan (UK BAP) list that are present in England with the addition of Hen Harrier. There are a small number of taxa below the species level (that is, sub-species) on the S41 lists. Such infra-specific taxa were only retained if the associated species was not included. This led to the removal of three sub-species and reduced the total taxa on the S41 list from 943 to 940. However, not all species on that list have suitable data available. The species in the priority species indicator are those species for which annual estimates of abundance are available, derived from national-scale monitoring schemes. Currently it contains data on 149 species. Four species of moth were excluded from the priority species indicator this year, due to the same failure of quality assurance checks in the all-species indicator. These species are: Haworth’s minor (Celaena haworthii), grey mountain carpet (Entephria caesiata), crescent (Helotropha leucostigma), and heath rustic (Xestia agathina).

There are 12 Section 41 species included in the all-species index that have not yet been included in this version of the priority species index (8 fish, 1 bumblebee, 1 vascular plant and 2 freshwater invertebrates). To retain consistency and comparability with the previous release of this indicator, we have not added them to this release. The impact of this on the results is small and these species will be included in future version of the indicator.

Method for creating a composite indicator of species abundance

The method for estimating the change in relative abundance for a group of species is complex and consists of many steps. The key steps taken to produce the estimates are as follows:

  1. Collection of observations in the field. Each scheme follows a set of standardised protocols to collect data on species abundance, typically involving counts of individuals across a fixed network of survey locations. In the case of vascular plants, abundance is measured in terms of percentage cover, rather than the number of individuals.
  2. Calculation of a national index of abundance for each species in each year. With a few exceptions, this involves the use of statistical methods that were developed specifically for that survey. For most datasets, this step is performed by the schemes that collect the data.
  3. Data cleaning.
  4. Pre-smoothing individual species trends to remove short-term fluctuations and reveal long term trends.
  5. Calculation of smooth multispecies (composite) indices and trends, accounting for missing values.

Steps 1 and 2 vary by monitoring scheme: in most cases the details are published on sampling scheme websites and summarised in Table 5. Steps 3, 4 and 5 are covered in detail below.

Recording scheme data Dataset Owner/Partners Survey Protocol Analytical techniques
BeeWalks Bumblebee Conservation Trust (BBCT), Bees, Wasps and Ants Recording Society, UKCEH, University of Kent Transect counts at approximately 600 non-random sites with replication during the season Statistical model accounting for seasonal variation (peer reviewed)
Breeding Bird Survey (BBS) BTO, RSPB, JNCC Counts on transects on approximately 3,000 randomly-selected sites, with two visits per year (data prior to 1994 used a different approach) Statistical model accounting for seasonal and spatial variation (peer reviewed)
Breeding Bird Survey (BBS) Mammals BTO, RSPB, JNCC Counts on transects on approximately 90% of the 3,000 randomly-selected sites, with two visits per year Statistical model accounting for seasonal and spatial variation (peer reviewed)
Freshwater Invertebrates (BIOSYS) EA (data collection) QMUL (analysis) Counts from kick samples at approximately 7,000 non-random sites (with repeat visits within the year) Statistical model accounting for seasonal variation (partially peer reviewed)
National Bat Monitoring Programme (NBMP) Bat Conservation Trust, JNCC Counts on randomly located transects and non-random hibernation and roosting sites (total number of sites over 1,000) Statistical model accounting for differences between methods (peer reviewed)
National Dormouse Monitoring Programme (NDMP) People’s Trust for Endangered Species (PTES) Counts at nest-boxes (2 visits per year) at approximately 400 known sites Statistical model (not peer reviewed)
National Fish Population Database (NFPD) and Transitional/Coastal waters Data (TRaC) EA (data collection) QMUL (analysis) Counts from approximately 1,000 non-random sites. Some sites use electrofishing, others seine netting. Statistical model accounting for seasonal variation (partially peer reviewed)
National Plant Monitoring Scheme (NPMS) UKCEH, Plantlife, BSBI, JNCC Percentage cover in quadrats on approximately 800 randomly-selected sites, with replication. Statistical model (peer reviewed) for each habitat, combined to a species-level trend.
Priority Moths Butterfly Conservation Counts at known sites using species-specific methods Statistical method (peer reviewed)
Rare Breeding Birds Panel (RBBP) BTO, RSPB, JNCC, RBBP secretariat Near-complete counts of the number of breeding pairs. No (species only included where data judged to be representative of complete counts)
Rothamsted Insect Survey Light Trap Rothamsted Research (RRes) (collection) UKCEH (analysis) Counts at light traps (mostly nightly) at approximately 80 non-random sites Statistical model accounting for seasonal variation (peer reviewed)
Seabird Monitoring Programme (SMP) BTO, JNCC in association with RSPB Near-complete counts at known colonies Not applicable (data are complete counts)
Statutory Conservation Agency and RSPB Annual Breeding Bird Scheme (SCARRABS) RSPB, JNCC, Natural England, NatureScot, Natural Resources Wales, NI Environment Agency Bespoke approach for each species, full census or random stratified sample. Species-specific approach (all peer-reviewed)
UK Butterfly Monitoring Scheme (UKBMS) Butterfly Conservation, UKCEH, BTO, JNCC Counts, mostly on transects, at over 1,000 non-random sites, most with weekly replication. Statistical model accounting for seasonal variation (peer reviewed)
Wetland Bird Survey (WeBS) BTO, RSPB, JNCC Counts at approximately 3,000 non-random sites Statistical model (peer reviewed)

Data Cleaning

The multispecies indicator is conceived as a geometric mean across species. One problem with this is that the geometric mean is undefined if any of the observations are zero. Several of the datasets used for this indicator contain cases where no organisms were observed in a particular year, resulting in zero counts or, in the case of the Rothamsted moths dataset, modelled counts that are extremely close to zero (for example, 0.000001 individuals). It is standard practice in this situation to add a small number to zero counts in order that the geometric mean is calculable. For species with zero counts (including moths with modelled zeros) we added a small number to every observation in that species’ time series. The value we added was equal to 1% of the mean value in that time-series.

Taxon names in the all-species indicator are defined by Schedule 2 of the Environmental Targets (Biodiversity) (England) Regulations 2023. Names in Schedule 2 were harmonized to the UK Species Inventory, which is an authoritative list maintained by the Natural History Museum. Taxon names for the Priority species indicator follow Schedule 41 of the Natural Environment and Rural Communities Act 2006. This part of the data cleaning step involves converting names of organisms to one of these standard lists.

Pre-smoothing

Species abundance of many organisms tends to fluctuate from one year to the next. These fluctuations make it difficult to reveal the underlying trends. For this reason, some schemes include a statistical smoothing to remove short term stochastic variation. A cross-validation exercise showed that our multispecies indicator approach is more robust if the species trends going into the method had already been pre-smoothed (see Technical Annex). We therefore applied a smoothing term to each species time series, except those for which a smoothed trend was already available (bats and most of the birds) and for four bird species where the number of abundance estimates was too few to smooth. We applied a thin plate spline with 0.3 degrees of freedom for each data point (Fewster et al., 2000) and did this on the log scale. The resultant smoothed trends were then taken forward to the next stage of analysis.

For vascular plants, the NPMS data series is just eight years. Smoothing species trends using the rule of 0.3 degrees of freedom per year produces trends that are linear, i.e. straight lines. This creates a situation in which the multispecies average for vascular plants would be estimated with extremely high precision that does not reflect the substantial uncertainty in the individual plant species trends. We therefore decided that a multispecies index of plants based on smoothed data would be misleading. For consistency, we used unsmoothed trends for vascular plants in both the all-species and plant-specific indicators.

To create the composite index, we used a method specifically developed for creating multispecies indicators from heterogeneous data (Freeman et al., 2020). The resulting index is an estimate of the geometric mean abundance. This is a relatively newly developed method and offers some advantages over older techniques: it is adaptable to different data types and can cope with the issues often presented by biological monitoring data, such as varying start dates of datasets and missing values.

A smoothing process is used to reduce the impact of between-year fluctuations - such as those caused by variation in weather - making underlying trends easier to detect. For this a penalised spline was used with the number of “knots” set to one of two values. Firstly, as has been done for previous iterations of the priority species indicator and as is standard elsewhere (Fewster et al., 2000), we used the total number of years of data divided by 3. Secondly, in order to reveal a more stable long-term trend in the data, we used the total number of years of data divided by 10. These two values were selected to demonstrate the range of plausible indicator values for the purposes assessing meaningful change in species abundance over time.

The Freeman method combines the individual species abundance trends taking account of the confidence intervals around the individual trends. The method is Bayesian, so it produces credible intervals to show the variability around the combined index, as well as in the trends of individual species.

The Freeman method was specifically developed to handle missing data, for example, species: year combinations where there is no estimate of species abundance available. These missing values occur for three reasons:

  1. The schemes contributing data to the indicator start at different points in time (Table 4), so the number of species contributing data has grown steadily over time (Figure 7).
  2. There is lag between data collection and the species abundance data becoming available, so the time series for some datasets terminate before 2022 (Table 4).
  3. There are internal gaps in the time series for some species, which happens when the number of sites contributing data falls below the levels required to reliably estimate abundance. This arises from a variety of reasons, including bad weather, natural turnover in volunteers, or if access to the countryside becomes temporarily restricted, as happened in 2001 (Foot-and-mouth) and 2020 (COVID-19) (both dips can be seen in Figure 7).

Missing values are handled by modelling species abundance as a multiplicative process of population growth. Growth rates for internal missing values are imputed by linear interpolation; growth rates for values outside the range of observed values (cases 1 and 2, above) are imputed based on the distribution of growth rates for species that do have data in that year. In other words, species with missing data at the start and ends of the series are assumed to behave in line with the average of all the other species.

Each species in the indicator was weighted equally. When creating a species indicator weighting may be used to try to address biases in a dataset, for example, if one taxonomic group is represented by far more species than another, the latter could be given a higher weight so that both taxonomic groups contribute equally to the overall indicator. Complicated weighting can, however, make the meaning and communication of the indicator less transparent. The main bias on the data is that some taxonomic groups are not represented at all, which cannot be addressed by weighting. For this reason, and to ensure clarity of communication, equal weighting was used, although other options were explored (see Technical Annex).

The overall trend shows the balance across all the species included in the indicator. Individual species within each measure may be increasing or decreasing in abundance (Figure 2). Estimates will be revised when new data or improved methodologies are developed and will, if necessary, be applied retrospectively to earlier years. Further details about the species that are included in the indicator, and the methods used to create the species indicator can be found in the Technical Annex.

When creating the indicator of priority species abundance, a subset of the data included in the process described above was used; only those 149 species for which we have adequate data and which appear on the Section 41 Priority Species in England list were used.

Assessment of change

Formal assessment of change is made on the basis of credible intervals for the time period; if the indicator value for the first year falls outside of the credible intervals for the final year then the indicator is deemed to have changed over that time period. This was done for three time periods; long-term (from the beginning of the time series to 2022), medium-term (the most recent 10 years) and short-term (the most recent 5 years).

To illustrate the variation in trends among individual species, an assessment of change is made for each species. Species are categorised into one of five categories on the basis of defined thresholds (Table 6). The five trend thresholds are based on average annual rates of change over the assessment period and are derived from the rates of decline used to assign species to the red and amber lists of Birds of Conservation Concern (Eaton et al., 2015). Asymmetric percentage change thresholds are used to define these classes as they refer to proportional change, where a doubling of a species index (an increase of 100%) is counterbalanced by a halving (a decrease of 50%).

Category Threshold Long term change
Strong increase An increase of more than 2.81% per annum Equivalent to an increase of more than 100% over 25 years
Weak increase An increase of between 1.16% and 2.81% per annum Equivalent to an increase of between 33% and 100% over 25 years
Little change Change is between +1.16 % and -1.14% per annum Equivalent to a change of between +33% and -25% over 25 years
Weak decrease A decrease of between 1.14% and 2.73% per annum Equivalent to a decrease of between 25% to 50% over 25 years
Strong decrease A decrease of more than 2.73% per annum Equivalent to a decrease of more than 50% over 25 years

The categorisation of species trends emerges from the multispecies model, rather than from the raw data. This is felt to be appropriate, because the multispecies model accounts for missing data and smooths out fluctuations among species with extremely variable trends. A side-effect of this treatment is that a species trend estimate from the all-species indicator model is slightly different from its trend estimate from the priority species indicator model. These differences are minute: the correlation between trends from the different models is extremely high (R squared > 0.999), but in a very small number of cases the difference is enough for species to switch categories. Specifically, two species switched category in the long-term assessment and three species in the short-term assessment.

Caveats and limitations

Collectively the datasets contributing to the all-species and Priority Species indicators are intermittent (contain missing data) and heterogeneous (they have different properties). We now have several examples of creating indices from multiple diverse taxa (for example, the Living Planet Index, the Living Planet Index for the Netherlands and Scotland’s Terrestrial Species Abundance Index), but this process still presents some challenges. A particular issue is that short-lived species (for example, insects) tend to fluctuate markedly in abundance from year to year, whereas long-lived species (birds and mammals) do not. This presents a statistical challenge to capture the signal of long-term change amidst the noise of fluctuating population numbers. The generic method described by Freeman et al. (2020) was designed specifically for this situation and has been assessed by independent experts as being appropriate for the task at hand. However, the method might be further refined in future to capture more completely the differences in population fluctuations between taxa.

Representativeness of the indicator

The all-species indicator was developed with the aim of producing an index to summarise trends in abundance for the broadest possible set of organisms that are representative of English biodiversity, although the species coverage is limited by data availability.

Taxonomic representation was limited prior to mid-2010 (Figure 7). Since publication of the early versions of the indicator, we have responded to stakeholder feedback and added further species groups (freshwater invertebrates, fish, vascular plants, bumblebees) to improve the representativeness of the indicator.

A summary of the species that are, and are not, well represented in the all-species indicator is presented below, and in the data sets accompanying the release of this publication. For more details, please see the Technical Annex. In addition to representation of different species groups, a discussion of how well the indicator represents habitats in England and provision of ecosystem services is also presented in the Technical Annex.

Species in the indicator

The all-species indicator currently includes a proportionally high number of species of moths, freshwater invertebrates, vascular plants and birds, – these groups account for 38%, 20%, 18%, and 14% of the species in the indicator, respectively. The reason for high representation of these groups is that data is collected for a large number of species through well-established monitoring schemes.

Moths are considered to be a useful indicator species for the status of the wider environment, as they are found in many different habitats across England and are highly sensitive to environmental change. The all-species indicator includes data for 19% of moth species in the UK, which makes them less well represented than many of the other taxonomic groups. The short life span of moths means that they quickly respond to change, and monitoring their abundance can provide important insights into the impacts of pressures such as agriculture, climate change, pesticides, and pollution.

  • Moths cover a diverse range of habitats, such as grassland, heathland, woodland and scrub habitat mosaics. These habitats are also important for a diverse range of other species that are not included in the indicator, due to insufficient data, and improvements to these habitats will benefit all of the species that depend on them.
  • Moths are a key part of the food chain, providing an important food source for birds, bats and other insectivorous species.
  • Moths are important pollinators for both wild plants and crops. Studies have demonstrated the importance of the pollination services provided by moths across a range of landscapes, including urban and agricultural (Ellis et al., 2023; Walton et al., 2020). Inclusion of moths, as well as butterflies and bumblebees, also means that pollination services are captured in the indicator (although these are only partially captured – see the Technical Annex for more detail of ecosystem service representation).
  • A review in 2021 by Dar and Jamal provides an overview of how moths have been used as ecological indicators for different environmental pressures (Dar & Jamal, 2021).

Bird species are well represented in the indicator, which includes data for 77% of bird species in the UK. Bird populations have long been considered to provide a good indication of the broad state of wildlife in England. This is because they occupy a wide range of habitats and respond to environmental pressures that also operate on other groups of wildlife. In addition, there are considerable long-term data on trends in bird populations, allowing for comparison between trends in the short term and long term. Because they are a well-studied taxonomic group, drivers of change for birds are better understood than for other species groups, which enable better interpretation of any observed changes. Birds also have huge cultural importance and are highly valued as a part of the UK’s natural environment by the general public.

Data for vascular plants comes from the National Plant Monitoring Scheme (NPMS), which was designed with the core aim of sampling plant communities within habitats of conservation value. The indicator specifically includes species that have been identified as positive indicators of habitat health, across a range of habitats, making these data a valuable component of the indicator. The indicator includes data for 14% of vascular plant species in the UK.

Invertebrates as an overall group are underrepresented in the indicator, which includes data for butterflies (93% of UK species), moths (19% of UK species), bumblebees (less than 5% of UK bee species), and a range of freshwater invertebrates. Freshwater benthic invertebrates make up 20% of the species in the indicator. The Environment Agency sampling scheme for freshwater benthic invertebrates covers all of England and is done in a systematic way, so we can assume that it reflects the abundance of wider benthic invertebrates. However, we can be less certain that these species represent trends across wider freshwater habitats (see the Technical Annex for more detailed discussion of habitat representation in the indicator).

Species not represented in the indicator

Notable taxa for which sufficient abundance data is not currently available include fungi, non-vascular plants (bryophytes and algae), microbes, amphibians and reptiles, and terrestrial invertebrates (other than moths, butterflies and bumblebees). As sufficient abundance data for these groups become available, meeting the criteria outlined in Background and Methodology, possible changes to the list of species in Schedule 2 may need to be considered and consulted on. As the indicator is for a terrestrial target, with the exception of seabirds and a small number of fish living in coastal waters, the indicator does not cover the marine realm.

Butterflies are considered to be a good indicator of terrestrial abundance, and inclusion of over 90% of the UK’s butterfly species in the indicator may partially compensate for the poorer representation of other large insect groups. However, we recognise that there are still key gaps and the lack of data for these groups means that some ecosystem services are not well represented in the indicator (particularly decomposition, pest control, and aspects of pollination) – see the Technical Annex for more detail of ecosystem service representation.

Source data

Table 4 details the datasets used in these indicators, they are generated largely from data collected by national monitoring schemes. In these schemes, data are collected in a robust and consistent manner and the geographical coverage is good, however there is variation in the degree to which each dataset is influenced by biases in the sampling protocols, as well as the methods used to account for that.

Most of the datasets that contribute to the indicator derive from national surveillance schemes with a high degree of spatial replication (for number of survey sites – see table 5). These are ideal for producing population time-series for widespread and common species; however, most of these schemes do not generate sufficient sample size to estimate the abundance of cryptic, rarer or more range-restricted species. Each scheme has a set of criteria to determine whether time-series can be generated for each species and if they are sufficiently robust to be included in the published results of the scheme. Further information about each monitoring scheme and the data analysis and results can be found on each recording scheme’s website.

A smaller number of datasets derive from targeted survey of known populations of rare species. In some cases, the data represent complete censuses of the English population (Table 5). Thus, the indicator has good representation of common species and, in some groups, of very rare species, but species that are neither very rare nor very common are largely absent (the exception to this rule is the butterflies).

Large national sampling schemes invest significantly in volunteer training, support and resources to enhance the accuracy of species records made by volunteers, many of whom have significant taxonomic skill. All records undergo automated and/or manual verification procedures to “clean” anomalies from datasets before analysis. Many schemes have collected long time series of data with consistent methods. Where changes to methods have occurred, the effect of these changes on the data series have been investigated to allow continuous trends to be evaluated.

It is known that structured scheme sampling involves bias. Some biases are accounted for in stratified sampling protocols; others are known because the nature of the schemes involves sampling self-selected, high-quality habitats using a set protocol over a long time period. Some bias is less controlled due to the necessity of giving volunteers some choice over where they record, to retain their interest. For example, there is a general trend in schemes for under sampling in more remote and inaccessible areas, some urban areas, and some areas perceived as “less interesting” for example, large homogeneous arable regions. The effect of these biases is less evident in England than other countries of the UK, given a more evenly distributed volunteer population. There is ongoing research and development work into improving the evidence generated from volunteer schemes, including reducing bias in volunteer datasets, enhancing verification methods, and integrating different types of volunteer datasets to better understand species trends at finer spatial scales.

The values going into the indicator are measures of species abundance at the national scale. The raw data that generate these measures vary from scheme to scheme, reflecting differences in the ecology of the species being monitored. In most cases, the raw data are counts of individual organisms, either from censuses, transect walks or light traps. Some data types are subtly different. For vascular plants, the raw data are measures of percentage cover within quadrats. For bumblebees and the field survey of the National Bat Monitoring Program, the data are counts, but not of individuals. In both cases, the survey takes place along a transect route that is split into sections: at each section the presence or absence of each species is recorded, and the count used for analysis is the number of sections on which the species was recorded. This method is more appropriate as a measure of abundance than the total number of encounters, due to the high probability of counting the same individual more than once. Counting organisms in this way can be thought of as a measure of local occupancy.

Method for including species

The trends of the taxonomic groups included within a multispecies indicator are often obscured by its composite nature. Indicator lines have been generated for a number of subgroups using the same method so that the trends for these groups can be seen more clearly. However, even within taxonomic groups species trends vary significantly.

As set out in Species Included, the all-species indicator includes all native species with suitable data, as well as species naturalised before 1500 and natural colonists from mainland Europe. This means that the all-species indicator does include some species that are associated with unfavourable habitat quality, that have negative interactions with rare and threatened species, or that are in some other way undesirable. We are aware that there is an argument for the exclusion of these species from the indicator, on the basis that increases in their abundance would not reflect improvements in the status of biodiversity. However, we have considered this question and have retained these species for the following reasons:

  1. Species typically thought of as undesirable are characterised by having high abundance in degraded or unnatural habitats (for example, polluted waterways). Under these conditions, species that we consider desirable typically have very low abundance. The value of the indicator represents both the relative abundance and the number of species in each group (desirable versus undesirable). In reality, the number of ‘undesirable’ species is small compared with the much larger number of species in the indicator that are considered to be desirable. Moreover, the indicator is calculated using the geometric mean abundance, the value of the index would generally decrease if habitats were to become degraded (where few species increase but most decrease) and increase if habitat quality were to improve (most species increase but few decrease).
  2. From a practical perspective, any decision about which species are desirable or undesirable to include in the indicator would need to be underpinned by a rigorous process that objectively defines the values by which ‘desirability’ would be assessed. Whilst “desirable” and “undesirable” species have been proposed for some groups in the indicator, a classification is not available for all groups in the indicator. Moreover, it is not clear whether the current methods used to identify “desirable” and “undesirable” species are comparable between taxonomic groups. In the absence of such a process, decisions to remove individual “undesirable” species would be subjective and risk undermining confidence in the indicator.

Confidence and uncertainty

The credible intervals around the multispecies index represent confidence in the degree to which average abundance in any given year is different from the baseline year (1970). They do not provide a clear guidance on the degree to which pairs of years (for example, 2000 versus 2022) differ.

The credible intervals capture uncertainty in the trends of individual species that contribute to the index. They do not capture uncertainty associated with the spatial locations of sample points, nor about the degree to which the species represent wider biodiversity.

The credible intervals partially capture uncertainty in the species abundance estimates, inasmuch as the method includes a term to estimate measurement error. However, our approach does not explicitly propagate information about relative uncertainty of different species or years.

It is standard practice for species trends to be smoothed to reduce the impact of between-year fluctuations - such as those caused by variation in weather - making underlying trends easier to detect, however the manner in which this is achieved varies by recording scheme. There is no existing protocol for smoothing a composite species indicator with as wide a species coverage as that included in these indicators. We explore this in a cross-validation exercise (see Technical Annex). Figure 8 shows just some of this variability for taxonomic groups in the indicator. As individual species trends are difficult to view for hundreds of species, we show just the taxonomic groups with fewer numbers of species.

Notes about Figure 8:

  • Figure 8 shows the smoothed trend from Figure 3 (solid line) with its 95% credible intervals (shaded area) compared against the raw species trends prior (lighter lines) to pre-smoothing and input into the final model.
  • For brevity, only one of the two smoothing options is shown (option 1, most smoothed) and only the taxonomic groups for which we have less than 50 species.
  • Index values represent change from the baseline value for each group; in 2010 for bumblebees, 2000 for fish and 1995 for mammals.
  • The y-axis has been truncated to 400 show the variability of trends around the smoothed index, but one species of bumblebee and several species of fish show trends that are considerably higher, contributing to their wider credible intervals.

Statistically accounting for missing data

As discussed above, not all species have data for every year of the indicator. Therefore, it is necessary to account for missingness using statistics, which means making an assumption about the behaviour of species that are missing. In this case, we assume that trends among missing species follow the same overall distribution as those with data. In other words, we assume that species are missing at random.

In reality, this is not true because missingness is mostly a function of when individual datasets start and finish. Many of the datasets in the all-species indicator started after the year 1995 (Figure 7, Table 4): most of the change in the indicator pre-date this time, so our estimates of long-term change reflect historical trends in birds, butterflies and moths.

Another concern are the species and datasets that finish earlier than others. In the current publication, most of the species with missing data for 2022 are rare species and/or of conservation concern (rare birds, wetland birds, seabirds, bats and priority moths). The index values for the years 2021 and 2022 should therefore be seen as provisional, reflecting assumptions about the 81 species in the all-species indicator (6.9% of the total) whose trend data has not yet been updated.

It is worth noting that any assumption about data missingness is likely to be problematic: in a previous version of the Priority Species Index, it was assumed that species whose time-series end early would remain fixed at constant abundance, which is likely to lead to an overly optimistic view of short-term trends.

Development plan

Developments planned for the next statistical release to be published in 2025:

  • We will work towards selecting the most appropriate smoothing option and producing a single indicator of species abundance, and priority species abundance, in England. We particularly welcome user feedback on the two options presented in this release.
  • A review of potential data sources was conducted as part of the development work supporting Schedule 2 of The Environmental Targets (Biodiversity) (England) Regulations 2023, and led to additional species being added to the all-species indicator. This suggests that data may be available for additional priority species that could be added to the priority species indicator. These data have not been included in this publication but we intend to include them in future releases if data allows.
  • We will develop an indicator of all-species distribution in England, which, alongside the priority species distribution indicator, will both be added to a future version of this publication.

Longer term development plans:

  • We will review on an ongoing basis new species abundance data that may become available.
  • We will continue to review the data that feeds into the indicator. This will include ongoing review of the status of monitoring schemes (including the schemes that provide data that is used in the current indicator, as well as those that may provide new abundance data in future).
  • We will continue to review the representativeness of the indicator. This will include reviewing how well the indicator represents the species groups that are already included in the indicator, as well as identifying opportunities to improve our evidence where there are specific gaps.
  • In this publication we have broken down the trend by taxonomic group only. In future, we will explore further options for breakdowns that may be useful for users of the statistic (for example, separate trends for generalist and specialist species).
  • We will work towards developing an indicator for the abundance of all-species at the UK scale.
  • We will review our methods for assessing change over short and medium time-scales in the indicators and, if appropriate, refine them further.
  • We will continue to improve the quality of the raw data, representation of the indicator, and methodology, in line with our commitment to the Code of Practice for Statistics.

Acknowledgements

Thank you to the many people and organisations who have contributed by providing data, the independent expert review panel who provided useful insights into developing the method and to the many colleagues who have helped produce these indicators.

Technical annex

Criteria for including source data

Three criteria were used to assess whether data were appropriate for inclusion in the indicator:

  1. Scheme uses standardised approach delivering annual abundance indices based on survey protocols and analytical methods that are appropriate for the organisms being studied.
  2. Spatially replicated survey design with coverage across England (or, for very rare species, the data captured should cover the vast majority of populations that are known to exist).
  3. Taxonomic resolution ideally to species level. In some cases, it was considered to be desirable to include data at a higher level to improve taxonomic coverage (for example, aggregated groups of species, or genus-level).

The rationale for these criteria is described below.

Standardised protocol: In order to assess change, it is essential that the abundance data are collected in a consistent manner across time. Structured schemes where data are collected annually, following a strict pre-determined protocol, allow reliable conclusions to be derived from the data on the national status of species and how their populations are changing in the long term. Any changes in protocol should be supported by extensive analysis to show that the resulting trends are robust to the change in methods (as happened when the Common Bird Census was replaced by the Breeding Bird Survey in the 1990s). The methods used vary by scheme to allow data collection to be appropriate for the target taxonomic group, but include transect walks, complete censuses and other approaches, usually with repeat surveys during each year.

Spatial representation: For the indicator to be representative of change across England, it is desirable for contributing datasets to represent the English landscape. To do this, data should have a spatially replicated survey design with coverage across England. Time-series of individual populations are not likely to be representative, except for species for which the vast majority of English populations are counted in these time-series (for example, where there is a single population in England).

Ideally the sample sites would also be a random sample of the English landscape. The datasets in the indicator include schemes that select sites at random (for example, Breeding Bird Survey) and those that are volunteer-selected (for example, UK Butterfly Monitoring Scheme). Allowing volunteers to select monitoring sites creates a number of potential biases in the resulting data (Boyd, Powney, & Pescott, 2023; Fournier, White, & Heard, 2019). However, even randomly selecting sites may not be sufficient to guarantee that the sites with data are wholly representative, because some sites in remote parts of the country may not have an available volunteer to collect the data. Some schemes may also weight sampling to areas of interest (for example, the NPMS sample locations are weighted towards sampling semi-natural habitats), but planned biases of this nature can be accounted for in analysis to understand national species trends.

Taxonomic representation: It is desirable that the data going into the index should measure the abundance of species, rather than some higher taxonomic group (for example, family). However, in some cases it was agreed to be appropriate to include data at an intermediate level (for example, species aggregate or genus level) to improve the taxonomic coverage.

A total of 15 datasets were assessed to meet the criteria for inclusion in the indicator, as summarised in Table 5. The species that these datasets cover are listed in the associated data file.

Version history

An early version of the priority species indicator was published in the State of Nature 2013 report and subsequently developed into an official statistic of priority species abundance in the UK, which was first published in 2014. The development of this indicator is described in Eaton et al., 2015. Between then and 2020 the methods were continually improved, and an indicator for priority species in England was developed. From this, work to develop an indicator for all-species in England began. A number of versions of the all-species indicator have been produced, each containing more species than the last (Table 7).

Version 1 (published in the Biodiversity Targets Consultation detailed evidence report, 2022)

UK Biodiversity Indicator C4a (Status of UK priority species – Relative abundance) was used as a starting point for developing the all-species abundance indicator. Version 1 of the indicator used data from seven datasets, which were the same as those that contribute to indicator C4a, covering butterflies, birds, mammals and moths. Of the species in these datasets, indicator C4a contains only the approximately 200 species that are on priority species lists in the UK. The all-species indicator, however, was expanded to include all the species that were included in the datasets (except for a small number of species that do not occur in England). This version of the indicator had 670 species.

Version 2 (published in the Biodiversity Targets Consultation detailed evidence report, 2022)

Based on Version 1, stakeholders and experts (including the Biodiversity Targets Advisory Group) recommended further exploration of representativeness of the indicator and potential to broaden species coverage. As a result, work was done to expand the indicator to include additional species and make the indicator as representative as possible (subject to the data available). A total of 164 vascular plant and 237 freshwater invertebrate species were added to the indicator to form Version 2.

Schedule 2 of The Environmental Targets (Biodiversity) (England) Regulations 2023

Following a consultation of the biodiversity targets in 2022, a review of the data included in the indicator, including new data sources, was carried out. As a result, additional species were considered to have suitable data to allow them to be added to the indicator: 11 bumblebees, 2 mammals, 38 freshwater and estuarine fish, 23 moths, and 83 vascular plants. A number of species were also removed from the indicator:

  • Two subspecies of the brent goose, Branta bernicula, were merged into one
  • Two moth species were excluded due to insufficient data to report a trend – basil thyme (Coleophora tricolor) and silky wave (Idaea dilutaria)
  • 28 vascular plants were excluded as they were found to occur on very few NPMS grid cells in England
  • Two freshwater macroinvertebrate species were removed due to their invasive status. The sideswimmer, Gammarus tigrinus, is invasive and should not have been included in Version 2. The species group orb mussels, Musculinium spp., includes data for both the native M. lacustre and the invasive M. transversum. Although it is believed that the majority of records are for the native species, there is a risk that the index value for this taxon could increase solely due to the expansion of the invasive species, and it was therefore decided to exclude this taxon entirely.

Following these updates, a list of the 1,195 taxa that should be monitored as part of the statutory species abundance targets was published in Schedule 2. The current publication includes data for all species in Schedule 2 for which data were ready for inclusion. Additional work is needed to prepare the data for a small number of species (10 plants, 6 moths, 1 fish and 1 mammal), so these are not included in this publication – see ‘Background and Methodology’ section for more details.

Table 7. Breakdown of species numbers by taxonomic group in each of four iterations of the all-species index. The number of datasets refers to datasets listed in Table 5.

Taxonomic Group Number of datasets Version 1 Version 2 Schedule 2 Current publication
Birds 5 169 169 168 168
Bumblebees 1 - - 11 11
Butterflies 1 55 55 55 55
Fish 1 - - 38 37
Freshwater invertebrates 1 - 237 235 235
Mammals 4 15 15 17 16
Moths 2 431 431 452 446
Vascular plants 1 - 164 219 209
TOTAL 15 670 1071 1195 1177

Notes about Table 7:

  • The total number of datasets is lower than the sum of this column (that is, 15 rather than 16) because one dataset (the Breeding Bird Survey) reports both birds and mammals.

Expert input to indicator development

Given the complex nature of measuring species abundance, expert input has been sought at various stages of the development of this indicator and previous related measures.

Expert groups were established to inform development of the Environment Act targets. The Biodiversity Targets Advisory Group (BTAG) was established in September 2020 to provide advice to Defra on developing the evidence base for legally-binding biodiversity targets. Details of the BTAG terms of reference, membership, and meeting minutes are published. The BTAG’s remit included providing expert advice on indicators used to measure progress towards the targets, and they provided useful input to the development of this species abundance indicator. Specific recommendations from the BTAG included completing work to broaden the species coverage and improve representativeness, following development of the initial measure with 670 species. This led to the addition of vascular plants and freshwater invertebrates to the indicator. The BTAG’s final meeting was in January 2022.

The BTAG was replaced by Defra’s new Biodiversity Expert Committee (BEC), which was established in September 2023. The BEC is a sub-committee of Defra’s Science Advisory Council, and its 12 expert members provide independent expert advice, challenge and scientific support to Defra specialists and policy makers in matters related to biodiversity. We have sought input from BEC on specific questions around the methodology and publication of the abundance indicator.

We also commissioned an independent expert review of the indicator methodology in Summer 2023. Three academic experts were asked to consider the suitability of the indicator methodology and make recommendations for its continued development. The panel made several recommendations for the methodology, particularly focussed on options to refine the smoothing. We have worked with the expert panel to implement these ahead of publication.

In publishing this release as an Official Statistic in Development, we welcome and invite further feedback from users and experts on the methods and presentation of the indicator that may help to improve future releases.

Model specifics

The Freeman method (Freeman et al., 2020) is a hierarchical Bayesian state-space model that was developed to create multispecies indicators from heterogeneous and intermittent data. Intermittent data refers to the fact that not every species has an observation for abundance in every year. The model deals with these missing values by treating the observations as deriving from a multiplicative growth process. In any one year, the multispecies average growth rate is the average of the growth rates for each species with data for that year. In this way, the missing data are assumed to have the same statistical properties as the non-missing data.

The multispecies abundance indicator in year t is simply the product of the multispecies growth rates from years 1 through t, scaled to have a value of 100 in the baseline year of 1970. Two additional features of the Freeman method are worth noting.

First, the model includes a smoothing term to remove short-term fluctuations in the indicator, such as might arise if many species are simultaneously responding to individual years with favourable (or unfavourable) weather conditions. The smoothing is applied to the growth rates, rather than the indicator itself, for computational reasons. The specific type of smoothing is known as a penalised spline. The degree of smoothing is controlled by a user-defined number of “knots”, which can vary from 2 (a straight line) to n, where n is the total number of years in the dataset. Because the degree of smoothing is defined by the user, rather than estimated from the data, we have chosen to present the indicators with two levels of smoothing. For the option with a greater degree of smoothing (option 1), we set the number of knots equal to one tenth of the number of years in the dataset: given that the full dataset incorporates 53 years of data from 1970 to 2022, this is 5 knots. In the variant with a lesser degree of smoothing (option 2), we used one knot for every three years of data, which is 18 knots. For the taxon-specific implementations of the model, we use the same values for the number of years per knot, which resulted in fewer knots for datasets with short time series. For bumblebees, freshwater invertebrates and vascular plants, which have 13, 10 and 8 years of data respectively, the model with a greater degree of smoothing (option 1) is based on two knots.

A second notable feature of the Freeman method is that, when estimating species-specific growth rates, the model does not treat the input data as perfect, but recognises they arise from a sampling process that is subject to measurement error. The method includes a facility to provide species-specific estimates for this measurement error, if available. However, for the current implementation, we have assumed that measurement error on species abundance estimates is constant. The magnitude of this error is a parameter estimated from the data.

The method is implemented in the BUGS language by the JAGS software (Plummer, 2003) using Monte Carlo Markov Chains (MCMC) and R version 4.3.1 (R Core Team). These are standard approaches for fitting statistical models in a hierarchical Bayesian framework. For further details of the Freeman method, including equations and information about choices for prior distributions, please refer to Freeman et al. (2020).

Model diagnostics

We investigated the performance of the model in a variety of ways:

  • General model diagnostics
  • Sensitivity analysis
  • Cross-validation:
    • Predictive accuracy
    • Influence of the degree of pre-smoothing on predictive accuracy

General model diagnostics

General model diagnostics included several model checks. Initially, we visually examined the MCMC trace plots for each parameter, encompassing parameters related to smoothing, growth rates and other model components. The total number of parameters varied depending on the number of knots used in the model. These plots exhibited no discernible trends, drifts, or irregular patterns, indicating satisfactory convergence. In addition to visual inspection of trace plots for each parameter, we conducted convergence diagnostics using the Gelman-Rubin statistic (Gelman and Rubin, 1992) for different taxonomic groups. The diagnostic values for all considered models based on different groups all indicated that convergence was acceptable.

We thoroughly examined the posterior distributions of the estimated parameters (including the growth rates and smoothing parameters) to evaluate their precision and credible intervals. While the majority of parameters exhibited satisfactory precision and credible intervals, we observed notable variations for certain parameters, particularly when the model was applied to the original data, without pre-smoothing. Specifically, some smoothing parameters displayed relatively wide credible intervals (and in some cases, larger estimates) in cases involving a large number of knots for certain taxonomic groups. However, despite the presence of these uncertainties, our cross-validation analysis, as detailed in Cross validation, revealed no apparent issues with predictive accuracy associated with parameters exhibiting large credible intervals or larger estimates.

Sensitivity analysis

Sensitivity analysis was performed to evaluate the robustness of our model against small perturbations in the parameters of prior distributions. Specifically, we systematically varied the parameters of prior distributions by ±5% of their initial values and observed the corresponding changes in model outputs, including parameter estimated mean and standard deviation values, as well as the values for the multispecies index. This range ±5% was chosen to encompass plausible fluctuations while remaining within a reasonable deviation from the original values. The results suggest that the model demonstrates stable behaviour and is not highly sensitive to these minor perturbations. This finding further strengthens our confidence in the model’s reliability.

Cross validation

Here we present the results of two cross validation exercises. The first exercise assesses the predictive accuracy of our model under different smoothing options, in order to assess whether the outputs are reliable. The second exercise explores the role of pre-smoothing on the model’s performance.

Blocked cross-validation (CV) (Snijders, 1988) provides valuable insights into the predictive capability of statistical models to unseen data, with its fundamental principle revolving around dividing the data into subsets: one for “training” the model and another for “testing” its predictions within temporal blocks.

To thoroughly assess the model’s predictive abilities across various temporal contexts, we employed four types of blocked cross-validation:

  • 5-Fold Cross-Validation (CV1), involving partitioned data into five distinct folds (20% of consecutive years) at a time
  • Leaving-one-year-out (CV2), which entails predicting an entire year at a time
  • 5 Years Block (CV3), involving predicting any 5 years, not necessarily consecutive
  • K-years Cross-Validation (CV4), which partitioned data into either five or two years

These methods evaluate the model’s ability to generalise over consecutive and non-consecutive time intervals. CV2 is computationally intensive and was therefore applied to a subset of comparisons, and CV3 couldn’t be implemented for vascular plants and freshwater invertebrates due to limited available data. All cross-validation types were applied separately to each taxonomic group and some of them to the combined dataset.

Further, we used six metrics to assess the model’s predictive accuracy:

  • RMSE (Root Mean Squared Error) which emphasises error magnitude
  • NRMSE (Normalised Root Mean Squared Error) normalises error by observed value scale
  • SI (Scatter Index) assesses symmetry and spread
  • MAE (Mean Absolute Error) focuses on absolute difference
  • MAPE (Mean Absolute Percentage Error) evaluates percentage difference
  • MASE (Mean Absolute Scaled Error) compares performance against a naive forecast

We present the results using two approaches: firstly, the results are shown as percentages, indicating the improvement of our model compared to an intercept-only model. Higher percentages denote better performance. Secondly, we provide evaluation metric values (such as RMSE) to offer a straightforward measure of model performance across taxonomic groups. Percentages alone may not provide a comprehensive understanding of the results, for instance, high percentages might not necessarily signify better outcomes if the absolute metric estimates are extremely high. Similarly, a minimal percentage improvement might be misleading if the metric estimates are close to zero, suggesting that the intercept-only model also performed well. On the other hand, percentages are more comparable between groups than metric values. This dual approach of presenting both the percentage-based analysis with direct metric estimates ensures a more thorough assessment of model performance.

Categorising metrics values presents a challenge due to their wide range of values, spanning from 0 to infinity. Here, we classify values below 0.1 as indicative of highly accurate predictions, 0.25 as moderately accurate, values below 0.5 as indicative of accurate predictions, and values above 0.5 as less accurate. Categorising percentages can also be subjective and may vary depending on the specific context and goals. Here, we classify improvement percentages as follows: greater than 90% is considered nearly perfect performance, above 75% as very good performance, greater than 50% as good, between 25% and 50% as moderate, and below 25% as limited or low improvement. Additionally, in cases where the model performs worse than an intercept-only model and the improvement percentage is zero (or negative), it is considered the worst (very bad) performance.

In order to evaluate predictive performance for individual species groups and the combined dataset, our model was compared to an intercept-only model in which the multispecies index is constrained to hold a constant value over time.

Part 1. Predictive accuracy

We assessed the model’s predictive capability under 2 different smoothing scenarios; 3 and 17 knots. The model with 3 knots is tailored to capture the general trends in the data, offering a broad overview of patterns while potentially overlooking finer details. Conversely, the model incorporating 17 knots is configured to provide a more intricate representation of the inherent patterns within the dataset. We compare our model’s percentage improvement to an intercept-only model across different cross-validation types (based on original, unsmoothed data).

Results for this exercise can be found in the published datafile. Percentages are shown for different metrics, with each group’s performance individually depicted. These files are given for both numbers of knots, demonstrating that the percentages are very similar for 3 and 17 knots, except for moths (CV1), where our model performed worse than the intercept-only model for 17 knots but showed good performance for 3 knots. Overall, our method outperforms the intercept model in most cases, except for moths and butterflies. Performance is consistently good (>50%) or better (> 75%) for other groups, demonstrating substantial improvement in relation to the intercept-only model, with similar results across different metrics, except for MAPE, which couldn’t be calculated for some combinations (which is not a cause for concern due to their sensitivity to small values).

In particular, the model demonstrated almost perfect predictions for mammals, very good for birds and invertebrates, good for plants and fish (with better results for CV2 and CV3), as well as moths in certain cross-validation types (CV2 and CV3), and a very bad performance for butterflies (CV1) or limited/low improvement (CV2, CV3). This suggests the model’s capability to predict better than the intercept-only model for one year or any 5 years but not for 20% of consecutive years, for both moths and butterflies. The influence of the moths group on the overall combined results led to slightly worse percentages for 17 knots compared to 3 knots.

Part 2. Influence of the degree of pre-smoothing on predictive accuracy

We conducted the blocked cross-validation twice: once with the original species trends and a second time with “pre-smoothed” trends. We also compare results from pre-smoothing on the log and measurement scale with one another. Note that pre-smoothing on the measurement scale created 38 species: year combinations (out of nearly 38,000) in which the smoothed abundance estimate was negative: for these we used the unsmoothed value instead.

The pre-smoothing of species trends was applied to all species equally, using the rule of thumb of 0.3 degrees of freedom per year. Although this rule of thumb has been applied for more than two decades (Fewster et al., 2000), it is appropriate to question whether different levels of pre-smoothing would be more appropriate.

To explore how the degree of data pre-smoothing affects the predictive accuracy of our model, we conducted an additional cross-validation analysis. This analysis mainly focused on four groups: butterflies, fish, invertebrates, and moths. However, it’s worth noting that other groups were also considered, albeit with only some model parameters (i.e. specific degrees of pre-smoothing). We considered butterflies and moths due to their high degree of variability from year to year.

We applied a thin-plate spline smoothing technique for data pre-smoothing, adjusting the degrees of freedom to various fractions (0.2, 0.25, 0.3, and 0.35) of the number of years. This allowed us to generate datasets with differing levels of smoothing, ranging from highly smoothed (0.2 fraction) to less pre-smoothed data (0.35 fraction). While we primarily pre-smoothed data on the measurement scale, we also conducted analyses using data pre-smoothed on the log scale for comparative purposes.

We chose CV1 and CV3 for their efficiency compared to the time-consuming CV2, and they were conducted with only one degree of freedom (0.3). We fully explored CV4 with all degrees of freedom (0.2, 0.25, 0.3 and 0.35). The application of a 5-year cross-validation scheme for invertebrates was not possible due to the limited availability of data spanning only 7 years.

Given the consistent findings across multiple metrics in previous results, we focussed on two primary metrics, Root Mean Squared Error (RMSE) and Mean Absolute Error (MAE), for CV4.

In addition to investigating the model’s performance with 3 and 17 knots, we also evaluated its performance with 10 knots for CV4, aiming to strike a balance between capturing general trends and retaining finer details.

Results from this exercise are presented in the published datafile. Using pre-smoothed data in general improved model performance across all considered taxonomic groups (and the combined dataset) under all considered cross-validation types, particularly benefiting variable groups like butterflies and moths. For less variable groups, the enhancement with pre-smoothed data may not be as apparent (especially in cases where the results already showed nearly perfect or very good performance), but notable improvements were observed, such as in fish data, where the improvement shifted from “good” to “very good” performance (CV1 and CV3; DF = 0.3). It’s worth noting that while the predictive performance for butterflies remained limited/low (approximately 20%) compared to the intercept-only model in CV1, there was a significant improvement for the butterfly group in CV3, shifting from “limited” (less than 25%) to “very good” (greater than 75%) (DF = 0.3). The predictive performance for moths in CV1 (17 knots) also showed significant improvement, shifting from poor performance (worse than the intercept-only model) to good performance (greater than >50%) (DF = 0.3).

The metrics estimates also reveal better model performance with pre-smoothed data, notably improving predictions for fish and invertebrates to “highly accurate” levels. Pre-smoothing substantially improved performance across metrics for both non-problematic and problematic species. Specifically, CV3 with pre-smoothed data demonstrated “highly accurate” predictions for all groups, including the problematic ones. While metrics values were similar for 3 and 17 knots, the 3 knot model outperformed the 17 knot model in certain circumstances, especially notable in butterflies and moths when CV1 was applied to the original data, but much less pronounced when applied to the pre-smoothed data. Additionally, while the MAPE index could not be estimated for most of the original data, it could be calculated from pre-smoothed data.

The results from both CV1 and CV4 showed similar outcomes, leading to identical conclusions. Due to variations in the number of years of data available for different groups, CV1 results for some groups were closer to those of CV4 with 5 years, while other CV1 results for other groups were closer to CV4 with two years. For non-problematic species, the differences between CV1 and CV4 were insignificant and negligible. However, for highly variable species such as moths and butterflies, we found that using CV4 with different years provided additional insights into model performance. Therefore, the results from CV4 are discussed here, focusing on how different degrees of pre-smoothing influence the outcomes, noting differences with CV1 if they occur.

Notably, both RMSE and MAE for CV4 exhibited similar values and patterns across different degrees of data pre-smoothing and numbers of knots, reinforcing the robustness of our findings.

In certain cases, an intercept-only model exhibited superior fit for data subjected to diverse degrees of pre-smoothing in CV4. Consequently, our focus in describing metrics values is not on comparing different pre-smoothing options (which will be addressed later through percentage comparisons), but rather on evaluating the predictive performance for different numbers of knots. The application of a 5-year cross-validation scheme for fish with 17 knots yielded highly unstable results and therefore is not included. This instability might be attributed to the small number of observations coupled with the model complexity.

Most models yielded nearly identical metric values across different numbers of knots and pre-smoothing levels, with insignificant differences of 0.01 to 0.02 in some cases (CV4). Notable deviation occurred in the freshwater invertebrates group, favouring smaller knot numbers (3 or 10, not 17), although also non-significant based on Wilcoxon signed-rank statistical test (p greater than 0.05). These findings underscore the challenge of determining the optimal knot number based solely on these results, suggesting the need for alternative approaches.

We observe that most results demonstrate accurate to highly accurate predictions, even for traditionally variable groups like butterflies and moths. While predicting consecutive five years resulted in larger absolute metric values for these groups, they still remained within the range of accurate predictions. Additionally, moderately accurate predictions were achieved for data with higher levels of pre-smoothing. Overall, increased pre-smoothing led to decreased absolute values across most groups, indicating improved model performance. However, for freshwater invertebrates, highly pre-smoothed data resulted in the worst model performance, suggesting the need for individualised selection of smoothing degrees for each group.

The results show that, overall, higher levels of pre-smoothing (degrees of freedom = 0.2 times the number of years) lead to better percentage improvement in both RMSE and MAE metrics when predicting either two or five years, except for invertebrates. Invertebrates demonstrated better results (96% to 97%) with a lesser degree of pre-smoothing applied to the data, while the worst outcomes (71% to 76%) were observed with a higher degree of pre-smoothing.

Both fish and moths achieved nearly perfect predictions when forecasting two years across all levels of pre-smoothing, and at least very good predictions when predicting five years (and at least good with CV1). Conversely, invertebrates showed only good performance with highly pre-smoothed data, while nearly perfect performance across other pre-smoothing degrees. The most significant variations were found in butterflies, where pre-smoothing greatly affected performance, ranging from moderate (RMSE: 34%-37%; 5 years CV4) to good (RMSE: 71%; 5 years CV4) or nearly perfect (RMSE: 90%; 2 years CV4) depending on the degree of pre-smoothing and the prediction year span.

Similar results were obtained when the model was applied to data pre-smoothed on a log-scale, leading to the same conclusion.

Conclusions

The cross-validation reveals key insights into our method’s performance, informing its strengths and limitations and guiding potential optimisations.

Our model consistently outperformed alternative models in predictive ability. We identified butterflies and moths as more difficult groups to model, where predictive performance sometimes lags behind or only shows limited improvement over an intercept-only model. This confirms the ability of cross-validation to diagnose problems with our models, giving confidence to the approach.

Determining the optimal number of knots to smooth the final model remains challenging due to similar performance across different knot numbers for all cross-validation types, requiring alternative methods or expert input. Our findings also reveal that models with varying degrees of smoothing performed differently across different taxonomic groups.

Pre-smoothing species data notably enhances model performance across all groups, particularly benefiting more variable ones. While some groups demonstrated better performance with models employing greater levels of data pre-smoothing, others showed better results with less data pre-smoothing. Our investigation showed that the degree of pre-smoothing emerges as particularly crucial for the highly variable groups such as butterflies, indicating its sensitivity to this parameter. Conversely, for less variable groups and moths, the degree of pre-smoothing appears to exert less pronounced effects on model performance. Consequently, selecting the appropriate degree of pre-smoothing requires careful consideration. One approach could involve selecting the degree of smoothing separately for each group based on their respective performance metrics. Alternatively, we may opt for the values that yielded the best percentage improvement across all degrees of freedom for smoothing and all knots.

However, given that our pre-smoothing analysis focused on only four groups, uncertainties remain regarding the optimal degree of freedom for broader applicability, necessitating further research and validation. Additionally, our analysis reveals that the choice between pre-smoothing on a log or original scale does not significantly impact the results, suggesting that the model’s performance remains consistent regardless of the pre-smoothing method employed. This study offers valuable insights into the performance of our method and sheds light on nuances surrounding the degree of data pre-smoothing.

Our two cross-validation exercises provide valuable insights into the performance of our method and sheds light on nuances surrounding the degree of data pre-smoothing. The results demonstrate that the method has good statistical properties, both in absolute and relative terms. Moreover, the cross-validation was able to detect poor performance among groups that were known to be problematic and confirm that performance was substantially improved by pre-smoothing. Overall, this provides confidence that the method is sound and fit for the purpose of producing multispecies indicators.

Exploring options for a weighted index

From the statistical perspective, giving different weights to the component parts of a composite index is straightforward. However, from a practical perspective, it is not, as there is no objective way of assigning weights. The pragmatic solution usually adopted is to give each component part the same weight. When there is precisely one component index per species, this corresponds to giving equal weight to each species, regardless of their abundance or range. Where species groups or genera have been included, usually because of the difficulties in resolving to species level, each group has been given a weight equal to the number of species in the group or genus.

By weighting species equally, the composite index is dominated by taxa with many species. Consequently, if there are small datasets with few species that perhaps do not meet the criteria for inclusion, but they are included regardless, the implications are minor, as they will have little effect on the composite index.

Another option is to set equal weights at a higher taxonomic level, such as family level instead of species, though this approach also has limitations. An approach similar to this has been done for the Living Planet Index (McRae et al., 2017) and we have also investigated this earlier on in the development of this indicator (Bane et al., 2022). Lacking any firm basis for setting weights objectively, the default option of weighting species equally has been taken, in line with other indicators in the England Biodiversity Indicators.

Representativeness of the species abundance indicators

It is estimated that the UK is home to around 55,000 native species of fauna, flora, and fungi. While it is unrealistic that any indicator could track all of these species, it is useful to consider which species are included in these indicators (which focus specifically on English Wildlife) and how representative they are of English biodiversity as a whole.

The all-species indicator tracks the abundance of 1,177 species. These species are intended to be representative of wider biodiversity in England, although the taxonomic coverage is limited by data availability. The number of species in the indicator by taxonomic group, and how representative they are of the number in the UK, is shown in the datafiles published alongside this release.

Taxonomic representation

A comprehensive list of species was only available for the whole of the UK, rather than for England specifically, so comparing the list of species in the indicator to the list of UK species does not give perfect insights about how well the indicator represents English wildlife. However, assuming that the proportions of species in different groups is largely similar in England as it is across the UK, useful insights about the taxonomic representativeness of the indicator can be made.

While vertebrate animals are always of great conservation interest, they make up only 0.7% of the UK’s species. It is unavoidable that vertebrates will be overrepresented in species indicators, as it would not be possible to monitor a truly representative sample of invertebrates given that there are so many. In the UK, there are 362 recorded vertebrate species (amphibians, birds, fish, mammals, and reptiles). Of these, 218 are bird species, making this the largest vertebrate group in the UK. The indicator tracks the abundance of 168 species of birds, equivalent to approximately 77% of the UK’s bird fauna. There are 49 species of mammals recorded in the UK, and the indicator tracks the abundance of 33% of these (16 species). However, 10 of these are bat species, so taxonomic diversity of mammals in the indicator is limited. The indicator also includes 37 species of freshwater and estuarine fish. Amphibians and reptiles are not monitored in the indicator – there are seven amphibian and six reptile species in the UK, each representing 0.1% of the total number of species in the UK, but the available abundance data for these species was not deemed to meet the criteria for inclusion in the indicator.

Over half of the UK’s species (54%) are invertebrates; this is primarily insects (23,947 recorded species) accompanied by other invertebrates such as arachnids, crustaceans and molluscs (an additional 5,369 species). The best represented invertebrate groups in the indicator are butterflies and moths; the indicator tracks 55 of the 59 butterfly species recorded in the UK (93%) and 446 out of the 2,345 moth species (19%). Although moths are the highest contributor of species to the indicator, they are less well represented in comparison to other groups, such as birds, butterflies and mammals. While inclusion of freshwater invertebrates helps to balance the invertebrate contribution to the indicator away from being driven by lepidopterans, compared with early versions of the measure, terrestrial invertebrates (other than moths, butterflies, and bumblebees) are a notable gap. The majority of invertebrate species in the UK come from three groups: Hymenoptera (bees, ants, and wasps), Diptera (true flies), and Coleoptera (beetles). These groups are underrepresented in this indicator.

Plants make up 9% of UK species. There are 209 species of plant in the indicator (4% of UK species), all of which are vascular plants (vascular plants make up 30% of the plant species in the UK). Non-vascular plants such as mosses are not represented in the indicator.

There are no fungi in the indicator, although they make up 32% of species in the UK. This is because there are currently no surveillance schemes that would provide the abundance data required to include non-vascular plants and fungi.

Representation of habitats

The indicator includes a large number of terrestrial taxa, sampled from across England, which should allow the indicator to be an acceptable indicator of terrestrial animal and plant abundance (noting the specific gaps highlighted under taxonomic and ecosystem service representation).

Since the first version of the species abundance measure, the addition of data for 235 taxa of freshwater benthic invertebrates, as well as 37 freshwater and estuarine fish, brings the proportion of freshwater species in the indicator to 25% (including 21 bird species from the Wetland Bird Survey). While the inclusion of these species has improved the representation of freshwater habitats in the indicator, the freshwater species are largely represented by benthic macroinvertebrates which means we can be less certain that the indicator will pick up trends across freshwater species as a whole. Specific gaps in the indicator include freshwater plants and non-benthic invertebrates.

With the exception of seabirds and a small number of fish living in coastal waters, the indicator does not represent marine habitats.

Representation of ecosystem services

The representativeness of the indicator for terrestrial ecosystem services was assessed in 2022 in the Evidence to Support Development of Biodiversity Targets: Technical Report, using the framework from Oliver et al. (2015). This assessment has been updated to account for more recent additions to the indicator, as follows:

Pest control: the high number of birds in the indicator captures some aspects of pest control, but the lack of terrestrial invertebrates (including beetles, spiders, centipedes, wasps, dragonflies, damselflies, hoverflies and ants) means this service is only partially captured by the indicator.

Pollination: the inclusion of a high number of butterflies and moths means that pollination services are captured. This representation was improved following the addition of bumblebees to the latest version of the indicator. However, only 11 bee species are included, and other pollinators such as hoverflies, wasps, and beetles are not represented, meaning this service is only partially captured by the indicator.

Decomposition: decomposition services are not captured by the indicator, due to lack of species that play a primary role in decomposition (for example, fungi, ants, isopods, myriapods, and annelids).

Carbon sequestration: The indicator includes 209 species of vascular plant, meaning that the carbon sequestration is captured in the indicator. However, in Oliver et al. (2015) all plants were assumed to have the function of carbon sequestration, so while the plants in the indicator certainly play a role in sequestering carbon, whether they are the most efficient at capturing carbon is not known.

Species in the indicator also provide freshwater ecosystem services. For example, the indicator includes 65 caddisfly taxa which are known to have the ecosystem functions of organic matter breakdown and substrate stabilization (Greenop et al., 2021). Further work is needed to explore the representation of freshwater ecosystem services in the indicator.

References

Bane, M. S., Cooke, R., Boyd, R. J., Brown, A., Burns, F., Henly, L., Vanderpump, J., & Isaac, N. J. B. (2023). An evidence-base for developing ambitious yet realistic national biodiversity targets. Conservation Science and Practice, 5(2), e12862. https://doi.org/10.1111/csp2.12862

Bat Conservation Trust & JNCC (2017). The state of the UK’s bats 2017: National Bat Monitoring Programme Population Trends. Boyd, R. J., Powney, G. D., & Pescott, O. L. (2022). We need to talk about nonprobability samples. Trends in Ecology & Evolution. doi:10.1016/j.tree.2023.01.001

Burns, F., Eaton, M.A., Hayhow, D.B., Outhwaite, C.L., Al Fulaij, N., August, T.A., Boughey, K.L., Brereton, T., Brown, A., Bullock, D.J., Gent, T., Haysom, K.A., Isaac, N.J.B., Johns, D.G., Macadam, C.R., Mathews, F., Noble, D.G., Powney, G.D., Sims, D.W., Smart, S.M., Stroh, P., Walker, K.J., Webb, J.R., Webb, T.J., and Gregory, R.D. (2018). An assessment of the state of nature in the United Kingdom: A review of findings, methods and impact. Ecological Indicators, 94(1), 226 to 236. doi.org/10.1016/j.ecolind.2018.06.033

Comont, R. F., & Dickinson, H. (2022). BeeWalk 10-Year Report. Bumblebee Conservation Trust, Stirling, UK

Comont, R. F., & Dickinson, H. L. M. (2023). BeeWalk Annual Report 2023. Bumblebee Conservation Trust, Stirling, UK

Dar & Jamal (2021). Moths as ecological indicators: A review. Munis Entomology & Zoology, 16 (2): 833-839

Eaton, M. A., Burns, F., Isaac, N. J. B., Gregory, R. D., August, T. A., Barlow, K. E., … Williams, J. (2015). The priority species indicator: measuring the trends in threatened species in the UK. Biodiversity, 16(2–3), 108–119. https://doi.org/10.1080/14888386.2015.1068222

Ellis et al., 2023. https://doi.org/10.1111/ele.14261

Fewster, R M., S T. Buckland, G M. Siriwardena, S R. Baillie, and J D. Wilson (2000). Analysis of Population Trends for Farmland Birds Using Generalized Additive Models. Ecology 81, 1970–84

Fournier, A. M. V., White, E. R., & Heard, S. B. (2019). Site-selection bias and apparent population declines in long-term studies. Conservation Biology, 33(6), 1370–1379. doi:10.1111/cobi.1337

Fox R, Dennis EB, Harrower CA, Blumgart D, Bell JR, Cook P, Davis AM, Evans-Hill LJ, Haynes F, Hill D, Isaac NJB, Parsons MS, Pocock MJO, Prescott T, Randle Z, Shortall CR, Tordoff GM, Tuson D & Bourn NAD (2021) The State of Britain’s Larger Moths 2021. Butterfly Conservation, Rothamsted Research and UK Centre for Ecology & Hydrology, Wareham, Dorset, UK

Freeman, S. N., Isaac, N. J. B., Besbeas, P., Dennis, E., B. and Morgan, B, J., T. (2020) A Generic Method for Estimating and Smoothing Multispecies Biodiversity Indicators Using Intermittent Data. Journal of Agricultural, Biological and Environmental Statistics, 26, 71 to 89. doi.org/10.1007/s13253-020-00410-6

Gelman, A., Rubin, D., (1992) Inference from Iterative Simulation Using Multiple Sequences. Statistical Science 7 (4) 457 - 472, https://doi.org/10.1214/ss/1177011136

Greenop, A., et al. (2021). Patterns of invertebrate functional diversity highlight the vulnerability of ecosystem services over a 45-year period. Current Biology

Hanfling B. & Harley M. (2003) A Molecular Approach to Detect Hybridisation between Crucian Carp (Carassius carassius) and Non Indigenous Carp Species (Carassius auratus and Cyprinus carpio) in UK Waters, including a Consideration of the Taxonomic Status of Gibel Carp (Carassius spp.). Environment Agency R&D Technical Report W2-077/TR

Heywood, J.J.N., Massimino, D., Balmer, D.E., Kelly, L., Noble, D.G., Pearce-Higgins, J.W., Woodcock, P., Wotton, S., Gillings, S. & Harris, S.J. (2023). The Breeding Bird Survey 2022. BTO Research Report 756. British Trust for Ornithology, Thetford

Isaac, N. J. B., van Strien, A. J., August, T. A., de Zeeuw, M. P. and Roy, D. B. (2014). Statistics for citizen science: extracting signals of change from noisy ecological data. Methods in Ecology and Evolution. https://doi.org/10.1111/2041-210X.12254

Mathews F, Kubasiewicz LM, Gurnell J, Harrower CA, McDonald RA, Shore RF. (2018) A Review of the Population and Conservation Status of British Mammals: Technical Summary. A report by the Mammal Society under contract to Natural England, Natural Resources Wales and Scottish Natural Heritage. Natural England, Peterborough

McRae L, Deinet S, Freeman R (2017) The Diversity-Weighted Living Planet Index: Controlling for Taxonomic Bias in a Global Biodiversity Indicator. PLoS ONE 12(1): e0169156. https://doi.org/10.1371/journal.pone.0169156

Nunn, A. D., Ainsworth, R. F., Walton, S., Bean, C. W., Hatton-Ellis, T. W., Brown, A., …Noble, R. A. (2023). Extinction risks and threats facing the freshwater fishes of Britain. Aquatic conservation : marine and freshwater ecosystems, https://doi.org/10.1002/aqc.4014

Oliver, T. H., Isaac, N. J., August, T. A., Woodcock, B. A., Roy, D. B., & Bullock, J. M. (2015). Declining resilience of ecosystem functions under biodiversity loss. Nature communications, 6(1), 1-8

Pescott, O.L., Powney, G.D. & Walker, K.J. (2019b). Developing a Bayesian species occupancy/abundance indicator for the UK National Plant Monitoring Scheme. 10.13140/RG.2.2.23795.48161

Pescott, O.L., Walker, K.J., Jitlal, M., Smart, S.M., Maskell, L., Schmucki, R., Day, J., Amos, C., Peck, K., Robinson, A. & Roy, D.B. (2019a). The National Plant Monitoring Scheme: A Technical Review. JNCC Report No. 622, JNCC, Peterborough, ISSN 0963- 8091

Plummer, Martyn. (2003). JAGS: A Program for Analysis of Bayesian Graphical Models using Gibbs Sampling. 3rd International Workshop on Distributed Statistical Computing (DSC 2003); Vienna, Austria. 124

R Core Team (2022). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. URL https://www.R-project.org/

Snijders, T.A. (1988). On cross-validation for predictor evaluation in time series. In: On Model Uncertainty and its Statistical Implications, pp. 56–69. Springer

Van Strien, A. J., van Swaay, C. A. M. and Termaat, T. (2013). Opportunistic citizen science data of animal species produce reliable estimates of distribution trends if analysed with occupancy models. Journal of Applied Ecology, 50(6), 1450 to 1458. https://doi.org/10.1111/1365-2664.12158

Walker, K.J., Stroh, P.A., Humphrey, T.A., Roy, D.B., Burkmar, R.J. & Pescott, O.L. (2023). Britain’s Changing Flora. A Summary of the Results of Plant Atlas 2020. Botanical Society of Britain and Ireland, Durham

Walton et al., 2020. https://doi.org/10.1098/rsbl.2019.0877

Wembridge, D., White, I., Al-Fulaij, N., Marnham, E. & Langton, S. (2019) The State of Britain’s Dormice 2019. People’s Trust for Endangered Species, London

Whitehorn, P. R., Seo, B., Comont, R. F., Rounsevell, M. & Brown, C. (2022). The effects of climate and land use on British bumblebees: Findings from a decade of citizen-science observations. Journal of Applied Ecology, 59, 1837–1851. https://doi.org/10.1111/1365-2664.14191