Guidance

Methods and limitations

Updated 6 December 2023

Applies to England

1. Survey Methodological Note

The Adults’ People and Nature Survey for England gathers evidence and trend data through an online survey relating to people’s enjoyment, access, understanding of and attitudes to the natural environment, and its contributions to wellbeing.

The Adults’ People and Nature is an online panel survey (self-completion questionnaire), sampling up to 25,000 adults in England on a continuous basis over 2020/2021. The survey launched in April 2020 and builds on the Monitor of Engagement with the Natural Environment (MENE) survey that ran from 2009 to 2019.

The adults’ People and Nature Survey consists of different modules. All modules are asked throughout the entire year, but certain modules (those that do not require an overall sample size of 25,000) are only asked to a randomly selected sub-sample of individuals. This means that some questions will have responses from everyone asked and some will have a smaller sample size.

Data is released quarterly and the survey started in September 2020. Additionally, in response to the coronavirus (COVID-19) pandemic, we produced monthly indicators of some indicative metrics using data from a subset of the survey questions between April 2020 and March 2022. These early releases enabled us, in a timely manner, to better understand the impacts and implications of COVID-19 on people’s engagement with nature, health and wellbeing, and have informed the Government’s and others’ plans for recovery.

Data from the Adults’ People and Nature Survey was initially released as Experimental Statistics, which are Official Statistics that are in the testing phase. However, after a year of data collection, we have developed a collaborative analysis framework, refined our data quality assurance processes and finalised an optimal specific weighting scheme approach. Products from the Adults’ People and Nature Survey are now therefore of sufficient statistical quality to merit ongoing production as Official Statistics. Publications are labelled according to their release status.

Experimental and Official Statistic statuses allow for the noteworthy change in data collection methodology between MENE and the People and Nature Survey to be fully evaluated before a planned transition to National Statistics designation. For more information on Official Statistics please visit the Office of National Statistics website.

In 2020, the Adults’ People and Nature Survey underwent a rapid regulatory review with the Office for Statistical Regulation to ensure compliance with the Code of Practice for Statistics, and its principles of:

• Trustworthiness: is about having confidence in the people and organisations that produce statistics and data.

• Quality: is about using data and methods that produce assured statistics.

• Value: is about producing statistics that support society’s needs for information.

More information about the People and Nature Survey and the latest questionnaire are available.

We welcome all feedback from users to aid the development of our publications by email to: people_and_nature@naturalengland.org.uk

1.1 Monthly Indicator Releases

Monthly indicator releases designated as Experimental Statistics have been published for the first year of the Adults’ People and Nature Survey (April 2020 – March 2021).

For the first year of the survey, indicators were generated using an interim weighting method. The interim weights were originally developed for the MENE survey and enabled Natural England to report timely data on people’s engagement with nature during COVID-19 while specific Adults’ People and Nature Survey weights were in development.

For the second year of the survey onwards (April 2021), indicators have been published using specific Adults’ People and Nature Survey weights as Official Statistics.

These publications report a set of weighted national indicators from the survey, which have been generated using data collected from April 1st 2020. Results generated using the interim weights are titled ‘Monthly Interim Indicators’, whereas results generated using the specific People and Nature weights are ‘Monthly Indicators’. Additionally, the monthly indicators for the first year have been updated with the specific Adults’ People and Nature Survey weights to allow comparisons between the first year and subsequent years.

National indicators include:

• Gross number of visits taken by adults (by age, income, ethnicity, and sex)

• % adults spending time outside in last 12 months by frequency

• % children spending time outside in last 12 months by frequency

• % adults visiting outside in last 14 days

• % adults visiting outside in last 7 days

• % adults visiting outside by place type (such as parks) in last month

• % adults agree that natural spaces are ‘Good places for mental health and wellbeing’

• % adults agree natural spaces are ‘Places that encourage physical health and exercise’

• % adults agree having access to a private or shared garden, outdoor space or allotment is important

• % adults agree ‘I feel part of nature’

• % adults agree ‘Being in nature makes me very happy’

• % adults agree that they are taking more time to notice and engage with everyday nature

• % adults engagement with nature since coronavirus restrictions began

• % adults behaviour and attitude changes in relation to green and natural spaces since coronavirus restrictions began

• % adults increasing amount of time on certain activities since coronavirus restrictions began

• % adults reducing the amount of; time they travel by car, food their household throws away or meat they eat since coronavirus restrictions began

• % adults prevented from spending time outdoors since coronavirus restrictions began due to certain barriers

• % children spending time outside and engaging with nature since coronavirus restrictions began, as reported by adults

1.2 Sampling

A sample of households was selected from the England subset of Kantar’s Lightspeed Panel. Samples are representative of the target population (adults aged 16 or older living in England) and quotas for age, gender, region, education and ethnicity are in place to ensure key population groups are adequately represented.

Sample sizes are included in all releases for each question, and in the supporting data. It should be noted that all respondents were not asked all questions, and this has been flagged where applicable.

1.3 Weighting

The interim weights are representative of the English adult population, according to the latest population estimate data available from the Office for National Statistics and based on the weighting developed for the MENE survey data. This approach was deemed appropriate given the similarities between MENE and the Adults’ People and Nature Survey, and the significant amount of MENE data supporting the key variables used to develop the MENE weights. These weights were applied to statistical releases up to December 2021.

Specific weights for the Adults’ People and Nature Survey for England have now been developed and applied to statistical releases from January 2021 onwards. These have been generated using regression modelling, which was able to highlight a large number of demographic variables significantly associated with number of visits to green and natural spaces.

We have included the following significant demographic variables in the Adults’ People and Nature Survey weight (Urban / Rural, Region, Age by Gender, Working Status by Gender, Long lasting health condition, Number of cars/vans, Education, Ethnicity, Dog ownership, Presence of children under 16) and ensured the weighted People and Survey data is representative of the population in terms of these variables. See the Technical Report for more information on how this weight has been generated.

There are small differences in figures generated by the interim and bespoke Adults’ People and Nature Survey weight. Therefore, direct comparisons between monthly statistics from the Adults’ People and Nature Survey that have been generated using the different weights is not advised. Previous results have been updated with the specific Adults’ People and Nature weight, and these should be used for comparisons.

2. Strengths & Limitations

2.1 Experimental statistics collected via an online panel

The Adults’ People and Nature Survey is conducted by Kantar using its online Lightspeed panel. The online panel brings benefits in terms of speed, ability to interview during lockdown and reducing social desirability bias. However, it is important to flag limitations:

• Since panels are opt-in there is the risk that panelists are not representative of the general population.

• Online panel surveys exclude the off-line population.

To minimise the risk of bias, Lightspeed uses a variety of recruitment methods. This includes opt-in email, e-newsletter campaigns, and social media.

Sample composition analysis during development of these statistics compared online panel methods and MENE. It showed that the online samples tend to be more representative than MENE in terms of age and working status. However, online samples tend not to represent the very oldest in the population well (as they are the least likely to have internet access). Both the online panel and MENE tend to achieve samples that are broadly representative of the population by region. Online samples tend to under-represent people from ethnic minority backgrounds, which is why an additional quota criteria for ethnicity was set for the Adults’ People and Nature Survey.

The survey was developed specifically to ensure it worked well on mobile devices. Quota objectives to achieve a nationally representative sample in terms of age, gender, region, ethnicity, and education level have been achieved (at least 95% of the target). Work is continuing to understand strengths and limitations of the panel method and further information will be published in due course. Details on how data is collected, analysed and reported on for specific sub-groups within the population are provided below.

Ethnicity

For the Adults’ People and Nature survey, we have collected data on ethnicity in line with the 2011 Census^{[footnote 1]}. The Office for National Statistics (ONS) runs the Census of England and Wales every 10 years. The decision on which ethnic groups to include is made in consultation with users of Census data (like government departments, local authorities, and the National Health Service) as well as people completing the Census, and the interest groups that represent them.

It is recognised that ethnic groups included in the Census do not represent how all people identify. Like the Census, the Adults’ People and Nature Survey offers an option for people to write in their ethnic group or background using their own words if they do not identify with any groups in the list.

Following government guidance we use the term ‘ethnic minorities’ rather than Black, Asian and minority ethnic (BAME) or Black and minority ethnic (BME).

While we consider all ethnic minorities in our analysis, we may not be able to report findings for individual groups owing to sample size constraints. In this instance, we will do our best to highlight where differences between ethnic minority groups and the White British group are occurring in relation to engagement with, and enjoyment of, the natural environment.

As our sample size grows, we will increasingly be able to analyse data and report findings that are specific to individual ethnic minority groups. Ultimately, we hope with this analysis to be able to reflect socio-cultural influences as well as other drivers of behaviour (including structural barriers, such as experience of discrimination) which may differ from the White British group.

Disability

For releases pre-September 2020 we can report data in consideration of whether the respondent has long term (12 months+) physical or mental health conditions or illness. From September 2020 we introduced a second element to this question which asks about whether this reduces the respondent’s ability to carry out day-to-day activities. This addition will ensure that we are able to report based on the Office for National Statistic’s definition of disability. We recognise that this bio-medical definition of disability, which focuses primarily on limitations of the individual body has restrictions . Alternative understandings are important, including those reflecting the role of physical and social environments in creating experiences of disability (for example, how settings are designed and managed, and how people with impairments are treated by others when in green and blue spaces). The survey team is considering how complementary research could reflect these broader understandings of disability.

Gender

Quotas have been set to compensate for known biases in online panels. Younger people and men are generally under-represented on panels (as they tend to be across all survey methods), so the quota by age and sex is interlocked. It is important to note that the population targets used only include male and female categories. As there are not yet robust population targets that allow for more than male/female categories, in order to manage survey fieldwork (and monitor quotas) anyone that selects a non-binary response is randomly assigned to either male or female in the quota targets and in the weighting. This includes the quota /fieldwork tables in reports. However, for analysis purposes the full response list is included in the dataset and any combining of response categories should be clearly stated in any reporting.

Between April and September 2020, the survey asked respondents about their sex, in October 2020 the wording was updated to ask about gender identity. The current survey response options are: ‘male’, ‘female’ or ‘in another way’.

Poverty

The Adults’ People and Nature survey asks respondents to indicate which of a range of income brackets best describes their total annual household income before tax. The lowest income bracket reported on for the Adults’ People and Nature Survey is £0-14,999 and in previous versions of the report, this was a good approximation of income below the relative poverty line. This is defined by ONS as households with less than 60% of contemporary median income before housing costs. However, prices and inflation change, and for 2022, the threshold for ‘relative poverty’ has risen to £17,628 ^{[footnote 2]}. The People and Nature Survey’s lowest income bracket of £0-14,999 now more closely reflects the bottom ten percent of households in terms of income before housing costs, rather than relative poverty.

2.2 Comparing indicators with other data

Where appropriate, indicators have been compared against other data sources and we report where similar or contrasting trends have been identified.

Although similar types of data are reported, direct comparisons of the Adults’ People and Nature Survey and MENE findings should not be made. The Adults’ People and Nature Survey has similar aims and many similar questions to MENE and it provides useful context, like the external survey data included in these monthly reports. However, it is important to note significant differences in methods could have an impact on results. The Adults’ People and Nature Survey uses an online panel approach whereas MENE used face-to-face interviews; there are differences in how respondents answer questions between these two different survey methods. Furthermore, for many questions, there have been other changes. These include question recall period and question length, plus new questions that have been added to the Adults’ People and Nature Survey.

2.3 Socio-demographic characteristics

Results from MENE indicated that there was inequality of environmental engagement between population sub-groups defined by several demographic characteristics (including: age, ethnicity, and socio-economic background, region, urban-rural and health).

The Adults’ People and Nature survey has been designed to report findings by socio-demographic characteristics. Reporting on a quarterly basis ensures larger sample sizes are reported against each question, thereby reducing standard errors.

1. In England and Wales, there are 18 ethnic groups recommended for use by government when they ask for someone’s ethnicity. These are grouped into 5 broad ethnic groups, each with an ‘Any other’ option where respondents can write in their ethnicity using their own words. For further information, please see the guidance. ↩

2. Source of 2022 income data: DWP (Mar 2023) Households Below Average Income ↩