Research and analysis

Evaluation of changes in dietary methodology in the NDNS: stage 3

Published 29 May 2025

Executive summary

Introduction

This report presents findings from the third and final stage of the formal evaluation of the changes in dietary assessment methodology in the National Diet and Nutrition Survey (NDNS). These changes were implemented from October 2019 (fieldwork year 12). The report updates the stage 2 analysis (covering 2019 to 2022) by including data collected from May 2022 to May 2023.

NDNS is a cross-sectional survey (that is data collected from participants at a single point in time) jointly funded by the Department of Health and Social Care (DHSC) and the Food Standards Agency (FSA). The survey is carried out by a consortium of the National Centre for Social Research (NatCen) and the Medical Research Council (MRC) Epidemiology Unit, University of Cambridge. It is designed to assess the diet, nutrient intake and nutritional status of the population aged 18 months and over living in private households in the UK. NDNS results are published for the age groups:

• 18 months to 3 years
• 4 to 10 years
• 11 to 18 years
• 19 to 64 years
• 65 to 74 years
• 75 years and over

Background to NDNS methodology changes

For the first 11 years of NDNS (2008 to 2019), dietary data was collected over 4 consecutive days using a paper food diary with estimated portion weights. In 2018, following a review of available digital tools, the dietary assessment method was moved to Intake24 and data was collected over 4 non-consecutive days. Intake24 is a web-based automated self-administered tool which collects detailed information on all food and drink a person consumed the previous day. Data collection using Intake24 began in October 2019.

Alongside the change to the dietary assessment method, there was also an increase in the number of participants selected per household, from the previous one adult and/or one child, to up to 2 adults and one child, or 2 children and no adults.

Carrying out a full comparison study of Intake24 alongside the paper diary, with a subset of individuals completing both methods, was out of scope due to budget constraints.

The original intention had been to evaluate the new dietary assessment method and associated changes to the fieldwork model over fieldwork year 12 (October 2019 to June 2020). However, the COVID-19 pandemic resulted in the suspension of fieldwork for 7 months which limited the amount of data collected in 2020 and continued to affect data collection for the remainder of the fieldwork period. So, the evaluation was carried out in a staged approach over a longer period.

Previous evaluation reports

The NDNS stage 1 evaluation report (PHE, 2021a) was published in 2021, based on data collected from October 2019 to March 2020. This initial evaluation aimed to gain an early understanding of any implications of the method change and identify any immediate issues.

The NDNS stage 2 evaluation report (OHID, 2023) was published in 2023 and sought to evaluate resolution of issues identified at stage 1 and to identify any new ones. It was based on NDNS data collected from October 2019 to May 2022 and data from the COVID follow up study in summer 2020.

The findings of the stage 1 evaluation indicated that introducing Intake24 was compatible with NDNS requirements. Comparability with previous data collected using the food diary appeared to be generally good, although a number of issues were identified for further review and action.

The stage 2 evaluation, based on a larger data set, did not identify any new concerns and suggested that some (though not all) of the issues identified in stage 1 had been resolved. The stage 2 evaluation also showed that use of Intake24 in NDNS is associated with a similar level of underreporting of energy intake to that found by previous NDNS studies using the food diary method.

What this stage 3 evaluation report includes

This stage 3 report updates the stage 2 analysis, including more data collected from May 2022 to May 2023. It includes:

• assessment of how the new fieldwork model is working in terms of participation rates for dietary recalls and spread across days of the week
• quality measures for use of Intake24, including completion time and number of foods reported
• analysis of the impact on dietary data of the fieldwork method change to select more individuals from the same household to participate in the survey
• analysis of the impact of changes to the dietary assessment method on time trends for foods and nutrients

Findings of the stage 3 evaluation

Dietary recall participation

Between October 2019 and May 2023, a total of 4,089 individuals (1,943 children and 2,146 adults) completed at least 1 dietary recall, and 76% of participants went on to complete all 4 dietary recalls. A dietary recall is a record of all food and drink consumed over the previous day (24 hours) completed retrospectively by the participant. The aim was to equally represent all days of the week across the data set as a whole. However, there was a small shortfall in recalls completed about a Saturday. This may be due to some participants being reluctant to complete recalls on a Sunday. This issue will continue to be monitored.

Impact of selecting more participants per household on dietary data (clustering)

A more cost-efficient model for selecting participants in a household was introduced in 2019. This change was implemented alongside the change to the dietary assessment method.

The change increased the number of participants selected from one adult and/or one child to 2 adults or 2 children (the children being in different NDNS age groups). This increased the proportion of multi-participant households in the data set from 33% to 59%. Two people who live in the same household are likely to have more similar diets on a given day than 2 people who live in different households.

The survey design meant that the first recall was usually completed by all participants in the household on the same day. However, completion days for recalls 2 to 4 were randomly assigned, so 2 participants in the same household were no more likely to be asked to complete their recalls on the same day than 2 participants living in different households.

This selection model change did result in some clustering of diets (that is less variation in diets within households than between households), but the analysis provides some reassurance that the degree of clustering is unlikely to be great enough to reduce confidence in the survey estimates.

Intake24 quality measures

Quality indicators, to help determine whether participants were using Intake24 as intended, were assessed for a total of 14,025 recalls completed from October 2019 to May 2023. Recall times, number of items and eating occasions and reporting of missing items were considered.

The quality assessment found that:

• 10% of recalls had at least 1 food reported by the participant as missing from Intake24
• 1% of all individual food items recorded were reported as missing foods
• 61% of foods reported by participants as missing could be matched from the food description to an existing food code in Intake24 by the research team
• the proportion of recalls with at least 1 missing food decreased with successive recalls
• the median recall completion time was 14 minutes (the mean was 33 minutes)
• 31% of recalls were completed in less than 10 minutes
• in total, 30% of recalls for children aged 11 to 18 years and 24% for adults aged 19 to 64 years contained fewer than 10 items
• for children 11 to 18 years the proportion of recalls that contained 9 items or fewer increased with each successive recall from 27% at recall 1 to 34% at recall 4

These findings indicate behaviour change between the first recall and later recalls. This may be due to participants becoming familiar with how to use Intake24 and so completing the later recalls more quickly. It could also be due to participants’ reduced compliance over time with the request to report everything that was consumed. The differences are greater in the 11 to 18 years age group. This could suggest reduced attention to detail and potentially increased underreporting in this age group. The proportion of recalls with very high or very low energy intakes was small and was comparable with the previous food diary method.

Evaluating impact on dietary data

To assess the impact of the method change on the food and nutrient intake data, individual average daily intake for energy and a selection of nutrients obtained using the paper food diary was plotted per quarter of a year for years 1 to 11 (2008 to 2019).

Data collected from October 2019 using Intake24 was added to the plots for the stage 1 and 2 reports. A linear regression line was shown for years 1 to 11, but at the time it was not possible to extend this line to the Intake24 data. These plots allowed any visually apparent changes in variation between the food diary and Intake24 data to be identified.

Step changes (that is changes that are different from what the years 1 to 11 trend data would suggest) were identified for some foods at stage 1. Following improvements to Intake24, some of these step changes were no longer apparent at the stage 2 evaluation, while some remained. No new step changes were identified.

An objective of the evaluation was to investigate the possibility of continuing the data set for monitoring trends over time. For this final stage of the evaluation, a trend analysis was carried out on selected foods and nutrients using the full set of weighted NDNS data from 2008 to 2023. A weighted regression modelling approach was used for the trend analysis. This approach included a break for the diet method change to Intake24 in year 12 such that regression lines were fitted separately for 2008 to 2019 (fieldwork years 1 to 11) and 2019 to 2023 (fieldwork years 12 to 15). This approach enabled any step changes to be quantified at the point of the method change. The analysis revealed a number of step changes, including:

• downward step changes in intakes and/or the percentage of consumers of fruit and vegetables, red and processed meat and oily fish for some or all age groups
• an upward step change in soft drinks with added sugar for young children
• downward step changes in confectionery, biscuits, buns, cakes and pastries and crisps and savoury snacks for young children

For energy intake, upward step changes were seen for children but downward step changes for adults. For saturated fat, free sugars and fibre, there were few step changes.

Changes to the analytical approach following the move to Intake24 may have contributed to these step changes. The move to Intake24 was accompanied by a change from collecting data on consecutive to non-consecutive days. This provided the opportunity to move to calculating ‘usual intakes’ for foods and nutrients, which is the accepted method to calculate population habitual intakes. This method reduces the impact of day-to-day variation in individual intakes and so reduces the extremes of the distribution while having a minimal impact on estimates of average consumption. This gives a better estimation of proportions above or below a recommended level and allows data to be included from participants who completed only 1 or 2 recalls, as well as those who completed 3 or 4. Participants who completed only 1 or 2 recalls were less likely to be consumers of infrequently consumed foods, for example oily fish. This may have contributed to changes seen in the percentage of consumers for oily fish and other less frequently consumed foods.

Conclusion

This stage 3 evaluation report builds on the findings of the stage 1 and 2 reports on the quality measures for Intake24 and the assessment of the impact of the method change on the dietary data. This report also presents data on the impact of selecting more participants per household on the dietary estimates (clustering).

The Intake24 method is more cost efficient and scalable than the food diary method. The evaluation has shown that the new method is compatible with NDNS requirements.

The evaluation identified a number of step changes in reported consumption of foods between Intake24 and the paper food diary. However, it is not clear in all cases if the observed step changes are a result of method changes, real changes in dietary intakes or a combination of both. So it is not appropriate to assess time trends in diet across the method change from paper diary to Intake24 in 2019. There is currently insufficient data to be confident about the time trend from 2019 to 2023, and it is too early to say whether or not the step changes seen represent a sustained change in long-term trends.

Data collected from 2024 will enable further assessment of trends and time trend data will be included in future reports when sufficient data is available.

Although this report marks the end of the formal evaluation of the method change from paper food diary to Intake24, the performance of Intake24 to capture diet will continue to be monitored throughout the next phase of fieldwork from 2024 to 2029 (years 16 to 20).

1. Introduction

1.1 Background

NDNS was set up in 2008 as a continuous cross-sectional survey designed to assess the diet, nutrient intake and nutritional status of adults and children aged 18 months and over living in private households in the UK. NDNS data is used by the government to:

• monitor progress towards the diet and nutrition objectives of UK health departments
• develop policy interventions
• provide evidence to support nutritional risk assessments by the Scientific Advisory Committee on Nutrition

NDNS is a government-commissioned survey jointly funded by DHSC and FSA. NDNS is carried out by a consortium comprising NatCen and the MRC Epidemiology Unit, University of Cambridge.

For the first 11 years of NDNS (2008 to 2019), dietary data was collected over 4 consecutive days using a paper food diary with estimated portion weights. This required paper-based, open-text entry by participants with review by field interviewers, and retrospective coding of foods and portions into the dietary assessment system DINO (Diet In Nutrients Out) (Fitt and others, 2014) by trained coders. In 2018 the decision was taken to move to an automated dietary data collection method to enable increased cost efficiency, while achieving similar data quality and the potential to scale the survey in the future.

Following a review of available automated tools and full evaluation of 3 shortlisted tools, Intake24, an automated, web-based, multiple pass, 24-hour recall tool, was selected to replace the paper food diary. The dietary data collection methodology change and associated fieldwork model changes were implemented in October 2019 (the start of fieldwork year 12). Due to budget constraints, the decision was taken not to conduct a full comparison study of Intake24 alongside the paper diary.

1.2 Content of this report

The following sections of this introduction provide an overview of the dietary data method change and associated fieldwork model changes for NDNS along with the summary findings from stages 1 and 2 of the evaluation.

Chapter 2 sets out how the fieldwork model is working in terms of participation rates for dietary recalls and spread across days of the week.

Chapter 3 presents intakes of key diet variables for multi-participant households to investigate clustering effects.

Chapter 4 presents a consideration of quality measures around use of Intake24, including completion time and number of foods reported.

Chapter 5 presents a time trend analysis to provide an assessment of step changes in order to consider the impact of the method changes on continuation of time trends for foods and nutrients.

Chapter 6 presents the conclusions and next steps.

1.3 Overview of evaluation

Since the purpose of NDNS is to provide information on food and nutrient intakes, including trends, for the UK population, it is important to understand the implications of survey methodological changes for data interpretation. Alongside evaluating the ability of the new tool to effectively capture dietary intake in the national survey setting, an assessment of the implications of the methodological changes on the quality, coverage (for example of days of the week) and detail of nutrient data collected and the comparability of results with previous years was also needed.

The primary objectives of the evaluation of the changes in methodology in the NDNS setting were to:

• describe how the new dietary method is performing in NDNS
• identify aspects of data discontinuity and assess the feasibility of continuing the time series data set for monitoring trends over time
• assess the degree of misreporting of energy intake (EI) by comparing EI from Intake24 in NDNS with total energy expenditure (TEE) measured by the objective biomarker doubly labelled water (DLW)
• compare differences in the ratio EI:TEE measured by the DLW sub-study using Intake24 and previous DLW sub-studies conducted in NDNS using the paper diary method

The original intention was to implement and evaluate the new dietary assessment method and associated changes to the fieldwork model over NDNS fieldwork year 12 (planned to run October 2019 to August 2020). However, year 12 fieldwork was suspended as a result of the COVID-19 pandemic in March 2020 and was not restarted. Due to the limited data collected in year 12, and the continued impact of COVID-19 on NDNS fieldwork, the evaluation has been conducted in a staged approach over a longer period.

A stage 1 evaluation report (PHE, 2021a) was published in September 2021 based on data collected during the 6 months of year 12, from October 2019 to March 2020. This initial evaluation aimed to gain an early understanding of any implications of the method change and identify any immediate issues.

A stage 2 evaluation report (OHID, 2023) was published in November 2023. This included the data from stage 1 along with data from October 2020 to May 2022. It sought to evaluate which issues identified at stage 1 had been resolved and to identify any new ones. Data from the NDNS follow-up study during the COVID-19 pandemic (August to October 2020) (PHE, 2021b) was also included to fill in the time gap resulting from the suspension of the main survey fieldwork due to COVID-19. The stage 2 evaluation also included findings of the NDNS DLW sub-study carried out across 2019 to 2022 (fieldwork years 12 to 14) to assess the degree of misreporting of energy intake using Intake24.

This stage 3 report concludes the planned evaluation of the changes in methodology. It updates the stage 2 evaluation analysis, including data from May 2022 to May 2023. It also presents data from 2008 to 2023 (fieldwork years 1 to 15) to provide an assessment of any step changes which may be associated with the move to Intake24. It also includes an assessment of the impact on the dietary data of the associated fieldwork method change to select more individuals per household to participate in the survey. 

Figure 1: Intake24 data collection periods included in first, second and third stage of evaluation

Figure 1 illustrates the data collection periods included in the stage 1, stage 2 and stage 3 evaluation reports and how these relate to survey fieldwork. Stage 1 covers October 2019 to March 2020 (fieldwork year 12). Stage 2 covers October 2019 to May 2022 (fieldwork years 12, 13 and 14). Stage 3 covers October 2019 to May 2023 (fieldwork years 12, 13, 14 and 15).

1.4 Overview of methodological changes

Intake24 is a web-based, automated, 24-hour dietary recall tool (Rowland and others, 2018; Bradley and others, 2016; Foster and others, 2019) designed to be self-completed. Participants are asked to record everything they ate and drank the previous day. The tool includes an embedded database of foods with linked portion sizes and corresponding nutrient composition data from which dietary intakes are automatically calculated.

Changes were made to Intake24 and to the underlying data before it was used in NDNS, to enable it to meet survey requirements^{[footnote 1]}. These included:

• updating the tool functionality and adding questions to provide supporting information, for example where food was obtained from
• updating the food list from which participants select the foods and drinks they consume
• rationalising and updating the linked nutrient composition information drawn from the NDNS Nutrient Databank (NDB) (see section 5.4)

These changes aimed to achieve a comprehensive and up-to-date database of foods to reflect the heterogeneity of foods in the UK, while remaining manageable for participants.

For NDNS, the change to Intake24 included moving to a more generic coding frame for mixed dishes (for example, recipes, salads and sandwiches), which would previously have been manually coded as individual components.

The move from a paper diary to a web-based recall is a significant method change for NDNS with a number of implications. It puts the onus on participants to use the online tool to search for and select foods rather than describing their foods freely and then their written entries being retrospectively coded. The web-based data collection model also allows for a much-reduced level of interviewer involvement with fewer interviewer visits to the household and significantly reduces the burden of dietary data coding. At the time of changing the dietary assessment method to Intake24, there was also a change in the selection method within a household, so that more NDNS participants could be included per household. All these changes were necessary to carry out the survey within budget.  

Table 1 gives an overview of the main methodological changes. Following a ‘dress rehearsal’ in April to June 2019 to test the new survey fieldwork model and processes, NDNS fieldwork for year 12 was launched using Intake24 (UK Locale, System Version 3, 2019, Cambridge University) in October 2019.

Table 1: summary of methodological differences in respect of changes introduced from year 12 (October 2019) of the NDNS

Component	Fieldwork years 1 to 11 (2008 to 2019)	Fieldwork years 12 to 15 (2019 to 2023)
Dietary assessment method	Paper diary completed prospectively with estimated portion sizes	Retrospective using online multiple pass 24-hour recall, Intake24.
Dietary data collection	Participant self-completion using open text with retrospective food and portion coding into the dietary assessment system DINO by trained coders.	Self-completion where participants search for foods against pre-set list and select best match for food names and portion options. Auto-linked to food codes and portion amounts. Nutrient information calculated within the tool.
Dietary data collection	Participants encouraged to report recipes, ingredients and, to aid coding, provide food packaging.	Participants encouraged to match to pre-defined list of foods. Limited reporting of recipes or individual ingredients. Option to report a missing food if match cannot be found.
Recording days	Consecutive over 4 days, start day randomly allocated (designed to include at least 1 weekend day).	Non-consecutive, total 4 days selected randomly (designed to include at least 1 weekend day).
Portion size assessment	Portion sizes mainly reported as household measures, often estimated as small, medium or large servings. Limited number of portion size photos available.	Large proportion of food codes in Intake24 linked to a range of portion-size photos from which the participant can select the best match to their portion size. Household measures also available for some foods.
Food coding	Individual coding of food items. Very few generic codes for mixed dishes, for example all reported recipes and sandwiches entered as individual ingredients or components.	Pre-set food list and embedded coding. More generic codes to allow a single code to represent a range of mixed dishes. Participants can report individual foods or recipes if they cannot find a match in the food list.
Reporting dietary supplement use	Dietary supplement use reported by free text description in the same way as foods.	Questions on dietary supplement use with pre-set list in the same way as foods, linked to generic supplement codes. Option to report a missing supplement if match cannot be found.
Dietary data set for analysis	Data set for analysis includes only participants providing 3 or 4 diary days.	Data set for analysis includes all participants with at least 1 recall.
Interviewer visit	2 to 3 visits or contacts to each household to complete all interviewer stage components. Additional visits for participants requiring assistance.	Single visit to household to complete all interviewer-stage components. Additional visits for participants requiring assistance (including those with no internet).
Interviewer involvement or support	Diary introduced at first visit and then reviewed at a mid-diary visit or telephone call. Reviewed and information clarified on third visit.	First recall completed by participant during interviewer visit. Subsequent recalls completed independently with no interviewer involvement except for households requiring assistance.
Participant selection	1 adult and (where present) 1 child selected in around one-third of addresses. 1 child (no adult) from the remaining two-thirds of addresses.	2 adults and (where present) 1 child selected in around one-third of addresses. Up to 2 children (no adult) from the remaining two-thirds of addresses.

Further details of the changes in methodology introduced from year 12 of NDNS and the ‘dress rehearsal’ can be found in chapter 2 of the stage 1 evaluation report (PHE, 2021a) along with details of the updates and modifications made to Intake24, including preparation of the embedded food list and rationalised NDB.

1.5 Summary findings of evaluation stage 1 and stage 2

The stage 1 and 2 evaluation reports covered the: 

• overall response to the survey, participation rates for completing dietary recalls and representation of weekdays and weekend days in the data set
• measures around use of Intake24, including completion time, number of foods reported
• impact of rationalisation and updating of the NDB on resulting dietary data
• impact on continuity of the NDNS trend data series

The stage 1 evaluation (PHE, 2021a) was based on a limited amount of data due to the impact of COVID-19 on NDNS fieldwork. However, it provided an early indication that overall, the introduction of Intake24 as a new dietary assessment tool was compatible with NDNS requirements. Based on a visual inspection of the data, comparability with the previous data collected using the food diary appeared to be generally good. However, a number of specific issues were identified for further review and action.

The stage 2 evaluation report (OHID, 2023) did not identify any new concerns and some of the specific issues identified in the stage 1 evaluation had been resolved by making changes to Intake24. However, some issues had persisted and these are described in this stage 3 report (see chapter 6).

The stage 2 evaluation also presented results of the DLW sub-study carried out across years 12 to 14, showing that use of Intake24 in the NDNS is associated with a similar degree of underreporting of energy intake to that found by previous NDNS DLW studies for the food diary method. Mean EI:TEE was 0.70 in this study and 0.71 in the previous years 6 and 7 sub-study. Misreporting (generally underreporting) is an inherent feature of any self-reported dietary assessment instrument.

2. Dietary recall participation

This chapter looks at the number of dietary recalls achieved and the spread across days of the week to assess how the new fieldwork model is working. Findings from the previous evaluation reports have been updated and this analysis uses combined data collected from October 2019 to March 2020 (year 12) and October 2020 to May 2023 (years 13 to 15). The following commentary is supported by appendix A.

2.1 Overview of fieldwork changes as a result of COVID-19

Data collection using the new fieldwork model using Intake24, with face-to-face participant selection and interviews, began in October 2019 (fieldwork year 12) and was due to run until August 2020. However, as a result of COVID-19, year 12 fieldwork was suspended on 18 March 2020 and did not resume.

Fieldwork subsequently re-started in October 2020 (year 13) with an adapted protocol to allow for remote data collection. Following participant selection on the doorstep, interviews were conducted by telephone, enabling the continuation of data collection through the ongoing COVID-19 pandemic. This approach was used for NDNS until August 2021.

Face-to-face interviewing restarted in September 2021 and continued for the remainder of the fieldwork for years 12 to 15. The option of a remote interview was retained for participants who did not wish to have a face-to-face visit. Full details of changes to fieldwork as a result of COVID-19 can be found in appendix A.

Response rates in NDNS, as in all surveys, fell during COVID-19 and have not recovered. So, it has not been possible to evaluate the impact of the new fieldwork model on participant response. Direct comparisons to response rates achieved in past NDNS years have not been made in this report.

2.2 Dietary data collection

In NDNS years 12 to 15 (for both face-to-face and remote interviews), interviewers introduced Intake24 and provided each participant with a unique URL that would be used to access Intake24 online for self-completion of all their recalls. Participants were told that once they accessed the link, they would be invited to watch a short tutorial video about Intake24.

For face-to-face interviews, the first recall was completed with the interviewer present. Interviewers then checked that participants had been able to complete and submit their recall. For remote interviews, interviewers asked participants to complete their recall following the telephone interview. A follow-up phone call was then scheduled on the same day or the next day for interviewers to check that participants had submitted their recall with no issues.

Participants were asked to complete 4 recalls in total. Subsequent recalls were completed independently by the participants who were notified when their next recall was due by text and/or email. Where participants were unable to complete recalls independently, for example due to internet access issues or lack of confidence with technology, assistance was available. Further details on the support procedures can be found in appendix A.

2.3 Participation rates for dietary recalls

Between October 2019 and May 2023, a total of 4,089 individuals completed at least 1 dietary recall (defined as productive participants): 1,943 children and 2,146 adults (appendix A, table A1).

The majority of productive participants went on to complete all 4 dietary recalls (3,107, 76%), 170 (4%) completed 3 recalls only, 274 (7%) completed 2 recalls only and 538 (13%) completed 1 recall only. The proportions of participants completing each of the recalls are similar to those reported in the stage 2 evaluation (OHID, 2023).

2.4 Recalls by day of the week

The NDNS design aims to provide an even representation of all days of the week in the overall dietary data set. In years 1 to 11 (2008 to 2019), the food diary could start on any day of the week, including weekend days, and would run for 4 consecutive days. The diary start day was randomly assigned for each participant at the first interviewer visit.

With Intake24 from year 12 (2019), the first recall was completed at the main interviewer visit (which could take place any day of the week but was less likely to take place at the weekend). The date for each subsequent recall was randomly allocated by the invitation system within 2 to 6 days after completion of the previous recall. If a participant did not complete their recall on the requested day, the recall invitation system sent up to 4 reminders (firstly in the evening of the initial requested day and then at intervals over the next 9 days), always requesting completion of the recall for the preceding day.

Figure 2: percentage of recalls by days of the week 2019 to 2023 (years 12 to 15)

Recalls	Monday	Tuesday	Wednesday	Thursday	Friday	Saturday	Sunday
Recall 1	19%	18%	17%	14%	10%	6%	16%
Recall 2	15%	12%	14%	15%	14%	15%	16%
Recall 3	14%	15%	12%	11%	23%	12%	12%
Recall 4	15%	16%	14%	12%	21%	11%	12%
All recalls	16%	15%	14%	13%	16%	11%	14%

Figure 2 shows the percentages of recalls obtained for each day of the week overall and for recall numbers 1 to 4 for years 12 to 15 (2019 to 2023). Data is shown for the day of the week each recall represents, rather than the day it was completed (as each recall represents the diet of the previous day). The pattern of interviewer fieldwork has resulted in fewer first recalls being completed for Fridays and Saturdays (as less interviewing took place at weekends) and Saturdays were underrepresented overall.

It should also be noted that some participants completed their recalls on a date that was different from the one they were initially allocated. Recalls were retained within the data set regardless of whether they were completed on the day requested or on a different day.

An adjustment was made to the recall invitation system in October 2020 to attempt to balance out the overall proportions of recalls completed for respective days of the week, such that third and fourth recall invitations were sent on the weekend if the participant had yet to complete a recall on a weekend day. This adjustment has increased the proportion of third and fourth recalls completed about a Friday (with an increase overall from 13% for the period October 2019 to March 2020 to 16% for October 2019 to May 2023). However, this change has had little impact on the proportion of recalls completed about a Saturday (with a slight decrease overall from 12% for the period October 2019 to March 2020 to 11% for October 2019 to May 2023). This may be due to participant reluctance to complete recalls on a Sunday.

No adjustment has been made in the analysis to account for the slight imbalance in days of the week.

The spread of days across the week will continue to be monitored with the aim to keep coverage of days as balanced as possible.

3. Household clustering effect on dietary data

This chapter presents an analysis to investigate the degree of clustering of dietary data within a household following the change in participant selection protocol for 2019 to 2023 (years 12 to 15). It also includes a consideration of any implications for reporting of dietary data.

3.1 Overview of participant selection

Along with the change in dietary assessment method, the other major planned change from 2019 (year 12) was the introduction of a more cost-efficient model for the selection of participants within a household.

As in previous NDNS years, the aim was to achieve an annual sample of 500 adults (aged 19 years and over) and 500 children (aged 18 months to 18 years) that was representative of the UK population. To achieve these numbers in years 1 to 11, one adult and one child were selected in around one-third of sampled addresses and one child (no adult) in around two-thirds of sampled addresses.

From year 12, the protocol was revised so that up to 2 adults and one child were selected from around one-third of addresses (referred to as ‘basic addresses’), and up to 2 children (each from a different NDNS age group: 18 months to 3 years, 4 to 10 years, 11 to 18 years) and no adults were selected from the remaining two-thirds of addresses (referred to as ‘young person addresses’). This change in selection meant that from year 12, the required number of adults and children were achieved from a smaller number of households than in previous years. For the first time the NDNS data set could include 2 adults or 2 children (from different NDNS age groups) from the same household.

A multi-participant household refers to a household where at least 2 members were NDNS participants. Prior to the method change in 2019 (years 1 to 11), 20% of households yielded more than one participant and 33% of participants overall were from multi-participant households. However as explained above, there was never more than one adult participant or more than one child participant in a household. In years 12 to 15 (2019 to 2023), as expected, clustering of participants was higher (approximately double) than was seen in years 1 to 11. The revised selection protocol resulted in 40% of households yielding more than one participant and 59% of participants were from multi-participant households. In years 12 to 15, 56% of productive adults were in a household with a second adult participant and 35% of productive children were in a household with another child participant (not in the same NDNS age group).

In 2019 to 2023 (years 12 to 15), the fieldwork model meant that the first recall was completed during or shortly after the interviewer visit, so participants within the same household would likely complete their recall on the same day. In the majority of cases, participants would then go on to complete subsequent recalls independently with no further involvement from interviewers. Days for recalls 2, 3 and 4 were randomly allocated on an individual basis, and invitations were sent via text or email. So, invitations for recall days 2, 3 and 4 were no more likely to coincide for participants in the same household than they were for participants in different households.

Overall, in years 12 to 15, 38% of households had 2 or more recalls completed on the same day. For multi-participant households this increased to 94%, but this is not surprising as the protocol was for the first recall to be completed at the main interviewer visit for all participants within the household. Across the overall data set, 32% of recalls matched the date of another recall within the same household. For multi-participant households this was 53% but the proportion decreased with increasing recall number (89% for recall 1 and 31% for recall 4).

3.2 Clustering of participants within households

To investigate the extent to which clustering of participants in years 12 to 15 resulted in a clustering of dietary data, the within-household variance (Wth) and between-household variance (Btw) components were calculated (by fitting a linear mixed-effects model) for a selection of dietary variables. This was to test the assumption that the diets of members of the same household will be more similar to each other than the diets of participants from different households.  

If there is a high degree of clustering of dietary data within households, Wth will be small compared with Btw, and so the ratio Wth:Btw will be very close to zero. If there is a low degree of clustering, Wth will be a similar size to Btw and so the ratio Wth:Btw will be very close to 1.

Table 2 shows the ratio of within and between-household variance components using the years 12 to 15 data for intakes of energy, a selection of nutrients, and fruit and vegetables. A small Wth:Btw ratio (close to zero) indicates a high degree of clustering of these variables within a household.

Table 2: Wth:Btw ratio for selection of dietary variables (years 12 to 15)

Diet variable	Adults (aged 19 years and over)	Children (aged 18 months to 18 years)
Energy (kcal/day)	0.72	0.65
Protein (g/day)	0.72	0.64
Protein (per cent of total energy)	0.64	0.56
Total fat (g/day)	0.66	0.65
Total fat (per cent of total energy)	0.56	0.73
Carbohydrate (g/day)	0.69	0.64
Carbohydrate (per cent of total energy)	0.51	0.69
Saturated fat (g/day)	0.64	0.60
Saturated fat (per cent of total energy)	0.52	0.66
Free sugars (g/day)	0.65	0.42
Free sugars (per cent of total energy)	0.64	0.39
Fibre (g/day)	0.54	0.59
Vitamin A (micrograms per day (mg/day))	0.49	0.51
Vitamin D (mg/day)	0.59	0.61
Total fruit (g/day)	0.54	0.49
Total vegetables (g/day)	0.45	0.52
Total fruit and vegetables (g/day)	0.38	0.47

Generally, the Wth:Btw ratios did not indicate a high or low degree of clustering for any of the selected dietary variables, even though participants could complete their recalls on the same day as others from the same household.

For roughly half of these dietary variables, the ratio was smaller for children than for adults, indicating children were more likely than adults to have more similar diets to each other within the same household. For adults, the lowest ratio was for total fruit and vegetables (0.38) while for children the lowest ratio was for free sugars (0.39). 

This analysis showed, as expected, that including more than one adult or more than one child participant per household in the survey from year 12 does result in some clustering of adult and child diets in the data set. This clustering, due to the different household selection model, was not present in the years 1 to 11 data set. While it is not possible to quantify the impact of this change on the survey results, the Wth:Btw ratios are not close to zero, which provides some reassurance that the degree of clustering is not so great as to reduce confidence in estimates. The random allocation of recall days to individual participants for recalls 2, 3 and 4 is likely to have helped mitigate the degree of dietary clustering within households.

4. Intake24 quality measures

This chapter provides an updated assessment of how the new dietary method is performing in NDNS by looking at a range of indicators to help determine whether participants are using Intake24 as intended.

For the stage 2 evaluation, the dietary data set included 6,715 recalls collected from October 2019 to March 2020 (year 12) and October 2020 to May 2022 (years 13 and 14). This third stage report includes additional recalls collected from May 2022 to May 2023 (years 13, 14 and 15), resulting in a final years 12 to 15 combined data set of 14,025 recalls (see figure 1). This larger sample increases the confidence in the evaluation of the data presented, compared with previous stages. The commentary in this chapter is supported by appendices A and B.

4.1 Missing foods

Intake24 automatically assigns food codes and weights in grams to the foods and portion sizes selected by participants, allowing the tool to auto-generate nutrient data. Participants are asked to select, wherever possible, foods from the food list provided in Intake24. If they cannot find an exact match for the food consumed, the tool prompts them to choose the closest matching item. If a participant still cannot find a suitable match, they can report their food as a missing food (by food name and/or providing ingredients of a recipe). Foods reported as missing are later manually assigned to an appropriate food code and portion size, using the missing food details and original free text search term provided by the participant. The raw Intake24 output is imported into a bespoke database to facilitate the coding of missing foods and further dietary data checks (see appendix A).

In the years 12 to 15 dietary data set, across all age groups, 10% of recalls had at least 1 food reported as missing from Intake24 by participants. The proportion of recalls with at least 1 food reported as missing decreased with successive recalls: 13% of first recalls had at least 1 missing food with 11% for recall 2, 9% for recall 3 and 8% for recall 4 (appendix B tables 1.1 and 1.2). This may be due to a number of factors including familiarity with using Intake24 and finding matches, or participants becoming less inclined to report missing foods.

One in 100 (1%) of all food items recorded were reported as missing foods by participants (data not shown). Of these foods, 61% (or 1,178 items) could be matched from the missing food description to an existing food code in Intake24 by the research team. Numbers for missing foods do not include:

• nutrient supplements
• duplicate entries
• items that did not require coding, for example non-nutrient supplements and cold and flu remedies

Similar percentages were seen when split by age group, except for older adults aged 65 years and over where 73% of foods reported as missing could be matched to existing food codes, and for children aged 18 months to 3 years where 53% could be matched.

For those missing foods that could not be matched to existing codes (39%, or 765) the food was allocated a ‘closest match’ code available in Intake24. Monitoring these ‘closest match’ codes is an ongoing task so that a decision can be taken on whether a new food needs to be added in Intake24 at a later stage, based on the reported amount and frequency of consumption. This helps to ensure that the food list in Intake24 reflects general consumption patterns within the population, while remaining proportionate and manageable for users.

4.2 Recall completeness

Given the nature of self-complete dietary assessment methods, it is not possible to know the extent to which participants are accurately and fully reporting the food and drink they have consumed. All methods are subject to reporting error and bias. Measures such as recall completion times and number of food items reported can provide an indication of recall completeness. For this evaluation, a number of thresholds were adopted for consideration. These were recalls with:

• 9 or fewer food or drink items
• 3 or fewer eating or drinking occasions
• completion time of under 10 minutes
• very low calorie intake (less than 400kcal) or very high (more than 4,000kcal)

These thresholds were not designed to classify recalls as complete or incomplete and it cannot be assumed that recalls that met any of these criteria were of poorer quality than recalls that did not. The thresholds were used to indicate where there may have been issues related to data collection that required further investigation, and to provide a set of checks that could be used to assess the tool’s performance across the age groups. A further purpose was to compare, where possible, with the same measures from the paper diary. The checks were observational rather than statistically assessing any associations.

For the years 12 to 15 combined data set, the median recall completion time was 14 minutes (mean was 33 minutes). Some long completion times were seen (9% of recalls took between 30 to 59 minutes and 7% of recalls took more than 60 minutes). Completion times are calculated within the Intake24 system from when a participant first logs in until they finally log out or submit their recall. To provide maximum flexibility for NDNS participants, Intake24 is configured so that a participant can complete their recall in one session or in stages over the day (including using different devices) providing it is submitted before midnight on the day the recall was started. So, if a participant takes a break during or before submitting their recall (passive time) and leaves their device logged onto Intake24, this will be reflected in their completion time.

Recall times have been included in the analysis without adjustment, but they may not necessarily reflect the time actively spent on recall completion, particularly for longer recall times.

Other similar online recall dietary assessment tools, MyFood24 and ASA24, have reported mean completion times of 16 and 24 minutes respectively, although system configurations are different (Albar and others, 2015; National Cancer Institute, 2023).

In Intake24, 39% of recalls were completed within 10 to 19 minutes, and 31% in less than 10 minutes. Eight per cent of recalls were completed in less than 10 minutes by adults aged 65 years and over, compared with 40% by those aged 11 to 18 years. Only 78 recalls (0.6%) were completed in under 3 minutes. Overall, the proportion of participants completing a recall in less than 10 minutes increased for each successive recall, with the biggest jump between the first and second recall. The percentage of participants who took less than 10 minutes for the recall was:

• 14% for the first recall
• 31% for the second recall
• 40% for the third recall
• 44% for the fourth recall

This pattern was observed in all age groups, particularly in children aged 11 to 18 years. The reduced completion time with successive recalls has been observed in similar tools (Subar and others, 2020) and could be related to a combination of factors including becoming more familiar with the tool (appendix B tables 2.1 and 2.2).

Thirty per cent of recalls for children aged 11 to 18 years and 24% of recalls for adults aged 19 to 64 years contained fewer than 10 items, compared with 6% to 14% of recalls in the other age groups. For children aged 11 to 18 years, the proportion of recalls containing fewer than 10 items increased with each successive recall (27% for the first recall, up to 34% for the fourth recall) while the proportions remained similar across the recalls for the other age groups (appendix B tables 2.1 and 2.3).

Figure 3: recall completion times and number of food items – all recalls 2019 to 2023 (years 12 to 15)

Figure 3 shows completion times plotted against number of food items reported for all 14,025 recalls. As noted above, completion times may include passive time and not necessarily only indicate time actively spent on completing the recall. The vertical line indicates the 10-minute threshold and the horizontal line indicates the 10 food items threshold. The plot shows that while 38% of the recalls completed in less than 10 minutes had fewer than 10 food items, up to 25 items were being recorded in under 10 minutes.

For children aged 11 to 18 years, the spread of data was similar to that seen overall even though this age group had the lowest mean number of food items (12 per recall) and highest percentage of recalls with fewer than 10 items. For this age group, 49% of recalls completed in less than 10 minutes had fewer than 10 food items.

These observations indicate behaviour change between the first recall and later recalls. This may be due to a combination of factors including positive learning effects, such as familiarity with using Intake24, and adaptive behavioural effects which may be related to study participation and compliance. These include reduced attention and reduced detail in the approach to reporting, resulting in potentially higher underreporting of foods and drinks. In other cases, fewer items may have been consumed. The differences between successive recalls are greater in the 11 to 18 year age group. However, results from the DLW sub-study show that while there is underreporting with Intake24 in older children, it is lower than the degree of underreporting seen in adults (OHID, 2023).

Consideration was given as to whether participants reported a similar number of food items using Intake24 compared with the paper diary. For the last 3 years in which the food diary was in use (2016 to 2019), it was possible to count items coded in the DINO dietary assessment system in a way that was similar to counting items recorded through Intake24. Some differences in number of food items were expected given the revised coding approach with Intake24 and the rationalisation of food codes. For example, in the paper diary, a cheese sandwich would be coded as at least 2 food items (bread and filling) whereas in Intake24, a generic cheese sandwich code would most likely be selected with the sandwich coded as 1 item.

Figure 4: percentage of diary days and 24-hour recalls by number of food items

Figure 4 shows the distribution of the number of food items per food diary day in fieldwork years 9 to 11 (based on 14,140 diary days) alongside the distribution of the number of food items per recall in fieldwork years 12 to 15 (based on 14,025 recalls). The figure shows that the 2 distributions differ slightly. The mean number of food items per recall was 17, while the mean number per diary day was 20. There was a higher proportion of Intake24 recalls at the lower end of reported number of food items, with 12% of 24-hour recall days having fewer than 10 food items reported, compared with 5% of diary days. For children aged 11 to 18 years, 24% of recall days had fewer than 10 food items reported, compared with 14% of diary days (appendix B table 2.1).

It was hypothesised that some participants might have tried to speed up recall completion by aggregating several different eating occasions together. For instance, recording all food items under breakfast and evening meal rather than reporting them at different times across the day. If this was the case, the data might show fewer eating occasions for 24-hour recalls completed in less than 10 minutes, perhaps along with an increase in the number of food items reported per eating occasion. For recalls completed in less than 10 minutes the mean number of eating occasions was 4.3, compared with 5.1 for recalls completed in 10 minutes or more. In addition, the mean number of food items recorded per eating occasion was 2.6, compared with 3.1. While there are differences, these figures suggest that this strategy was not being widely used.   

Six per cent (903) of all recalls were completed in less than 10 minutes, had 3 or fewer eating occasions and had fewer than 10 food items.

Overall, the proportion of 24-hour recalls with very high or very low energy intakes was small and was comparable with the proportion of diary days from years 1 to 11 (combined) (appendix B table 1.1). In the years 12 to 15 (combined) data, 128 recalls (0.9%) had energy intakes less than 400kcal per day, in 61 of which the participant reported eating less than usual. The percentage of paper diary days below 400kcal in years 1 to 11 (combined) was similar (0.6%). In the years 12 to 15 (combined) data, after ‘winsorization’ of pizza portions 87 recalls (0.6%) had energy intakes more than 4,000kcal per day, in 27 of which the participant reported eating more than usual. Winsorization is a statistical technique that involves recoding extreme values to the nearest ‘reasonable’ value. In this case, a 1,000g cut-off was applied to the pizza portion data. This was also similar to the percentage of diary days above this cut off (0.5%). For both recalls and diary days, the majority of records with energy intakes more than 4,000kcal per day were in the age groups 11 to 18 years and 19 to 64 years.

Figure 5: recall completion times and energy intake – all recalls

Figure 5 shows recall completion times plotted against energy intake for all recalls (14,025) in fieldwork years 12 to 15. The vertical line indicates the 10-minute threshold. It shows a wide spread of energy intakes for recalls completed in under 10 minutes, with 0.5% of these recalls exceeding 4,000kcal per day and 1.9% being below 400kcal per day. This pattern was similar across all age groups.

5. Evaluating impact on dietary data

The changes in survey methodology implemented from October 2019 (fieldwork year 12) are summarised in section 1.3. The objective of the evaluation has been to identify and understand any differences observed between the dietary data collected before and after these changes and to consider whether there are potential implications for continuing the NDNS trend data series.

The objective has not been to align data collected before and after the method changes. It is accepted that measurement error applies across all self-report dietary assessment methods and that different methods will have different error profiles. So, it is likely that the error profiles will be different for the paper food diary and Intake24 used in NDNS.

5.1 Step changes observed in stage 1 and 2 of the evaluation

At stage 1 and stage 2 of the evaluation, data for a set of key foods and nutrients were visually inspected, selected on the basis of the following considerations:

• importance for government policy and monitoring over time
• indicators which have been relatively constant over time and where change would not necessarily be expected
• items commonly omitted
• items which may be misclassified by participants when using Intake24

These are listed in tables D1 and D2 of appendix D in the stage 2 evaluation report (OHID, 2023)

For the selected foods and nutrients, for each NDNS age group (18 months to 3 years, 4 to 10 years, 11 to 18 years, 19 to 64 years, and 65 years and over), individual level average daily intake (obtained using the paper food diary) was plotted per quarter of a year for years 1 to 11 (2008 to 2019). Then a weighted linear regression line was presented along with combined year weighted means. Details on weighting the NDNS data can be found in appendix B of ‘NDNS: results from years 9 to 11’ (PHE, 2020). Individual-level average daily intakes for the data obtained using Intake24 (collected from October 2019 onwards) were added to the plots but the regression line was not extended and no statistical testing was performed between the 2 data collection periods. This was because at the time, survey design information (such as weightings and stratification) was not available for the more recent data. While it was not possible to judge any shift in the centre of the distribution from these plots, they showed the range of intakes so that any visually apparent changes between the years using Intake24 and previous years using the paper diary could be identified.

In addition, for selected foods, percentage of consumers per quarter of a year (unweighted) was plotted alongside population intakes for years 1 to 11 using the paper diary for comparison with data from Intake24 to identify any changes. Due to the small number of participants in some quarters of a year between October 2020 and May 2022, data for this period was combined and plotted for half of a year instead of quarters. For foods where there was a large proportion of non-consumers (more than 20%), percentage of consumers and intakes for consumers only were presented instead of population intakes.

The stage 1 evaluation report presented this analysis using data from October 2019 to March 2020 (fieldwork year 12). It showed that generally the spread of energy and nutrient intake data collected using Intake24 appeared to be similar to that for years 1 to 11 which used the paper diary. Step changes were observed for soft drinks, fat spreads, vegetables, sugar confectionery, buns, cakes and pastries and dietary supplements. These were likely to be due to methodological factors rather than other factors. However, numbers were small in some age groups, limiting data interpretation.

After further review, some amendments were made to Intake24 to address issues seen at stage 1. The analysis was repeated for the stage 2 evaluation adding data from October 2020 to May 2022 (fieldwork years 13 and 14) and from the NDNS follow up study carried out between August 2020 and October 2020 (PHE, 2021b) to the plots (see section 1.2). This data collection period included the COVID-19 pandemic and rising inflation in the UK. Some of the discrepancies seen in stage 1 were no longer apparent on visual inspection (soft drinks, fat spreads, dietary supplements) and some had persisted (vegetables, sugar confectionery, buns, cakes and pastries). However, no new step changes in foods or nutrients were observed at stage 2 that were not seen at stage 1. This suggested that the changes in survey methodology are likely to have contributed to the differences seen. However, it could not be ruled out that some of the observed changes were real rather than methodological.

5.2 Change in analytical approach

The change from paper diary to online 24-hour recall was accompanied by a move to collecting dietary data on non-consecutive days rather than consecutive days. This provides the opportunity to calculate ‘usual intakes’, which is the accepted method to estimate population habitual nutrient and food intakes, in place of calculating ‘day average’ intakes (the method used for consecutive days of data collection). However, it is not possible to calculate usual intakes for foods with insufficient numbers of consumers. In these cases, the day average method is used.

When calculating ‘day average’ nutrient and food intakes, the variance of the group’s usual intake is inflated by day-to-day variation in individual intake, resulting in inflated estimates of the prevalence of low or high intakes. With the collection of repeated time-spaced 24-hour recalls, it is possible to eliminate the intra-individual variability of the data and as a result obtain an estimate of the population usual intake distribution (Souverein and others, 2011) (see appendix D). The primary effect of shifting to the calculation of ‘usual intakes’ is to reduce the extremes of the distribution, drawing them closer to the mean, while having minimal impact on estimates of average consumption. This enables a more appropriate estimation of ‘percentiles’ or ‘proportions above or below a threshold’.

With the introduction of Intake24 and the move to calculating ‘usual intakes’, it has been possible to include all participants with at least 1 recall in the dietary data set, even if they did not complete further recalls. This differs from the protocol in NDNS years 1 to 11 where only the participants who completed 3 or 4 diary days were included (a diary with fewer than 3 days was considered incomplete). This is possible because the ‘usual intakes’ calculation removes the day-to-day variation and reduces the influence of extreme values, both of which may result from using fewer days of data collection (see appendix D).  

The extremely low values (zero consumption) for some infrequently consumed foods, for example oily fish, would have an impact on the percentage of consumers. This is because a participant with 1 or 2 recalls is less likely to be a consumer than someone with 3 or 4 recalls.

In the years 12 to 15 data set, 20% of participants completed only 1 or 2 recalls. However, the plots in appendix D of the stage 2 evaluation report (OHID, 2023) do not show a consistent decrease in the proportion of consumers of infrequently consumed foods when including those with only 1 or 2 recalls.

5.3 Change in participant selection within a household

Chapter 3 presents an analysis to investigate the degree of clustering of dietary data within a household following the change in participant selection protocol for years 12 to 15 (2019 to 2023) and consideration of any implications for reporting of dietary data. This concludes that the new participant selection model does result in some dietary clustering, but it is not so great as to reduce confidence in the estimates.  

5.4 Updates to the nutrient databank

The NDB is a bespoke database of nutrient composition information maintained for NDNS by the survey consortium.

For Intake24, the food codes and associated nutrient composition data are imported from the NDB into databases that sit in the tool. The development of Intake24 for the NDNS required a major review and rationalisation of the linked NDB (PHE, 2021a). To measure the impact of this on monitoring trends over time, dietary data from NDNS year 10 (2017 to 2018) was recalculated after matching paper diary entries to the foods available in the year 12 rationalised NDB. This exercise, reported in the first stage of the evaluation (PHE, 2021a), aimed to test if any changes observed in the NDNS data collected before and after the move to the new dietary assessment method could be the result of changes in the food codes used. Mean daily intake of selected foods and nutrients using the original year 10 data was compared with the daily intake based on recoded year 10 data, calculated with the rationalised NDB, for all ages combined.

While overall no major differences were observed following the year 10 code replacement, a few differences were observed as a result of the change from individual recipe coding in the paper diary to using more generic recipe codes in Intake24. There was:

• an increase in mean total energy intake (47kcal per day)
• an increase in mean total fat intake (2.8g per day or 0.6% of total energy)
• a decrease in mean fruit and vegetable intake (13g per day or 0.2 portions per day)

These comparisons were not statistically tested for the stage 1 report, but subsequent analysis showed no statistical differences (Amoutzopoulos and others, 2022).

Following the stage 1 evaluation report, a further review was carried out on the recoded year 10 data to look at individual diary days where the largest differences were seen in key foods and nutrients. The findings suggested some improvements were needed to the rationalised NDB and identified that better recipe matches could have been made for some foods in the recoding of the year 10 diary data for the comparison. Differences between DINO and Intake24 in the disaggregation values for smoothies used in calculating total fruit and vegetables were also identified. This meant that smoothies contributed more to fruit and vegetable intake with the diary method than with Intake24. After further updates and re-matching, recalculation of nutrients showed a slight narrowing of the differences seen between the original year 10 data and the updated recoding of the year 10 data. However, it cannot be ruled out that the move to generic codes may be a factor in the differences seen between years 1 to 11 using the diary and years 12 to 15 using Intake24.

Further changes following this review have included improving the detail of recipes and alignment with standard recipes (FSA, 2017) and reinstating some foods that were removed in the original rationalisation.

For NDNS, the NDB is regularly reviewed so that it is up-to-date and, as far as possible, reflects the nutrient composition of the food supply for each survey year reported. In years 1 to 11 the NDB was updated and applied to the dietary data annually. With the move to Intake24, the programme of updates has moved to every 2 years. However, due to the work to update the NDB for the transition to Intake24, and the impact of COVID-19 on survey fieldwork, there were 2 NDB updates during the period of this report, initially for year 12 and then for years 13 to 15. This included updates to sugar values and micronutrient values for milk and non-wholemeal wheat flour.

5.5 Time trend analysis and quantifying the step change between years 1 to 11 (2008 to 2019) and years 12 to 15 (2019 to 2023)

One of the main objectives of the evaluation was to investigate the possibility of continuing the data set for monitoring trends over time. A trend analysis was carried out on selected foods and nutrients, using NDNS data for years 12 to 15 alongside data for NDNS years 1 to 11.

Survey weights were applied to the combined years 12 to 15 data set (2019 to 2023). This is to remove any bias in the observed results which may be due to differences in the probability of households and individuals being selected to take part; and to attempt to reduce non-response bias.

A weighted regression modelling approach was used for the trend analysis. This included a break for the diet method change in year 12 such that regression lines were fitted separately for years 1 to 11 and years 12 to 15 based on having common slopes but different intercepts. This assumption of common slopes between years 1 to 11 and years 12 to 15 seemed reasonable from a looking at the dietary data and without any further dietary data after year 15 to disprove it. It was not possible to plot 2 independent regression lines due to insufficient time-points for years 12 to 15 and because over 50% of the data for the year 12 to 15 period was collected in 2022 and 2023 (after COVID-19 restrictions had been lifted and in-home interviewing resumed). This approach enabled the quantification of any step changes (the difference between the intercepts) between years 1 to 11 and years 12 to 15. Appendix D provides a full explanation of the analytical approach.

The analysis identified the presence of step changes in key foods and nutrients. This may reflect differences due to the move to Intake24 and associated methodological changes such as those described above, but also other factors. Both the COVID-19 pandemic and rising inflation have coincided with much of the data collection period for years 12 to 15. These events have resulted in changes to the availability and price of food with likely consequences for lifestyle and eating behaviours for many people (Hoenink and others, 2024; The Food Foundation, 2023; PHE, 2021b).

As described above, there have been some changes which are likely to be attributable to the diet method changes because they were observed before the pandemic and before the cost-of-living pressures. However, it cannot be ruled out that some of the step changes observed in the evaluation are real rather than methodological. More years of data are needed, and it is too early to say whether or not the step changes seen will come to represent a sustained change in long-term trends.

The time trend analysis has been carried out for all NDNS age groups and presented as plots in appendix C. The commentary focuses on a selection of those foods and nutrients analysed and describes step changes seen from year 12 that are considered nutritionally meaningful.

 5.6 Foods

The foods discussed in this section have been selected based on public health relevance. In this section, reference is made to other non-NDNS data sources where available or relevant.

Fruit and vegetable 5 A Day

Appendix C tables 1.1 and 1.2 show that for adults aged 19 to 64 years, there was evidence of a downward step change between years 1 to 11 (2008 to 2019) and years 12 to 15 (2019 to 2023) for both the number of 5 A Day portions (1.1 portions per day: confidence interval (CI) 0.8, 1.3) and the percentage achieving the 5 A Day recommendation (17 percentage points: CI 12, 22). Step changes were not seen in children aged 11 to 18 years or older adults ^{[footnote 2]}. The step changes seen are largely attributable to a downward step change in reported vegetable consumption (70g per day for men and 55g per day for women) (appendix C table 1.5).

The last published NDNS report using the diary method showed that in years 9 to 11 combined (2016 to 2019), 33% of adults aged 19 to 64 years and 12% of children aged 11 to 18 years met the 5 A Day recommendation (PHE, 2020). As the 5 A Day recommendation is in the middle of the intake distribution, many participants have intakes just above the recommendation. This means that even small decreases in the amount of fruit and vegetables reported results in a shift in the number of participants no longer meeting the recommendation.

Methodological factors could be contributing to this age-differential drop in reported vegetable consumption seen from year 12. Following the stage 2 evaluation of updates to the NDB (section 5.4), more vegetable-based recipe food codes and more associated prompts for fruit and vegetables commonly eaten with other foods were added to Intake24 to improve data capture for vegetables consumed in recipe dishes or as accompaniments. These updates were implemented in the last 6 months of data collection (November 2022 to May 2023), but the changes were small and not expected to bring about any major uplift in reported intake. It is also possible that the large amounts of vegetables and vegetable-based dishes reported in some paper diaries were overestimated and that Intake24 provides more accurate estimates.

While it is likely that there are method differences, the prolongation of the downward step change may also reflect a real reduction in fruit and vegetable consumption over this period due to a combination of lack of availability during the COVID-19 pandemic and cost of living pressures during this data collection period. Other data sources suggest that there has been a reduction in fruit and vegetable purchases during this period (Defra, 2023).

Red and processed meat

Appendix C table 1.7a indicates a downward step change between years 1 to 11 (2008 to 2019) and years 12 to 15 (2019 to 2023) in the percentage of consumers of red and processed meat for children aged 4 to 10 years (9 percentage points: CI 4, 13) and women aged 75 years and over (8 percentage points: CI -2, 17).

Appendix C table 1.7c indicates downward step changes between years 1 to 11 and years 12 to 15 in the percentage consuming more than 90g of red and processed meat per day for girls aged 11 to 18 years (7 percentage points: CI 1, 13) and men aged 65 to 74 years (8 percentage points: CI -7, 22).

Household purchase data for all meat shows a reduction in purchases since 2021 (Defra, 2023).

Oily fish

There was a downward step change between years 1 to 11 (2008 to 2019) and years 12 to 15 (2019 to 2023) in the percentage of consumers of oily fish in all age and sex groups (appendix C table 1.10a). This may be because the Intake24 data included participants with 1 or 2 recording days, while the diary data included only participants with 3 or 4 days (see section 5.2). As this type of food is not frequently consumed, a participant with 1 or 2 recalls is less likely to be a consumer than someone with 3 or 4 recalls. Step changes were generally larger in adults than children with the largest in men aged 75 years and over (21 percentage points: CI 2, 40). There was no evidence from purchasing data to corroborate a fall in oily fish consumption.

For children aged 11 to 18, there was an upward step change between years 1 to 11 and years 12 to 15 in the amounts of oily fish consumed by consumers (appendix C table 1.10b).

Soft drinks with added sugar

Appendix C table 1.12a shows that there was an upward step change between years 1 to 11 (2008 to 2019) and years 12 to 15 (2019 to 2023) in the percentage of consumers of sugar-sweetened soft drinks for children aged 18 months to 3 years (10 percentage points: CI 1, 19) and 4 to 10 years (9 percentage points: CI 2, 17) but not for children aged 11 to 18 years. A rise in the percentage of consumers of sugar-sweetened soft drinks (children only) and a drop in the percentage of consumers of low-calorie or no added sugar soft drinks (children and adults) in data reported using Intake24 compared with the diary was seen in the stage 1 evaluation.

Action was then taken to address inconsistencies in the naming of these types of drinks in an effort to be clearer to the participant as to the drink type, particularly for concentrated and ‘ready to drink’ soft drinks. Errors identified in some food codes linked to soft drinks were also corrected. Work is ongoing to simplify the food lists to improve confidence that participants are making the correct selection.

There were also upward step changes between years 1 to 11 and years 12 to 15 in the amount of sugar-sweetened soft drinks consumed by consumers for all 3 child age groups (appendix C table 1.12b). It is possible that the apparent upward step changes are indicating a plateau from years 9 to 11 to years 12 to 15, rather than a continuation of the downward trend and that, in more recent years, there has been a levelling off for both the percentage of consumers of sugar-sweetened soft drinks and the amount consumed in the population.

Confectionery

Appendix C table 1.13a shows there was a downward step change between years 1 to 11 (2008 to 2019) and years 12 to 15 (2019 to 2023) in the percentage of consumers of sugar confectionery for children aged 18 months to 3 years (24 percentage points: CI 15, 34) and 4 to 10 years (21 percentage points: CI 14, 29) and adults aged 65 to 74 years (16 percentage points: CI 9, 22).

A similar pattern was seen with chocolate confectionery (appendix C table 1.14a).

This is likely to be in some part due to the diet method change. There is no evidence from purchasing data of a drop in these types of food (OHID, 2022). As the step changes were seen in young children it could be that parents or other proxies are having more difficulty in recalling confectionery for children compared to the diary method.

In year 15, participants were asked in the interview how many days in the last month they had consumed sugar and chocolate confectionery. There was a weak correlation between these food frequency questions (FFQ) and the number of days a participant reported consuming these foods in Intake24. This was mainly due to participants saying they consumed confectionery in the FFQ at rates that would suggest it would be likely to be consumed during the recall days but then not reporting it in Intake24, although there were also participants who misclassified in the opposite direction. Sugar confectionery had a lower FFQ versus Intake24 correlation than chocolate confectionery.  

Biscuits

Appendix C table 1.15a shows there was a downward step change between years 1 to 11 (2008 to 2019) and years 12 to 15 (2019 to 2023) in the percentage of consumers of biscuits in all 3 child age groups: 15 (CI 4, 25), 11 (CI 5, 17) and 12 (CI 5, 19) percentage points for those aged 18 months to 3 years, 4 to 10 years and 11 to 18 years respectively. There was also a downward step change for women aged 65 to 74 years (15 percentage points: CI 2, 29). For younger children it may be that parents or other proxies are having more difficulty in recalling biscuits using Intake24 compared to the diary method.

As with confectionery, there was a weak correlation between the FFQ and the number of days a participant reported consuming sweet biscuits in Intake24. This was mainly due to participants saying they consumed sweet biscuits in the FFQ at rates that would suggest they would be likely to be consumed during the recall days but then not reporting it in Intake24, although there were participants who misclassified in the opposite direction. There is no evidence from purchasing data of a drop in these types of food (OHID, 2022).

Buns, cakes and pastries

Appendix C table 1.16a shows there was a downward step change between years 1 to 11 (2008 to 2019) and years 12 to 15 (2019 to 2023) in the percentage of consumers of buns, cakes and pastries in children aged 4 to 10 years (22 percentage points: CI 15, 28) and boys aged 11 to 18 years (14 percentage points: CI 3, 24).

Crisps and savoury snacks

There was a downward step change between years 1 to 11 (2008 to 2019) and years 12 to 15 (2019 to 2023) in the percentage of consumers of crisps and savoury snacks for children aged 18 months to 3 years (15 percentage points: CI 5, 25), girls aged 11 to 18 years (20 percentage points: CI 10, 30) and men aged 75 years and over (18 percentage points: CI 1, 36) (appendix C table 1.17a). For younger children, as with other foods, it may be that parents or other proxies are having more difficulty in recalling crisps and savoury snacks using Intake24 compared to the diary method.

Step changes between years 1 to 11 and years 12 to 15 were seen in the opposite direction in the amounts of crisps and savoury snacks consumed by consumers, particularly in men aged 75 years and over (appendix C table 1.17b).

5.7 Energy and nutrients

In this section, intakes of saturated fat and free sugars are expressed as a percentage of energy intake excluding ethanol (alcohol). This has been shortened to percentage of energy intake for simplicity.

Energy

Upward step changes between years 1 to 11 (2008 to 2019) and years 12 to 15 (2019 to 2023) were seen in children aged 18 months to 3 years (172kcal per day: CI 112, 231), girls aged 4 to 10 years (140kcal per day: CI 69, 212) and boys aged 11 to 18 years (143kcal per day: CI 28, 258) (appendix C table 1.18). However, these upward step changes are calculated from a model which assumes that the downward year-on-year trend in energy intake for years 1 to 11 continues after year 12. This would lead to an over-estimation of the upward step change if in fact there was no year-on-year trend.

There was a downward step change between years 1 to 11 (2008 to 2019) and years 12 to 15 (2019 to 1023) in adults aged 19 to 64 years (121kcal per day: CI 55, 187) and men aged 75 years and over (152kcal per day: CI -43, 347). There was no step change in the other age groups.

As noted in section 1.5, the results of the DLW sub-study carried out across years 12 to 14 showed that use of Intake24 is associated with underreporting of energy intake (30% overall) and to a similar degree to that found by previous DLW studies for the food diary method carried out in years 1 and 3 and years 6 and 7. However, it should not be assumed that underreporting with Intake24 has the same characteristics as underreporting with the food diary.

Saturated fatty acids

Appendix C table 1.21 shows a downward step change between years 1 to 11 (2008 to 2019) and years 12 to 15 (2019 to 2023) in the percentage of energy from saturated fatty acids in children aged 18 months to 3 years (1.1 percentage points: CI 0.3, 1.9). No step changes were seen in any of the other age groups.

Free sugars

Appendix C table 1.23 shows that there was a downward step change between years 1 to 11 (2008 to 2019) and years 12 to 15 (2019 to 2023) in the percentage of energy from free sugars for boys aged 4 to 10 years (1.2 percentage points: CI 0.3, 2.2). There were no downward step changes in free sugars in other age groups and for men aged 19 to 64 years there was an upward step change (1.0 percentage points: CI 0.0, 1.9).

Fibre

Appendix C table 1.24 shows that there was an upward step change between years 1 to 11 (2008 to 2019) and years 12 to 15 (2019 to 2023) in fibre intake for children aged 18 months to 3 years (2.4g per day: CI 1.6, 3.2). Downward step changes were seen in adults aged 19 to 64 years (3.6g per day: CI 2.7, 4.5), adults aged 65 to 74 years (2.4g per day: CI 0.8, 4.1) and men aged 75 years and over (2.0g per day: CI -0.6, 4.6) which are likely to be linked to the lower reported consumption of vegetables (a key contributor to fibre intakes).

6. Conclusions and next steps

This third and final stage of the formal evaluation builds on the findings of the stage 1 and 2 reports with respect to the quality measures for Intake24 and the assessment of the impact of the dietary method change on resulting data. It also presents an analysis of the impact of selecting more participants per household on the dietary data (clustering).

In self-reported dietary assessment methods, it is not possible to know the extent to which participants have fully reported their consumption and if not, what types and quantities of items have been missed. Misreporting is a feature of all dietary assessment methods, and results from the DLW study show that the degree of underreporting with Intake24 is substantial in adults and older children (OHID, 2023). There are no measures that can be applied to recalls to definitively and consistently establish to what extent any individual recall is an accurate and complete record of what was consumed.

A range of indicative thresholds based on completion times, number of items, eating occasions recorded and reported calorie intakes can give an indication of potential issues with the performance of the assessment tool. Evaluation data suggests that completion times reduce with each successive recall in most age groups. Other quality measures indicate that successive recalls are associated with fewer foods recorded, and fewer missing foods. This is most apparent among children aged 11 to 18 years and may be due to a learning effect and/or changes in engagement. 

There has been a small but persistent shortfall in the proportion of recalls completed for a Saturday which may be due to participant reluctance to complete recalls on a Sunday. This will continue to be monitored.

Increasing the number of participants per household to achieve greater cost efficiency resulted (as would be expected) in some clustering of dietary variables, although not to the degree as to reduce confidence in estimates of intake. The impact may be partly mitigated because recall days were assigned randomly to individual participants, so participants in the same household were not necessarily completing recalls on the same days.

A key objective of the evaluation has been to try to identify and understand any differences observed between the dietary data collected before and after the method change and implications for continuing the NDNS time trend data series. The evaluation period coincided with the COVID-19 pandemic and then the period of cost-of-living pressures in the UK, both of which impacted diets due to changes in lifestyle, food availability and affordability. So, it has not been possible to definitively distinguish between differences in the data due to the method change and those that reflect real changes in diets. 

Some step changes seen in the data were consistent with other data sources or reports, for example the downwards step change for intakes of vegetables in adults is consistent with data on purchases of vegetables. Others, including the reduction in the percentage of consumers of oily fish, were not corroborated by purchasing data and may be explained by the method changes, in particular the move to include participants with one or 2 recalls. In other cases, step changes were seen that could not be easily explained by household purchase data or the method change, for example the drop in the percentage of consumers of confectionery and biscuits for young children.

Some differential changes were seen between age groups. For the youngest children (aged 18 months to 3 years), a step up in energy intake was observed alongside a step down in consumption of many high in fat, salt and sugar (HFSS) food groups. It may be that method differences, for example changes in the way that portion sizes are estimated by the 2 methods, has contributed to the higher recorded energy intake in young children.

Overall, the new Intake24 fieldwork model is effective in capturing dietary intake, as assessed by the quality measures. There is some evidence of a decline in data quality over successive recalls in the 11 to 18 years age group. Misreporting is a feature of recalls and food diaries and does not appear to be method specific.

Regression analysis shows step changes in amounts consumed or percentage consumers for many of the foods we considered. Taken together, the number and size of the step changes seen in this evaluation have led to the conclusion that it is not appropriate to continue the time trend series across the method change from paper diary to Intake24. The time trend plots are presented in this report to aid the evaluation and have informed this conclusion. Because there is insufficient data to assess the trend in years 12 to 15, and we cannot be confident that the trend lines shown on the plots reflect the true situation, the plots will not be presented as results in the year 12 to 15 report. More data is needed to be confident of the trend from year 12, and time trend data will be included in future reports when sufficient data is available.

Although this report marks the end of the formal evaluation of the method change from diary to Intake24, work to keep Intake24 up-to-date and to take advantage of opportunities for continual improvement is ongoing. The tool’s performance to capture diet will continue to be monitored alongside wider survey performance throughout the next phase of fieldwork from 2024 to 2029.

Acknowledgements

The NDNS is funded by DHSC and FSA and is carried out by a consortium comprising NatCen and the MRC Epidemiology Unit, University of Cambridge.

This report was prepared by Caireen Roberts, David Collins and Polly Page (MRC Epidemiology Unit), Beverley Bates (NatCen) and Gillian Swan and Jo Nicholas (Office for Health Improvement and Disparities (OHID)).

The authors would like to thank all of those who gave up their time to be interviewed as part of the NDNS. The authors would also like to acknowledge the professionalism and commitment of the fieldworkers who worked on the survey and who are so important to the survey’s success.

The authors would like to thank everyone who contributed to the work behind and production of this report, in particular:

• colleagues at the MRC Epidemiology Unit: Suzanna Abraham, Birdem Amoutzopoulos, Anila Farooq, Jackie Foreman, Dan Griffiths, Albert Koulman, Steph Moore, Angela Mulligan, Elise Orford, Toni Steer, Kirsty Trigg, Michelle Venables
• colleagues at NatCen: Suzanne Hill, Jess Melling and Dhru Shah
• members of the Project Advisory Group: Jean Adams and Nita Forouhi (MRC Epidemiology Unit), Robin Clifford (FSA), Adrienne Cullum, Natasha Powell and Celia Sabry-Grant and Rachael Wall (OHID),
• members of the NDNS Project Board: Susan Fairweather-Tait, Mairead Kiely, Julie Lovegrove and Sian Robinson (Scientific Advisory Committee on Nutrition), Jayne Woodside (independent), Paul Niblett (OHID), Joseph Shavila (FSA)

This research was also supported by the National Institute for Health and Care Research (NIHR) Cambridge Biomedical Research Centre (NIHR203312).

References

Albar SA, Carter MC, Alwan NA, Evans CEL and Cade JE on behalf of the myfood24 consortium group. Formative evaluation of the usability and acceptability of myfood24 among adolescents: a UK online dietary assessments tool. BMC Nutrition. BMC Nutrition 2015: volume 1, article number 29.

Amoutzopoulos B, Steer T, Roberts C, Collins D, Trigg K, Barratt R and others. Rationalisation of the UK Nutrient Databank for Incorporation in a Web-Based Dietary Recall for Implementation in the UK National Diet and Nutrition Survey Rolling Programme. Nutrients 2022: volume 14, issue 21, page 4551.

Binder DA. ‘On the Variances of Asymptotically Normal Estimators from Complex Surveys’. International Statistical Review 1983: volume 51, pages 279-292.

Bradley J, Simpson E, Poliakov I, Matthews JNS, Olivier P, Adamson AJ and Foster E. Comparison of INTAKE24 (an Online 24-h Dietary Recall Tool) with Interviewer-Led 24-h Recall in 11–24 Year-Old. Nutrients 2016: volume 8, issue 6, page 358.

Department for Environment, Food and Rural Affairs (Defra). Family Food FYE 2023. (viewed on 29 October 2024).

Fitt E, Cole D, Ziauddeen N, Pell D, Stickley E, Harvey A and Stephen AM. DINO (Diet In Nutrients Out) - an integrated dietary assessment system. Public Health Nutrition 2014: volume 18, issue 2, pages 234-241.

The Food Foundation, 2023. The Broken Plate 2023 (viewed 28 August 2024).

Foster E, Lee C, Imamura F, Hollidge SE, Westgate KL, Venables MC and others. Validity and reliability of an online self-report 24-h dietary recall method (Intake24): a doubly labelled water study and repeated-measures analysis. Journal of Nutrition Science 2019: volume 8, eLocator 29.

Hoenink JC, Garrott K, Jones NRV, Conklin AI, Monsivais P and Adams J. Changes in UK price disparities between healthy and less healthy foods over 10 years: An updated analysis with insights in the context of inflationary increases in the cost-of-living from 2021. Appetite 2024: volume 197, article 107290.

Holt D, Smith TMF and Winter PD. Regression analysis of data from complex surveys. Journal of the Royal Statistical Society 1980: volume 143, issue 4, pages 474-487.

MRC Human Nutrition Research. Food Standards Agency Standard Recipes Database, 1992-2012. UK Data Service, 2017 (viewed on 29 October 2024).

National Cancer Institute. ASA24 Respondent Website Features (viewed on 11 July 2023).

OHID. Sugar reduction programme: industry progress 2015 to 2020. 2022 (viewed on 18 December 2024).

OHID. Evaluation of change in dietary methodology in NDNS rolling programme: stage 2. 2023. (viewed on 13 June 2024).

Public Health England (PHE). National Diet and Nutrition Survey Rolling programme Years 9 to 11 (2016/2017 to 2018/2019). 2020. (viewed June 2024)

PHE. Evaluation of change in dietary methodology in NDNS rolling programme: stage 1. 2021a (viewed on 11 July 2023).

PHE. NDNS: Diet and physical activity – a follow-up study during COVID-19. 2021b (viewed on 11 July 2023).

Rowland MK, Adamson AJ, Poliakov I, Bradley J, Simpson E, Olivier P and Foster E. Field Testing of the Use of Intake24—An Online 24-Hour Dietary Recall System. Nutrients 2018: volume 10, issue 11, page 1690.

Souverein O, Dekkers A, Geelen A and others. Comparing four methods to estimate usual intake distributions. European Journal of Clinical Nutrition 2011: Volume 65, Supplement 1), pages S92–S101.

Subar AF, Potischman N, Dodd KW, Thompson FE, Baer DJ, Schoeller DA and others. Performance and Feasibility of Recalls Completed Using the Automated Self-Administered 24-Hour Dietary Assessment Tool in Relation to Other Self-Report Tools and Biomarkers in the Interactive Diet and Activity Tracking in AARP (IDATA) Study. Journal of the Academy of Nutrition and Dietetics 2020: volume 120, issue 11, pages 1,805-1,820.

Appendix A: background, fieldwork and data processing

A1: fieldwork changes due to the COVID-19 pandemic impacting the evaluation period

For the stage 3 evaluation report, NDNS data has been included from October 2019 to May 2023, spanning fieldwork years 12 to 15. In year 12 of NDNS (October 2019 to March 2020), interviewers conducted a face-to-face interview with all selected participants within a household, during the same visit if possible. This visit included a computer-assisted personal interview (CAPI), dietary recall, physical activity questionnaire, height and weight measurements, spot urine sample, and seeking agreement for a biomedical fieldworker (BMF) to visit for collection of a blood sample and other physical measurements.

In response to the COVID-19 pandemic, there was a full suspension of fieldwork activity for around 6 months and it was decided not to resume year 12. This meant that just over half of the planned year 12 interviewer fieldwork, and a quarter of biomedical fieldwork, was completed.

Subsequently, changes to the fieldwork model were introduced to enable data collection to resume during periods of government restrictions. When year 13 started in October 2020, restrictions meant that in-home interviewing was not possible and interviewing was conducted remotely over the telephone, after participant selection at the household’s doorstep. Some adaptations had to be made for remote data collection, for example spot urine sample collection was moved to the BMF visit (for participants who agreed), and self-reported rather than interviewer-measured height and weight were collected. Changes to the dietary recall process are discussed in section A3 below.

Year 14 fieldwork started in April 2021 initially under the same remote interviewing approach as year 13. Face-to-face interviewing was re-introduced in September 2021 for households who were content with an in-home visit; the telephone option remaining available for those who were not. From the start of year 15 fieldwork in April 2022, face-to-face interviewing was the preferred mode, with interviewers offering this option first.

In summary, COVID-19 impacted fieldwork across all the survey years included in the evaluation:

• year 12 fieldwork was truncated (resulting in a lower number of issued addresses)
• year 13 interviewer fieldwork was conducted via telephone only in Great Britain, with some face-to-face visits in Northern Ireland which, for practical reasons, were conducted at a later date
• years 14 and 15 fieldwork was conducted using a mixture of telephone and face-to-face modes (the proportion of households interviewed in person varied by fieldwork year, from 100% in year 12, 15% in year 13, 67% in year 14 and 90% in year 15)

During August, September and October 2020, fieldwork was carried out for the NDNS follow-up study during COVID-19 study (PHE, 2021b). The study aimed to describe and assess the impact of the COVID-19 pandemic on the diet and physical activity of people in the UK by following up participants who had previously taken part in NDNS. Participants in the study (363 children, 567 adults) completed up to 4 recalls using Intake24.

A2: Intake24 support

During the initial interview (face-to-face or telephone), participants were asked to complete their first Intake24 recall independently. In year 12, interviewers reviewed the first recall before submission if possible. From year 13 this review moved to a check that participants did not have any issues completing and submitting their recall (for face-to-face visits this was a verbal check; for telephone interviews a follow-up telephone call was scheduled). In the majority of cases, participants would then go on to complete subsequent recalls independently with no further involvement from interviewers.

For participants who were unable to complete their first recall independently, for example due to internet access issues or lack of confidence with technology, face-to-face recall assistance was offered by interviewers where in-home interviews were carried out. If a participant indicated that they were unable to complete the second recall independently, a follow-up visit would be arranged where the interviewer would schedule a date and time to return to the household and complete the recall with the participant. At the end of each visit, the participant would indicate if they could complete the following recall independently (so some participants may have had assistance with the second recall but not the third and fourth).

For remote interviews, or where internet access was poor in the area, telephone recall assistance was scheduled with research staff at the MRC Epidemiology Unit. Participants were sent a hard copy food photograph atlas prior to the scheduled phone call to help them estimate portion sizes during completion of the recall. The researcher read out the instructions and prompts in Intake24 to facilitate the participant to provide the information on their food and drink consumption which the researcher then entered into Intake24. At the end of the recall, the researcher arranged another telephone appointment for the next dietary recall if assistance was still required.

A3: number of completed recalls

Table A1 shows the number of completed recalls for NDNS fieldwork years 12 to 15 (2019 to 2023).

Table A1: number of completed recalls for NDNS years 12 to 15 (2019 to 2023)

Number of completed recalls	Adults (n)	Adults (%)	Children (n)	Children (%)	Total (n)	Total (%)
1 recall only	250	12%	288	15%	538	13%
2 recalls only	123	6%	151	8%	274	7%
3 recalls only	73	3%	97	5%	170	4%
4 recalls	1,700	79%	1407	72%	3,107	76%
Productive participants	2,146	Not applicable	1,943	Not applicable	4,089	Not applicable

Note: Productive participants are those who completed at least 1 recall.

A4: quality checks

As explained in section 4.2 in the main report, recall data were examined according to a series of quality checks informed by the research team’s experience with processing and checking dietary data, and published studies using Intake24 and other similar dietary assessment methodology. This included monitoring the number of recalls with:

• 9 or fewer food or drink items
• 3 or fewer eating or drinking occasions
• completion time of under 3 minutes
• very low calorie intake (less than 400kcal) or very high (more than 4,000kcal)

In addition to the counts, more detailed checks were undertaken on a subset of 10% of recalls to understand performance of the tool. Such checks are useful at the start of a study, where the tool is used in a new population or setting, and at times of methodological change. In NDNS, they were undertaken to monitor performance of Intake24 on introduction of the new method to the survey and where there was a change of fieldwork protocol due to COVID-19. Checks included:

• multiple food items in the participant’s search term (for example, ‘toast cereal yoghurt’) and only one food item was coded
• inconsistencies between the search term and the food code selected, for example searched for chicken stir-fry but selected prawn stir-fry
• ‘orphan’ foods (a reported food that appeared to have been eaten on its own, for example beef steak when it would commonly be eaten with other foods such as chips or potatoes, or salad)

Specifically, these checks were carried out on the year 12 data (October 2019 to March 2020) and on the first 4 months of data collected in year 13 (October 2020 to February 2021) following the move to remote interviews (see section A1).

Undertaking and reporting on the above checks was part of the monitoring of Intake24, to identify potential issues and improvements for tool functionality and usability. The objective was to identify the frequency of known issues in the NDNS data set and to consider improvements to Intake24. For example, additional food prompt questions, improved portion estimation, clarification around the naming of foods. Such changes were implemented at intervals during NDNS data collection. No adjustments were made to the dietary data itself as a result of the above checks as it was not possible to apply the checks and adjustments systematically to the overall years 12 to 15 data set. Bias may have been introduced if selected adjustments had been made.

When all reported missing foods had been coded (see section 4.1 in the main report), individual recalls were reviewed where the total energy intake was less than 400kcal and where the participant had not stated that they consumed ‘less than usual’, or that they were on a weight loss diet. Seven recalls were considered ‘incomplete’ and excluded from the evaluation and also from the years 12 to 15 data set.

Portion size boxplots were generated by age and sex groups (18 months to 3 years, 4 to 10 years, 11 to 18 years and 19 years and over) to identify any extreme outliers within each food group. Extreme outliers were identified from the boxplots as individual data points separate from the box and whiskers where they were more than 3 times the inter-quartile range (IQR) (75th percentile to 25th percentile) from the nearest quartile for that intake (either the 25th or 75th percentile). These were examined on a case-by-case basis and reviewed in the context of the participant’s overall reported consumption. Portion sizes that were considered to be implausible, and likely to be the result of errors in portion size selection, were adjusted. Adjustments were carried out in the bespoke dietary database by changing the portion code at the individual level.

Finally, boxplots were generated by age group to identify any infeasible or extreme energy and nutrient values. Extreme outliers were identified as described above for portion sizes and looked at on a case-by-case basis. Extreme intakes that were considered to be the result of errors in portion size estimation or food composition in the NDB were adjusted, otherwise values were left in the data set as they were assumed to reflect consumption by participants.

As a result of the extreme outlier checks for portion size, energy and nutrients, 0.2% of all food and drink entries in the final years 12 to 15 data set were adjusted.

Appendix B: Intake24 quality measures

See publication page Evaluation of changes in dietary methodology in NDNS: stage 3 for the spreadsheet with the Appendix B tables.

Appendix C: NDNS year 1 to 15 time trend analysis UK

See publication page Evaluation of changes in dietary methodology in NDNS: stage 3 for the spreadsheet with the Appendix C tables.

Appendix D: statistical analyses

The time trend analyses have been carried out on key foods and nutrients and are presented as plots in Appendix C. This section outlines the statistical methods used to estimate the ‘average change per year’ for years 1 to 15 (2008 to 2023) and the ‘diet method change’ in each outcome. The same weights and design variables were applied in these analyses as those used in the years:

• 1 to 4 (combined)
• 5 and 6 (combined)
• 7 and 8 (combined)
• 9, 10 and 11 (combined) reports

Additional weights and design variables were applied for years 12, 13, 14 and 15 (combined).

The weights for each data set were re-scaled based on sample size, such that each set of data is in the correct proportion (4:2:2:3:4) to give a standardised sample size per survey year ^{[footnote 3]}.

The ‘average change per year’ and ‘diet method change’ for the continuous variables were estimated through linear regression models. Proportions (such as the percentage of the sample meeting the 5 A Day guideline for fruit and vegetable intake) were estimated through logistic regression models. These models included a common slope across years 1 to 15, but a different intercept for years 1 to 11 and years 12 to 15 across 6 age groups, overall and by sex (for all but the youngest age group).

The age groups were 18 months to 3 years (sex-combined only), 4 to 10 years, 11 to 18 years, 19 to 64 years, 65 to 74 years and 75 years and over. Participants were grouped into quarters of a calendar year according to when their dietary data was collected, and this time- variable was used as the explanatory variable in the regression models.

The statistical analyses were undertaken using the following 3 stages:

1. Exploratory analyses.
2. Estimation of ‘changes per year’.
3. Diagnostic procedures (assessment of model assumptions and goodness of fit).

All the analyses, including the graphical tools and diagnostic procedures, took into account the complex survey design.

Exploratory analyses

The observed distributions of the continuous variables were screened through histograms, Q-Q plots and boxplots. These graphical tools showed the shape of the distribution and highlighted the presence of outliers. These were investigated as well as their impact on the regression analyses.

Estimation of the ‘average change per year’ and the ‘diet method change’ for continuous variables

Linear regression models were used for continuous measurements of foods, nutrients and blood and urine analytes. The regression coefficients (which estimate the intercept and slope parameters for each age and sex group) use probability weighted least squares (Holt and others, 1980) and their covariance matrix was estimated using a Taylor linearization method (Binder, 1983). The slope parameter (along with the associated 95% confidence interval) estimates the ‘average change per year’ for each variable. The difference between the intercept terms for years 1 to 11 (2008 to 2019) and years 12 to 15 (2019 to 2023) estimates the ‘diet method change’.

Estimation of the ‘average change per year’ and the ‘diet method change’ for proportions

Logistic regression models (with an identity link function) were used for binary variables. The regression coefficients (which estimate the intercept and slope parameters for each age and sex group) use a pseudo-likelihood approach (Holt and others, 1980) and their covariance matrix was estimated using a Taylor linearisation method (Binder, 1983). The slope parameter (along with the associated 95% confidence interval) estimates the ‘average change per year’ for each variable. The difference between the intercept terms for years 1 to 11 (2008 and 2019) and years 12 to 15 (2019 to 2013) estimates the ‘diet method change’.

Diagnostic procedures

The ‘goodness of fit’ of the linear models was examined using the concept of explained variation (R-squared).

Usual intakes estimation of nutrient and food intakes

Dietary data for years 1 to 11 (2008 to 2019) was collected over 4 consecutive diary days, and so nutrient and food intakes were estimated by calculating the ‘day average’. The move to non-consecutive day dietary recalls for years 12 to 15 (2019 to 2023) provided the opportunity to estimate nutrient and food intakes by calculating ‘usual intakes’ which is the accepted method to estimate population habitual nutrient and food intakes.

When calculating ‘day average’ nutrient and food intakes, the variance of the usual group intake is inflated by day-to-day variation in individual intake, resulting in misleading estimates of the prevalence of low or high intakes. With the collection of repeated 24-hour recalls in years 12 to 15, it is possible to eliminate the intra-individual variability of the data and thereby to obtain an estimate of the population usual intake distribution (Souverein and others, 2011). Several statistical procedures for estimating the usual intake distribution from repeated 24-hour recalls are available to enable the estimation of ‘habitual’ intakes. This enables more appropriate estimation of ‘percentiles’ or ‘proportions above/below a threshold’ compared with the ‘day average’ method. For more information see, Usual Dietary Intakes: the National Cancer Institute method at the USA National Cancer Institute website.

Usual intakes of frequently consumed foods such as meat, fruit and vegetables and sources of protein, can be estimated well with short-term measures (2 or more days of a 24-hour recall). Short-term measures for infrequently consumed foods such as fruit juice, fish and sugar-sweetened soft drinks can result in zero intake being reported for many participants. Including an additional long-term measure such as a Food Frequency Questionnaire can help to more accurately capture usual intake for such foods. For example, with a dietary assessment protocol of collecting up to 4 recalls, a participant may record zero intake for a particular food on all recording days but may record in a Food Frequency Questionnaire that they consume the same food on average once a week. Food Frequency Questionnaire information has therefore been collected in CAPI for a limited number of infrequently consumed foods (fish, white meat, fruit juice and sugar-sweetened soft drinks) and will be factored in alongside the short-term measure to improve the estimation of ‘usual intake’.

The Multiple Source Method is a web-based application which uses R code to perform the statistical analysis and was used to estimate ‘usual intakes’ of nutrients and foods for years 12 to 15. All valid recalls from all participants, regardless of the number of recalls obtained, were included in the analysis. The inclusion of participants with only 1 or 2 dietary recalls was made possible by the usual intake method ‘borrowing’ day-to-day variation estimates from other similar-aged participants with 3 or 4 recalls. This provides an advantage over the previous diary method which required at least 3 consumption days for a participant to be included.

For some foods there were insufficient participants with 2 or more consumption days, and so it was not possible to estimate the day-to-day variation required in the calculation of ‘usual intakes’. The threshold set for this was 15% or more participants with 2 or more consumption days. For foods which did not meet this threshold intakes were estimated by calculating the ‘day average’ (for example, oily fish and sugar confectionery).

1. Intake24 was developed by Newcastle University, originally with funding from Food Standards Scotland and is licenced under the Open Government Licence. The tool is maintained by and developed in collaboration between Cambridge University, Monash University (Australia) and Newcastle University (UK). The version of Intake24 used for NDNS is provided and adapted by the University of Cambridge, based on the original, with technical advisory input from Newcastle University. A demo version is available at Intake24.org. The demo version of Intake24 is updated regularly but is not exactly the same as the version used in NDNS years 12 to 15. ↩

2. The number of portions of fruit and vegetables consumed per day has only been calculated for adults and children aged 11 to 18 years from disaggregated data in line with the 5 A Day criteria, including up to one portion each of fruit juice and baked beans or pulses per day. While children under 11 years should also eat at least 5 portions of fruit and vegetables a day, the above portion size estimates are aimed at adults and older children. For younger children portion sizes should be proportionally less and have not been quantified. For more information about 5 A Day, including portion sizes, see 5 A Day: what counts?. ↩

3. Although the weights were not specifically designed for this type of sub-group analysis, it was possible to use the years 1 to 15 weights and design variables for just 2 to 4 years’ data (years 1 and 2, years 3 and 4, years 5 and 6, years 7 and 8, years 9, 10 and 11 or years 12, 13, 14 and 15), as the selection weights correct for any differences in sampling strategy across survey years, and there was no evidence that response behaviour had changed significantly between the 6 survey periods. However, to use subsets of any other combination of years of the data set, the weights and design variables would have to be reviewed to ensure that the subset of data is still representative of the UK population when the years 1 to 15 weights and design variables have been applied. ↩