Guidance

Family Resources Survey: quality assessment report

Updated 26 March 2026

1. Introduction

This report contains information on the Family Resources Survey (FRS) data sources used by the Department for Work and Pensions (DWP), as well as quality assessments on each of them. The assessment considers the journey of the data from its collection, through processing and analysis and ultimately to publication.

The UK Statistics Authority have published a regulatory standard including a Quality Assurance of Administrative Data (QAAD) toolkit for administrative data. A QAAD report is designed to set out how the producer has explored the administrative data source and assured themselves that the data are of sufficient quality to produce statistics.

The Code of Practice for Statistics emphasises that producers of statistics, regardless of whether they are administrative or survey based, must clearly communicate information on the quality of statistics to users.

Quality is about using suitable data and appropriate methods to produce reliable statistics that meet user needs. Statistics should inform rather than mislead, and producers must uphold high standards of transparency and quality assurance. 

Quality is dynamic and evolving, not fixed or absolute. It requires curiosity and a mindset that is open to seeing how statistics could be wrong. Maintaining quality involves continuous monitoring, innovation, collaboration and effective communication with users to ensure statistics are developed, understood and used appropriately.

Statistics produced from administrative data can use the Quality Assurance of Administrative Data (QAAD) toolkit to provide assurance of data quality to users.  This report provides users of the FRS data with a similar framework of assurance for survey data. As such, FRS data has been assessed against the four specific areas for assurance included in the QAAD toolkit. These have been adapted to apply to survey data.

This report also recognises the development of the FRS in making use of administrative data to improve the survey data quality. In summary, the main strengths and limitations of the FRS data are:

Current strengths

• The FRS has a long history of review and development, having been run in Great Britain for over 30 years and in Northern Ireland for over 20 years.
• The FRS remains a large-scale survey; the achieved sample is large enough that standard errors of key estimates are routinely less than one percentage point. This has remained the case irrespective of recent declines in response rates. See FRS futures position on response rates
• A stratified, tailored sample design is used to reduce sampling error.
• Face-to-face interviewing enables the collection of data on a wide range of topics. These include many personal and family characteristics which are either not available from administrative sources or are sensitive in nature and thereby unsuited to other collection methods.
• Effective modes of communication between data collectors, data suppliers, dataset producers and data security teams are well established.
• The suppliers of survey data obtained from FRS interviews have established and agreed data processing procedures. Careful attention is paid to the accurate collection of survey information, followed by meticulous data processing, editing, and quality assurance.
• DWP has access to a range of high-quality administrative datasets covering the main benefits reported in the FRS dataset. These data are used to edit and, where appropriate, replace survey reported benefit receipt and amounts so that they align with administrative records.
• DWP has an ongoing quality assurance dialogue with policy experts and academic researchers, who are users of the FRS data and statistics. This includes independent quality assurance of the datasets by the Institute for Fiscal Studies.

Current limitations

• Response rates have struggled to return to those of pre-pandemic years. The survey’s response rate has declined since the 2015 to 2016 survey year. As the lower the response rate to a survey, the greater the likelihood that those who responded are significantly different to those who did not, we recognise the greater risk of systematic bias in our survey results
• Robust analysis of the data is only applicable at regional level
• The FRS questionnaire is lengthy and demanding and a key concern is, where possible, to reduce (or at least not increase) its length. This is to mitigate the risk of overburdening respondents or interviewers
• With over two thousand variables and their associated values collected per interview, not every data item can receive the same level of individual quality assurance

1.1 Background

The primary objective of the FRS is to provide DWP with information on the development, monitoring and evaluation of social welfare policy. Detailed data is collected on respondents’ incomes from all sources including earnings, benefits and pensions; expenditure on housing, and type of housing tenure; caring needs and responsibilities; disability; education; childcare; family circumstances; child maintenance; material deprivation, household food security and food bank usage.

The FRS datasets and published information are accredited official statistics, which are called National Statistics in the Statistics and Registration Service Act 2007. They were independently reviewed by the Office for Statistics Regulation (OSR) in 2011 and confirmed as National Statistics by the OSR in November 2012. This designation means they comply with the highest standards of trustworthiness, quality and value in the Code of Practice for Statistics and are labelled ‘accredited official statistics’.

The OSR sets the standards of trustworthiness, quality, and value in the Code of Practice for Statistics that all producers of official statistics should adhere to and all our statistical practices and outputs are regulated by the OSR. You are welcome to contact the TEAM.FRS@DWP.GOV.UK directly with any comments about how we meet these standards.

1.2 List of Datasets

These datasets are used in the production of FRS accredited official statistics. A description of them and their uses is provided below.

Survey data sources

The FRS is a continuous survey which collects data on the incomes and circumstances of individuals living in a representative sample of private households in the United Kingdom. The survey has been running in Great Britain since October 1992 and was extended to cover Northern Ireland from the survey year 2002 to 2003.

DWP directs, funds and is the data controller for the survey. The Office for National Statistics (ONS), National Centre for Social Research (NatCen) and Northern Ireland Statistics and Research Agency (NISRA) conduct the operational aspects of the FRS, which are not handled in-house within DWP. This includes implementing questionnaire changes requested by DWP, drawing the survey sample, managing day-to-day fieldwork, collating data across the three organisations, and delivering the combined dataset to DWP. This also considers region-specific variations required for circumstances in Northern Ireland, for which DWP liaises with the Department for Communities NI.

Benefit-related administrative data sources

Processing of state-support data used to be a purely manual task, with the FRS analytical team using eligibility guidelines and information about the individual or benefit unit (family) circumstances to determine a value for any missing or incorrect benefit amounts.

Changes to the lawful basis for data linking in 2018, combined with targeted investment in improving linking methodology, have enabled at least 95% of FRS respondents to be linked to their administrative records. This creates opportunities significantly to improve data quality, timeliness, and cost efficiency through the use of administrative data within the FRS. DWP has access to a range of high-quality administrative data sources covering the main benefits reported in the FRS survey dataset.

Figure 1: Administrative data sources used for linking benefit records

Benefit	Admin Data Source
Child Benefit	HMRC
Attendance Allowance	GMS/CPS
Disability Living Allowance	GMS/CPS
Personal Independence Payment	Personal Independence Payment (PIP)
Carer’s Allowance	GMS/CPS
Employment and Support Allowance	GMS/CPS
Income Support	GMS/CPS
Jobseeker’s Allowance	GMS/CPS
Universal Credit	UCFS
Housing Benefit	SHBE
State Pension	GMS/CPS
Pension Credit	GMS/CPS
Tax Credits	HMRC
Industrial Injuries Disablement Benefit	GMS/CPS

2. Quality assurance of data assessment

2.1 UK Statistics Authority toolkit

The assessment of the FRS data sources has been carried out in a way which is parallel to the approach in the QAAD toolkit. The toolkit sets out four levels of quality assurance that may be required of a dataset:

• A0 – no assurance
• A1 – basic assurance
• A2 – enhanced assurance
• A3 – comprehensive assurance

Please, see the QAAD toolkit for more context on the grading matrix.

Level A1 – basic assurance

The statistical producer has reviewed and published a summary of the survey data quality assurance (QA) arrangements.

Level A2 – enhanced assurance

The statistical producer has evaluated the survey data QA arrangements and published a fuller description of the assurance.

Level A3 – comprehensive assurance

The statistical producer has investigated the survey data QA arrangements, identified the results of independent audit and published detailed documentation about the assurance and audit.

To determine the appropriate assurance level for a statistical publication, it is necessary to consider both the level of data quality risk and the public interest profile of the statistics. The UK Statistics Authority states that the A0 level is not compliant with the Code of Practice for Statistics.

Figure 2: UK Statistics Authority quality assurance of administrative data (QAAD) risk and profile matrix

Level of risk quality concerns	Public interest profile: Lower	Public interest profile: Medium	Public interest profile: Higher
Low	Statistics of lower quality concern and lower public interest [A1]	Statistics of low quality concern and medium public interest [A1/A2]	Statistics of low quality concern and higher public interest  [A1/A2]
Medium	Statistics of medium quality concern and lower public interest [A1/A2]	Statistics of medium quality concern and medium public interest [A2]	Statistics of medium quality concern and higher public interest [A2/A3]
High	Statistics of higher quality concern and lower public interest [A1/A2/A3]	Statistics of higher quality concern and medium public interest [A3]	Statistics of higher quality concern and higher public interest [A3]

Source: Office for Statistics Regulation

2.2 Assessment and justification against the QAAD risk and profile matrix

The FRS survey data, as the primary source for the FRS (along with supporting administrative data on benefits) has been evaluated using the QAAD toolkit’s risk and profile matrix (Figure 2), which reflects both the level of data quality risk and the public interest profile of the statistics.

FRS survey data is regarded as being medium risk of quality concern.

There is a clear formal agreement, between DWP and its data suppliers, of what data will be provided, when, how and by whom. It is recognised that risks are increased when there are multiple data collection sources; these being ONS, NatCen and NISRA. However, the risk is mitigated by each organisation using the same questionnaire format, interviewer instructions and editing software. Also, the length of time that each organisation has been involved in the FRS, and the experience they have built up adds considerable value and provides assurances that lower the risk. This has also added to the quality of regular and effective communication between all partners in the data supply process.

Whilst every effort is made to collect data to the highest quality, as with all survey data, it is dependent on suitable data sources, robust methods and assured quality. Checks are carried out throughout the process from the survey sample design, questionnaire consultation, collection of the data to producing the statistical dataset. Nonetheless, some respondent errors and both sampling and non-sampling errors may occur.

FRS datasets are regarded as higher public interest.

FRS data informs poverty measures both income-based and non-income-based, produces statistics that are reported widely in the media and impacts upon state support policies. FRS data underpins DWP’s Policy Simulation Model (PSM) which is used for the development and costing of policy options. The dataset is also used extensively by academics, and research institutes for economic and social research purposes.

The FRS contains information used across a wide range of government departments. This includes, but is not limited to, its use in tax and benefit policy analysis by His Majesty’s Treasury and His Majesty’s Revenue and Customs. Other users include the Ministry of Justice, the Department for Education and the Department for Environment, Food and Rural Affairs.

Therefore, as defined by the risk and profile matrix (Figure 2), the combination of a medium level of risk of quality concern and a higher public interest profile indicates that an enhanced level of assurance (A2) is the minimum required for FRS statistics.

Practice areas of quality assurance

Evidence for how the FRS data meets the requirements of an enhanced level of quality assurance is outlined in the remainder of this report, sections 3 to 6 below.

FRS data are assessed under four distinct practice areas:

• operational context and data collection
• communication with data supply partners
• quality assurance principles, standards and checks applied by data supplier
• producer’s quality assurance investigations and documentation

Each of the four practice areas are evaluated separately, and the respective level of assurance is stated. This approach provides:

• an in-depth investigation of the areas of particular risk of interest to users
• a demonstration of evidence of how identified risks are managed and mitigated
• transparency in how communication with data suppliers ensures a common understanding of any quality issues
• a clear explanation of the strengths and limitations of the data

3. Operational context and data collection (matrix score A2)

This section demonstrates our understanding of the environment and processes in which the data are being collected and the factors identified, that might increase the risks to the quality of the survey data.

3.1 Population and sample selection

The FRS sample is designed to be representative of all private households in the UK. The sampling frame excludes people who are living in communal settings, e.g. nursing homes, halls of residence, barracks or prisons, and people living in temporary (bed and breakfast) accommodation. This is intentional in the survey design, as these properties are difficult for interviewers to access and the purpose of the FRS is to capture information on household income and circumstances.

A “household” is defined as “one person living alone, or a group of people (not necessarily related) living at the same address, who share cooking facilities and share a living room or dining area”, according to the harmonised definition used in the Census and ONS social surveys.

One of the strengths of the FRS is that it collects many personal and family characteristics which are not available from administrative sources. This means that the FRS can be used to analyse income and benefit receipt in ways which are not possible from administrative data sources alone.

One of the known limitations of surveys is non-response. The lower the response rate to a survey, the greater the likelihood that those who responded are systematically different to those who did not, and so the greater the risk of systematic bias in the survey results. The FRS stratified sample structure is designed to minimise the impact of non-response being different for different types of households in the achieved sample.

The sampling frame in Great Britain

The Great Britain FRS sample is drawn from the Royal Mail’s small users Postcode Address File (PAF). One of the main advantages of using PAF is that it organises address information into a standard format and is updated daily. The small users PAF is limited to addresses which are not flagged with Royal Mail’s “organisation code”. To support drawing the FRS sample, an updated version of this list is obtained twice a year.

By focusing only on small-user delivery points, most large institutions and businesses are excluded from the sample. Delivery points identified as small business addresses are also removed. However, some small businesses and other ineligible addresses may still remain within the sampling frame. If such addresses are selected, they are classified as ‘ineligible’ once the interviewer confirms that no private household occupies the premises.

Sample design in Great Britain

The Great Britain FRS uses a stratified, two-stage probability sample of addresses. First, 1,963 postcode sectors (called Primary Sampling Units or PSUs) are selected from about 9,300 sectors in Great Britain, with selection probability proportional to size. These PSUs are stratified by 27 regions and three Census-based variables to ensure the sample reflects the population.

Within each region, postcode sectors are grouped into eight bands based on the proportion of households where the Household Reference Person (HRP) is in National Statistics Socio-economic Classification (NS-SEC) classes 1–3. Each band is then split into two, based on the proportion of economically active adults aged 16–74, creating 16 bands per region. Finally, these bands are ranked by the proportion of unemployed economically active men aged 16–74. These sets are known as “stratifiers”. These stratifiers improve accuracy for key variables like household income and housing costs.

Within each selected PSU, a sample of addresses is drawn. This is typically 28 households per PSU each year.

Figure 3: A representation of the FRS sampling frame

• The PSUs (postcode sectors) are represented by the small squares. They are each assigned to strata, represented by the shading (note that only 6 strata are depicted here)
• A sample of these PSUs from each stratum is drawn at random (non-shaded PSUs are not visited)
• A single PSU is expanded, to show its households. A random sample of those (shown brighter) are selected to be interviewed

The FRS sample stratification variables for Great Britain are as follows:

Regions:

• 19 in England (inc. Metropolitan vs non-Metropolitan split, 4 in London)
• 2 in Wales
• 6 in Scotland

The proportion of households where HRP is in NS-SEC 1 to 3:

• 8 equal bands

The proportion of economically active adults aged 16-74:

• 2 equal bands

The proportion of economically active men aged 16-74 who are unemployed:

• Sorted within above bands

Each year, half of the PSUs are retained from the previous year’s sample, but with new addresses chosen. For the other half of the sample, a fresh selection of PSUs is made (which in turn will be retained for the following year). This is to improve comparability between years.

Sampling in Northern Ireland

The sampling frame used for Northern Ireland is derived from the NISRA Address Register (“NAR”). The NAR is developed within NISRA and is primarily based on the Land and Property Services (LPS) POINTER database, the most comprehensive and authoritative address database in Northern Ireland, with approximately 752,000 address records available for selection. A systematic random sample is selected for the Northern Ireland FRS from the NAR. Addresses are sorted by local council and ward, so the sample is effectively stratified geographically.

Owing to its sample design, the FRS cannot be used to provide robust estimates at Local Authority/Unitary Authority/Council level, meaning that the lowest level of geography for which the FRS can be used to provide estimates is Region.

3.2 Sampling Error

All survey estimates have some degree of sampling error attached to them, which stems from the variability of the observations in the sample. From this, a margin of error (confidence interval) is estimated, which indicates the likely range of results that would appear if the same survey were to be conducted many times with a different sample whilst maintaining the same approach as the current sample.

It is this confidence interval, rather than the estimate itself, that is used to make statements about the likely ‘true’ value in the population; specifically, to state the probability that the true value will be found between the upper and lower limits of the confidence interval. In general, a 95% confidence interval is calculated, within which there is a 95% chance that the true value of the population is found. A narrower confidence interval is generally indicative of a more precise estimate of where the true value lies.

Figure 4: How confidence intervals communicate the precision of an estimate

The FRS sample in Great Britain, as described earlier, is selected using a stratified design, based on addresses clustered within postcode sectors. As a result, FRS sampling error is not just dependent on the variability between sample units (households, benefit units, individuals), but also on the variability between postcode sectors.

For example, if a sample characteristic is unevenly spread by postcode sector (i.e. is clustered) the representativeness of a given sample is reduced, and the variability of repeated samples would be greater overall than would occur in a simple random sample of the same size. Therefore, the mathematical accounting for clustering causes the (actual) sampling error to be greater than a sampling error calculated under the assumption of simple random sampling.

Stratification attempts to account pre-emptively for some of the variation between clusters using information at a cluster level known prior to the survey. Its effect is to reduce the sampling error, relative to what it would otherwise have been. Clustering is not used in Northern Ireland, but households are contacted using systematic random sampling, which has a similar effect to stratification.

Following the survey fieldwork, far more is known about those sampled. For certain characteristics, this information can be compared to known population totals and weighted to ensure proportionate representation of those otherwise disproportionately sampled (see the Grossing Information in Section 6 of the FRS Background Information and Methodology). This can be thought of as increasing the representativeness of the sample or reducing the variability that would be observed if making repeated samples. The effect is therefore similar to stratification and reduces the sampling error. For this reason, this process is also called post-stratification.

Communicating uncertainty within FRS-based estimates

Whilst the FRS sample is designed to minimise sampling error, naturally there remains a level of uncertainty to the estimates produced from the survey. To help quantify the level of uncertainty associated with a selection of FRS-based estimates, standard errors, design factors and confidence intervals are produced. A larger standard error or a wider confidence interval means there is more uncertainty about the FRS estimate. Similarly, a higher design factor shows that using a complex sample design with post-stratification has reduced precision compared to the same estimate formed from a simple random sample.

Standard errors, design factors and confidence intervals vary from survey estimate to survey estimate.  A new standard error methodology was introduced from the 2021 to 2022 publication, following a similar method to that used in the HBAI publication since the financial year ending 2016. Standard errors, design factors, and confidence intervals on estimates are now calculated using a bootstrap resampling method that accounts for the complex survey design and post-stratification weighting as fully as possible. An overview of this methodology, written by the Institute for Fiscal Studies, can be found in this methodological note from 2017.

A perfect method for calculating variability in survey estimates would be to have performed the survey many times, independently, and measure the range of estimates observed. As the survey fieldwork has been performed only once, the method of estimating uncertainty is as follows:

• Uncertainty is approximated by treating the achieved sample as if it were the population and repeatedly drawing sub-samples at random from that achieved sample.

• This process is referred to as ‘resampling’ and the result is a series of ‘resamples’. Resamples are drawn to mimic the original sampling methodology and replicate its effects on the reliability of any results. This means that stratification and clustering information and systematic random sampling processes are used to replicate the original FRS household selection process.

• Households can be selected multiple times, since resamples are drawn with replacement, and the unequal probability of selection of households is accounted for by using a grossing factor. Each FRS resample is smaller than the original sample size by about two thirds. The magnitude of the estimates of uncertainty is driven by the ratio between the size of the original FRS sample and the size of the resamples. In this case, since the resamples have a smaller sample size than that of the original FRS sample, it is likely that the estimates of uncertainty presented in the Methodology and Standard Error tables are more likely to be overestimates of the true uncertainty in the FRS sample rather than underestimates.

• Once households are selected, each FRS household is assigned a grossing factor in a process identical to the full sample. Altogether, this produces a series of alternative samples from which to calculate a series of alternative estimates.

The variability in these alternative estimates is used to quantify the uncertainty in the original estimate in two ways:

• The standard deviation of these alternative estimates is the approximate standard error of the original estimate; and

• The series of estimates produced are ranked by ascending size, and the 2.5th and 97.5th percentiles extracted. These are then used in calculating the approximate 95% confidence interval around the original estimate.

The size of the actual standard error relative to the standard error calculated under an assumption of simple random sampling is represented by the design factor, which is calculated as the ratio of the two. Where the standard errors are the same, the design factor equals one, implying that there is no loss of precision associated with the use of a complex sample design with post-stratification.

Conversely a design factor of less than one implies the FRS estimate is more precise than would be obtained from a simple random sample. In many cases, the design factor will be greater than one, implying that FRS estimates are less precise than those of a simple random sample of the same size due to the clustered sampling used within the FRS sample design.

Published Methodology and Standard Error (SE) Tables provide standard errors, design factors and confidence limits for a selection of variables from the survey. An example of how to interpret figures in this table is as follows:

Example: Uncertainty measures for household composition

Suppose that published tables show that 73% of households did not contain any children, and the standard error is estimated as 0.1 percentage points. These estimates form the final point estimates for the proportion of households without children and this group’s associated standard error.

The design factor for this variable is 0.3. This means that the effect of using a complex survey design and post-stratification, rather than a simple random sample, has led to a reduction in uncertainty of 70%, when using standard error as the measure of uncertainty. In contrast, a design factor of 1.5 would have denoted an increase in such uncertainty of 50%. Among smaller groups, such larger design factors are not uncommon.

The 95% confidence interval is given as 72.4% to 72.7%. This means that if sampling error is the sole source of error present, there is a 95% chance that the true percentage of households without children lies within this range. Whilst it appears that the estimate is not within the confidence interval, this is generally due to the rounding applied to the published estimate. It is also important to note that the confidence limits are drawn from the sampling distribution of proportions that the bootstrapping process generates, whilst the point estimate is derived from the ‘original’ FRS sample.

A methodology paper is available for information on estimating variance and confidence intervals in special circumstances e.g. where the number of occurrences of a response in the sample are very small.

3.3 Non-sampling error

Non-sampling errors are systematic inaccuracies in the sample when compared with the population. Non-sampling errors arise from the introduction of some systematic bias in the sample compared with the population it is supposed to represent.

As well as response bias, such biases include inappropriate definition of the population; misleading questions; data input errors; data handling problems; or any other factor that might lead to the survey results systematically misrepresenting the population. There is no simple control or measurement for such non-sampling errors, although the risk can be minimised through careful application of the appropriate survey techniques from the questionnaire and sample design stages through to analysis of results.

It is not possible to eliminate non-sampling error completely, nor can it be easily quantified. However, non-sampling error is minimised in the FRS through:

• effective sample design (as described in Section 3.1)
• authoritative questionnaire design (as described below)
• active fieldwork management (as described below)
• strategies to improve response rates (as described below)
• the use of skilled and experienced interviewers (as described below)
• extensive quality assurance of data (as described later in Sections 5 and 6)
• use of DWP/HMRC administrative data (as described later in section 6)

Data collection and fieldwork management

Data is collected by other organisations on behalf of DWP. In Great Britain, ONS and NatCen conduct fieldwork for the FRS. In Northern Ireland the sampling and fieldwork for the survey are carried out by the Central Survey Unit at NISRA. These organisations have operational responsibilities for drawing the sample, programming the survey questionnaire, enacting annual changes which are specified by DWP, contacting the selected households, and some initial data processing.

With fieldwork, each month the Great Britain sample is randomly divided between the two sets of interviewers, with 35% assigned to ONS and 65% assigned to NatCen. The UK set of selected addresses is then assigned to the relevant interviewers. Before interviewers visit the selected addresses, a letter is sent to the occupier explaining that they have been chosen for the survey and that an interviewer will visit the address soon. The letter and accompanying leaflet emphasise that information given in the interview will be treated in the strictest confidence and used only for research and statistical analysis purposes. Further information is provided on the ONS website Family Resources Survey - Office for National Statistics and NatCen website Family Resources Survey - National Centre for Social Research which also explains that the survey relies on the voluntary co-operation of respondents.

The main face-to-face contact with respondents is via doorstep contact. If contact is not made on the first attempt, the interviewer is required to make additional visits to an address. These visits must be made at different times of the day and on different days of the week, including at least one weekend attempt. If more than one household receives mail at an address, a single household is interviewed.

Addresses returned as non-contacts or partial refusals can sometimes be re-issued to another interviewer where appropriate, in the hope that an interview at the non-responding household can still be achieved. Interviewing at re-issued addresses can be carried out at any point in the remaining survey year.

Response

To apply a quality measure of how reliable an interview is, a household is defined as fully co-operating when:

• an interviewer has been able to interview all adults aged 16 and over
  • except those aged 16 to 19 who are classed as dependent children
• there are fewer than thirteen ‘don’t know’ or ‘refusal’ answers to monetary amount questions in the benefit unit schedule
  • excluding the assets section of the questionnaire

Proxy interviews are accepted when a household member is unavailable for interview. All data in the FRS dataset and the published statistics in all FRS-based publications refer only to fully co-operating households.

Response rates are calculated as follows:

Response rate =   (Number of fully cooperating households /  Number of all eligible households) x 100      

For a UK survey of the size and complexity of the FRS, a response rate of around 50% was considered reasonable, prior to the Covid-19 pandemic. Response rates of around 30% have become more prevalent since the 2020 to 2021 survey year. In recent years, the FRS has had a markedly higher issued sample so as to offset the lower response rates now seen. This ensures that the FRS still achieves a sizeable sample, which is representative of the population.

The Background Information and Methodology document accompanies this and every recent FRS publication, providing details of such changes; together with a technical paper, as required, outlining further quality assurance of the processing of data and production of statistics, particular to each survey year. Methodology tables are also published alongside the main publication, summarising the UK household response rate, regional response rates and the reasons given for refusal if provided.

Households that are not fully co-operating are classified as partially co-operating, refusals, or unable to make contact. A partially co-operating household is one where a full interview has been obtained from the Household Reference Person’s (HRP’s) family (benefit unit), but others living in the household have not co-operated at all, or only to a very limited extent.

Refusals include those residents of an address that contact head office to refuse to participate and residents who refuse to participate in the survey when contacted by the interviewer. Those who were not available are recorded as “non-contacts”, as the interviewer is unable to establish whether they would have chosen to participate if they had been available. The category of “non-contact” only includes those addresses in the issued sample where the interviewer has confirmed that the address is eligible for the survey, but they are unable to contact residents to ask them to participate.

Any information that can be obtained about non-respondents is useful both in terms of future attempts to improve the overall response rate and potentially in improving the weighting of the sample results. Direct information about the non-responding households is valuable although, by definition, difficult to obtain. Monitoring of the components of non-response including the rate of refusals and non-contacts is carried out by the FRS team, using information from monthly performance indicators provided by ONS (see later section). Further investigation into the breakdown of the numerous categories of both refusal and non-contact recorded by interviewers can assist data suppliers in designing and evaluating methods for increasing response.

Interviewer training

Interviewers are trained in the running of an FRS interview prior to commencing interviews. They also receive training in the collection of financial and other sensitive information. This has the advantage over other modes of survey collection, because during a face-to-face interview trust is built and interviewers can assist respondents with understanding complicated questions. The main emphasis is on collecting accurate information, with respondents asked to consult documentation wherever possible to verify figures. This aids the consistency of data capture across different survey years, which is essential.

Interviewers new to the FRS are briefed on the questionnaire and an annual re-briefing is given to all interviewers on changes to the questionnaire. All interviewers working on the survey have the opportunity to describe their experiences with specific parts of the questionnaire and comment on how changes were received in the field. This feedback is provided at the end of each interview, and it is collated into a written report.

Questionnaire design

As part of the process of agreeing annual questionnaire changes, suggestions from users are also considered, as well as those arising from an evaluation of feedback from interviewers. Any changes to the questionnaire are checked for consistency with the harmonised standards for social surveys across government.

Each year, DWP runs a questionnaire consultation and draws up a list of possible questionnaire changes. Users are asked to identify individual questions or sections which are no longer of interest. The FRS questionnaire is lengthy and demanding and a key concern is, where possible, to reduce (or at least not increase) its length, so as not to overburden respondents or interviewers.

New questions are added with the expectation that they will produce useful data that can be delivered to users through additional variables and used to support future policy analysis. Some changes are made to improve the interview experience or to support improvements to data processing.

Operationally, changes are tested on-screen by ONS, NatCen and NISRA once coded; and several test versions of the questionnaire are provided to DWP during the survey year, each with a successively greater number of the year’s changes encoded, culminating in a final version which includes all changes. Change control systems record all changes to the questionnaire, with individual forms for each change documenting the request for a change, reasons for decisions taken and how it has been implemented. Changes to the resulting dataset are documented in an output spec which assists DWP with quality assuring the dataset once delivered.

Completion and development of the change control arrangements is a joint initiative between all parties with regular communication between DWP and data suppliers. The output spec, along with the updated dataset metadata, document in detail the conversion process from questionnaire variables to output variables and therefore form an integral part of the change control documentation.

The new variables are released in the published dataset subject to successful quality assurance, but not all are added to the main FRS publication on GOV.UK; or to the set of FRS tables available from the Department’s Stat Xplore tool. This decision is made considering user interests and the need for disclosure control.

Strengths

• A specific tailored, stratified sample design is used. Stratification can pre-emptively account for some of the variation between clusters, so its effect is to reduce sampling error
• Effective data collection processes, with trained interviewers
• Face-to-face interviewing enabling the collection of data on a wide range of topics, including many personal and family characteristics which are not available from administrative sources, together with sometimes sensitive information
• The questionnaire consultation process is integral to the annual development of the survey, involving collaboration between users, suppliers and producers

Limitations

• Owing to its sample design, the FRS cannot be used to provide robust estimates below the level of Region
• The lower the response rate to a survey, the greater the likelihood that those who responded are significantly unlike those who did not, and so the greater the risk of systematic bias in the survey results
• The FRS questionnaire is lengthy and demanding and a key concern is, where possible, to reduce (or at least not increase) its length, so as not to overburden respondents or interviewers

4. Communication with data supply partners (matrix score A2)

This section provides evidence of how the FRS maintains effective relationships with data suppliers. It includes the provision of regular performance reports and documentation of change management processes and the consideration of statistical needs when changes are being made, for example to either the sampling design or the questionnaire. Several reports are used by ONS and the DWP FRS team to communicate information during the survey year:

• Field Report
• Monthly Performance Indicator Reports
• Issues Log
• Annual Report

Data supply from the interview

As noted above, DWP are the Data Controller for the FRS. DWP determines the interview content (questionnaire), the funding, sample size and address selection policy, data quality assurance procedures and the timing and content of the FRS annual publication.

The collection, processing and transfer of data is governed by a Data Protection Impact Assessment (DPIA), and a Security Assurance for Research and Analysis (SARA). UK GDPR Principle 7 – accountability requirements are met. Data are shared under section 45A of the Statistics and Registration Service Act 2007. The Lawful Basis for Processing is UK GDPR Article 6(1)(e) - Public Task. It is recognised that Special Category data is captured, such that the respective Lawful Basis for Processing is UK GDPR Article 9(2)(j) - Archiving, research and statistics.

A memorandum of understanding (MOU) is in place between DWP and ONS for the provision of the FRS. The MOU states that all data processors, and where applicable sub-processors, must have in place procedures for storing and transferring FRS data using appropriately secure methods. Transfer is either via PGP encrypted email or MoveIT or GlobalScape, with oversight applied by DWP Data Security to all inbound receipts.

Data supply from administrative sources other than the interview

See technical paper for details.

4.1 Field Report

To assess how well any new questions or other changes are performing in the field, feedback is collected from interviewers at the end of the survey. The report summarises the feedback from interviewers and fieldwork operations staff; it is usually written by ONS then provided to DWP in early summer. Its key objective is to provide qualitative feedback on the questionnaire, directly from the people who collect data from respondents.

Interviewers are advised that their feedback will be taken into consideration for changes to the FRS questionnaire in the future.

Outside this reporting, DWP analysts are invited to shadow interviewers in the field. This enables analysts to observe the practicalities of how the data are collected. Potential areas for confusion in questions, and how trained interviewers address these, is important in understanding how well the data values reflect a household’s actual circumstances.

4.2 Monthly Performance Indicator Reports

The aim of these reports is to keep DWP informed on the key metrics which may affect quality, in terms of number of productive interviews successfully conducted, compared to the numbers issued, by each organisation. Response rates are mainly determined by the capacity of available interviewers, their ability to contact a sampled household and the willingness of the members of that household to participate in the interview

The main metrics in the monthly report are response rates, broken down by Full, Partial, Proxy and Follow-up responses.

Other metrics that are provided are:

• Ineligible addresses
• Regional response rates
• Interview timings by organisation
• Encashment rates of incentive vouchers
• Cumulative Target vs Achieved cases
• (Quarterly update on) trained interviewer numbers

These reports provide input to regular discussions between the DWP FRS Team Leader and the ONS Family Resources Survey Lead. Representatives from NatCen and NISRA are also involved in discussions when appropriate. During these meetings there is the opportunity to discuss any newly emerging quality concerns and/or updates on actions taken to mitigate previously identified risks. Wider bilateral meetings between senior leaders in ONS and DWP allow for the opportunity to address any issues as they emerge, discussing the potential for possible mitigation strategies.

On a monthly basis the regional response rates are analysed by the DWP FRS team, looking at changes over a longer time-series of 2-3 years or more. Comparisons are also made of the regional distribution of the sample, compared to the UK population; to examine how representative the achieved sample is likely to be for the survey year. The identification of any region that may have substantially lower response rates than the UK average, either consistently or in a particular month allows early investigation and possible actions to be taken to address the issue.

These Performance Indicator reports, are communicated to senior DWP colleagues. Colleagues in the Devolved Governments of Scotland, Wales and Northern Ireland receive quarterly reports on response rates for their geographic area, including how these compare to the UK average and to historic years.

Communicating the differences between regions and between survey months also assists the processing team with understanding possible risks to data quality.

4.3 Issues Log

This is a detailed log of any issues that are uncovered within the data, by either ONS or DWP. Each item is logged by description, date and identifier. Any supporting information is linked / embedded.

Regular updates to this are made by DWP analysts, with discussions being held at the 6-month and 12-month stage to enable resolution of issues as necessary. Items that remain live at the 6-month stage are flagged to be followed up before the wider 12-month Data QA meeting.

Some issues can be resolved within the existing development dataset, but others require further work by ONS that can only be resolved by a re-issue of the dataset to DWP. A later redelivery (resupply) of both the 6-month and the 12-month datasets is standard, recognising the adjustments required for a dataset with many inter-dependencies. In circumstances where issues are discovered, which require fundamental changes to the underlying data structure, a further resupply would then be made by ONS.

4.4 Annual Report

The annual report focuses on four areas:

• Fieldwork summary for the year just concluded
• Quality assurance
• Staffing levels
• Overall successes and areas for improvement

The quality assurance section covers, amongst other things, a breakdown of the number of individual checks carried out at each validation stage.

Strengths

• Effective lines of communication between data collectors (interviewers) and data suppliers (ONS, NatCen and NISRA) and onwards to DWP, have been established and developed over the lifetime of the survey and continue to evolve
• Weekly discussions between the DWP FRS Team Leader and the ONS FRS Lead provide an opportunity to discuss any newly emerging quality concerns and/or updates on actions taken to mitigate previously identified risks
• Wider bilateral meetings between senior leaders in ONS and DWP allow for the opportunity to address any issues as they emerge, discussing the potential for mitigation strategies and future developments
• Formal agreements, whether they be contracts or MOUs, are carefully scrutinised by Data Security, Data Protection, Legal and Commercial teams to ensure that they are fit-for-purpose. This provides a clear line of accountability

Limitations

• No material limitations have been identified in the communication with data supply partners

5. Quality assurance principles, standards and checks by data supplier (matrix score A2)

This section relates to the validation checks and procedures undertaken by the data supplier, any process of audit of the operational system and any steps taken to determine the accuracy of the data.

Careful attention is paid to the accurate collection of data followed by meticulous processing, editing, and quality assurance. This is especially important as microsimulation is central to DWP’s use of the FRS data.

5.1 Timeline of interview and post-interview quality assurance

ONS, NatCen and NISRA carry out a range of editing tasks on the captured survey response data, before its transmission to DWP. An overview of these stages with an indicative timeline is given below:

Figure 5: Survey data supplier processing

The image is a timeline-style process flowchart showing how survey data moves from ONS, NatCen, and NISRA to DWP across several months.

Left‑side (April to July): ONS / NatCen / NISRA workflow

• April: ONS, NatCen, and NISRA conduct April mainstage interviewing.
• May: April data undergo editing.
• June: ONS merges April data and produces tables for monthly quality assurance.
• July: A blue box notes that reissued interviewing and editing can happen in future months.

A note states that this process repeats for all months from April to March, with March editing finishing in June.

Right‑side (September to December): DWP workflow

• September: April–June data is delivered to DWP. DWP loads and runs checks on data, including key variables, record creation, and type checks. An issue log is created and sent back to ONS. If possible, start processing, otherwise wait for immediate data re-supply.
• October: ONS investigates April–June data issues and agrees changes. July data is sent to DWP. DWP loads July data to interface.
• November: April–June resupply data is quality assured and August data is delivered to DWP. DWP loads August data to interface.
• December: Resupply of April–September data is delivered to DWP. DWP loads April–September data to the FRS interface and checks it against the issue log, then start processing.

5.2 Checks at interview

One of the benefits of interviewing using Computer Assisted Personal Interviewing (CAPI) is that in-built checks can be made at the interview stage. This helps to check respondents’ answers and that interviewers do not make keying errors. There are checks to ensure that amounts are within a valid range and cross-checks which make sure that an answer does not contradict a previous response.

However, it is not possible to check all potential inconsistencies, as this would slow down the interview to an unacceptable degree, and there are also capacity constraints on interviewer notes. FRS interviewers can override most checks if the answers are confirmed as accurate with respondents.

A problem inherent in all large surveys is item non-response. This occurs when a household agrees to give an interview, but either does not know the answer to certain questions or refuses to answer them. This does not prevent them being classified as fully co-operating households if there is enough known data to be of good use to the analyst (although see the first paragraph of the Response section above for information about non-response to monetary questions).

Interviewers encourage respondents to consult documentation at all stages of the interview to ensure that the answers provided are as accurate as possible. For some items whether certain documents are consulted or not is recorded on the questionnaire. This assists FRS users in assessing the accuracy of the data.

5.3 Checks post-interview

Interview data is stored on interviewers’ encrypted, password protected laptops or tablets. Interviewers are instructed to transmit completed interviews as soon as possible so that data is not stored on the laptop or tablet unnecessarily.

Once an interview has taken place, data is returned to ONS, NatCen, or NISRA respectively. At this stage, editing takes place, based on any notes made by interviewers. Notes are made by the interviewer when a warning has been overridden, for example, where an amount is outside the expected range, but the respondent has documentation to prove it is correct. Office-based staff make editing decisions based on these notes.

Checks and enhancements to collated data

Data is collated and edited from all interviews on a monthly basis. A limited number of edits are made at this stage, which include:

• interviewer edits completed post-interview, such as occupation / industry coding etc.
• adding certain categorical variables, such as educational level and socioeconomic group
• adding certain geographical variables, such as Broad Rental Market Area (BRMA), Lower Super Output Area (LSOA); and Council Tax and NI Rates information
• either imputing or suggesting the imputation of various missing items such as net pay, tax, etc using algorithms supplied by DWP

Before further validation, FRS data is converted from CAPI format into SAS-readable tables. Using DWP specifications, SAS-readable tables are created by ONS, with each table displaying information from different parts of the questionnaire.

Both DWP and ONS then carry out checks on key input and output variables to ensure that the data have converted correctly to the new format. Checks include ensuring that the number of adults and children recorded is correct, and that records are internally consistent. ONS conduct the first round of credibility checks on a monthly basis and these are sent with the initial data delivery. These flag potentially problematic cases and sometimes a suggested edit.

If an error is identified in the data delivered to DWP, specifically one identified in the script that translates the questionnaire answers to SAS-readable files, it can take several attempts to revise the output script to ensure all issues have been addressed.

Strengths

• ONS, NatCen and NISRA have established and agreed data assurance processes that evolve as needed to deliver quality data
• Careful attention is paid to the accurate collection of survey information, followed by meticulous data processing, editing, and quality assurance

Limitations

• If an error is identified in the data delivered to DWP, reviewing of outputs is often a manual checking task. It is necessary to ensure that variable categories that are output as final are as expected, given the respondents’ answers to all other questions within the associated question block
• With over two thousand variables and their associated values collected per interview not every item can be quality assured by the supplier. For a survey of this magnitude there will always be the risk of unidentified errors. If these are identified by the producer, a re-issue of the dataset from the supplier (ONS) to the producer (DWP) may be required

6. Producers’ quality assurance investigation and documentation (matrix score A2)

This section demonstrates the quality assurance conducted by DWP analytical teams, including corroboration against other data sources.

The FRS dataset is used for a wide range of analyses beyond the published tables. For many users, the dataset is more important than the statistical releases themselves. The use of the FRS dataset for policy modelling places a premium on accuracy, in that an inaccurate dataset could lead to policy costs or benefits being incorrectly assessed, and/or a suboptimal choice of policy option. As small groups of cases could affect the results of user analyses, a thorough examination of case-specific information is made.

6.1 The FRS Interface

The original interface for processing FRS data was developed in the 1990s, as a SAS AF/SCL based application. As this technology had passed end-of-life in support terms, it was replaced with a new, HTML-led data management solution, built to modern standards. This was an important investment for the future of the FRS project. This solution (FRESCO) recreates many of the old interface’s functions, but also offers improved code version control, improved data viewing, on-screen editing and a part-automated anonymisation setup.

6.2 Producer expertise

The FRS Team at DWP has analysts from a mix of professions and experience working across all elements of the project together. Team members have distinct, but integrated roles; whilst someone may be responsible for the questionnaire consultation or managing the publication process, every member of the team is involved in an aspect of dataset processing and leading a topic of publication.

This team structure means that there is expertise across the whole project, so that people can be assigned to development projects (alongside routine processing), to investigate and suggest improvements.

Desk instructions for all aspects of data processing are easily accessible and routinely updated, so expertise is not lost when team members change. These not only include how to carry out a processing function, but also why these actions are taken. This allows the rationale to be monitored and where necessary, the action improved over time. A flow chart of the producer’s checking processes is presented below, and following sections give further detail on the main steps:

Figure 6: Data producer processing

6.3 Pre-processing checks

On receipt from the suppliers, the initial load and validation of data involves:

• Check that the data contains the tables and variables that are expected
• Check that the data content for every variable is valid, that it is not missing and that the values fall within certain minimum and maximum limits
• Producing a Changes document, showing the changes in the data content since the last delivery of data
• New variables and changed variables have values that are sensible
• Manual data content validation for all variables and record types that have been changed since the previous data delivery, especially if this was for a previous survey year
• Record Creation checks: checks between tables in the dataset to ensure (for example) high-level parent records can be linked to all expected lower-level child records, and vice versa
• Set Type checks: investigates sets of variables that have a response pattern of “Yes”, “No”, “None” for invalid patterns
• Skipped Important Variables check: a routing check for many of the most important variables on the FRS dataset, to identify where respondents may have been incorrectly routed to the wrong questions
• Variables are skipped where expected to be
• Variables do not have values where skipped values would be expected
• Records in tables have been created in line with their parent-level ‘flags’

Initial validation takes place after any new delivery of data and will always consider both new changes and previously outstanding issues.

6.4 State support validation – Benefit Editing

DWP validates all state support records within the FRS dataset. Information on benefit receipt is one of the key areas of the FRS, and it is very important that this section is thoroughly validated and cleaned. It is not appropriate to use imputation methods, such as hot-decking, algorithms or bulk edits (see below) for benefits data so instead a separate procedure of validation and editing is used.

A wide range of checks are applied to all benefit records. These look for instances where respondents have stated that they were receiving some form of benefit; and where the pound amount reported was in some way nonstandard or otherwise questionable (including that they may not be eligible).

Benefits where administrative data is used

As outlined in the DWP Statistical Work Programme – section 2.4, the department is committed to transforming its surveys through the integration of administrative data to enhance its data quality.

This is in the wider context of the UK Statistics Authority’s Strategy for data linking Joining Up Data for Better Statistics – Office for Statistics Regulation and OSR recommendations in their 2021 review of income-based poverty statistics, that DWP should explore the feasibility and potential of social survey and administrative data integration Review of Income-based poverty statistics – Office for Statistics Regulation.

A technical report on FRS Transformation work to date, with illustrative results for DWP benefits is available at: Family Resources Survey Transformation: integrating administrative data into the FRS. As developments are implemented, they are communicated to users via the FRS Release Strategy.

Each year, Methodology Table M.6a compares the grossed number of benefit recipients in the FRS data with the total caseload on benefit from administrative data sources. Typically, the FRS numbers in receipt are below those seen in administrative data. However, the difference varies by benefit, and for benefits such as State Pension it has always been just a few per cent.

It is acknowledged that some part of the benefit undercount in the FRS dataset is due to an under-representation of benefit recipients in the achieved FRS sample (as opposed to incorrect data capture from those in the sample).

For several years, the FRS has made use of administrative data to check on the accuracy of benefit receipt (yes/no) and pound amounts reported during the interview. These sources have been used for various kinds of state support. For example, with Universal Credit responses, where a pound figure was reported as being received, all such figures have been replaced with their administrative data figures for their Universal Credit.

The releases of FRS data in 2026 build upon this by expanding the use of administrative data. Changes applied for the 2024 to 2025 survey year, with past FRS years edited in the same way and re-released. See the technical paper for more details

Process of data linking to administrative data sources for benefit editing

A detailed account of the FRS Transformation work to date, including the development of the lookup file, data linking processes, quality assurance, and illustrative results for DWP administered benefits, is set out in the FRS Transformation: integrating administrative data into the FRS – implementation in March 2026 technical paper.

Benefits where administrative data is not used

The FRS dataset is intended to record benefit receipt across all benefits, including small-caseload state benefits and also private benefits such as health/unemployment insurance payouts. Where administrative data is not available for these benefits, quality checks are still applied. Computer programs are run to carry out a check for benefit entitlement and amounts paid, and to output any cases that look unreasonable. All cases detected are individually checked and edited where necessary. These would include the near-zero and missing amount checks of past years.

In particular, there is no access to administrative data for the Armed Forces Compensation Scheme and War Widow’s/Widower’s Pension; nor for benefits paid only in Scotland such as the Adult Disability Payment, Child Disability Payment or Pension Age Disability Payment. The same is true of the Scottish Child Payment (SCP). For these benefits a decision has been made on level of receipt – especially for SCP, with a specific imputation of receipt – based upon all the available evidence of that person’s circumstances.

Overall strengths and limitations

Strengths

• Improves the survey data quality by using administrative data to correct a substantial proportion of erroneous benefit amounts and benefits receipt reported in the FRS
• More efficient process compared with the previous manual benefit‑editing approach
• Reduces processing time, as administrative data reduces the amount of manual editing required
• Timely availability of data, since most administrative extracts are available shortly after the survey year ends and can be used as soon as FRS processing begins
• Robustly scrutinised, with the methods reviewed by the FRS and the FRS Transformation team responsible for data linking

Limitations

• Does not resolve all errors, as administrative data addresses many but not all inaccuracies in reported benefit amounts and benefits receipt
• Dependent on administrative data quality and completeness, which may vary across benefit types
• Linkage constraints, as successful linking relies on matching information being available and accurate

6.5 Other pre-imputation cleaning

Apart from state benefits, DWP also validates the other data items on the FRS dataset.  Many other checks and edits are applied:

Weekly amounts

In the FRS, most monetary amounts are converted to a weekly equivalent. To calculate this, respondents are usually asked the amount, then the length of time this amount covered. The latter is known as a “period code”. Period codes are used in conjunction with amounts to derive weekly figures for all receipts and payments. Some variables, such as interest on savings accounts, refer to the amount paid in the whole of the past year. These are also converted to a weekly amount.

Sometimes the period code relates to a lump sum or a one-off payment. In these cases, the corresponding value does not automatically convert to a weekly amount. For the data to be consistent across the survey, edits are applied to convert most lump sums and one-off payments to weekly amounts. In the same way, where period codes are recorded as ‘don’t know’ or ‘refused’, these are imputed so that the corresponding amount can be converted to a weekly value in the final dataset.

Near-zero amounts

Any cases of near-zero amounts in other variables are examined individually, and an edit decision is made.

Outliers

Statistical reports of the data are produced to show those cases where an amount was greater than four standard deviations from the mean. These relate to outliers, data that is beyond the expected value range of the variables being explored, based on the other data in the set. It is important that outliers are transformed so that they can validly contribute toward the analysis (or be omitted). Although if the outliers are omitted this could increase the risk that false conclusions are drawn.

For up to seven values which are all over four standard deviations from the mean, the individual record is examined and where necessary – but only if a value looks unrealistic – the case is edited. Outliers are verified by examining other relevant data for that household to establish whether the amount aligns to other answers. Compared with earlier FRS years, a relatively low number of these edits are now carried out, because of the many range checks in the computerised questionnaire.

Credibility checks

A wide spectrum of checks is carried out for the internal consistency of certain variables; and values which are otherwise found to be unreasonable. Most tables in the FRS dataset will have several checks applied to them each year; the overall number of checks is more than 100. For example, one check on mortgage payments ensures that payments to the mortgage from outside the household are not greater than the mortgage payment itself. Such cases are examined and edited where necessary. These checks are reviewed annually to edit for changes to variables in the first instance, but more widely reviews are undertaken to add new credibility checks to test for errors that were found during the previous year’s processing and quality assurance.

6.6 Imputation

The main objective of imputation is to maximise the information available to users; the imputation carried out simplifies the analysis for users and helps to secure the uniformity of analysis created from the FRS data. If missing data were not imputed on the FRS, then it would be impossible to calculate accurately household income for many households surveyed.

The responses to some questions are much more likely to have missing values than others. For example, it is very unlikely that a respondent will refuse to give or will not know their age or marital status; whereas it is much more likely that they will not be able to provide precise information on the amount of interest received from their investments.

Areas where missing values are a problem are typically income values, such as employee earnings, income from self-employment and income from investments. This is because these values are required in the calculation of derived variables, used for reporting total Individual Income [INDINC], Benefit Unit Income [BUINC] and ultimately Household Income [HHINC], with these concepts heavily influencing the Households Below Average Income (HBAI) publication.

Results in the FRS published tables include imputed values. Elsewhere however, values are left to remain as missing in some variables (such as hours of care). Methodology Table M.4 is published alongside the main report each year to illustrate the extent of missing values. The main imputation methods are summarised below, in the order in which they are applied:

Closing down routes

As with any questionnaire, a typical feature of the FRS is a gatekeeper question positioned at the top of a sequence of questions, at which a particular response will open the rest of the sequence. If the gatekeeper question is answered as ‘don’t know’ or ‘refused’ then the whole sequence (route) is skipped.

A missing gatekeeper variable could be imputed such that a further series of answers would be expected. However, these answers will not appear because a whole new sequence (or route) has been opened. For example, if the amount of rent is missing for a record and has since been imputed, any further questions about rent would not have been asked. From the post-imputed dataset, it will appear that these questions should have been asked because a value is present for rent.

For this reason, where the gatekeeper question has been skipped the onward routes should be closed. In most cases, gatekeeper variables are of the ‘yes or no’ type. If missing, these would be imputed to ‘no’, on the basis that if a respondent does not know whether an item is received or paid, then it is likely that it was not received or paid.

Hot-decking

This process looks at characteristics within a record containing a missing value to be imputed and matches it up to another record with similar characteristics for which the variable is not missing. It then takes the known variable and copies it to the missing case. For example, when imputing the earnings of a person, the type of work they do is used to search for a case with a similar record. This method ensures that imputed solutions are realistic and allows for a wide range of outcomes which maintain variability in the data.

Algorithms

These are used to impute missing values for certain variables, for example variables relating to mortgages. The algorithms range from very simple calculations to more sophisticated models, based on observed relationships within the data and individual characteristics, such as age and gender.

‘Mop-up’ imputation

This is achieved by running a general validation report of all variables and looking at those cases where missing values are still present. At this stage, variables are examined on a case-by-case basis to decide what to impute. Credibility checks are re-run to identify any inconsistencies in the data caused by imputation, and further edits are applied where necessary.

All imputations, by each of the methods above, are applied to the un-imputed dataset via a transaction database. This ensures auditability in that it is always possible to reproduce the original data.

Points to note with imputed data

• Whilst several processes are used to impute missing values, it should be remembered that they represent only a very small proportion (typically two per cent) of the dataset
• Imputation will have a greater effect on the distribution of original data for variables that have a higher proportion of non-response, as proportions of imputed data will be higher
• As mentioned above, in certain situations, imputed values will be followed by ‘skipped’ values. It was decided in some cases that it was better to impute the top of a route only, and not large amounts of onward data. For a small proportion of imputed values, it is not possible to close down a route. These cases are followed by ‘skipped’ responses (where a value might otherwise be expected).

6.7 Derived variables

Derived variables (DVs) are those which are not created by the original interview, but instead are made by combining information, both within the survey and from other sources.

They are created at the FRS user’s request. Their main purpose is to make it easier for users to carry out analysis and to ensure consistent definitions are used in all FRS analyses. For example, INDINC is a DV which sums all components of income to find an individual’s total income. This is possible because of the various sources collected by the survey.

As new information is collected in the survey, the relevant DVs are updated as necessary, and a record of these updates is available for users of the End User Licence and Safe Room datasets held at the UK Data Service (UKDS) and also the Secure Research Service at ONS.

6.8 Grossing

Grossing or grossing-up is the term given to the process of applying factors to sample data so that they yield estimates for the overall population. The system used to calculate grossing factors for the FRS divides the sample into different groups. Grossing factors attempt to correct for differential non-response, at the same time as they scale up sample estimates. The groups are designed to reflect differences in response rates among different types of households. The software used to make the final weighted sample distributions matches the population distributions through a process known as calibration weighting.

Details of the control variables used in the grossing regimes for Great Britain and Northern Ireland are published annually in the Background Information & Methodology accompanying the main report.

In developing the FRS grossing regime, careful consideration has been given to the combination of control totals, and the way age ranges, Council Tax bands and so on, are grouped together. The aim has been to strike a balance so that the grossing system will provide, where possible, accurate estimates in different dimensions without significantly increasing variances. The published Methodology Table M.3 shows the extent to which the FRS grossing regime controls for this bias in the achieved sample.

6.9 Methodological reviews

Grossing

A review of the FRS Grossing Methodology was carried out by the ONS Methodological Advisory Service in 2013. Several relatively minor methodological improvements were made as a result, with the grossing calculations updated to use 2011 Census data at that point. Further details on the methodological changes were published as an explanatory paper of revisions made to the Family Resources Survey grossing methodology in 2014.

Inflation

Since the 2014 to 2015 survey year, the Consumer Price Index (CPI) has been used to adjust for inflation. More information concerning this methodological change was published as a statistical notice in 2016.

Material Deprivation

In December 2021, DWP commissioned a review of FRS material deprivation questions. The review was conducted by the Centre for Analysis of Social Exclusion (CASE) at the London School of Economics and Political Science (LSE). The review recommended a pilot of a short-list of 35 items and activities. Full details of the test questions and changes to methodology are given in Section 4 of the published review.

Following a pilot in 2022 to 2023, the FRS questionnaire introduced 29 updated questions in 2023 to 2024 for the whole of the survey year. Around 75% of the sample were asked the updated questions, with the remaining 25% being asked the old questions. The division of questions was made at random.

From the 2024 to 2025 survey onwards, only the updated questions have been used.

6.10 Output validation

Internal quality assurance: QA group

Internal users of the FRS dataset, who are authorised to have pre-published access, for either their own publication production and/or assisting with QA, are kept up to date with changes.

The FRS dataset is checked by a group of stakeholders, from DWP, other government departments and the Devolved Governments. This adds valuable insight from subject-matter experts, which given the breadth of topics covered in the FRS is essential to support the knowledge and experience of the analytical team. Checks are made at both the 6-month and 12-month processing stages to allow any concerns to be fully investigated before release of the 12-month data.

After the FRS team’s validation, checks and cleaning of the data have been completed, stakeholders are presented with a summary of changes to the data and any issues that the FRS team have identified. The test dataset is shared with these stakeholders. Any further issues are then dealt with by direct discussion with the stakeholder. There may be a further issue of a revised test dataset, before the data is declared to be final, and ready to use for publication of analysis.

DWP has an ongoing dialogue with expert users of the FRS-based statistics in relation to several data issues. DWP expects this to continue in the future as part of its long-term work programme.

External assessors

Third-party quality assurance is provided by the Institute for Fiscal Studies (IFS) under contract (which focuses on HBAI and grossing aspects). The aim of the independent validation is to discover and correct inconsistencies in the Households Below Average Income (HBAI) estimates. IFS runs independent checks on the HBAI dataset and all key inputs (e.g. population estimates) and replicates the DWP estimates for each publication year.

IFS has over many years used the FRS, alongside HBAI data to build and maintain a tax and benefit micro-simulation model, which it uses to estimate the distributional impacts of tax and benefit policies. They are therefore ideally placed to act as authoritative quality assurers of the FRS and HBAI data.

IFS works with the DWP team to ensure data issues are resolved. The process is iterative, involving investigation of the causes of the differences (if necessary) by both parties, agreement on revision of processing code, implementation of those changes, and a further round of cross-checking to confirm that differences have been eliminated.

DWP has also established an Expert Advisory Group on Survey-based Income Statistics to support its development work. This is in line with the User Engagement Strategy for Statistics released by the GSS. The purpose of the Group is to provide advice to the Chief Statistician on plans to implement the integration of administrative data into the FRS and related outputs and other technical issues as they arise. Members of the Group include regular users of the FRS and its related outputs, including academic experts, users from third-sector organisations and methodology input from ONS.

Strengths

• Investment has been made into the development of the upgraded interface processing tool – FRESCO
• A wide spectrum of checks, totalling more than 100, is carried out for the internal consistency of certain variables and values.
• The FRS team has the strength of experience and perspective from different analytical professions
• The benefit amounts extracted from administrative sources, for the purpose of editing benefits data, are judged to be very accurate and therefore add to the quality of the final FRS dataset
• An Expert Advisory Group provides knowledge and views on developments
• DWP has an ongoing dialogue with expert policy and academic research users of FRS-based statistics
• Independent quality assurance has been provided by the Institute for Fiscal Studies (IFS) for many years

Limitations

• With over 250 derived variable codes to review each year for input changes there is a risk of error. On the rare occasions that these have been identified after release of the data, revisions have been made and users informed as soon as possible. DWP analysts note lessons learned and continue to develop processes to minimise risk of recurrence
• The use of administrative data for editing benefit amounts is limited by the availability and access to the most appropriate administrative dataset, some of which are not DWP owned. Data sharing agreements are required to obtain access to data from other government departments

7. Summary

FRS accredited official statistics have been assessed as being assured to level A2 [Enhanced Assurance], as aligned to the UK Statistics Authority QAAD toolkit.

In constantly seeking to improve FRS accredited official statistics, steps will be taken to mitigate the limitations identified in this report, and progress will be communicated to users via the Release Strategy or Background Information & Methodology accompanying the main published report.

If you are of the view that this report does not adequately provide this level of assurance, or you have any other feedback, please contact us via team.frs@dwp.gov.uk with your concerns.