Guidance

Quality statement: DWP benefits statistical summary

Updated 13 February 2024

This quality statement covers the statistics published by quarterly release of Department for Work and Pensions (DWP) Benefit Statistics; a collection brings together summary National and Official DWP statistics.

Data are available through Stat-Xplore for the following benefits:

• Attendance Allowance (AA)
• Benefit combinations
• Bereavement benefits (BB)
• Bereavement Support Payment (BSP)
• Carer’s Allowance (CA)
• Disability Living Allowance (DLA)
• Employment and Support Allowance (ESA)
• Housing Benefit (HB)
• HB flows
• Incapacity Benefit (IB)
• Income Support (IS)
• Industrial Injuries Disablement Benefit (IIDB)
• Jobseeker’s Allowance (JSA)
• Pension Credit (PC)
• Severe Disablement Allowance (SDA)
• State Pension (SP)
• Widow’s Benefit (WB)

We also publish the following official statistics on a quarterly basis via data tables as part of this release:

• Maternity Allowance (MA)

We have previously published National and Official statistics on the following benefit breakdowns twice a year in May and November via data tables within this collection:

• State Pension 5% sample

These data tables were suspended from August 2021 due to the source data not being representative following the introduction of a new DWP computer system. For more information on this issue see the background information note.

Introduction

The Department for Work and Pensions (DWP) is committed to producing accurate, timely, high quality official statistics publications that take into account user needs and which are produced and disseminated in accordance with the UK Statistics Authority’s (UKSA) Code of Practice. The 3 pillars of the Code of Practice are trustworthiness, quality and value. The quality pillar of is further split into these areas:

This document describes the ways in which the DWP benefit statistics publication meets each of these areas. Its focus is on the suitability and quality of the data used to produce statistics, but we also discuss their relevance, usefulness and presentation in section 3. The release brings together key National and Official statistics on DWP administered benefits. Monthly and quarterly statistics are published quarterly on our main dissemination tool Stat-Xplore.

There is more information available on the production of this release in the Methodology statement.

Separate methodology documents are available for:

• Universal Credit
• Personal Independence Payments
• Bereavement Support Payment
• Housing Benefit and Housing Benefit Flows

Their statistics are published separately, but are referenced in the DWP benefit statistics narratives to enable a fuller overview of DWP benefits.

This document should be read in conjunction with the Department for Work and Pensions policy statement on quality guidelines.

1. Suitable data sources

The statistics published in the DWP benefits statistics are based on administrative data sources which have been evaluated for quality and suitability.

List of Administrative Datasets

100% Quarterly Frozen Datasets

As detailed in our methodology document, the Frozen Datasets are a snapshot of the benefit system at a certain point in time, the last day of the quarter. They are created using a combination of spells data, which tell us when claims start and end, with other data with some rules applied to assure quality and check for problems.

5% Sample Data

The 5% data tables are taken from the backend of DWP computer systems, again with some consistency rules being applied. The data offer a wide set of variables and features, not all of which are available on the 100% sources. Where it is used, it is subject to sampling error and other issues around precision.

Single Housing Benefit Extract (SHBE)

The SHBE dataset is the primary source used to create HB statistics. It contains data about all HB and Council Tax Benefit claims, and is collated from returns from administering Local Authorities. Local Authorities use a range of software suppliers to provide the systems to administer HB and to provide data to DWP.

Data are collected using a well-defined set of specifications which helps to ensure both consistency and quality. The data returns are monthly and cover a range of different characteristics about the status of each claim, the personal characteristics of claimants, payable amounts and any deductions that may have been made. The data also provides information about which Local Authority is administering each claim.

Customer Information Service (CIS)

The Customer Information System (CIS) is a system used by DWP to collect information about customers. It provides the latest customer information including:

• personal details such as date of birth, gender and geographic information
• benefit awards

We use two different data feeds from CIS in the production of caseload statistics. Firstly, we use an address history file to provide residential based geographic information. Secondly, we use date of birth and gender fields from a weekly view of CIS data to verify the age and gender data supplied from our main data extracts.

Central Payment System (CPS)

The Central Payment System is an integrated payment and accounting system for the Department. It is used in DWP benefit statistics to verify the payment status of Attendance Allowance claims, and in quality assurance of other data such as State Pension and Jobseeker’s Allowance where appropriate.

Determining the suitability and quality of data sources using the Quality assurance of administrative data (QAAD) framework

The UK Statistics Authority have published a regulatory standard for the assessment of administrative data quality including a Quality Assurance of Administrative Data (QAAD) toolkit. The standard was developed in response to concerns about the quality of administrative data and in recognition of the increasing role that such data is playing in the production of official statistics.

UK Statistics Authority QAAD toolkit

In line with the QAAD toolkit, a review of the appropriate assurance level required for the DWP Benefit Statistics data sources has been carried out.

• A0 – no assurance

• A1 – basic assurance

• A2 – enhanced assurance

• A3 – comprehensive assurance

The UK Statistics Authority states that the A0 level is not compliant with the Code of Practice for Statistics. The assessment of the assurance level is in turn based on a combination of assessments of data quality risk and public interest. The toolkit sets out the level of assurances required as follows:

Level A1 – basic assurance

The statistical producer has reviewed and published a summary of the administrative data quality assurance (QA) arrangements.

Level A2 – enhanced assurance

The statistical producer has evaluated the administrative data QA arrangements and published a fuller description of the assurance.

Level A3 – comprehensive assurance

The statistical producer has investigated the administrative data QA arrangements, identified the results of independent audit and published detailed documentation about the assurance and audit.

To determine which assurance level is appropriate for a statistics publication it is necessary to take a view of the level of risk of quality concerns and the public interest profile of the statistics.

Each administrative data source has been evaluated according to the toolkit’s risk and profile matrix (Table 1), reflecting the level of risk to data quality and the public interest profile of the statistics.

Table 1: UK Statistics Authority quality assurance of administrative data (QAAD) risk and profile matrix

	Lower public interest profile	Medium public interest profile	Higher public interest profile
Low level of risk of quality concerns	Statistics of lower quality concern and lower public interest [A1]	Statistics of low quality concern and medium public interest [A1 or A2]	Statistics of a low quality concern and higher public interest [A1 or A2]
Medium level of risk of quality concerns	Statistics of medium quality concern and lower public interest [A1 or A2]	Statistics of medium quality concern and medium public interest [A2]	Statistics of medium quality concern and higher public interest [A2 or A3]
High level of risk of quality concerns	Statistics of higher quality concern and lower public interest [A1 or A2 or A3]	Statistics of higher quality concern and medium public interest [A3]	Statistics of higher quality concern and higher public interest [A3]

Source: Office for Statistics Regulation

Assessment and justification against the QAAD risk and profile matrix

The data risk of quality concern and public interest profile in DWP benefit statistics are rated by assessing firstly the possibility of quality concerns arising in the administrative data that may affect the statistics’ quality, and secondly the nature of the public interest served by the statistics.

The data quality concerns for DWP benefit statistics currently carry a low level of risk. The data are thought be a good representation of the caseload across the full range of benefits, with some usual caveats about how data are input onto the system. Data from multiple sources are combined using a well-defined rule set that ensures all claimants are correctly classified, with appropriate spells data showing when their claims began and ended. Additional data sources are used to verify personal characteristics such as age, gender and geography.

The DWP benefit statistics release covers a wide range of DWP benefits with varying degrees of public interest. However, given the most important benefits, the State Pension for instance, have very high levels of public interest, the statistics should be regarded as carrying the higher public interest profile.

Therefore, as defined by the risk and profile matrix (Table 1), the combination of low level of data risk concerns, and higher public interest profile indicate that the enhanced assurance level [A2] is the appropriate level required for the quarterly DWP benefit statistics release.

Operational context and administrative data collection

The vast majority of administrative data used within DWP is derived from the department’s own computer systems – in particular the various benefit processing and payment systems such as the Income Support Computer System (ISCS), the Jobseeker’s Allowance Payment System (JSAPS) and the Pension Service Computer System (PSCS) together with the Labour Market System which is used by Jobcentre Plus.

All IT systems are subject to scheduled service management reviews of key performance metrics where the department meets with IT suppliers to discuss incidents, problems, availability, planned changes and warranty of the delivered services.

Additionally, the DWP Standard Operating Model (SOM) assists staff and managers by providing detailed process maps for Benefit Centres, Contact Centres, Jobcentres, Pensions Centres and Disability & Carers delivery linked to procedural guidance.

Each Working Age (Jobcentre Plus) SOM is a collection of job roles, process maps and “must do” activities which link to supporting guidance. For Pensions, Delivery & Carers Service Delivery the SOM is represented by a number of components comprising of business services and components with some links to process maps regarding the process.

To ensure compliance with operational procedures, in addition to local checking, Assurance Officers within DWP Service Excellence Group undertake checks across a range of benefits. Claims are randomly sampled and will be checked for payment accuracy and compliance with operational procedures. The accuracy of information held on departmental systems is a central part of the checking methodology.

The above processes, tests and requirements aim to ensure the accuracy of data input, data processing and data output for administrative purposes. For most of the benefits covered by this release (with the exception of Housing Benefit Industrial Injuries Disablement Benefit, Maternity Allowance and Bereavement Support Payments) data from different computer systems are cross-referenced with one another for consistency. General Matching Service (GMS) provides rules to highlight inconsistencies with the data held by the DWP when compared with our customer’s cases. It is primarily a tool used to identify potential fraud and error on customer cases but it provides an excellent way of ensuring our data are coherent and consistent.

In relation to the DWP benefits statistics release, it is worth bearing in mind some of the key limitations in the application of administrative data:

• Data are typically collected to fulfil the administration of the benefits and not for the purposes of statistical production
• The way fields are input for individual claims onto a computer system may not always be exactly consistent with the intended meaning
• Operational coding systems may not be suitable for publication and may require further interpretation
• On some occasions errors are made in the recording of claimant details
• Prioritisations in the way cases are managed by operational colleagues may lead to inconsistencies with how data are recorded
• Interruptions and temporary easements to the standard models of business operation can lead to gaps or changes to the way data are collected and interpreted by data users
• Other timing issues may present that might lead to potential counting errors if not accounted for in the production of statistics
• For a small number of claims, clerical case or special rules handling means that no administrative dataset is able fully capture every single claimant

Specific details of limitations with the data and processes are footnoted on data tables in Stat-Xplore, or included on the publications collection pages on GOV.UK. More detailed explanations of limitations are always included in the Background information note or discussed as appropriate in our statistical summaries.

Communication with data supply partners

This relates to the need to maintain effective relationships with suppliers (through written agreements such as service level agreements or memoranda of understanding), which include change management processes and the consideration of statistical needs when changes are being made to relevant administrative systems.

As detailed in our methodology document, a close collaborative partnership exists between the two strands responsible for data supply and statistics production – Data as a Service and Data as Statistics. These relationships are underpinned by appropriate memoranda of understanding, and there is a change mandate process for any variations to our working relationship so that all parties can consider the impacts of any changes.

Data Delivery Team and Data as Statistics maintain stakeholder relationships with a wide range of internal data users, analysts and policy colleagues to ensure that the data they supply are relevant and accurate.

In relation to this release, the strength of these relationships were demonstrated by recent changes to the administration of State Pension data and our User Acceptance testing of a new data feed. We also have recent examples of working collaboratively to accommodate changes to Finance coding changes that impacted Central Payment System information for claimants living in Scotland.

Data as a Service will help us to investigate changes to the data where our own quality assurance highlight unexplainable differences or where clarifications of the meaning of data are required. We benefit from access to a Service Gateway Team within Data and Analytics who provide support to analysts where there is a query on the content of data presented or where changes to data volumes or context require explanation and where Data and Analytics may need to return directly to the source of the data.

2. Assured quality

Quality Assurance principles, standards and checks by data suppliers

Our principal data suppliers within Data as a Service, the Data Delivery Team (DDT), maintain a series of quality assurance checks on all of the raw data scans they produce and the processes they follow to combine data into the Frozen Data scans we use for publication. We work collaboratively with DDT to ensure that our 100% Frozen Data program codes run successfully. Also, each quarter, we work together to audit trail and test our coding rules to ensure that case retention, data cleaning and case removal are in line with benefit rules and any policy changes.

The supply of Housing Benefit data is accompanied by detailed reports from DDT that show breakdowns for each of the key variables used in the production of statistics and details of any gaps in coverage. This is particularly important as data are collected from multiple sources throughout a four-week period.

Producer’s quality assurance investigations and documentation

Quality assurance of the 100% Frozen Datasets is a largely automated process carried out in SAS with a manual analysis of output. Before the automated phase can begin, analysts will check SAS production logs to ensure the Frozen Datasets were created free of any known errors and that the coding has referenced all of the data sources correctly.

The automated program aims to check for various features within the datasets compared to the previous version, such as missing variables, new decodes, missing values, avoidance of any duplicated cases, and changes to the distribution of variables. The results are stored away in a lookup file for use in future quarters and are presented in the form of an html report. Members of the team assess these reports and look for any potential issues within the datasets. All potential issues and suspect movements in the datasets are raised and investigated. It is often the case that movements can be attributed to change in benefit policy and are fully expected.

Types of tests include:

• caseload numbers which are checked for trends
• number of cases leaving the datasets since last quarter
• number of cases joining the datasets since last quarter
• check that the number of duplicated values is zero
• consistency checking such as the distributions over different variables (including missing values)
• Chi-Squared tests
• Kolmogorov-Smirnov two sample tests
• reports of which variables have changed the most since the previous quarter

Where necessary, additional longitudinal or time-series analyses are performed on relevant variables to check for consistency across a wider timeframe.

The lead statistician is responsible for ensuring all checking has been done and then signs off the statistical datasets. All actions are recorded on a Quality Assurance Toolkit, and are checked off once complete to ensure that no steps are missed, in line with Aqua Book principles.

Data features or concerns that have stood out in the assurance process or that have led to further investigation are scored using Red, Amber, Green (RAG) risk ratings in terms of their initial impacts on data quality, and then a secondary RAG rating applied to demonstrate that follow up action has been completed to resolve or explain any data issues.

Similar quality assurance activity takes place for Housing Benefit, Industrial Injuries and Disability Benefit and Maternity Allowance statistics.

Checking data on Stat-Xplore

Stat-Xplore is an online tool that allows the creation and download of customised statistical tables, and data visualisations through interactive charts. Stat-Xplore has also has a useful User Guide. Stat-Xplore is maintained by a central team within Client Statistics who co-ordinate the monthly and quarterly releases across a broad range of benefits and measures.

On a quarterly basis, once all of our datasets have been uploaded onto Stat-Xplore’s development server, statisticians will perform quality checks on the breakdowns, notation and presentation in order to satisfy ourselves that the process of hosting data has worked without errors or any other unintended consequences.

Stat-Xplore data are “perturbed” to hide the true value of data, as a form of disclosure control. However, provided the differences between the raw Frozen Data and the Stat-Xplore output fall within a specific range then the data tables can be signed off by the lead statistician. This type of checking is performed in Microsoft Excel. Additional checks of titles, footnotes and background guidance held on Stat-Xplore is also performed each quarter.

Quality indicators for input data and output statistics

In accordance with Aqua Book principles, the lead statistician maintains a data register that describes each of our source datasets, their usage and provides an assumption based risk rating against each. The risk rating is based on the stability of each assumption made against a data source and its likely impact on our publication should anything fail.

Policy changes, changes to the operating model or the introduction of new IT systems can all have an impact on the stability of data supplied and their interpretation through to the final published statistics.

By maintaining an assumptions based risk register, the team can ensure that sufficient priority is given to the right areas by maintaining strong relationships with data suppliers and other internal stakeholders. We use this as the basis for informing our users of any changes to the context or interpretation of statistics covered by our release. Where appropriate messages are displayed:

• on Stat-Xplore
• the Background information note: DWP benefits statistical summary
• Methodology statement: DWP benefits statistical summary

• Housing Benefit caseload and flows statistics: methodology statement

• on the collection page for the release

The risk and impact of quality issues on statistics and data can therefore be minimised to an acceptable level for the intended uses.

3. Sound methods

This section provides details about how case management and data production can impact on quality and timeliness of our statistics. It also describes some of the ways we ensure consistency in the statistical outputs we produce.

We have produced separate methodology documents for DWP Benefit Statistics, Housing Benefit and Bereavement Support Payments.

Timeliness versus Completeness

Going back to how benefit claims are managed, it may take some time for an initial claim a for benefit to result in a ‘live’ claim. Entitlement may also begin before the claim process starts, and some claim types are backdated. It can also take some time to establish the correct time period for when a claim should be considered as closed and ended. Also, some changes of circumstances can take time to appear on our data after they have been accounted for on our computer systems.

There are also timing issues in the provision of some of the General Matching Service datasets, and the data for benefits managed by Pension Service Computer System (PSCS) is only made available once every six weeks.

Statistics derived and underpinned by the National Benefits Database carry a natural time-lag of three months before spells data can be reliably produced. This time period is used to enable us to capture any backdating of claims ending and starting in the prevailing time period or other changes relevant to accuracy. However, in terms of our statistics releases, the time lag translates into a six-month delay before we can publish data.

An internal analysis of the trade-off between accuracy and timeliness was conducted by DWP analysts some time ago, and reviewed recently in relation to the introduction of New State Pension. It was found that without a period of retrospection, the accuracy of some payment breakdowns and the completeness of data could not be sustained. If we were to publish more timely data, then potentially we would need to continually revise some of our previously published data, which would have further implications for users.

In comparison, while similar considerations are at play for Housing Benefit data, analysis has shown that we can produce reliable HB statistics by combining just two monthly scans of HB data. There are no discernible enhancements to the quality of HB statistics by using a scans from other time periods. When we release HB statistics, they only carry a three-month time lag and no further need to revise estimates going forward.

A key weakness in carrying the six-month delay to our release cycle is that users will not be able to compare our statistics for legacy benefits against newer benefits that are replacing them. Universal Credit (UC) is replacing a raft of income related working age legacy benefits, while Personal Independent Payment (PIP) is replacing Disability Living Allowance. Both UC and PIP statistics are released with a greater degree of timeliness. This can lead to some UC and PIP figures being released as provisional and then undergoing further revision. However, the absence of a consistent approach means some analysts and users then need to wait before they can interpret the UC and PIP figures against any changes to their legacy benefits.

Coherence and comparability

As described in earlier sections, data are combined against other sources to validate and ensure their accuracy. As part of the General Matching Service, in the production of the National Benefits Database and in the production of the 100% data, cross-referencing helps to ensure that claimants’ details are represented accurately in terms of when claims were active, their claim’s payment status and in the recording of personal and protected characteristics.

As an example, it was shown that the payment status field for Attendance Allowance claims can misrepresent the number of claims in payment, and to maintain its accuracy it is necessary to use supplementary data from the Central Payment System to provide corroborating evidence.

Hierarchical cleaning rules are applied to 100% Frozen Data codes to remove cases that violate rules or present as being out-of-date. Deletion reason codes are applied to any cases being removed from our statistics so we can directly assess why they were removed from our statistics and can keep an eye on the numbers to ensure there are no adverse impacts on quality. Cleaning rules are especially useful where we know that people are likely to transition from one benefit to another, and where timing issues in the data may lead to contradictory evidence that we then need to manage.

Context and Interpretation

An important part of our role as statisticians is to maintain close working relationships with our Analytical Community and Policy experts. We hold regular stakeholder groups with internal experts to ensure that our statistics are representative of caseloads, and to make sure we have kept abreast of any policy developments that might have an impact on caseloads and average payment amounts. We can then relay the context of any such changes to users through either our statistical summaries or through the background and information note. 24-hour pre-release access to our statistical commentaries is provided to a limited number of key internal stakeholders across policy, analysis and communication functions as well as to ministers.

On some occasions we also network directly with operational colleagues to check the consistency in how data are being captured. Recent examples of variations to how some data features were captured over the coronavirus (COVID-19) pandemic are referenced in our background and information note.

Dissemination

Our preferred dissemination channel for data releases is Stat-Xplore. Data on Stat-Xplore allow users to produce bespoke breakdowns to meet their needs, and aim to drive consistency in outputs across a range of different benefit statistics and breakdowns. For instance, data carry the same levels of disclosure controls and are protected in a consistent fashion.

A good example of this is in the presentation of geographical hierarchies, where a common reference file is used to underpin their presentation in terms of coding, naming and ordering of the data.

Where possible, standardised coding systems from across Government are adopted in the dissemination of our figures. Examples include:

• Use of the National Statistics Postcode Lookup to present residential geographies
• Use of Standard Occupation and Standard Industrial codes in the presentation of industrial injuries statistics
• Use International Classification of Disease (ICD) for the presentation of medical condition grouping for incapacity benefits

Other data definitions either refer to the DWP operational standard or where data is derived or combined, our metadata provide details of their derivation, as agreed by our policy colleagues.

Metadata is publically available on Stat-Xplore detailing the meaning of fields and where appropriate metadata may carry specific caveats and quality statements in relation to the breakdowns provided.

Data as Statistics have recently published an accessibility statement regarding dissemination of statistics. Further accessibility guidance available to statisticians on the Government Statistical Service (GSS) website has been applied to our statistical summary and spreadsheet releases as necessary.

Summary of strengths and limitations

The following strengths and limitations of the data have been identified:

Strengths:

• Data collections are underpinned by an operating model
• Consistency in how computer systems collect and hold data
• Data sources are cross-referenced against other sources to ensure accuracy
• Supply chain is robust, dependable
• Quality checking is thorough and uses appropriate statistical techniques to identify any underlying problems in the data
• Close contacts with internal data supply and policy colleagues ensure the interpretation of statistics and any analysis is relevant and accurate
• Retrospection is used to maximise coverage and to ensure accuracy. Data are frozen to a specific time period and do not need further revision

Limitations and risks:

• Data coverage can never fully reach 100% of eligible caseload
• DWP can only collect data from customers where the information is relevant and proportionate
• The way data are recorded and structured may not always be totally suitable for the publication of statistics without further interpretation and manipulation
• Structural changes to the DWP systems and data redevelopments can be hard to implement, and any lags in data development for new IT systems may create gaps in data
• Interruptions in data collection can impact the data we receive – instance fewer face to face interviews with customers over the initial COVID-19 period
• Timeliness of data may not meet user requirements, and there is a natural trade-off between complete accuracy and timeliness. This can be particularly felt where comparable statistics are published with a greater degree of timeliness
• When set in a time-series, quarterly data may miss some details provided by monthly figures

Notice of changes

We commit to consulting with users before making substantial changes to the statistics we release. If it is necessary to make changes to the publication, release notices are published in advance to inform users of the nature and extent of the change, usually on the collection page and in the background and information note.

Periodic reviews

The production and publication processes are periodically reviewed by DWP statisticians to streamline and improve processing and assess the content of the publication based on user requirements. New quality assurance processes were introduced in 2018 to add context to the checks being performed. The use of newer versions of SAS statistical software meant that a deeper level of analysis was possible.

The statistics are periodically reviewed by the Office for Statistics Regulation (OSR), the regulatory arm of the UK Statistics Authority, to ensure they maintain the standards required for badged National Statistics. This review usually takes place approximately every 4 years. The most recent review was published in November 2020 and found that the statistics continue to meet National Statistics standards, subject to implementing certain improvements. The DWP action plan for achieving this was published in April 2021: Response to Office for Statistics Regulation report: DWP benefits statistical summary .

Further information

Please see the policies and statements for this release including the background information note for more details about changes and revisions.

Please also see wider Policies and Statements for DWP statistics

Find contact information and more about DWP statistics on the Statistics at DWP page.

Find future benefits statistics publication dates in our statistics release calendar.

Contact information and feedback

For more information about this statement please contact: benefits-statistics@dwp.gov.uk

DWP would like to hear your views on our statistical publications. If you use any of our statistics publications, we would be interested in hearing what you use them for and how well they meet your requirements.

Email benefits-statistics@dwp.gov.uk with your feedback. You may like to use this questionnaire when providing feedback

You can also join the Welfare and Benefit Statistics community on StatsUserNet. DWP announces items of interest to users in this forum, as well as replying to users’ questions.