Research and analysis

Effectiveness Assessment of Universal Credit Advances Model

Published 9 July 2026

Executive summary

The Universal Credit (UC) Advances machine learning model is used to identify advance requests predicted to be high risk. These requests are then referred for a fraud prevention intervention. This effectiveness assessment is informed by statistical analysis covering the period 1 April 2025 to 31 March 2026 of the live Advances model, which is set out in the Statistical Annex. This is experimental analysis so we will continue to iterate the process and methodology in future years, as well as how the findings of the analysis are communicated.

It is the Department’s assessment that there are minimal concerns about effectiveness, unfair treatment or detrimental impact on legitimate claimants arising from the Advances model given that:

• there are a suite of safeguards in place, including crucially that a human always reviews the evidence and makes a decision; the model is only used to support prioritisation
• the model is 2.5 times more effective at identifying high-risk advances than a random control group, increasing the precision of referrals and enabling human decision-makers to focus on the riskiest cases; a random approach would generate more false positives and avoidable checks
• there is minimal impact on payment timeliness for legitimate claimants

Given the factors set out above, the Department has determined it remains reasonable and proportionate to continue operating the Advances model as a fraud prevention control.

The Department committed to retraining the model to improve its performance and alignment with the latest Advances fraud risks. The model has been retrained, but further testing is required ahead of the retrained model going live to ensure it works effectively and efficiently. Once the retrained model has been deployed and sufficient data is collected, we will assess its effectiveness. It should be noted that model retraining is an activity undertaken in the normal course of maintaining and improving the model to ensure it remains optimised to identify fraud risk.

Purpose of the Effectiveness Assessment

1. The purpose of this Effectiveness Assessment is to assess whether it is reasonable and proportionate to continue operating the model as a fraud prevention control.

2. The assessment considers the results of the statistical analysis, alongside other factors, to review the extent to which any statistical disparity may represent a risk of discrimination, unfair treatment or detrimental impact on claimants. Other factors taken into consideration in this Effectiveness Assessment include:

• the size and nature of Advances fraud
• the consideration of using machine learning as a control to mitigate the identified fraud risk
• the safeguards in place for the design, development and operation of the model
• the impact of the model on claimants, including the timeliness of Advances payments for requests referred by the model

Size and nature of Advances fraud

3. The Department provided 1.3 million UC advances to new UC claimants in 2025 to 2026, with a total value of £700 million. As set out in the Official Statistics, we estimate that for 2025 to 2026 the monetary value of fraud and error on UC advances lies between £20m and £90m^{[footnote 1]}. Advances fraud was the subject of a National Audit Office report in March 2020^{[footnote 2]} setting out the challenge of UC Advances fraud. The machine learning model has been part of the response to this type of fraud and contributed to more than halving the estimated scale of the problem. The persistent nature of the fraud threat requires appropriate controls to tackle Advances fraud risk, such as the Advances fraud prevention model.

Consideration of the use of machine learning as a fraud control

4. It is a government manifesto commitment to reduce waste and safeguard public money. The Department has a clear obligation to protect public money and tackle fraud and error. It is not an option for the Department to do nothing where fraudsters seek to steal money from the Department for Work and Pensions (DWP).

5. To achieve this, the Department could mandate all citizens provide evidence to DWP, for scrutiny and verification, of all relevant circumstances before an Advance is paid. Given the number of people who need to access support from the Department each day, this would be inefficient for DWP and add additional steps in the process for all legitimate claimants.

6.The Department is therefore developing machine learning capability to tackle fraud and error. The UC Advances model is designed to risk assess requests for Advances. It is the only fraud and error machine learning model currently deployed at scale in live service.

7. This approach enables the Department to focus its efforts on reviewing and verifying UC Advances assessed to have the highest risk of fraud. This risk-based approach:

• minimises impact on the customer experience of legitimate claimants
• optimises the use of taxpayers’ money in delivering the fraud control
• optimises the effectiveness of the control at tackling fraud

8. The performance information for 2025 to 2026 demonstrates the model is 2.5 times more effective at identifying fraud risk than a randomised control group sample.

Design, development and operation of the model

9.The Advances model has been developed in a way that ensures compliance with data protection legal obligations, including the fairness, lawfulness and transparency requirements set out in Article 5(1)(a) of the UK GDPR, as well as having due regard to the public sector equality duty under section 149 of the Equality Act 2010. The Department’s Personal Information Charter (privacy policy) informs claimants their data may be used for the prevention and detection of fraud and protecting public funds. This includes detailing the type of data which may be used and a specific reference to artificial intelligence as a method by which data may be processed.

10.There is a suite of safeguards to minimise the risk of unfair treatment or detrimental impact on legitimate customers, irrespective of their protected characteristics, including:

• The protected characteristics of UC Advances claimants are not used to train the model, except for age since this directly affects the financial award.
• A blend of high-risk and control group referrals are sent for human intervention to mitigate against human bias. In addition, the DWP employee delivering the intervention does not receive a risk rating in the referral nor are they made aware the referral has been generated by the model.
• A decision to decline an Advance request does not prevent the same claimant from making a further Advance request and does not automatically result in the associated new UC claim being refused. Decisions on eligibility and entitlement for a new UC claim are a separate consideration.
• Regular effectiveness assessments, including statistical analysis, are conducted to identify any concerns of unfair treatment or detrimental impact on customers.
• We also conduct analysis of any impact on payment timeliness on legitimate Advances requests.

11.The ultimate safeguard in place is that there is always a human intervention and decision, with no automated decision making by the model. Advance requests identified as high risk by the model are referred to a DWP employee, who reviews all available and relevant information, to decide whether to approve or decline the request.

Performance of the model as a fraud prevention control

12. Performance monitoring of the Advances model confirms it is an effective fraud prevention control and more efficient than an untargeted approach. The model is 2.5 times more effective at identifying high-risk advances than a control group sample. It has delivered and continues to deliver measurable savings. Therefore, the model enables the Department to reduce fraud and protect the public purse effectively.

13.Analysis confirms the payment of Advance requests predicted as high risk by the model and subsequently approved by a human decision maker are not unduly delayed. The median payment delay is one day longer compared to Advance requests that are approved automatically, which is in line with the delay experienced by Advance requests that are subject to other fraud controls that are distinct from the model.

Assessment of statistical effectiveness analysis

14. In traditional equality analysis, a ‘good’ outcome is typically defined as one in which all groups have an equal likelihood of experiencing a given outcome, assuming all other factors are equal. This would mean no measurable disparity between groups, which may indicate that the process or system is operating without bias. However, the UC Advances model is designed to assess claim characteristics associated with higher fraud risk. Due to fraudsters misrepresenting their circumstances and because fraud risk is not evenly distributed across all claim types, some statistical disparities between groups should be expected. These disparities do not imply that any group is inherently more likely to commit fraud, and all disparities are subject to monitoring and review.

15. The statistical analysis for the UC Advances model uses 2 metrics to support our understanding of what a ‘good’ outcome looks like:

• Referral disparity — proportion of Advances risk-scored by the model that were predicted to be high risk. This disparity tells us the difference in the proportion subjected to human review.

• Outcome disparity — proportion of Advances predicted as high risk by the model that were confirmed to be fraudulent by a human decision maker. This is the true positive rate which we use as an indicator for the underlying fraud in a cohort.

16. DWP’s assessment is that, where these 2 metrics are consistent, this is an indication of a ‘good’ outcome. It suggests that the rate at which Advance requests were predicted by the model to be high risk is consistent with the true positive rate of fraudulent Advance requests for a subgroup. To note, fraudsters misrepresent their true circumstances and therefore a fraudulent Advance request may share common claim characteristics with legitimate requests from a specific group. A higher referral and outcome disparity, relative to the comparator group, does not imply any group is more likely to commit fraud.

17. Where there are inconsistencies between the measured disparities, that is a signal to explore the disparity further. It might be the disparity is considered reasonable because we assess the model is successfully targeting the fraud risk found within a group, again noting fraudsters misrepresent their circumstances and fraudulent Advance requests may share common claim characteristics with legitimate requests within that group. Alternatively, a disparity may be an indication it is appropriate to consider further action, e.g. retrain the model.

18. To determine whether there are differences between groups, we have adopted ‘relative likelihoods’ as the chosen statistical methodology, following best practice recommended by the Cabinet Office on equality analysis^{[footnote 3]}. Further information on the methodology can be found in the Statistical Annex.

19. Due to the limited availability of protected characteristic data for the high-risk cohort, age and nationality (included within the definition of race) are the only protected characteristics that have been measured. For other protected characteristics, no appropriate data sources could be identified. To better understand the behaviour of the model, effectiveness analysis has also been undertaken based on whether the claim is from a UC couple and if the claimant reported an illness. Further information on the data used can be found in the Statistical Annex.

Results

20. For the characteristic ‘reported illness’, there was consistency between referral and outcome disparity for claimants. The metrics are moving in the same direction; therefore, this is an indication that the model is working effectively to identify fraud risk – we have assessed this as a ‘good’ outcome.

21. The likelihood of non-UK nationals being referred by the model was higher than UK nationals. However, the likelihood that the Advance was rejected by a human decision maker following a referral was equivalent for non-UK and UK nationals. This indicates the model is not working as effectively as it could for this cohort.

22. Claimants in UC couples were less likely to be correctly referred by the model compared to those in single claimant contracts, despite being referred at similar rates overall. This suggests the model is not working as effectively as it could for this non-protected characteristic.

23. For the age characteristic, there was consistency between referral and outcome disparity for claimants aged 16 to 24, as the likelihood of being referred by the model and the likelihood of those referrals being correct are both similar to the 25 to 34 years-old comparator group.

24. However, for other age groups an increased likelihood of being referred by the model is inconsistent with either a similar or reduced likelihood of those referrals being correct compared to the comparator group. The evidence suggests the model is not working as effectively as we would expect for these age bands.

25. The Advances model has been retrained to improve its performance and, consequently, to reduce the disparities identified in the assessment of the live model. However, the retrained model requires further testing and evaluation ahead of being deployed into live service. As a result, it is not currently possible to determine whether a human decision-maker would approve or reject the Advances it will identify as high risk. Once the model has been deployed and sufficient data is available, we will assess the effectiveness of its actions.

The full statistical analysis is set out in the Statistical Annex: Ad hoc statistical release of the effectiveness analysis for the Advances model.

Assessment

26. It is the Department’s assessment it remains reasonable and proportionate to continue operating the UC Advances model because:

• Safeguards — there are a suite of safeguards in place protecting all legitimate claimants irrespective of protected characteristics to prevent detrimental impact, crucially that a human always reviews the evidence and makes a decision.

• Assessment of Statistical analysis findings — in last year’s Effectiveness Assessment the Department determined that it remained reasonable and proportionate to continue operating the UC Advances model as a fraud control, with only minimal concerns identified regarding discrimination, unfair treatment, or adverse impacts on claimants. In light of the factors considered in this year’s assessment, this conclusion remains unchanged, particularly as the Department has retrained the UC Advances model to address these concerns. Further statistical analysis will be completed, after the retrained model has been deployed and once sufficient data is available.

• Impact on payments — there is minimal impact on payment timeliness for legitimate claimants. The median impact on payment of UC Advances referred by the model, and subsequently approved by a human decision maker, is one day (compared to an Advance not assessed as high risk by any fraud control measures). The payment timeliness of the UC claim is not affected by the model.

• Performance of the model — the model minimises the impact on the whole Advances claimant population by focussing the additional fraud prevention intervention on the riskiest (rather than randomly selected) Advance requests, reducing unnecessary interventions for the broader claimant population. The model is 2.5 times more effective at identifying high-risk advances than a random control group selection.

27. Given the factors set out above, the Department has determined it is reasonable and proportionate to continue operating the model as a fraud prevention control until the retrained model is ready to be deployed. There is minimal concern of discrimination, unfair treatment or detrimental impact on claimants. Further statistical analysis will be completed, after the retrained model has been deployed and once sufficient outcome data is available.

Statistical Annex: Ad hoc statistical release of the effectiveness analysis for the Advances model

Ad Hoc Statistics

1. This annex is a statistical publication, presenting the statistical analysis that has been conducted to understand the impacts on groups with protected characteristics. This is a complicated process and there are limited precedents available, hence we expect to further develop the methodology in the future. This ad hoc statistical release has been produced in accordance with the principles set out in the Code of Practice for Statistics, ensuring transparency, integrity, and quality.

Equality Act 2010

2. Under the Equality Act 2010, a protected characteristic refers to specific attributes or traits that are safeguarded against discrimination. There are 9 protected characteristics: age, disability, gender reassignment, marriage and civil partnership, pregnancy and maternity, race, religion or belief, sex and sexual orientation. The definition of each protected characteristic is given in legislation. Characteristics or attributes that do not meet the definition of a protected characteristic are referred to as a ‘non-protected characteristic’ or simply a ‘characteristic’ in this report.

Equality data availability

3. Universal Credit claimants are asked to answer optional equality questions when making their claim to support disparity analysis to identify differences between protected characteristic groups. In line with the department’s approach to using UC equality data for published analysis, we only conduct disparity analysis for characteristics with a completion rate of at least a 70%, to mitigate the risk of non-response bias. Among claimants referred by the Advances model as high risk, the proportion who answered the UC equality questions with a response other than ‘prefer not to say’ did not meet the 70% threshold. As a result, we have not used this data source on this occasion.

4. We considered other sources of protected characteristics data that were available to us. The age of a claimant at the time they request an advance can be calculated from the date of birth they provide in their UC application. This source of age data directly corresponds to the definition within the Equality Act. Similarly, the nationality of a claimant, which is included in the Equality Act 2010 within the definition of race (which is a protected characteristic), is collected as part of their UC application. We do not have enough data that directly corresponds to any other protected characteristic as defined in the Equality Act 2010 for claimants referred by the Advances model as high risk. Response rates for equality data in this release may be lower than those reported in other published statistics because the 70% completion rate must be reached for cases identified as high risk by the Advances model, as well as the majority of cases that are not. See the Universal Credit statistics: background information and methodology for more information about response rates.

5. In the interest of better understanding the characteristics of claims referred by the model we have conducted disparity analysis for some non-protected characteristics: whether the claim is from a UC couple contract and if the claimant reported an illness. The disparity analysis of these non-protected characteristics cannot be used as an approximation for any protected characteristics. Listed below are the reasons these characteristics do not meet the definition of a protected characteristic:

• Whether the claimant is part of a UC couple contract or not is not an approximation for marriage and civil partnership. The relationship between the claimants in a couple contract may not have the legal status of marriage or civil partnership (as referenced in the Equality Act 2010).

• The claimant’s response to the question “Do you have any disabilities, illnesses, or ongoing conditions?” which is asked as part of the UC claim process, can include declarations of illness which may not meet the definition of disability under the Equality Act 2010.

Data used in analysis

6. All UC Advances risk-scored by the model between 1 April 2025 and 31 March 2026 were considered in the analysis. Where a claimant or contract had more than one advance request during the period, one advance was randomly selected to ensure each observation was independent and to avoid bias from multiple requests by the same claimant.

7. We have considered 2 measures of disparity:

• Referral: The dataset used to calculate Referral disparity contains UC Advances that were requested between 1 April 2025 and 31 March 2026 and risk-scored by the model (limited to at most one advance per claimant and contract).

• Outcome: The dataset used to calculate Outcome disparity contains UC Advances, that were requested between 1 April 2025 and 31 March 2026, risk-scored by the model and were referred to a human decision maker because they were predicted to be high risk (limited to at most one advance per claimant and contract).

Relative likelihood and statistical significance

8. Relative likelihoods is a statistical technique carried out to identify any differences between groups and indicates the extent to which 2 groups differ in their likelihood of experiencing an event. It follows the methodology used as the standard approach for assessing disparities of outcomes by ethnicity, developed and recommended for use across government by the Cabinet Office (Race Equality Unit) published in 2020. It is used in other government departments for considering potential disparities of outcomes by ethnicity and by DWP in its benefit sanctions statistics. To calculate a relative likelihood, we use the following formula: Relative likelihood = percentage (or proportion) of one group experiencing an outcome, divided by percentage (or proportion) of comparator group experiencing an outcome.

9. To calculate a measure of relative likelihood, a comparator group must be chosen. We have chosen the group with the largest number of UC advance requests as the comparator. It is noted in the report published by the Race Equality Unit that the estimates are more robust when the largest group is chosen as the comparator. While the comparator group provides a useful benchmark for assessing relative outcomes, it is important to note that it does not necessarily represent an ideal or optimal standard. Rather, it serves as a reference point to highlight differences or disparities. The comparator may itself be subject to limitations, biases, or systemic issues, and should not automatically be interpreted as the group to emulate.

10. The closer a relative likelihood is to 1, the greater equality there is between the 2 groups. A relative likelihood greater than 1 suggests the outcome is more likely in the group that was compared to the comparator group. A relative likelihood less than 1 suggests the outcome is less likely than in the comparator group.

11. We have used significance testing, in the form of 95% confidence intervals, to test whether a relative likelihood is statistically significantly different from parity with the comparator group. In simple terms, if a difference is statistically significant it represents a real difference rather than being solely due to chance.

The size of disparity

12. Some difference in outcomes between groups are to be expected due to natural variation. As such it is essential to not only understand if such differences are statistically significant, and unlikely to be the result of chance alone, but also whether the scale of impact of those differences is meaningful. With larger sample sizes, significance testing has the power to detect small effects. Considering the scale of the effect size allows us to assess whether such differences would have a notable materially meaningful impact. This then guides interpretation as to whether such differences require monitoring, investigation or action.

13.To test the effect size (i.e. the scale of the difference between the relative likelihood and parity with the comparator group), a four-fifths rule has been applied to the relative likelihood estimates. Any relative likelihood estimates that fall outside a range of 0.80 to 1.25 indicate the impact of the disparity is meaningful and is described in this report as “notable”. The four-fifths rule has been used by DWP in its benefit sanctions statistics and stems from guidance from the Race Disparity Unit.

Referral disparity

14. The model classifies claims as high risk, based on a risk scoring. We calculate the high-risk classification rate as the number of UC advances the model predicted to be high risk, divided by the total number of advances risk-scored by the model.

15. High-risk UC Advances are referred to a human decision maker. The relative likelihood of the predicted positive rate tells us how likely a group was to be referred by the model compared to the comparator group. For example, a relative likelihood of 1.2 indicates that group was 20% more likely to be scored as high risk by the model than the comparator group.

Outcome disparity

16. The UC advances identified by the model as high risk are referred to a human decision maker along with a random sample of low-risk cases. The cases subsequently denied payment by a human decision maker are considered correct referrals. To assess disparities in the proportion of referrals correctly predicted by the model, we calculated the true discovery rate for each characteristic. The true discovery rate is defined as the number of UC advances correctly predicted by the model to be high risk, divided by all the advances it predicted to be high risk.

17. The relative likelihood of true discovery tells us how likely a group was to be correctly referred by the model compared to the comparator group.

18. It is not unexpected for a fraud prevention model to have measurable disparities and it is currently unknown whether it is feasible to design an effective fraud prevention model where referral disparity and outcome disparity are identical at all times.

Results

19. Age and nationality, as a component of race, are the only protected characteristics that have been measured. For other protected characteristics, no appropriate data sources could be identified. To better understand the behaviour of the model, effectiveness analysis has also been undertaken based on whether the claim is from a UC couple contract and if the claimant reported an illness. The disparity analysis of these non-protected characteristics cannot be used as an approximation for any protected characteristics.

Claimant age

20. The age group that had the largest number of Advances risk-scored by the model was the 25 to 34 year old group, hence it has been chosen as the comparator group.

Table 1: Relative likelihoods of referral by age group, compared to the 25 to 34 year old age group

Age Group	Relative Likelihood of Referral	Size of Disparity	Difference in Relative Likelihood
16 to 24	1.24	Not Notable	Statistically Significant
25 to 34	1	Not applicable	Not applicable
35 to 44	1.65	Notable	Statistically Significant
45 to 54	1.71	Notable	Statistically Significant
55 to 65	1.36	Notable	Statistically Significant
66 and over	27.24	Notable	Statistically Significant

Table 2:  Relative likelihoods of correct referral by age group, compared to the 25 to 34 year old age group

Age Group	Relative Likelihood of Correct Referral	Size of Disparity	Difference in Relative Likelihood
16 to 24	1.06	Not Notable	Not Statistically Significant
25 to 34	1	Not applicable	Not applicable
35 to 44	0.93	Not Notable	Not Statistically Significant
45 to 54	0.89	Not Notable	Statistically Significant
55 to 65	0.58	Notable	Statistically Significant
66 and over	0.36	Notable	Statistically Significant

21. There is consistency between referral and outcome disparity for claimants aged 16 to 24. For those claimants, the likelihood of being referred by the model and the likelihood of the Advance being rejected by a human decision maker following a referral are both not notable compared to the comparator group.

22. However, for other age groups an increased likelihood of being referred by the model is inconsistent with a reduced likelihood of the Advance being rejected by a human decision maker following a referral, with the disparity of correct referrals being notable for claimants aged 55 to 65 and 66 and over. This inconsistency requires further attention and will be evaluated in the retrained model, once sufficient data is available.

23. Claimants aged 66 and above were 27.24 times more likely to be referred by the model compared to claimants aged 25 to 34 years old. It should be noted that claimants aged 66 and above are an unusual group as they only include those eligible for UC through their partner or move to UC cases from tax credits, which represent only 0.1% of observations in the data analysed. As such, the results for this group should be treated with caution due to the small sample size.

Nationality

24. The group with the largest number of Advances risk-scored by the model were UK nationals, compared to non-UK nationals. Therefore, we have chosen UK nationals as the comparator group.

Table 3: Relative likelihood of referral for non-UK nationals compared to UK nationals

Nationality	Relative Likelihood of Referral	Size of Disparity	Difference in Relative Likelihood
Non-UK national	3.35	Notable	Statistically Significant
UK national	1	Not applicable	Not applicable

Table 4: Relative likelihood of correct referral for non-UK nationals compared to UK nationals

Nationality	Relative Likelihood of Correct Referral	Size of Disparity	Difference in Relative Likelihood
Non-UK national	1.04	Not Notable	Not Statistically Significant
UK national	1	Not applicable	Not applicable

25. The relative likelihood that a non-UK national was referred by the model is higher than a UK national, but the likelihood that a non-UK national was correctly referred by the Advances model is equivalent to the likelihood for UK nationals. This analysis shows an inconsistency between referral and outcome disparity.

26. In the interest of better understanding the characteristics of claims referred by the model we have conducted disparity analysis for some characteristics that are not protected characteristics under the Equality Act 2010. We cannot draw any conclusions about the disparity that may or may not exist for any protected characteristic from the following analysis.

UC couple contract

27. The group with the largest number of Advances risk-scored by the model were requested from single claimant UC contracts, compared to couple contracts. Therefore, we have chosen single claimant contracts as the comparator group.

Table 5: Relative likelihood of referral for UC couple contracts compared to single claimant contracts

Couple Contract Status	Relative Likelihood of Referral	Size of Disparity	Difference in Relative Likelihood
Couple Contract	0.98	Not Notable	Not Statistically Significant
Single Claimant Contract	1	Not applicable	Not applicable

Table 6: Relative likelihood of correct referral for UC couple contracts compared to single claimant contracts

Couple Contract Status	Relative Likelihood of Correct Referral	Size of Disparity	Difference in Relative Likelihood
Couple Contract	0.36	Notable	Statistically Significant
Single Claimant Contract	1	Not applicable	Not applicable

28. The likelihood that an Advance request from a couple contract was referred by the model was equivalent to the likelihood for a single claimant contract. However, the likelihood that the referral resulted in an Advance being refused by a human decision maker was lower for couple contracts compared to single claimant contracts. This analysis shows an inconsistency between referral and outcome disparity.

Reported Illness

29. The group with the largest number of Advances risk-scored by the model were requested by claimants who had not reported an illness, compared to those who did. Therefore, we have chosen claimants who reported an illness as the comparator group.

Table 7: Relative likelihood of referral for claimants who have reported an illness compared to claimants who have not reported an illness

Reported an Illness	Relative Likelihood of Referral	Size of Disparity	Difference in Relative Likelihood
Claimant has reported an illness	0.32	Notable	Statistically Significant
Claimant has not reported an illness	1	Not applicable	Not applicable

Table 8: Relative likelihood of correct referral for claimants who have reported an illness compared to claimants who have not reported an illness

Reported an Illness	Relative Likelihood of Correct Referral	Size of Disparity	Difference in Relative Likelihood
Claimant has reported an illness	0.73	Notable	Statistically Significant
Claimant has not reported an illness	1	Not applicable	Not applicable

30. There is consistency between Referral and Outcome Disparity for claimants that have reported an illness compared to those that have not. The likelihood of claimants with no reported illness being referred by the model and the likelihood of those referrals resulting in the Advance being refused by a human decision maker are both higher compared to claimants that have reported an illness.

Other relevant statistics considered in the effectiveness assessment process

31. A random sample of Advances predicted by the model to be low risk are sent to human decision makers to safeguard against confirmation bias and monitor false negatives. This is combined with a random sample of high-risk Advances to produce a single stratified random sample. The proportion of Advances predicted by the model as high risk and confirmed to be fraudulent by a human decision maker is compared to the proportion of Advances in the random sample that are confirmed to be fraudulent. This comparison finds the model to be 2.5 times more effective at identifying Advances that are fraudulent.

32. Advances predicted as high-risk by the model are referred to a human decision maker, who reviews all relevant and available information to decide whether to approve or decline the Advance. Payment is made if the Advance is approved. Advances requested between 1 April 2025 and 31 March 2026 that were risk-scored by the model and not referred by any fraud control had a median payment delay of 0 days, so were paid on the same day as the Advance request. Whereas Advances identified as high-risk by the model during the same period and approved by a human decision maker had a median payment delay of one day, so payment was made the day after the Advance request. This median payment delay is the same for Advances that were not referred by the model but were subject to other fraud controls.

1. Fraud and error in the benefit system: financial year ending (FYE) 2026 estimates - GOV.UK ↩

2. Universal Credit advances fraud - National Audit Office report. ↩

3. Using relative likelihoods to compare ethnic disparities - GOV.UK. ↩