HMRC: Business Propensity to Pay
Predict business' likely engagement with tax debt recovery process.
1. Summary
1 - Name
Business Propensity to Pay
2 - Description
Business Propensity to Pay (PtP) uses machine learning to analyse past debt repayment behaviour to predict businesses’ likely engagement with the debt recovery process. The output from the model helps to segment debtors and informs the debt journey, including the frequency and content of letters sent by HMRCs Debt Management function. This helps to optimise the recovery of overdue tax.
3 - Website URL
N/A
4 - Contact email
Tier 2 - Owner and Responsibility
1.1 - Organisation or department
HM Revenue & Customs
1.2 - Team
HMRC Debt Management Campaigns and Remissions
1.3 - Senior responsible owner
Head of Campaigns and Remissions and Head of Debt Analytics
1.4 - Third party involvement
No
1.4.1 - Third party
N/A
1.4.2 - Companies House Number
N/A
1.4.3 - Third party role
N/A
1.4.4 - Procurement procedure type
N/A
1.4.5 - Third party data access terms
N/A
Tier 2 - Description and Rationale
2.1 - Detailed description
Business Propensity to Pay (PtP) analyses previous debt repayment behaviour and predicts likelihood of future payment using a gradient boosted decision tree.
Debts from VAT, PAYE and CT are first linked at a debt customer level. A gradient boosted decision tree model analyses historical data, including data on past debt repayment behaviours and predicts the likelihood of a customer repaying their debt in the near future. The initial output is a score from 0 to 1 which is then segmented and fed into downstream IT systems and along with other data is used to determine which debt collection journey a debtor will receive.
2.2 - Benefits
Improved debt journey by enabling Debt Management to tailor the initial direct customer communications, including timing, frequency and content
2.3 - Previous process
The Business PtP model superseded a simple model which segmented debtors based on how impacted they may have been by the COVID-19 pandemic. Prior to this, the majority of customers received the same debt collection journey instead of a more tailored one.
2.4 - Alternatives considered
Simpler methods (like logistic regression) were not found to be suitable for the dataset. A variety of supervised machine learning techniques were considered, specifically logistic regressions, decision tree based models and neural networks. Decision tree models were chosen over neural networks due to being explainable and less computationally expensive, The performance of different types of decision tree algorithms (single decision tree, random forest and gradient boosting) were compared, and gradient boosting was selected due to having the highest performance (accounting for the risk of overfitting).
Tier 2 - Deployment Context
3.1 - Integration into broader operational process
The Business Propensity to Pay (PtP) segment score helps to determine the debt collection journey a customer will receive. However, this is only one of the factors which determines the debt journey.
The PtP score is visible on the Debt Management debt collection system and HMRC staff may refer to it to provide them with additional insight when they’re reviewing individual cases.
3.2 - Human review
Model outputs are monitored and reviewed at an aggregate level on a monthly basis via a dashboard. Automated checks are in place to assess monthly change in scores
3.3 - Frequency and scale of usage
The model is run on a monthly basis and creates scores for approximately 4 million customers each month.
3.4 - Required training
Collectors have training and guidance to understand what the PtP score that they can see on their system means, and how they might want to take this insight into consideration to support their conversations with debtors
3.5 - Appeals and review
Members of the public cannot appeal decisions made by the tool, as the model output is not an outcome and no decision is made as a direct result of the PtP model. The debt collection journey would happen regardless of the tool and the PtP model is only one factor in determining the tailored part of the debt collection journey. At any point the customer engages we will agree an appropriate outcome, such as payment or instalments.
Tier 2 - Tool Specification
4.1.1 - System architecture
The algorithm is using the decisiontree.gbtreetrain CAS action in SAS Viya. Hyperparameter tuning was carried out manually. There was a balance found between performance and evidence of low levels of overfitting. The resulting model was calibrated using a logistic regression to make final scores interpretable.
The model is run entirely in SAS Viya. The final scores are outputted as CSVs.
4.1.2 - System-level input
Historical structured tabular data such as debt value, debt payment history, etc.
4.1.3 - System-level output
Final output is a likelihood to repay categorisation for each business debtor.
4.1.4 - Maintenance
Model outputs and outcomes are monitored at an aggregate level on a monthly basis via a dashboard. Automated monitoring and checks are in place to assess drift in input data over time to trigger an ad-hoc review. Otherwise, model is reviewed on an annual basis.
4.1.5 - Models
A separate linking process is used upstream of the Business PtP model to link debts across VAT, CT and PAYE.
Business Propensity to Pay model is a gradient boosted decision tree model built in SAS Viya.
Tier 2 - Model Specification
4.2.1. - Model name
Business Propensity to Pay
4.2.2 - Model version
Model version 1.0 is currently used. We monitor input variable drift and model performance on a monthly basis, which indicate when the model needs to be refreshed.
4.2.3 - Model task
Predict likelihood of a business engaging with the debt recovery process
4.2.4 - Model input
Historical structured, tabular data on debts and debt payment
4.2.5 - Model output
A percentage estimate of the business’ predicted willingness to engage with the debt repayment process
4.2.6 - Model architecture
The model is trained using a supervised machine learning process using gradient boosted decision trees and has been developed in SAS Viya.
Monotonicity is imposed and hyperparameter tuning developed in house to improve model performance.
4.2.7 - Model performance
Three different decision-tree based methods (single decision tree, gradient boosting and random forest) were compared. Mean squared error was used to assess training and validation datasets, and was compared between training and validation datasets to assess overfitting, showing that the gradient boosted model performed best.
Metrics used to support hyperparameter tuning were mean squared error, F1 score and Chi squared goodness of fit test, using an 80-20 split between training and validation. The 7 highest performing models were tested using 5-fold cross-validation. All top candidates had similarly low standard deviations of performance across the five folds. Therefore, the model with the strongest performance across all the 5 criteria was chosen and the final set of hyperparameters was tested against hold-out data. This was made up of two sections: one randomly chosen set of records from the months of data used for training, and the 2 most recent months of data that was not included as part of the training/validation data. Confusion matrices to visualise the distribution of successes and failures in each band were also used to monitor and assess performance.
4.2.8 - Datasets and their purposes
The data used to develop the model comes from HMRC debt data warehouse. An 80-20 split between training and validation were used to develop and test the model.
2.4.3. Development Data
4.3.1 - Development data description
ADePT - HMRC debt data warehouse
4.3.2 - Data modality
Tabular
4.3.3 - Data quantities
The model’s hyperparameters were tuned on 9.5m rows. Initial tuning used an 80-20 split between training and validation. The strongest candidates were further tested using 5-fold cross-validation on the same rows to select the strongest model. Once the final hyperparameters were chosen, actual performance was recorded against 1.2m records of hold-out data.
The current model (using this set of hyperparameters) was trained on 11.5m records, with a holdout dataset of 400k records to understand performance.
4.3.4 - Sensitive attributes
No personal data or protected characteristics used in the model.
4.3.5 - Data completeness and representativeness
Data is sourced from ADePT, HMRC debt data warehouse, data is structured and there are no missing data. Same data sources are used for training and inference, the data is representative of the target population.
4.3.6 - Data cleaning
Structured dataset from a controlled data source. Data preparation techniques such as one-hot encoding, pivoting, binning and grouping are performed on data to ensure that the data is in suitable format for model algorithm. Pre-processing involves filtering to eligible debts and linking debts across VAT, CT and PAYE.
4.3.7 - Data collection
Internal debt data warehouse used to process and manage debt recovery and repayments.
4.3.8 - Data access and storage
The data warehouse is accessible by Knowledge, Analysis and Intelligence (KAI), Debt Engineering and Analytics (DEA) and CDIO (Chief Digital and Information Office) Data Science analysts. Access is controlled and managed by DEA and requires security clearance.
4.3.9 - Data sharing agreements
N/A
Tier 2 - Operational Data Specification
4.4.1 - Data sources
HMRC debt data warehouse
4.4.2 - Sensitive attributes
No personal data or protected characteristics are input into the model.
4.4.3 - Data processing methods
Structured dataset from a controlled data source. Data preparation techniques such as one-hot encoding, pivoting, binning and grouping are performed on data to ensure that the data is in suitable format for input into the scoring model. Pre-processing involves filtering to eligible debts and linking debts across VAT, CT and PAYE.
4.4.4 - Data access and storage
Datasets used for scoring are created in a secure environment, with no personal or sensitive data included. Interim datasets are stored for a maximum of 30 days, and only accessible to those with access to the debt data warehouse. Final scores are stored for a longer period to allow historical analysis and performance monitoring, also using the same data warehouse.
The final set of scores is ingested into the system which determines customer journeys. A set of IDs and associated scores is downloaded from the secure environment and uploaded to the system, with the interim downloaded copy deleted.
4.4.5 - Data sharing agreements
N/A
Tier 2 - Risks, Mitigations and Impact Assessments
5.1 - Impact assessments
DPIA: Screening completed 19/11/2024 Full DPIA published 16/12/2024 Review completed 17/12/2025
The DPIA covers the relevant data processing activities for the data engineering and machine learning modelling.
5.2 - Risks and mitigations
If debts are linked incorrectly as part of the pre-processing or if the boundaries used to determine the PtP segment from the numerical score is incorrect, there is a risk the predicted Propensity to Pay segment score could over or underestimate a debtors likely engagement with the debt recovery process. This could lead to increased/decreased frequency of letters and different letter tone and content. However, all letters have been deemed to be suitable to any debtor and performance is monitored via a monthly dashboard and segmentation thresholds can easily be adjusted to mitigate this risk if an issue is identified.