Deeploy: Ensuring transparency & good governance in anti-money- laundering and other high-risk applications

A case study of Dutch neobank Bunq and ML Ops start-up Deeploy

From:
Centre for Data Ethics and Innovation and Department for Science, Innovation and Technology
Published
15 December 2023
Use case:
Data-driven profiling, Machine learning, Deep learning
Sector:
Financial and Insurance (SIC Code Section K), Administrative & Support Services (SIC Code Section N), Public Administration & Defence (SIC Code Section O)
Principle:
Safety, security and robustness, Appropriate transparency and explainability, Fairness, Accountability and governance, Contestability and redress
Key function:
R&D, Product and service development, Risk management, Strategy and corporate finance
AI Assurance Technique:
Compliance audit, Performance testing, Risk Assessment, Bias Audit
Assurance Technique Approach:
Technical

Background & Description

Regulated industries like banking, insurance and healthcare struggle to safely deploy their AI models in ways that ensure transparency and proper governance. This case study zooms in on ways to safely deploy AI for transaction monitoring, with concepts being applicable to other use cases too. Core concepts include explainability to end users, using different explainability techniques like SHAP to help analysts understand the outcome of the models, and monitoring of the feedback loop to actively improve the model, and avoid bias and discrimination by flagging case unintended behavior of the model.

We followed five steps to build the explainer for the AI model being used for transaction monitoring:

  1. Define the Objective & Design the Explainer Output: Identify the specific explanations that would be valuable for end users. For instance, is the goal to clarify why a particular transaction was scored as high-risk?

  2. Choose an Explainability Method: Evaluate various explainability methods and frameworks such as SHAP, LIME, Anchors, Partial Dependency Plots (PDP), MACE, and many others, that all follow different ways of explaining an AI model. Depending on the objective, opt for a method that suits your needs. In this case, we’ve chosen SHAP to determine the most significant attributes influencing risk, which we translated into a top 10, including weights.

  3. Train or Configure the Explainability Method: train or configure the chosen explainability method. While some methods require training on the same data as the AI model, others, like integrated gradients, are configurable and non-trainable. Deeploy supports a wide range of those models. This process results in an explainer object ready for deployment.

  4. Deploy and Integrate the Explainer: Implement the explainer as a micro-service, integrating it into the backend of the AML application. This can be done easily using Deeploy, which covers lots of explainer methods. Utilize explainer endpoints to trigger the method. Consider the orchestration strategy—whether to calculate explanations in real-time or in advance. Additionally, decide whether to provide explanations for every transaction or only upon user request, bearing in mind computational costs and application performance.

  5. Enable Feedback Mechanism: Leverage the explainability feature to solicit user feedback. Implement methods to receive and process feedback, monitoring the loop for insights into algorithm enhancements. This not only refines the algorithms but also mitigates risks of bias or discrimination by empowering users to correct unintended model behaviors.

Relevant Cross-Sectoral Regulatory Principles

Safety, Security & Robustness

By making AI explainable to end users, transaction monitoring experts are able to understand, give feedback and correct AI models, therefore ensuring safe & secure use, and improving the performance of AI models, while reducing bias and discrimination.

Appropriate Transparency & Explainability

Transparency is a key concept in Deeploy, and is being used to explain red flags in transaction monitoring to internal experts. Furthermore, the feedback loop is being monitored, such that performance of AI models is transparent for everyone. The feedback loop gives clues where the model underperformance, for example if feedback often occurs in a subset of the data (let’s say in a given geography), the developers of the model directly know where to improve the model.

Fairness

By providing explanations, experts are able to give detailed feedback on cases of bias and discrimination, steering models in the right direction.

Accountability & Governance

Both accountability and ownership of models and predictions are clearly defined, making sure that there is always an owner accountable for model performance, and a human-in-the-loop for transactions being flagged by an AI model. This means an analyst in the anti-money-laundry team is checking every flag and either approves (sending the transaction to the FIU) or clearing the flag, giving feedback to the model why it was a false positive.

Contestability & Redress

Experts assess a small sample of non-flagged transactions to minimize the risk of false negatives. The transactions that are selected for assessment are anomalies, which allows humans to screen the most unusual transactions to identify new behaviour.

Why we took this approach

It’s crucial for banks to use AI safely for transaction monitoring. On the one hand, people could lose access to their bank accounts, not being able to pay their bills. On the other hand, criminal activity could be missed by missing behaviour in AI models. Therefore, using AI safely and transparently, always having a human in the loop, is crucial, to avoid both false positives (people losing access to their bank account for wrong reasons), and false negatives (missed cases of fraud or money laundering). Furthermore, given transaction monitoring is regulated, it’s key to be able to trace back every decision, including the feedback and overrules by experts.

Benefits to the organisation using the technique

Bunq benefited from (responsible) AI in multiple ways:

  • By using AI, both false negatives (missed cases of fraud and money laundering) are reduced significantly.

  • Time spent on cases of false positive was reduced by about 80% by using more accurate models.

  • By making AI explainable, the time used per case was reduced by almost 90%, as analysts understand how to interpret the outcomes of AI, and finish their research much quicker.

  • The risks are being reduced, by actively being in control (governance, compliance)

  • The accuracy of the AI models improved drastically by using a feedback loop with experts.

Limitations of the approach

Explainability is computationally heavy, meaning the costs for cloud increased and analysts don’t get an explanation right away.

https://www.deeploy.ml/case/banking/

https://www.deeploy.ml/product/

https://siliconcanals.com/news/startups/fintech/amsterdams-bunq-wins-aml-appeal-against-dnb/

Further AI Assurance Information

Published 15 December 2023
Contents