Nvidia: Explainable AI for credit risk management: applying accelerated computing to enable explainability at scale for AI-powered credit risk management using Shapley values and SHAP
Case study from Nvidia.
Background & Description
Explainability, defined in alignment with the OECD’s AI principles as “enabling people affected by the outcome of an AI system to understand how it was arrived at,” is one of the five values-focused cross-sectoral principles described in the Science and Technology framework. AI systems are often described as “black boxes,” but a growing portfolio of techniques exists to provide insights into how AI systems make decisions. SHAP (SHapley Additive exPlanation), based on the Shapley values concept used in game theory, is a popular explainable AI framework.
Following the introduction of SHAP in 2017, the financial services industry has explored using this approach to create explainable AI models for applications such as credit risk management. However, while the approach proved effective, it was too time consuming and expensive in terms of compute, infrastructure resources and energy, to be commercially viable at scale.
This case study focuses on the use of graphics processing units (GPUs) to accelerate SHAP explainable AI models for risk management, assessment and scoring of credit portfolios in traditional banks, as well as in fintech platforms for peer-to-peer (P2P) lending and crowdfunding. This approach has the potential to operationalise AI assurance at scale, enabling financial institutions to generate the explainability profile of entire portfolios in minutes rather than days.
How this technique applies to the AI White Paper Regulatory Principles
More information on the AI White Paper Regulatory Principles.
Appropriate Transparency and Explainability
Explainability is fundamental to the functioning of a values-based cross-sectoral regulatory approach. Where the use of AI could pose a risk to fairness, such as financial services, it is appropriate that regulators’ explainability requirements should be proportionately high. For AI systems to be accepted and trusted by the full spectrum of stakeholders they impact, both the technical processes of an AI system and related human decisions must be explainable. The Explainable AI (XAI) approach to credit risk management detailed in this case study allows practitioners in the financial services industry to conduct explainability performance testing and verification, building a narrative and explanation for machine learning-based decisions which meets business, regulatory, and customer needs, and drives AI adoption.
The Explainable AI (XAI) approach to credit risk management was designed to overcome the explainability gap of AI models in the financial services industry. It uses GPU accelerated Shapley values and SHAP. These approaches were used to help identify the decision-making criteria of an AI system in the form of variable importance (individual input variable contributions). This means they were used to find the relative contribution of different variables to the predictions made by an AI system.
The SHAP Clustering approach enables users to understand why a decision was made. The ‘why’ is not causal but expressed as numeric contributions of input variables. It’s possible to look at a specific data point or cluster and see the input variables, the variable’s contributions to the prediction, as well as the prediction itself. In doing so users could provide explanations for the predictions made by AI systems used in credit risk assessment and management, as well as in other financial areas like investment portfolio construction.
This approach also supports traceability, when combined with a mechanism of documentation to the best possible standard. This includes, among other things, the documentation of data sets used for training and validation of models, the process and outputs of any data labelling, and an unambiguous record of the decisions made by the AI system. The SHAP Clustering approach makes it possible to trace back and document the variable contributions to decision making. The clustering of SHAP information is one of the new pieces of information added by this approach which can be used to enrich the traceability and documentation. To further enhance traceability, the steps used to improve the model based on the new information can also be documented.
Model explanations generated using this approach can inform a wide range of people and roles including model builders, model checkers, compliance and governance officers, risk managers, product owners, senior managers, supervisors, customers, and regulators.
In addition to NVIDIA and Hewlett Packard Enterprise, more than 20 universities were involved in the project, as well as the European supervisory and financial service community, including the Gaia-X Financial AI Cluster (FAIC) project.
Why we took this approach
In early academic research, SHAP demonstrated great potential to enable relevant parties to access, interpret and understand the decision-making processes of an AI system. However, without GPU acceleration the technique was too slow for real-world commercial deployment and consumed an unfeasible amount of energy in the datacentre, as well as blocking costly compute resources. GPU acceleration enabled SHAP to run multiple orders of magnitude faster, accelerating the technique to the point it becomes commercially viable for financial institutions to deploy explainable AI at scale and facilitate AI adoption in the industry.
Financial supervisors are adjusting their approaches and skills to support the introduction of AI in banking. Banks must be clear about where humans fit in the oversight of the model and give supervisors sound explanations of what their AI systems actually do, as well as to what end. The decisions must be informed, and there must be human-in-the-loop oversight. This approach provides plausible explanations to reconcile the machine-based decisions with a human narrative that “makes sense”. The model can be better controlled as it delivers feedback on how it comes to all decisions both at the global level (global variable importance) and local levels (data points).
By maintaining a proper model pedigree, adopting an explainability method that provides clarity to senior leadership on the risks involved in the model, and monitoring actual outcomes with individual explanations, AI models can be built with clearly understood behaviours. This enables relevant life cycle actors to provide explainability information in the form and manner required by regulators, supporting transparency.
Benefits to the organisation
Using an accelerated computing platform means that this technique delivers explainability at scale quickly and reliably within a reasonable energy, time and resource budget. These factors are critical to enabling commercial deployment of explainable AI within the financial services industry, and therefore to industry AI adoption. In addition, these analytic capabilities and tools plus the interactive visual exploration enable a much better understanding of the outputs of an entirely black box model. Because the approach deploys techniques with which the financial services industry data science community is already familiar, the barrier to entry for adoption is low. Better understanding leads to more effective control, which supports regulatory compliance.
The approach also helps financial institutions assess risk more effectively, improve processing, accuracy and explainability, and to provide faster results to consumers and regulators. AI explainability techniques such as this one will be fundamental to the ability of financial institutions to deploy AI in ways which enable them to comply with current and forthcoming regulatory frameworks. Explainability techniques will be essential for financial institutions to benefit from the competitive advantage offered by AI in terms of increased revenues, reduced costs and improved customer service.
Limitations of the approach
SHAP techniques are computationally expensive. However, this issue is addressed through the use of GPU acceleration. Without significant effort during the training of the model, the results can be very sensitive to the input data values. Some also argue that because data scientists can only calculate approximate Shapley values, the attractive and provable features of these numbers are also only approximate, which reduces their value.
Further Links (including relevant standards)
- OECD AI Principles: Transparency and Explainability
- A Unified Approach to Interpreting Model Predictions
- Accelerating Trustworthy AI for Credit Risk Management
- Machine learning explainability in finance: an application to default risk analysis
- Financial Risk Management and Explainable Trustworthy Responsible AI
- Explainable Machine Learning in Credit Risk Management
Further AI Assurance Information
-
For more information about other techniques visit the OECD Catalogue of Tools and Metrics: https://oecd.ai/en/catalogue/overview
-
For more information on relevant standards visit the AI Standards Hub: https://aistandardshub.org/