Deeper Insights: Enhancing AI Explainability in Turtle Detection using Grad-CAM
Case study from Deeper Insights.
Background & Description
This project involved developing an AI model for detecting a particular marine species within their natural habitat, distinguishing them from marine debris and other aquatic life forms. The primary challenge was the complexity of the marine environment where the target species may be obscured or camouflaged.
The project aimed to apply AI for a specific aspect of marine conservation. Grad-CAM (Gradient-weighted Class Activation Mapping) was incorporated into the project to provide an interpretable layer for the AI model, aligning to ensure that the AI’s operational mechanisms were transparent and understandable.
As we progressed down the solution, the model’s performance was not completely understood. Due to the issues with marine life being obscured by other materials from the sea, we needed to introduce further explainability into the model.
To address these issues, Grad-CAM was incorporated into the convolutional neural network model to visually highlight the areas in input images most influential to the model’s predictions. This feature aids in understanding which aspects of the imagery the AI model focuses on when making its determinations allowing for greater interpretability of the model’s performance and results.
How this technique applies to the AI White Paper Regulatory Principles
More information on the AI White Paper Regulatory Principles.
Appropriate Transparency & Explainability
Grad-CAM was integrated into the AI model to provide a layer of transparency and explainability, revealing which aspects of the input imagery were most influential in the model’s decision-making process. This approach aligns with the need for clear and understandable AI actions, particularly in sensitive environmental applications.
Why we took this approach
The decision to implement Grad-CAM was driven by the necessity to build trust in the AI model among a range of stakeholders, including environmental scientists, data scientists, and regulatory bodies. The visual explanations offered by Grad-CAM ensured that the model’s decisions could be easily interpreted, fostering confidence in its reliability and effectiveness.
This transparency also allowed us to identify artefacts that were causing the model to make incorrect predictions, for which further development was required. Hereby the use of Grad-CAM supported further improvements in the accuracy of the model.
Benefits to the organisation using the technique
-
Enhanced Model Accountability: Grad-CAM’s visual explanations enable the organisation to demonstrate the AI model’s decision-making process, fostering a higher level of accountability.
-
Facilitated Stakeholder Communication: The visual aspect of Grad-CAM aids in effectively communicating how the model works to non-technical stakeholders, promoting greater collaboration and understanding.
-
Guided Model Improvement: Insights provided by Grad-CAM allow the technical team to identify and address any shortcomings in the model, leading to continuous improvement in detection accuracy.
Limitations of the approach
-
Surface-Level Insights: Grad-CAM provides basic visual insights into the AI model’s decision-making process but lacks depth in explaining the underlying computational logic. For in-depth technical analysis or diagnosing specific model issues, Grad-CAM may provide too general information.
-
Dependence on Visual Inputs: Grad-CAM’s effectiveness is limited to scenarios with visual inputs and may not fully explain decisions influenced by non-visual factors.
Further AI Assurance Information
-
For more information about other techniques visit the CDEI Portfolio of AI Assurance Tools: https://www.gov.uk/ai-assurance-techniques
-
For more information on relevant standards visit the AI Standards Hub: https://aistandardshub.org/