Citadel AI: Citadel and BSI partner to provide comprehensive AI assurance techniques
Case study from Citadel.
Background & Description
Citadel AI, a provider of automated AI testing and monitoring tools, and the British Standards Institution (BSI), the global standards and business improvement company, partnered to ensure AI is used ethically, safely, and responsibly in industries that pose risks to humans. The partnership is designed to ensure that the AI used to help people in a variety of ways, from healthcare to autonomous driving, become more reliable, transparent, and responsible.
Through Citadel AI’s tools, as part of their assessment process, BSI can measure AI compliance against technical standards, supported by in-depth technical analysis including fairness testing, bias detection and robustness testing. They can also automatically generate comprehensive analysis and reporting for supporting the assessment process, assuring the safety and reliability of AI systems.
How this technique applies to the AI White Paper Regulatory Principles
More information on the AI White Paper Regulatory Principles.
Safety, Security and Robustness
Citadel AI’s tools help AI developers and auditors ensure that AI systems function in a safe, secure and robust way throughout the AI life cycle, from model development to deployment.
Citadel Lens, our model testing product, automatically stress tests AI models to reveal reliability issues, including robustness and safety under a wide variety of unexpected inputs and external conditions. For example, Lens can automatically assess AI systems against the standard ISO/IEC TR 24029, Assessment of the robustness of neural networks.
On the other hand, Citadel Radar, our model and data monitoring product, automatically detects and protects AI systems against real-world data problems that cause performance deterioration in production, such as data drift, model drift, and outlier/invalid data points.
Appropriate Transparency and Explainability
Citadel Lens, our model testing product, provides a model-agnostic toolkit to inspect the explainability and transparency of AI systems, providing human-interpretable explanations at multiple levels: individual model predictions, model predictions on a sub-population, and global model behaviour. Our model-agnostic explainability technology allows Lens to explain models regardless of internal architecture and across modalities: for example, for object detection systems, Lens can generate multiple explanations per input, one for each bounding box in an image.
Fairness
Citadel Lens, our model testing product, directly assesses the fairness of AI models and datasets to avoid inconsistent performance across different sub-groups of a population. For example, Lens can slice model error rates across demographic categories, such as gender, race, and age – even across intersectional categories such as gender × race. Additionally, Citadel Lens can assess AI systems against the standard ISO/IE TR 24027, Bias in AI systems and AI aided decision making, and generate a report for conformity assessments.
Accountability and Governance
Citadel Lens, our model testing product, helps AI auditors, accreditation bodies, and developers comply with regulations on AI such as the forthcoming EU AI Act and international technical standards for AI. Since our technology is model-agnostic, it can be used to automatically test a wide variety of AI models and datasets developed within an organisation and across organisations. From BSI’s perspective, this capability is required by BSI for conducting regulatory conformity assessments as part of their future role as an AI Notified Body.
Why we took this approach
We provide two products, Citadel Lens (used during the training stage) and Citadel Radar (used during the serving stage). This enables us to cover the end-to-end AI lifecycle from research to production. Additionally, our model-agnostic technology allows our products to test, monitor, and explain a wide variety of AI systems, even as the internal architecture of those models change over time and between organisations.
Benefits to the organisation
Ensuring the development of reliable and responsible AI is a key challenge of this decade, and delivering safe AI products is an important corporate responsibility and reputational risk for AI developers. Citadel AI provides automated testing and monitoring products for AI applications, to help organisations minimise reliability risks and maximise performance from research to deployment.
Our products are also used by AI auditors to assess the reliability, safety, and robustness of third-party AI systems. In light of the upcoming AI regulation, BSI is working towards gaining notified body status, and their work will be underpinned by the technical expertise of Citadel AI. This partnership will benefit not only the UK population but society across the globe.
Limitations of the approach
Citadel AI’s tools can automatically test and monitor a wide range of AI systems, algorithms, and datasets. There are, however, some limitations. For example, our tools do not yet support LLMs and generative AI models as of May 2023 (under development).
Further Links (including relevant standards)
- BSI and Citadel AI partner to solve the AI challenges of tomorrow
- ISO/IEC TR 24027, Bias in AI systems and AI aided decision making
- ISO/IEC TR 24029, Assessment of the robustness of neural networks - Part 1: Overview
- DigiARC-TR-2022-01: Machine Learning Quality Management Guideline
Further AI Assurance Information
-
For more information about other techniques visit the OECD Catalogue of Tools and Metrics: https://oecd.ai/en/catalogue/overview
-
For more information on relevant standards visit the AI Standards Hub: https://aistandardshub.org/