Armilla Verified: Third-party AI Product verification
Armilla Verified is Armilla’s third-party verification of AI-powered products. It empowers AI vendors and enterprises alike to assure the quality and reliability of their AI solutions.
Background & Description
Armilla Verified is Armilla’s third-party verification of AI-powered products. It empowers AI vendors and enterprises alike to assure the quality and reliability of their AI solutions. Armilla Verified is a holistic, socio-technical assessment that has been derived from global best practices and standards for AI testing and risk management. The system leverages proprietary model evaluation technology to comprehensively, efficiently and cost-effectively validate the data sets, performance, fairness, interpretability, robustness and security of a given AI solution considering its wider context and business requirements. Eligible clients receive the Armilla Verified quality assurance seal as well as an independent expert report containing the results of the assessment.
Relevant Cross-Sectoral Regulatory Principles
Safety, Security & Robustness
Robustness testing is critical to ensuring that an AI model can be relied upon to produce accurate and reliable results in a variety of real-world scenarios, or that it is not overly sensitive to small variations or errors in the input data. Some of the approaches we have employed to evaluate robustness include looking at:
-
Adversarial attacks: We have evaluated performance under various types of adversarial attacks, including data poisoning attacks and model evasion attacks.
-
Data perturbation: We have tested the model’s robustness to different types of data perturbation, including adding noise, flipping pixels, and changing the intensity of the input data.
-
Hyperparameter tuning: We have evaluated the model’s performance under different hyperparameters settings, including learning rate, batch size, and regularisation strength.
-
Distributional shifts: We have tested the model’s performance under different data distributions, including different classes, domains, and environments.
Appropriate Transparency & Explainability
We rely on a variety of techniques to evaluate and increase the transparency and explainability of machine learning models, including feature importance analysis, visualisation tools, and model interpretability algorithms:
-
Permutation-based feature importance: this model inspection technique can be used for any fitted estimator when the data is tabular. This is especially useful for non-linear or opaque estimators. The permutation feature importance is defined to be the decrease in a model score when a single feature value is randomly shuffled. It measures the increase in the prediction error of the model after we permute the feature’s values, which breaks the relationship between the feature and the true outcome.
-
PDP and 2D PDP testing: A Partial Dependence Plot (PDP) is a visualisation that shows the impact of a feature on the predicted outcome. It shows the dependence between the target response and a set of input features of interest, marginalising over the values of all other input features. It can be used to analyse interaction between the target response and a set of input features.
-
Model sensitivity: We use a variety of sensitivity analysis approaches (including SOBOL, FAST and RBD FAST). These global sensitivity analyses work within a probabilistic framework, decomposing the variance of the output of a model or system into fractions which can be attributed to inputs or sets of inputs. These methods are often used to identify key parameters that drive model output.
-
Fingerprint comparability: An approach to reverse-engineer the model into an inherently explainable model. Simpler models are more successfully reverse-engineered, and regions where the models disagree correspond to risk in complexity corresponding to the lack of understanding of those regions.
We also use overlapping methods to understand the complexity of the model, including:
-
Contextual feature elimination: Feature elimination helps determine the unimportant features and can allow a model to perform better by weeding out redundant features and features that are not providing much insight. We utilise recursive feature elimination, which works by eliminating the least important features, to determine feature importance. Contextual feature elimination increments feature in descending order of feature importance and attempts to fit a surrogate model on the subset of features and the target variable. An estimation bias and variance is computed for each surrogate model and then compared with the performance of the base model. An optimal model is suggested with a subset of the features when the increment of the model performance between successive models is no more than 0.05%
-
Fingerprint complexity analysis: This approach interrogates the reverse-engineered model to determine the number of features and interactions required to create a well-behaved proxy to the original model
Fairness
Achieving model fairness can be challenging, particularly in cases where the data used to train the model contains biases or reflects existing inequalities. Fairness is not always a clear-cut or objective concept, and different stakeholders may have different opinions or definitions of what constitutes fairness in a particular context. Therefore, achieving model fairness may require careful consideration and engagement with a variety of stakeholders, including those who may be affected by the model’s predictions or recommendations.
There are a variety of approaches and techniques that we use to assess the fairness of a machine learning model, including: counterfactual analysis, fairness constraints, and metrics, including demographic parity, equality of odds, disparate impact, the four-fifths rule, group data bias and over 40 types of fairness tests.
Accountability & Governance
Some examples of governance practices that we follow to support Responsible AI include stakeholder engagement, independent frameworks, and oversight and accountability mechanisms.
-
Stakeholder engagement helps us ensure that the perspectives and concerns of diverse stakeholders, including affected communities, are considered during the assessment process.
-
Independent assessment frameworks. We leverage leading independent assessment frameworks for AI, such as the Responsible AI Institute’s certification framework, to help us evaluate the potential implications of an organisation’s AI systems and make informed recommendations about their development and deployment.
-
Oversight and accountability mechanisms, such as audits and impact assessments, can help organisations ensure that their AI systems are transparent and accountable, and that they are being used in a way that aligns with ethical principles and respects human rights.
Why we took this approach
As AI capabilities continue to accelerate, so do the risks for enterprises. Our third-party verification for AI products provides organisations with the confidence they need to unlock the potential of AI while mitigating risk whilst meeting emerging compliance obligations.
Benefits to the organisation using the technique
Our third-party AI model verification is a powerful quality assurance and risk mitigation tool for vendors of AI solutions and the enterprises that procure them.
Vendors of AI-powered products leverage Armilla Verified to:
-
Assure their systems through independent validation and trustworthiness;
-
Get ahead of evolving enterprise procurement requirements for AI;
-
Prepare for compliance with AI regulations and industry standards.
Enterprises procuring third party AI products leverage Armilla Verified to:
-
Evaluate AI risk specific to the business use case;
-
Quickly and efficiently weed out underperforming or immature vendors before they are onboarded;
-
Mitigate risk of downstream AI-related damages and liability;
-
Assure compliance with emerging AI regulations, and corporate policies and procedures.
Limitations of the approach
Based on the results of the assessment, we provide expected limitations of the model.
Assessment dependencies include the availability of a sufficiently complete, representative sample of training and test data supplied by the solution developer, which also forms part of Armilla’s verification and has direct impact on its results. Another dependency consists in Armilla’s ability to access the model in question, typically via an API endpoint.
Further Links (including relevant standards)
RAI Institute Certification Framework
ISO/IEC 23894:2023 Information technology — Artificial intelligence — Guidance on risk management
NIST AI Risk Management Framework
Further AI Assurance Information
-
For more information about other techniques visit the CDEI Portfolio of AI Assurance Tools: https://www.gov.uk/ai-assurance-techniques
-
For more information on relevant standards visit the AI Standards Hub: https://aistandardshub.org