Advai: Streamlining AI Governance with Advai Insight for Enhanced Robustness, Risk Management and Compliance

Case study from Advai.

From:
Centre for Data Ethics and Innovation and Department for Science, Innovation and Technology
Published
12 December 2023
Use case:
Big data analytics, Data-driven profiling, Machine learning, Deep learning
Sector:
Manufacturing (SIC Code Section C), Energy & Utilities (SIC Code Sections D & E), Retail (SIC Code Section G), Transportation & Storage (SIC Code Section H), Accommodation and Food Service (SIC Code Section I), Digital & Comms (SIC Code Section J), Financial and Insurance (SIC Code Section K), Real Estate (SIC Code Section L), Professional, Scientific & Professional Activities (SIC Code Section M), Administrative & Support Services (SIC Code Section N), Public Administration & Defence (SIC Code Section O), Education (SIC Code Section P), Healthcare & Social Work (SIC Code Section Q), Other Services (SIC Code Section S)
Principle:
Safety, security and robustness, Appropriate transparency and explainability, Fairness, Accountability and governance, Contestability and redress
Key function:
R&D, Risk management, Strategy and corporate finance
AI Assurance Technique:
Data assurance, Compliance audit, Formal verification, Performance testing, Risk Assessment, Impact Assessment, Impact Evaluation, Conformity Assessment, Bias Audit
Assurance Technique Approach:
Technical, Procedural, Educational

Background & Description

Advai Insight is a platform for enterprises that transform complex AI risks and robustness metrics into digestible, actionable insights for non-technical stakeholders. It bridges the communication divide between data science experts and decision-makers, ensuring that the management of AI risk and AI regulation compliance, is efficient and informed.

How this technique applies to the AI White Paper Regulatory Principles

More information on the AI White Paper Regulatory Principles.

Safety, Security & Robustness

The dashboard provides a comprehensive overview of AI risks, enabling proactive measures to ensure robustness and secure AI applications. It alerts users to deviations from expected or required results, enhancing security by flagging potential adversarial behaviour or model drift.

Appropriate Transparency & Explainability

Visualisation of complex data science concepts enables clear communication to non-technical stakeholders, ensuring transparency in AI operations.

Fairness

Insight helps to ensure that organisations have confidence that they will be alerted to deviations to fairness-related metrics. AI applications are continually monitored as they operate.

Accountability & Governance

Insight’s dashboard tracks compliance and risk, allowing for a high degree of accountability and governance in AI deployment and maintenance.

Contestability & Redress

The tool enables stakeholders to understand when and why to retrain, hold, withdraw or redeploy models, thereby offering avenues and justification for contestability and redress.

Why we took this approach

This approach was taken to empower senior stakeholders with the necessary insights to make informed decisions about AI applications without requiring them to have a technical background. Thus, governance and compliance are enhanced and risk is managed more effectively.

This creates a positive knowledge-action feedback loop. Senior stakeholders are enabled to provide technical teams with directives in the language of business objectives and monitor the technical progress towards these objectives in equivalent terms.

Benefits to the organisation using the technique

  • Facilitates informed decision-making at the senior level, improving governance and strategic planning.

  • Provides a ‘full picture’ of an organisation’s portfolio of AI models.

  • Enhances understanding and oversight of AI risks and performance across the entire AI estate.

  • Streamlines the compliance process, aiding in lawful and ethical AI deployment.

  • Aids in efficient resource management by enabling organisations to schedule resources for AI model retraining, deployment, analysis, etc. against the priority and risk level of each model. In other words, a full picture helps managers consider the big picture and prioritise where resource is needed.

Limitations of the approach

  • It may not fully replace the need for technical expertise in interpreting and acting on AI risks and compliance issues. At least, the coordination of risk, compliance and engineering resources falls to the organisation.

  • Dashboard insights are only as good as their connectedness to the organisation’s portfolio of algorithms. The organisation must ensure new models across the organisation are connected and old models are disconnected to preserve the accuracy of the dashboard.

  • Understanding nuanced aspects of AI performance may still require detailed explorations by technical experts whom would provide explanations beyond the dashboard’s capabilities.

Further AI Assurance Information

Published 12 December 2023
Contents