NayaOne’s AI Sandbox 

NayaOne is a Sandbox-as-a-Service provider to tier 1 financial services institutions, world-leading regulators, and governments.

From:
Centre for Data Ethics and Innovation and Department for Science, Innovation and Technology
Published
9 April 2024
Use case:
Natural language processing and generation, Image recognition and video processing, Machine learning, Deep learning, Virtual agents or artificial conversational interfaces
Sector:
Energy & Utilities (SIC Code Sections D & E), Retail (SIC Code Section G), Digital & Comms (SIC Code Section J), Financial and Insurance (SIC Code Section K), Professional, Scientific & Professional Activities (SIC Code Section M), Public Administration & Defence (SIC Code Section O)
Principle:
Safety, security and robustness, Appropriate transparency and explainability, Fairness, Accountability and governance
Key function:
R&D, Human Resources, Marketing and sales, Customer services, Risk management, Strategy and corporate finance
AI Assurance Technique:
Compliance audit, Performance testing, Risk Assessment, Impact Assessment, Impact Evaluation, Bias Audit
Assurance Technique Approach:
Technical, Procedural, Educational

Background & Description

NayaOne is a Sandbox-as-a-Service provider to tier 1 financial services institutions, world-leading regulators, and governments.

Creating a sandbox that serves as a secure environment for government departments, regulators, and regulated entities to test data and models is crucial in ensuring the ethical and responsible use of artificial intelligence (AI).

This sandbox is designed to address key concerns in AI deployment by providing a single environment where AI can be evaluated and procured while also enabling collaboration and access to world-leading tools.

  • The sandbox provides a controlled setting where entities can securely test data and AI models, simulating real-world operations without the risk of affecting actual operations or exposing sensitive information.

  • The sandbox is a secure environment that does not connect to the regulator, regulated entity, or government network ensuring they can safely and rapidly procure and evaluate AI models that they wish to use in the organisation.

  • The setup comes preinstalled with performance and stress testing capabilities as well as advanced AI tools including but not limited to:

  1. Detecting bias within datasets and models, helping entities identify and mitigate unfair biases that could lead to discriminatory practices.
  2. Understanding how their models make decisions, ensuring transparency and accountability in AI operations.
  3. Built-in functionalities to monitor for model drift, the sandbox helps ensure that AI models remain accurate and relevant over time, adjusting to new data and changing environments.
  4. The prevention of label leaking is a core feature, safeguarding against the inadvertent use of information in training data that unfairly influences the model’s performance.
  5. For testing Generative AI models, the sandbox includes specific tools to detect and analyse hallucinations and accuracy, ensuring that the generated outputs are accurate and reliable.

  • The environment supports a wide range of AI models and data types, catering to the diverse needs of government sectors, regulatory bodies, and regulated entities.

  • Collaboration tools within the sandbox enable teams and third parties to work together on testing and refining their AI systems, fostering an environment of shared learning and improvement. This can be done privately using wider TechSprint and Hackathon models.

  • Comprehensive reporting tools provide detailed insights into test results, highlighting areas of concern and recommending improvements.

  • It allows for the testing of models under a variety of conditions and scenarios, including stress tests to evaluate performance under extreme or unexpected data conditions.

  • Regular updates and support ensure that the sandbox remains at the cutting edge of AI testing technology, incorporating the latest research and tools.

How this technique applies to the AI White Paper Regulatory Principles

Safety, Security & Robustness

The sandbox environment is  secure and safe and remaining disconnected from the regulators/regulated organisations.

Appropriate Transparency & Explainability

The sandbox allows for the organisation that performs the testing to demonstrate why, or why not, they have chosen to adopt an AI solution. Testing for transparency and explainability are, by default, included in the AI sandbox.

Fairness

This can be tested for by default in the sandbox using a suite of tests and synthetic data.

Accountability & Governance

Financial organisations are adopting the sandbox as a means of implementing AI – both when procuring AI vendors and testing internally developed AI.

Why we took this approach

The sandbox approach means that all regulated entities, regulators, and governments can test and validate AI with a single secure environment which is disconnected from their production and internal systems.

Benefits to the organisation using the technique

Technical Benefits – while procuring AI:

  • Rapid procurement of AI models.

  • Secure access to deploy and test AI models and data to evaluate them.

  • Out of the box access to leaders in the AI and GenAI space – ready to test and procure.

  • Select from the NayaOne AI testing suite or bring your own evaluation criteria.

Educational Benefits

  • The tool educates developers on the best practises with AI ensuring responsibility and ethical design.

  • The tool enables execs to better understand AI and the benefits while also immersing them with hands on experience.

Procedural Benefits

  • Consistent and world-leading product development and risk evaluation on all AI tools being tested or procured through the sandbox.

Limitations of the approach

As a sandbox any techniques that the organisation wishes to add to the AI sandbox’s suite of tests can be employed. However, the sandbox requires the ability to access a version of the AI (whether it is developed in the sandbox, by the regulator/regulated organisation, or a third party).

Nayaone.com

Further AI Assurance Information

Published 9 April 2024
Contents