Weights & Biases: The AI Developer Platform

Weights & Biases provides an MLOps platform to help organisations gain auditable and explainable end-to-end machine learning workflows for reproducibility and governance.

From:
Centre for Data Ethics and Innovation and Department for Science, Innovation and Technology
Published
9 April 2024
Sector:
Agriculture, Forestry and Fishing (SIC Code Section A), Manufacturing (SIC Code Section C), Energy & Utilities (SIC Code Sections D & E), Transportation & Storage (SIC Code Section H), Financial and Insurance (SIC Code Section K), Professional, Scientific & Professional Activities (SIC Code Section M), Public Administration & Defence (SIC Code Section O), Healthcare & Social Work (SIC Code Section Q), Arts, Entertainment & Recreation (SIC Code Section R)
Principle:
Safety, security and robustness, Appropriate transparency and explainability, Fairness, Accountability and governance
AI Assurance Technique:
Risk Assessment, Bias Audit

Background & Description

Weights & Biases provides an MLOps platform to help organisations gain auditable and explainable end-to-end machine learning workflows for reproducibility and governance. The system creates a single system of record for machine learning teams, tracking every detail of model training—from hyperparameters and code to model weights and dataset versions.

How this technique applies to the AI White Paper Regulatory Principles

Safety, Security & Robustness

The platform provides users with a reliable and scalable way to track all files in the ML workflow with flexible artefacts. This allows teams to trace the flow of data through their pipeline with a clear understanding of which datasets feed into which model(s). Data and role access controls are also available and can be asserted at the team and project levels to regulate and monitor access to sensitive data. For teams with strict compliance requirements, the secure storage connector is available to meet greater security and privacy needs, without undermining efficiency or accessibility.

Appropriate Transparency & Explainability

The platform allows for continuous insights into model behaviour and detection of bias, helping ensure explainability and transparency in the ML process which in turn can satisfy demands from regulators and provide clarity for internal and external stakeholders. The solution allows teams to easily document each component of building an interpretable model and easily share it with key personnel leading to real time oversight with AI activities across an organisation. By documenting the entire model lifecycle from initial concept to deployment, teams can reduce their risk, shorten the time it takes to debug models, and more easily explain their model and pipeline lineage.

Fairness

Models should be fair and free of biases that negatively affect individuals or groups. The W&B platform includes functionality for users to interactively explore their data and create custom charts and dashboards. This can help to enhance trust in AI systems and eliminate bias by helping identify the reasons for biased outcomes or data drift. Robust tooling allows for granular examination of model and data pipelines to uncover the root causes of detrimental model behaviour and facilitate faster debugging.

Accountability & Governance

Weights & Biases provides a centralised system-of-record for maintaining visibility and establishing accountability. A centralised model registry enables lineage tracking and full auditing capabilities on all actions performed on model versions. It acts as a secure and organised repository that’s accessible across an organisation, streamlining model development, evaluation, and deployment. Role-based access controls allow for organisations to partition access, ensuring that critical systems and models are available to the most appropriate people or teams.

Why we took this approach

Regulations surrounding AI are becoming increasingly stringent. They require organisations to remain adaptable and flexible to evolving regional or sector-specific constraints. Weights & Biases was designed from the ground up as a platform to ensure transparency and explainability across the ML lifecycle, providing end-to-end AI oversight. As models become increasingly complex, robust governance practices are no longer optional but required for scaling AI.

Benefits to the organisation using the technique

Weights & Biases is committed to help organisations responsibly deploy AI to advance their business objectives. Within the secure environment our platform provides, we offer tools that make models more explainable and freer from bias, inaccuracies, incompleteness and other harmful errors, as well as enabling greater accountability and helping companies meet compliance standards.

Limitations of the approach

Weights & Biases is best known for its experiment tracking capabilities. While the platform has functionality for a majority of the steps in a traditional machine learning pipeline, features around production monitoring and data labelling (or relabelling) are not present. However, the platform is extensible and integrates well with the commonly used tooling for these steps of the machine learning life cycle. Additionally, organisations with sensitive data (such as health or financial records) typically deploy on-premises or in a dedicated cloud environment. Weights & Biases employs a team of experts that can help integrate or stand the product up in complex or bespoke environments.

Further AI Assurance Information

Published 9 April 2024
Contents