MoJ: PolyAI / Luna
PolyAI provides an AI-powered conversational voice assistant that automates routine telephone interactions and supports users in accessing services more efficiently.
1. Summary
1 - Name
PolyAI / Luna
2 - Description
Virtual voice assistant deployed to the Prison Enquiry Centre for five prisons to deal with frequently asked process questions (‘how to’) from prisoner friends and families.
3 - Website URL
N/A
4 - Contact email
CCT_Service_Management@Justice.gov.uk
Tier 2 - Owner and Responsibility
1.1 - Organisation or department
Ministry of Justice
1.2 - Team
Shared Services Client Team
1.3 - Senior responsible owner
Deputy Director, Shared Services Client Team
1.4 - Third party involvement
Yes
1.4.1 - Third party
SSCL (Shared Services Connected Ltd), PolyAI Ltd
1.4.2 - Companies House Number
SSCL - 8460577 PolyAI - 11048129
1.4.3 - Third party role
SSCL - Service provider, provided information relating to the FAQs and provided designs for the conversational flows. PolyAI - Technology provider, developed the tool alongside SSCL utilising information/data from SSCL.
1.4.4 - Procurement procedure type
PolyAI provided as a change request under the existing MoJ Call Off to the Independent Shared Services Centre 2 (ISSC2) Framework Agreement.
1.4.5 - Third party data access terms
SSCL is a data processor, acting on behalf of MoJ as data controller.
Tier 2 - Description and Rationale
2.1 - Detailed description
PolyAI’s current production deployment uses an intent-based conversational AI architecture to automate well-defined service interactions at scale. Its purpose is to handle high-volume, repeatable contact-centre queries accurately and consistently, while giving operational teams clear control over behaviour, escalation and outcomes. The intended users are end users contacting a service by phone or chat, plus operational teams who configure, monitor and improve the assistant.
In practice, a caller’s speech is transcribed, the utterance is classified into a predefined intent, relevant entities are extracted, and a deterministic dialogue manager decides the next action. That action may be to answer a question, collect further information or hand off to a human agent. Key functions therefore include intent recognition, entity extraction, disambiguation, conversation state tracking, response and handoff. This design is especially effective where the organisation wants predictable behaviour, audibility and strong policy control.
The scope of the intent-based solution is well suited to structured service journeys where user goals can be represented through a manageable set of intents, entities and business rules. Typical examples include information requests, appointment information, status checks, triage, and scripted transaction support. It is strongest when the organisation needs tight control, clear audit trails and consistent handling across a large volume of similar interactions.
Its limitations are that coverage must be explicitly designed and maintained. When a caller phrases a request in a completely new way, combines many goals in one utterance, or moves into an area outside the defined journeys, the assistant may need to clarify, redirect or escalate. It is also less suitable as the only solution for highly unstructured knowledge work or broad open-domain conversation.
2.2 - Benefits
The tool is expected to deliver several benefits, it reduces wait times by handling high volume repetitive queries that do not require specialist human input. It improves consistency by ensuring callers receive the same approved information every time. It supports staff by freeing up time to focus on complex or sensitive cases that require human judgement. It also helps to maintain service availability during peak periods enabling callers to obtain simple information quickly without queueing for an agent. Over time the system also provides insights through anonymised analytics helping to identify areas of improvement or where services can be streamlined.
2.3 - Previous process
The legacy process relied on a human agent for all query types from simple to complex/sensitive, with no intent detection calls are handled in the ordered queued which could mean complex or sensitive calls are queued whilst the simple FAQ type queries are being answered.
2.4 - Alternatives considered
Other tools were explored as part of this project all of which used similar intent based model to identify FAQs. The product was chosen due to it’s quality of conversational audio and market strength in conversational voice assistants.
Tier 2 - Deployment Context
3.1 - Integration into broader operational process
The tool is used as part of the telephony journey, when a user calls the service they are met with the virtual assistant, the user is asked about their query and the assistant aims to provide a response to a recognised FAQ, where a FAQ is successfully identified the information is provided to the user, where no FAQ is identified or where the intent is identified as a potential safeguarding issue or if the user explicitly asks for an agent the call is transferred to the next available human agent.
3.2 - Human review
The SSCL product owner and operational leads review the outputs through analytics dashboards on a daily/weekly basis using intent recognition accuracy, fallback rates and user handoff frequency, reviews ensure that FAQ content remains accurate and that intents reflect current service need. Significant service changes trigger updates to content or configuration.
3.3 - Frequency and scale of usage
The tool currently handles circa 2,900 contacts per month. The tool will handle roughly 60,000 contacts per month from citizens whether contained or handed off.
3.4 - Required training
N/A
3.5 - Appeals and review
The tool does not explicitly make decisions, an introduction message is played advising the user that they will be connected to a virtual assistant and can request a human advisor at any point.
Tier 2 - Tool Specification
4.1.1 - System architecture
PolyAI’s platform is deployed within AWS infrastructure in the UK, ensuring data residency and alignment with public sector requirements. The system is architected across multiple Availability Zones within the AWS region to provide high availability and resilience. This means that core services are distributed so that if one zone experiences disruption, traffic is automatically routed to healthy zones without impacting service continuity.
The platform is designed to be horizontally scalable, allowing it to handle significant fluctuations in call volumes, including peak demand periods often seen in public services. Stateless services and containerised components enable rapid scaling and failover, while stateful data is stored using managed AWS services with built-in redundancy and replication.
From a resilience perspective, PolyAI employs automated monitoring, health checks, and failover mechanisms to detect and respond to issues in real time. Disaster recovery strategies are in place, with backups and replication ensuring that critical data can be restored quickly in the event of a failure. This architecture supports high uptime requirements and ensures that the service remains available even under adverse conditions.
Security is integrated throughout the deployment, with encryption applied both in transit and at rest, and strict access controls enforced through AWS Identity and Access Management (IAM). This approach ensures that the platform meets the reliability, security, and operational standards expected for government-facing services. See technical documentation here - https://docs.poly.ai/home
4.1.2 - System-level input
Inputs include voice audio or text, together with any available contextual metadata such as channel, customer identifier, or call routing context and the response FAQs & knowledge base provided by the client. In voice deployments, the audio is transcribed before language understanding takes place.
4.1.3 - System-level output
Outputs include an identified intent, extracted entities (e.g. name, address, as selected by the client), dialogue-state updates, backend actions where applicable, a customer-facing response, and handoff metadata if the interaction is transferred. In operational terms the tool can also produce transcripts, summaries, analytics signals and structured event logs for reporting and monitoring.
4.1.4 - Maintenance
Intent-based deployments are maintained through a regular operating rhythm that combines live monitoring, transcript review, issue triage, test updates and periodic model or dialogue improvements. The exact cadence depends on the maturity and change rate of the service, but in practice review happens continuously, with more formal release cycles for tested changes into pre-production and production environments. Re-training is carried out where new data, emerging phrases, policy changes or performance trends justify it.
4.1.5 - Models
The intent-based tool includes several model and logic components rather than a single model. These typically include speech recognition, intent classification, entity extraction and rule or policy-based dialogue management. Depending on deployment scope, there may also be models or heuristics supporting disambiguation, call summarisation, analytics or routing.
Tier 2 - Model Specification
4.2.1. - Model name
For the intent-based solution, PolyAI would typically describe the relevant components by function rather than presenting them as a single branded model. The principal models are the speech recognition layer, the intent classification model and the entity extraction model, supported by deterministic dialogue policies.
The intent-based solution is primarily self-hosted or operated within PolyAI’s managed enterprise environment, depending on the agreed deployment model. The intent and entity models are part of the PolyAI platform rather than exposed as a standalone public API model in the way that consumer foundation models are.
4.2.2 - Model version
Versioning for the intent-based system is handled through PolyAI’s normal release and MLOps processes. Model updates, dialogue updates, configuration changes and integration changes are version-controlled and promoted through sandbox, pre-release and production environments. There is not typically a single customer-facing version number for the overall assistant, because performance depends on the combined configuration of models, policies, flows and integrations.
4.2.3 - Model task
The intent-based models are designed to determine what the user is trying to do, extract the details needed to complete that task, and guide the interaction through the correct service workflow. In practical terms, that means recognising requests such as checking status, updating details, rescheduling an appointment, asking a policy question or requesting transfer to a human.
4.2.4 - Model input
Typical input: a text utterance or Auto Speech Recognition transcript, optionally accompanied by metadata such as channel, language, customer context or current dialogue state.
4.2.5 - Model output
Typical output: an intent label, extracted entities, confidence signals, updated conversation state and the next system action or response.
4.2.6 - Model architecture
The intent-based model is a supervised language-understanding system designed to map utterances to a defined set of intents and entities. The dialogue layer is policy-driven and aligned to MoJ/SSCL policies for our service delivery, it uses explicit control logic to determine how the conversation progresses. This is a classic hybrid enterprise architecture: machine learning handles language variation, while deterministic logic handles business process execution.
The model type is primarily supervised machine learning for intent classification and entity recognition, plus deterministic rule or policy logic for dialogue management.
Methods used include transformer-based text classification, named-entity extraction, confidence scoring, disambiguation routines, state management and optimisation through transcript review and retraining. Operationally, PolyAI also uses extensive testing and dialogue tuning to improve real-world performance.
There is no simple published fixed weighting scheme that determines every output. In practice, outcome prioritisation is shaped by the learned classification model, confidence thresholds, dialogue state, explicit business rules and escalation logic. Where certain rules must take precedence for safety, compliance or operational reasons, those are encoded in the orchestration layer rather than left to model behaviour alone.
Publicly accessible material is available via PolyAI’s documentation and product pages https://docs.poly.ai/home These resources describe the platform and integration surfaces, although they do not expose proprietary models or internal logic in full.
4.2.7 - Model performance
Readiness for deployment is evaluated through a combination of offline testing, scenario-based validation, transcript review, integration testing and user-acceptance testing. PolyAI assesses whether the assistant handles the intended call types accurately, follows policy, fails safely and provides an acceptable user experience. Deployment readiness is not judged on a single metric alone; it is based on overall performance against the agreed service objectives and test cases.
Validation usually includes curated test sets, scripted conversation scenarios, edge-case testing, regression testing after each change, and pre-production user acceptance testing with the client team. Live-call review is then used after launch to confirm that actual behaviour matches the tested behaviour.
Typical metrics include intent accuracy, entity precision/recall or F1 (F1 is a single metric that measures how well an AI model balances precision and recall), completion or task-success rate, containment rate, appropriate escalation rate, average handling time and customer satisfaction measures where available.
Privacy and security are supported through controlled data handling, encryption in transit and at rest, role-based access, audit logging and environment segregation. Fairness is addressed through testing, review of failure patterns and operational monitoring to identify groups or journeys that may be underserved.
Graphs and tables are generally supplied as part of project evaluation, UAT reporting or service review packs rather than embedded in this narrative response. In practice, PolyAI commonly reports on measures such as containment, task success, recognition accuracy, handling time and reasons for failure.
4.2.8 - Datasets and their purposes
PolyAI uses datasets sourced from conversation text and audio derived from real or representative service interactions for model development and finetuning.
The intent-based system is typically developed using a combination of anonymised historical customer-service interactions, curated domain examples, synthetic training utterances, labelled intent datasets, entity examples and test scenarios. Different subsets are used for training, validation, regression testing and user-acceptance testing.
2.4.3. Development Data
4.3.1 - Development data description
Development data is sourced from conversational text and audio derived from real or representative service interactions. PolyAI augments real-world data with synthetic data where necessary to broaden coverage or improve specific system capabilities. These datasets are confidential and not publicly available.
4.3.2 - Data modality
Audio, text transcripts, conversational logs, and label/scores for training & evaluation purposes.
4.3.3 - Data quantities
For PolyAI’s in house LLM, the size of the dataset is hundreds of thousands of conversational turns across tens of thousands of conversations.
For deployment specific testing pre production, the exact testing dataset size will vary by deployment, user acceptance testing capacity, and by call type.
4.3.4 - Sensitive attributes
PolyAI employs advanced, privacy by design redaction to remove direct identifiers from both transcripts and audio recordings before use in development datasets. PolyAI also breaks up voice recordings into ‘snippets’ which are on average on 3 seconds long to make them harder to identify an individual from.
4.3.5 - Data completeness and representativeness
PolyAI’s models are trained on high quality, real-world conversations to ensure diverse and representative training data, including across different genders, race, accents, dialects and speech impediments. Where necessary, PolyAI augments real-world data with synthetic data where necessary to broaden coverage or improve specific system capabilities.
4.3.6 - Data cleaning
Development data undergoes multiple processing steps to ensure quality, safety, and suitability for developing customer service agents, including:
- Redaction - removal of direct identifiers
- Filtering - selection of desired data distributions to improve specific service capabilities
- Labelling - annotation to provide efficient learning signals during system training and evaluation.
4.3.7 - Data collection
Development data is collected from real world service interactions to the extent contractually authorised by PolyAI customers and permitted by applicable law, or otherwise synthetically generated by PolyAI. Use of data collected from the operation of an AI tool is inherently compatible with AI development where such purpose is within the reasonable expectation of, and brought to the attention of, the data subject. PolyAI only uses data for AI development where contractually authorised by PolyAI customers and on the condition that they have provided all transparency notices to data subjects, and obtained consent where applicable. Data from real world service interactions is necessary to ensure that the development data reflects real world conditions.
4.3.8 - Data access and storage
Development data is stored for as long as necessary for model analysis and training and access is limited to authorised personnel only and aligned to data retention policies. Technical controls include encryption, segregation, access management and audit logging, supported by operational controls such as least-privilege access and approval processes. We also conduct ongoing vulnerability scanning, yearly pentests by third parties, and regular security training. PolyAI maintains ISO27001 and SOC-2 Type 2 certifications. Privacy enhancing measures include data minimisation (by limiting voice data to snippets), redaction of direct identifiers, and pseudonymisation.
4.3.9 - Data sharing agreements
Development data is stored in restricted access in PolyAI’s Amazon Web Services (AWS) S3 buckets. Data is shared with approved processors or infrastructure providers such as AWS, only where necessary to deliver services to PolyAI and subject to contract, data processing terms and the principle of least necessary access. Development data is not shared with any third parties for their own purposes and PolyAI does not sell development data for any third parties.
Tier 2 - Operational Data Specification
4.4.1 - Data sources
A user will state the reason for their call to the assistant, the assistant will check against the predefined list of FAQ’s when a matched intent is found the appropriate response is played back to the user, where no FAQ is identified or where a trigger word that could relate to a safety concern is identified the contact is automatically routed through to a human advisor.
4.4.2 - Sensitive attributes
The tool does not act on or store any personal/sensitive data from the datasets.
4.4.3 - Data processing methods
No pre processing is completed by the tool, the tool only refrences a pre defined list of FAQs that are fixed responses where an FAQ is matched, where no FAQ is matched the user is handed to a human advisor.
4.4.4 - Data access and storage
Interaction logs are stored in line with SSCLs retention schedule and are used solely for monitoring and service improvement, logs are accessible to the SSCL product owner and Operational Leads for MI and Quality assurance purposes.
Recordings and transcripts may contain personal data only if the caller voluntarily provides it during the call no personal data is used to train the model.
PolyAI also have access to the logs for technical support/incident logging , this data is not used for general model training.
SSCL remains responsible for the data and no sensitive data is stored.
4.4.5 - Data sharing agreements
N/A
Tier 2 - Risks, Mitigations and Impact Assessments
5.1 - Impact assessments
Following ISSC2 contract governance protocols which include impact assesments etc. key dates below:
- Impact Assesment Jan 2024
- Business Continuity Review June 2024
- Data Protection Impact Assesment Jan 2025
- Delivery Assurance Jan 2025
5.2 - Risks and mitigations
Risks:
- Misclassification of caller intent
- Difficulty with certain accents, dialects or speech impairments
- Providing incomplete or out of date information
- Failure to detect safeguarding related statements
- Caller frustration if the assistant cannot resolve the query
Mitigations:
- Human in the loop escalation at any time
- Safety keyword triggers for immediate handoff
- Weekly reviews of intents and FAQ content
- Monitoring of fallback rates and escalation patterns
- Clear communication that callers are speaking with an assistant
- Ability for callers to request a human agent at any point
This tool does not make decisions that could result in discriminatory outcomes or affect legal rights.