Skip to main content

DWP: DWP Ask

DWP Ask is a question & response tool designed to support users in finding information within guidance documentation more quickly.

1. Summary

1 - Name

DWP Ask

2 - Description

DWP Ask is a question & answer tool designed to help Working Age Services work coaches in quickly finding information within policy and guidance stored on DWP’s internal intranet.

3 - Website URL

N/A

4 - Contact email

Tier 2 - Owner and Responsibility

1.1 - Organisation or department

Department for Work and Pensions (DWP)

1.2 - Team

Transformation and Delivery, Working Age Services, Digital & Transformation group

1.3 - Senior responsible owner

Transformation & Delivery Director

1.4 - Third party involvement

Yes

1.4.1 - Third party

IBM United Kingdom Ltd Microsoft Ltd

1.4.2 - Companies House Number

00741598

1.4.3 - Third party role

IBM United Kingdom Ltd - algorithmic tool delivery partner Microsoft Ltd - technology provider

1.4.4 - Procurement procedure type

Competitive, Artificial Intelligence Dynamic Purchasing System (RM6200)

1.4.5 - Third party data access terms

N/A

Tier 2 - Description and Rationale

2.1 - Detailed description

The DWP Ask tool provides internal users (DWP agents) with quick access to relevant guidance. Users ask their guidance related questions to the tool and the Generative Pre-trained Transformer (GPT) Large Language Model (LLM) returns responses which includes a summary of the guidance information and source links to which guidance documentation the information was retrieved from. Every response includes a notice for users to check the accuracy of the information by clicking on the source links provided. Users use this information to help the better answer customer queries.

2.2 - Benefits

The tool reduces the time taken to search for up to date guidance by searching the internal DWP intranet and providing relevant information related to the user’s query.

2.3 - Previous process

N/A

2.4 - Alternatives considered

N/A

Tier 2 - Deployment Context

3.1 - Integration into broader operational process

DWP guidance is complex with multiple source documents often needing to be referred to, to find the appropriate information. DWP Ask brings all the relevant information at the users’ point of need quickly to enable better quality conversations with their customers.

3.2 - Human review

Users use the tool to find information within guidance more quickly. The users’ existing training on guidance equips them with the ability to understand and review the outputs. The tool’s output is always reviewed by internal agents. It is not citizen facing and is not a decision-making tool. It enables users to have better appointments with their customers.

3.3 - Frequency and scale of usage

The tool is currently being scaled to all job centres across Great Britain. Around 18,500 internal users will have access to the tool to support with their daily work.

3.4 - Required training

Before gaining access to the tool, users must complete online training on how to use the tool responsibly. Users also complete regular internal training on guidance topics that are included in the tool.

3.5 - Appeals and review

The tool does not make any decisions.

Tier 2 - Tool Specification

4.1.1 - System architecture

Users access the tool through a web application hosted on DWP’s internal shared Azure platform. Users enter their question into the web application which subsequently calls out to AI and OpenAI services. An appropriate response is generated and returned to the user.

4.1.2 - System-level input

Users input their questions into the tool and an appropriate response is returned with relevant source links included from the DWP intranet.

4.1.3 - System-level output

AI generated answers based on the question asked and relevant guidance document links retrieved.

4.1.4 - Maintenance

There are multiple reviews depending on the changes to the data and necessary re-training will take place once a month following guidance updates.

4.1.5 - Models

This tool uses the GPT-4o: LLM to generate answers to the user’s question. The tool also uses the BERT model to classify users’ question into one of the 7 applicable topic areas to support in retrieving relevant documentation required to generate the answer.

Tier 2 - Model Specification:

4.2.1. - Model name

GPT-4o Pre-Trained (no refinement) Hosted on Azure OpenAI Service

4.2.2 - Model version

2024-11-20

4.2.3 - Model task

Generates natural language answers to users’ questions about guidance.

4.2.4 - Model input

Input is a structured prompt containing multiple components (system prompt, user question, context/retrieved chunks, additional constraints and token limit) in a text string template specific format

4.2.5 - Model output

Output is a natural language text response

4.2.6 - Model architecture

Large Language Model (LLM) - multimodal transformer architecture (though used for text-only in this application)

Method:

  • Autoregressive text generation using transformer architecture with attention mechanisms
  • Trained using supervised learning and reinforcement learning from human feedback (RLHF)

Temperature: 0

Prompt engineering approach:

  • Retrieval-Augmented Generation (RAG) pattern
  • System prompts enforce response constraints
  • Instructions to refuse to answer (“I don’t know”) when information is insufficient

4.2.7 - Model performance

This is a pre-trained, off-the-shelf model used without modification or fine-tuning. The model’s effectiveness is evaluated through end-to-end answer generation quality assessment.

4.2.8 - Datasets and their purposes

A dataset with Subject Matter Expert (SME) approved data based on the guidance is used to test the accuracy of the model.

Tier 2 - Data Specification

4.3.1 - Development data description

A dataset with SME approved data based on the guidance is used to develop the tool.

Public accessibility: Not publicly accessible - these are internal DWP guidance documents containing operational information for work coaches and staff.

4.3.2 - Data modality

Modality: Text only (no images): Stored as HTML docs on DWP Intranet, converted into raw Markdown after ingestion.

Content type:

Structured documentation including:

  • Policy guidance: based on official policy and re-worded to define how the policy should be used/acted on
  • Spotlights: deep-diving on specific areas of policy

4.3.3 - Data quantities

Dataset of 314 guidance documents across 7 topic areas. Total size: 40MB

For RAG pipeline (Azure AI Search indexing):

  • Chunking approach: Documents chunked based on text hierarchy - i.e., split into sections by Headings/Subheadings
  • Chunk size: Variable, dependent on section sizes within documents
  • Total Number of chunks stored in vector DB: 2400 Chunks, 55MB data (including text, embedding vectors + metadata)
  • Purpose: All chunks indexed for semantic search and retrieval

For topic classification model fine-tuning:

  • Chunking approach: Documents chunked at 100-character intervals
  • Total samples: 32,716 chunks (post-chunking)
  • Training set: 31,080 samples (95%)
  • Test set: 1,636 samples (5%)
  • Split method: Stratified random split to ensure proportional representation of all 7 topics
  • Sampling strategy: Mean sampling applied to training data to address class imbalance

4.3.4 - Sensitive attributes

No personal data or sensitive attributes are used.

4.3.5 - Data completeness and representativeness

Completeness:

  • Missing data: N/A
  • Documents represent current internal guidance and updates are reflected on a daily basis

4.3.6 - Data cleaning

Guidance documents are converted into a raw markdown, navigation elements and anchor links are removed. Documents are split into small sections (chunks) based on heads and subheadings. These are vectorised and stored in a vector database with metadata.

4.3.7 - Data collection

Guidance documents are collected from the DWP intranet and updated daily. These are intended to be used by trained, internal DWP users.

4.3.8 - Data access and storage

Role-based access control (RBAC) is provided to required DWP development teams and stakeholders. All data is stored in the UK only.

4.3.9 - Data sharing agreements

N/A

Tier 2 - Operational Data Specification

4.4.1 - Data sources

User inputs (user questions) determines what guidance data is returned.

4.4.2 - Sensitive attributes

There is no sensitive data present in the guidance documentation.

Users are trained to not input personal data into the tool. Additional design and technical mitigations are in place in case sensitive data is inputted into the tool by mistake by users.

4.4.3 - Data processing methods

Query rephrasing, personal information masking and query embedding pre-processing is applied to every user question to enable the most appropriate guidance to be returned by the tool.

4.4.4 - Data access and storage

Operational data is logged and stored with roles-based access control applied for use. Users are able to view their chat record for audit trail purposes. Operational staff and the development team are able to view an anonymised visual of chat history to enable system-wide analysis of the tool.

4.4.5 - Data sharing agreements

N/A

Tier 2 - Risks, Mitigations and Impact Assessments

5.1 - Impact assessments

A Data Protection Impact Assessment has been completed and reviewed annually. Latest review was conducted 15/12/2025.

Equality Impact Assessment completed on 12/01/2026.

5.2 - Risks and mitigations

This is not a decision making tool, human intervention by DWP users is incorporated prior to any information being relayed to customers. Notices are included in the tool to remind users that the responses are AI generated and that all accuracy should be validated against source links.

The product has been through several stages of delivery e.g. proof of concept and small scale piloting prior to full roll out. At each stage testing has been carried out to ensure accurate presentation of results.

Prior to gaining access to the tool, users must complete the required training.