Care Quality Commission: Risk Categorisation
Using data and information from and about health and care organisations to understand the risk of poor quality care in the services that the Care Quality Commission (CQC) regulates.
1. Summary
1 - Name
Risk Categorisation
2 - Description
Using data and information from and about health and care organisations to understand the risk of poor quality care in the services that the Care Quality Commission (CQC) regulates.
3 - Website URL
N/A
4 - Contact email
Tier 2 - Owner and Responsibility
1.1 - Organisation or department
Care Quality Commission
1.2 - Team
Data and Insight
1.3 - Senior responsible owner
Deputy Director
1.4 - Third party involvement
No
1.4.1 - Third party
N/A
1.4.2 - Companies House Number
N/A
1.4.3 - Third party role
N/A
1.4.4 - Procurement procedure type
N/A
1.4.5 - Third party data access terms
N/A
Tier 2 - Description and Rationale
2.1 - Detailed description
The tool uses data and information from and about health and care organisations to understand the risk of poor quality care in the services that CQC regulates.
A Risk Category is a score on a three point scale of Medium, High or Very High which is awarded against an assessment service group (ASG) for a registered CQC location or Provider. The score is created using a rules based PowerBI model which takes a risk model output score (where available) and ratings based rules to produce a Risk Category Score. There are currently four risk models, Adult Social Care, General Practice, Independent Healthcare, and Urgent and Emergency Care. Where we do not have a risk model we solely use the ratings based rules.
We are currently producing approxiately 46000 refreshed scores each month.
The risk category is presented to Operational staff (inspectors) through CQC’s Regulatory Platform based on Microsoft Dynamics 365. It is available on a service’s Overview page as well as on the Data and Insight tab.
Operational staff use the risk category to prioritise assessment activity. Local insight and information can override the score, if required.
The tool uses data and information from and about health and care organisations to understand the risk of poor quality care in the services that CQC regulates
2.2 - Benefits
This approach allows sectors or specialisms to have specific data items or sources that are relevant to create a risk score, and then overlays a series of generic rules to increase the score where required. This means that we can be nuanced to the sector but also have a level of consistency of how we approach awarding a risk score to different sector or specialisms.
The approach allows CQC to identify where there is higher risk, as well as prioritise assessment activity and target resource in those areas.
CQC needs a consistent and transparent approach to determining risks within services. Risk categorisation is focused on the risk that people are receiving poor care in a service.
2.3 - Previous process
Previously some risk models included ratings rules but this was not consistent across sectors. Also where sectors or specialisms did not have a risk model no risk ‘score’ was generated.
2.4 - Alternatives considered
We are developing a Machine Learning model to prioritise inspections of care homes but it is not yet in Production
Tier 2 - Deployment Context
3.1 - Integration into broader operational process
The risk category is presented to Operational staff (inspectors) through CQC’s Regulatory Platform based on Microsoft Dynamics 365. It is available on a service’s Overview page as well as on the Data and Insight tab.
Operational staff use the risk category to prioritise assessment acitivtiy. Teams will review the awarded score and potentially undertake a site visit or another form of review.
Teams are also able to see the main data drivers that underpin the scores via the same dashboard mentioned above.
3.2 - Human review
The scores are reviewed each month prior to the load to the Regulatory Platform. If all rules have followed and pass the QA, then they are uploaded.
Inspectors review the tool on the Regulatory Platform. Usage (views and visitor numbers) can be tracked on the Data and Insight tab.
The output from the model is part of priorities for each sector.
3.3 - Frequency and scale of usage
The model is refreshed monthly for all services CQC regulates. This is ensure that any new data is included which might change the risk score provided. For example, if a service was a medium score in August. However a series of different contact has come into CQC over the past month, when the score is re-run in September the score could move to High.
3.4 - Required training
Guidance is provided to inspection colleagues. FAQ’s have been created. Internal online meetings held to share information on the approach and demo’s of how to access the scores has been provided. This is in the form of Product Campuses, written guidance in Word and PDF format as well as a dedicated route for raising queries.
3.5 - Appeals and review
N/A - the risk category score is not shared externally.
Tier 2 - Tool Specification
4.1.1 - System architecture
The Risk Category score is generated using a PowerBI report. This uses a series of rules to give a score. 5 tables are created in the report. These are combined to give the final list of scores against locations or provider assessment service groups (ASGs).
4.1.2 - System-level input
Risk Model Scores, CQC ratings data, CQC ASG data, CQC registration data.
4.1.3 - System-level output
Combined Excel file of scores. Loaded to regulatory platform.
4.1.4 - Maintenance
Run monthly to ensure all data we have is included in the risk calculation.
4.1.5 - Models
ASC risk model, General Practice risk model, Independent Health risk model, and Urgent and Emergency Care risk model.
Tier 2 - Model Specification
4.2.1. - Model name
Risk Categorisation model
4.2.2 - Model version
3.0
4.2.3 - Model task
Identify the risk of poor quality care in the services CQC regulates.
4.2.4 - Model input
CQC and external data.
4.2.5 - Model output
A risk category; “Very High,” High” or “Medium”.
4.2.6 - Model architecture
Rules based algorithm that creates the Risk Category score is generated using a PowerBI report. This uses a series of rules to give a score. 5 tables are created in the report. These are combined to give the final list of scores against locations or provider assessment service groups (ASGs). The rules are:
Very High Service has a Data and Insight (D&I) risk score of Very High or Service has a current overall rating of “Inadequate” or has a current overall rating of “Requires Improvement” that is over 5 years old or Service has been registered for 2 years or more and not yet rated
High Service has a D&I risk score of High or Service has no D&I risk score and either has a current rating of “Requires Improvement” that is over 1 year old but less than 5 years old or a current rating of “Good or Outstanding” that is over 6 years old. or Service’s previous overall rating is “Insufficient Evidence to Rate”
Medium Service has a D&I risk no higher than medium (or low) or Service has no D&I risk score and either has a current rating of “Requires Improvement” that is under 1 year old or a current rating of “Good or Outstanding” that is less than 6 years old.
4.2.7 - Model performance
The model’s output is tested against inspection outcomes to track how closely the score mirrors ratings issued.
We actively monitor feedback from inspectors on the risk models and categories.
4.2.8 - Datasets and their purposes
Data used within the models includes the following;
- Statutory notifications
- Information from Whistleblowers
- Safeguarding alerts and concerns
- Feedback from people who use services, professionals and friends and family through our Give Feedback on Care (GFOC) service
- Data from partners bodies such as NHS England
- Registration information about the service, such as the presence of a Registered Manager (a statutory requirement for some services)
Tier 2 - Operational Data Specification
4.4.1 - Data sources
CQC owned data including registration information, statutory notifications, cases from the Regulatory Platform.
External data from bodies such as NHS England.
4.4.2 - Sensitive attributes
No personal data is used in the model or its output.
While Personal identifiable data (PID) is within some of the data sources, for example the age, name and sex of a person using a service, this data is not used within the models.
4.4.3 - Data processing methods
Data quality checks are undertaken on all data used within the models.
4.4.4 - Data access and storage
Data taken from CQC systems including iHub2 (our indicator repository) and our Enterprise Data Platform (EDP) and loaded into Power BI to create the risk model outputs.
These outputs are then stored on SharePoint and used within another PBI report to create the risk categories. This file us then uploaded to the Regulatory Platform.
Risk category and model outputs have only been produced since November 2023. A retention period is being developed as part of the annual asset register/retention schedule review.
4.4.5 - Data sharing agreements
Yes there is a Data Sharing Agreement (DSA) in place for data we receive and use from NHS England.
Tier 2 - Risks, Mitigations and Impact Assessments
5.1 - Impact assessments
DPIA not required and confirmed by our Information Access colleagues
5.2 - Risks and mitigations
Risk - Inspectors don’t use the output from the tool to make decisions about inspections. This risk is surfaced at unit level (D&I)
Mitigation - Regular engagement sessions with operational staff and full guidance available.