The Patient Care Aggregator Service
This is the report from the alpha assessment of the Patient Care Aggregator service, on 16th June 2022.
Service Standard assessment report
The Patient Care Aggregator Service
| From: | Central Digital & Data Office (CDDO) |
| Assessment date: | 16/06/2022 |
| Stage: | Alpha |
| Result: | Met |
| Service provider: | NHS England |
Service description
A patient engagement portal within the NHS app to identify national interventions that would support patients waiting for treatment, and to mitigate the impact of longer waits on patients and the NHS itself. An aggregation service layer to collate patient data (appointments, referrals and wait times) and present the results back on the NHS App for patients to access and amend.
Service users
NHS App users with appointments at local hospitals that are using the partner Patient Engagement Portals (PEPs)
1. Understand users and their needs
Decision
The service met point 1 of the Standard.
What the team has done well
The panel was impressed that the team:
- deeply understood the problem space, the different user groups, and the context in which problems for users are occurring
- triangulated the problems they’ve identified in qualitative research with other data sources to validate problems and needs and avoid biases
- identified the root causes of problems for users in a detailed and rigorous way
- researched with users in context, and balanced the needs of patients against the impact on clinicians and administrators when considering solutions
- iterated based on research insights, and fed discovery insights into alpha design principles to ensure they were taken into account
- focused their research on vulnerable users and identified risks to them that will need to be managed through beta
What the team needs to explore
Before their next assessment, the team needs to:
- explore the impact of the approach to non-digital channels on users with lower digital skills, and test their assisted digital pathways
- test the end-to-end journey with participants
- go further in depth in usability testing with users with access needs, particularly those using assistive technologies
- test the impact over time of removing the GP as a main point of contact for users
2. Solve a whole problem for users
Decision
The service met point 2 of the Standard.
What the team has done well
The panel was impressed that:
- the service design started by considering the process of managing referrals as a whole, and have built up a good understanding of the challenges and opportunities in this space
- the team scoped the project well and were realistic and pragmatic about which part of this huge area they can best have an impact on
- the team dealt well with the many constraints that surround the project. They considered a range of options to solve this problem, but have chosen a solution that lets them deliver value to users faster. They could also explain how they made sure the constraints were not leading them to miss opportunities, and were able to talk about the ‘ideal’ version of the service and what their tradeoffs have been
- the team demonstrated a comprehensive understanding of the wider NHS transformation landscape their service fits into, and could clearly explain where it relates to other initiatives
What the team needs to explore
Before their next assessment, the team needs to:
- make sure they continue to question technical constraints to make sure the user needs are still the main priority in design decisions
- monitor whether the incremental approach to onboarding portal providers causes more confusion for users, as not all appointments will be available in the app initially
- explore in greater detail how individual hospital trusts as users benefit from the data collected by the app to drive overall service performance on triaging patients more effectively to reduce overall waiting times for patients
3. Provide a joined-up experience across all channels
Decision
The service met point 3 of the Standard.
What the team has done well
The panel was impressed that:
- the team has investigated the back office processes that inform this service, considered the impact the changes would have on staff, and have thought about how they’ll prepare trusts to support it
- the team has worked well with the portal teams to join up their user journeys, meaning fewer clunky handoffs to third party websites for users
- if users cannot use the app, they can still manage their referrals via their trusts’ existing routes
- the team has considered support channels from the start and have made sure there is relevant contact information in the app for those that need it
What the team needs to explore
Before their next assessment, the team needs to:
- work closely with trusts to monitor the parts of the journey where users could fall through the cracks
- continue to build relationships with the portal providers, and think about how this good practice can be sustained in the long-term. If relationships were to wane, there could be struggles later down the line to get necessary improvements made and the journey could break down
- consider what they can do to mitigate the risk of information in the app being incorrect and out of date - even though this sits with trusts to manage, it will have a significant impact on the user experience and reputation of the service
- ensure that GPs are automatically sent hospital discharge summaries of their patients to complete the user journey
4. Make the service simple to use
Decision
The service met point 4 of the Standard.
What the team has done well
The panel was impressed that the team:
- utilised the NHS design system, including for the portal sections
- redesigned the portal sections for this different user base
- carried out usability testing and iterated the design and content in response
- focused on the handoff to the portals in early testing and it does not seem to confuse users
- made sure users see consistent language throughout the service
- simplified jargon in many places, and built in features like declarations to double check users understand the action they’re about to take
What the team needs to explore
Before their next assessment, the team needs to:
- get dedicated content and interaction design skills on the team so the design and language choices are as simple as possible for users and follow best practices - for example, using the active rather than the passive tense
- continue to research and test users’ awareness of what’s happening when they’re going to the portals, to make sure this does not introduce additional confusion in the long run (for example, raising questions about where their data is stored, or creating confusion over when they should use the app vs a portal)
- continue to iterate the design of the screens and look for opportunities to simplify things further - in particular, the screen that shows all a users’ referrals. It is quite complicated and may benefit from being split into multiple screens. This could make it more clear at a glance what actions a user needs to take
5. Make sure everyone can use the service
Decision
The service met point 5 of the Standard.
What the team has done well
The panel was impressed that the team:
- has been designing to meet the WCAG 2.1 AA standard and has been carrying out testing to make sure the designs are compliant
- tested with a range of users in groups that may struggle to access the service, for example, people with complex health conditions and people who speak English as an additional language
- thought about who the app-based part of the service is not suitable for, and demonstrated that those users will still be able to manage their referrals through other methods determined by individual trusts, based on their circumstances
What the team needs to explore
Before their next assessment, the team needs to:
- demonstrate that the service works for disabled users with a range of cognitive and physical disabilities, and assistive technologies
- do usability testing with users with low levels of digital confidence and be able to explain what you learnt from this
- clearly define the user support model, including assisted digital support
- test the name of the service with users, ensuring it is easy to find
6. Have a multidisciplinary team
Decision
The service met point 6 of the Standard.
What the team has done well
The panel was impressed that:
- the team in place is passionate about the work they are doing, and it showed throughout the assessment
- the team has worked well together to define the problem space, and how the service can deliver better outcomes for users
- a specific Wayfinder tiger team will ensure that NHS E/I, NHSD and other key stakeholders have been actively involved during the build and test process
What the team needs to explore
Before their next assessment, the team needs to:
- consider that as the programme director is due to be leaving, this will require a comprehensive handover to ensure continuity
- get support from internal governance structures to expand at pace for beta in order to provide effective coordination of the tiger team to reach the target of roll-out to 40% of hospital trusts by summer 2022. Specifically in areas such as user research and QAT
7. Use agile ways of working
Decision
The service met point 7 of the Standard.
What the team has done well
The panel was impressed that:
- the service team has utilised agile ways of working where possible, running effective two-week sprint cycles to build the service in an agile environment
- the service team has overcome restraints in shared tech and software to ensure they have the right common tools to deliver an effective service
- the team fixed core technology choices and use of the NHS app, whilst remaining agile and open to other options based on the discovery and ongoing user research
- the team worked effectively remotely during COVID, and have since blended this with in-person meetings
What the team needs to explore
Before their next assessment, the team needs to:
- as feedback from clinicians and patients grows in beta, the team will need to ensure against scope creep
- the team in beta will need to run an effective backlog, that can be continuously worked towards once live, moving from an MVP towards to desired ‘To Be’ state that was described in the assessment
8. Iterate and improve frequently
Decision
The service met point 8 of the Standard.
What the team has done well
The panel was impressed that:
- the team demonstrated a well defined requirements gathering process and showed how early prototypes had been iterated upon based on user feedback
- the team has actively identified a desired To Be state to improve the service beyond the initial MVP
What the team needs to explore
Before their next assessment, the team needs to address the following:
- there is a large gap between the MVP and To Be state, the service moving into beta will need to manage users expectations about what to expect in the first few live iterations of the service
- the gap to the To Be state means the service will need to maintain a large team once in public beta to work through the technical debt. Plans for a team in place to work on continuous improvement should be explored in greater detail during the beta assessment
9. Create a secure service which protects users’ privacy
Decision
The service met point 9 of the Standard.
What the team has done well
The panel was impressed that:
- the new aggregation component doesn’t permanently store user data
- the team consults business and information assurance teams to make sure the service meets NHS’s security requirements
What the team needs to explore
Before their next assessment, the team needs to:
- be fully aware and contribute to establishing the application’s threat model by follow closely the work of partner teams within NHS on issues related to the application’s cybersecurity (for example pen testing)
- engage with the relevant information assurance teams on data protection issues. Find and engage with the data controller and the DPIO. Address other questions related to GDPR. Delegating complex data protection questions is essential, yet the team should be aware of those questions and their answers, as provided by IA teams
- implement OAuth before public beta
10. Define what success looks like and publish performance data
Decision
The service met point 10 of the Standard.
What the team has done well
The panel was impressed that:
- the service team had a well defined list of key performance indicators linking each to a spending objective
- the service team has plans for performance testing in beta, with active monitoring of user feedback through smart surveying via the app
What the team needs to explore
Before their next assessment, the team needs to:
- consider in greater detail how growing waiting times in the NHS will impact users perception of the app
- consider how the data collected via the app can be used to help drive performance improvements within individual trusts including key metrics such as waiting times and bed availability. Consider for example whether the app could show users some of this key data to help drive better choices about where to have treatment
11. Choose the right tools and technology
Decision
The service met point 11 of the Standard.
What the team has done well
The panel was impressed that:
- the service team works closely with the supplier designing the Patient Care Aggregator and is also engaging with the NHS app team
- the team has built a sound technical architecture based on modern design principles
- the service will run on the public cloud
What the team needs to explore
Before their next assessment, the team needs to:
- make sure the design work of the Patient Care Aggregator is closely monitored, in particular making sure that if at any point the supplier was to change, the new supplier would be able to pick up where the old one left off. The team did identify this as a risk
- consider that the risk also exists with AWS. In the event of the hosting platform changing (possibly for contractual or commercial reasons), the service should be able to be migrated to other platforms by limiting its reliance on proprietary technology
- maintain a decision log where any change in technology (either by the service team or any supplier) is documented and approved/refused
- always make sure that the Aggregator’s supplier is able to iterate as fast as the team would like, for instance by ensuring that processes around updating requirement documents are kept to a minimum
12. Make new source code open
Decision
The service met point 12 of the Standard.
What the team has done well
The panel was impressed that:
- the service team plans to release the code source in the open
- the code is in github and therefore the work of the team over the complete timeline of the implementation of the service will easily be made available to the public
What the team needs to explore
Before their next assessment, the team needs to:
- make the source code available publicly, following code publication practises like removing any secrets or PII present in current or former versions
- continue development in the open by working directly in the public code repositories, once the code is published
- do their best to work in the open, following the government design principles
13. Use and contribute to open standards, common components and patterns
Decision
The service met point 13 of the Standard.
What the team has done well
The panel was impressed that:
- the team and its suppliers use open technologies and languages, such as NodeJS, OpenAPI, OAuth (upcoming) and FHIR
- the team will make use of the NHS design system when adding the service’s functionality to the NHS App
What the team needs to explore
Before their next assessment, the team needs to:
- contribute any useful research findings to the NHS design system community backlog
- contribute any changes to the NHS design system back to the community, and ideally report any issues encountered with BaRS/FHIR to NHS standards team or HL7
14. Operate a reliable service
Decision
The service met point 14 of the Standard.
What the team has done well
The panel was impressed that:
- the team already has plans for active monitoring, performance testing and IG reviewing at alpha stage
- the team has a clearly defined list of technical metrics
What the team needs to explore
Before their next assessment, the team needs to:
- not just rely on AWS for platform reliability and expect NHS Digital CSOC to take care of all monitoring just by sending them application logs (presumably they need to be told what to look out for). Instead the team should develop in-depth knowledge of what makes their platform reliable, how that reliability should be compromised, and what the processes are around detection and remediation of the service malfunctioning or going offline. This includes the team itself measuring service performance and forecasting traffic volume for private beta, public beta and live. Likewise, all reviews carried out by other teams (for example pen testing and code checking) should be systematically reviewed
- far greater collaboration and extra support from wider NHS digital teams will ensure best practice is followed, building a more robust service
Next Steps
This service can now move into a private beta phase, subject to implementing the recommendations outlined in the report and getting approval from the CDDO spend control team. The service must pass a public beta assessment before launching into public beta. Please speak to your Digital Engagement Manager to arrange it as soon as possible.
Before the beta assessment we recommend particular focus on the following areas highlighted in the report:
- greater testing of the end-to-end journey with participants across a range of trusts and users with a range of access needs, particularly those using assistive technologies
- grow the team to meet the ambitious timeframes for delivery. This includes greater user research capability, and dedicated content and interaction design skills on the team
- engage with the relevant information assurance teams across NHSX/Digital on data protection issues
- make the source code available publicly
- contribute any useful research findings to the NHS design system community backlog
Published 26 July 2022