Guidance

Diabetic eye screening: participation in the grading test and training system

Updated 25 November 2020

Public Health England (PHE) has agreed that:

• images presented for the test and training scheme (TAT) will be clearly defined as being for either test or training purposes
• all graders will be made aware of the participation requirements
• full participation in the test set will be 200 image sets per year
• minimum and achievable standards of sensitivity and specificity for detection of retinopathy and referable retinopathy will be set
• the local manager/clinical lead will receive a report of test performance for individual graders and for the programme
• the report will show each grader’s test results as sensitivity and specificity compared to the system grade
• the local manager/clinical lead will receive a scatter plot graph which will graphically plot the position of the individual against unidentified national and programme peers
• an anonymised version of the test report will be available to QA teams
• training sets will be made available
• guidance in the management of grading accuracy, feedback of results and grader support will be developed

1. Scope

Any grader who is registered on the local programme management software and grades regularly is expected to comply with this guidance. This includes optometrists, medical staff, consultant ophthalmologists and consultant diabetologists.

The professional standards committee of the Royal College of Ophthalmologists supports this level of participation from its members as representing good practice.

2. Participation in the test and training (TAT) scheme

All regular graders in the NHS DES Programme (NDESP) should participate in TAT. Clinical leads need to assure programme stakeholders that all their graders operate within nationally accepted standards.

2.1 Categories of grader

Regular graders

Regular graders work at all grading levels including arbitration and referral outcome grading (ROG).

Trainee graders

Trainee graders are subject to direct supervision of their grading. They are expected to fully participate in TAT. Results from trainee graders are reported separately to those of regular graders.

Guest graders

Graders who are not required to participate in TAT but want to undertake the tests can be registered as guest graders. They will receive feedback, but their results are not included in TAT performance reports. Individuals requesting guest status must be associated with a local service for it to be granted.

Clinical leads who are not regular grading staff

Clinical leads who are not registered on the local programme management software as graders are exempt from participating in TAT. It is recommended that they undertake 6 tests each year as a guest grader.

Ophthalmologists who are not regular grading staff

Ophthalmologists working occasionally in the NHS DES Programme and the Hospital Eye Service (HES) offering diabetic medical retinal advice are exempt from participating in TAT. They may undertake the tests as a guest grader.

2.2 Minimum number of test sets

All regular graders and trainee graders should complete a minimum of 10 monthly test sets of 20 images during the year. Twelve sets are an achievable level of participation.

The figure of 200 is considered statistically robust to detect grading that does not meet national standards. There is a low likelihood when grading this number of test sets that this result will be obtained by random chance.

2.3 When insufficient tests are completed

Programme boards should ascertain why a grader has not met the minimum standard of participation. TAT participation reports will indicate where this has occurred. Where there are fewer than 6 tests the results are not statistically robust and should not be used for assurance purposes.

2.4 Access to individual performance data

The clinical lead and programme manager will have access to individual identifiable TAT results. TAT results are also available to the grading college management if the grader is a member.

Identifiable data is also available to the NHS DES Programme manager responsible for the TAT system, the TAT provider and the external software provider. It is not available to other NDESP staff or to the QA team.

Identifiable TAT reports can only be made available outside of the local programme with the written consent of the manager or clinical lead. This might be requested as part of a serious incident investigation.

2.5 Access to anonymised performance and participation reports

QA teams and commissioners will have access to anonymised TAT reports. These reports should be reviewed at the programme board. The board will want to check that that the programme is participating in TAT and that grading levels are meeting national standards. The board should ensure that appropriate recovery facilities are in place for graders who are not meeting the standard.

2.6 If a grader falls below the agreed standard

Graders will have access to guidance explaining how the system works. The guidance will include the steps that would be taken if grading accuracy falls below the agreed standard and what additional support would be available to the grader.

2.7 Extended periods of absence

The grader, clinical lead or manager should email the TAT provider if a grader is on extended sick leave or maternity leave. The manager or clinical lead can define the period of absence on the TAT system. The reporting system will take account of this absence and assign a flagged leave of absence for this individual. The programme should notify the TAT provider when the grader returns to work. The grader should take the current test set as soon as possible, prior to recommencing grading for the screening service.

2.8 Regular graders who move programme or work in more than one programme

Clinical leads and managers should verify previous test results for graders moving to their programme to ensure the grader is meeting national standards.

Graders working in more than one programme should ask for their test set results to be available to the clinical lead and programme manager at each programme.

2.9 Grading college members

College graders agree to maintain TAT participation according to the grading college membership agreement. The system will make their results available to the college management and to their local programme(s).

2.10 Self-testing and training sets

TAT will provide a number of image sets for training purposes. Graders can use these image sets to learn about specific grading issues.

The local programme will decide on participation in these training sets. The results will only be used by the local programme and the grader.

2.11 Implementation

All regular graders have been required to complete a minimum of 10 tests per year since April 2014. The TAT reporting system will produce rolling year reports of test sensitivity and specificity. The availability of at least 1 year of TAT results at go live means the reports will be robust.

3. Background

The NHS DES Programme uses the TAT system to present images to graders under test conditions. Most graders in the NHS DES Programme participate in TAT. The results were only available to the local manager/clinical lead and the grader as an internal quality assurance check.

TAT results show an individual’s ability to grade in test conditions. This is not a measure of their ‘live’ grading standard. The results should form 1 component of the assurance system that local services have in place to assess grading quality.