Guidance

Gynaecological cytopathology EQA scheme

Updated 11 October 2023

The NHS cervical screening programme (NHS CSP) is committed to maintaining its internationally recognised track record in innovation, service quality, use and development of evidence and data, education and public information.

Achieving high quality outputs across all cancer screening activities underpins every aspect of the programme. A clear professional accountability and governance structure supports this philosophy.

One way the programme achieves this is through the gynaecological cytopathology external quality assessment (EQA) scheme.

The requirements of ISO/IEC 17043 drive the scheme’s management system as defined in the quality manual.

This document provides guidance on the:

• design of the scheme
• interpretation of results
• general requirements and responsibilities for laboratories and participants
• principles and methodology of operation for a consistent delivery across the nation

It applies to NHS cervical screening laboratories in England, Wales, Scotland, Northern Ireland and laboratories outside the UK which have signed up to the EQA agreement.

1. The EQA scheme

The scheme centres on the assessment of slide sets by all staff who report cervical cytology on behalf of the NHS CSP. It compares individual reporting to consensus reporting to determine if a participant’s performance falls within an acceptable range.

There are 2 rounds (2 circulations of slide sets) in each EQA year. An EQA year runs from 1 April to 31 March.

There are 10 slides in each slide set per round used for performance analysis. The set may include up to 2 additional ‘special interest’ slides. These are for educational interest and are not included in the calculation of final assessment results.

1.1 Quality assurance (QA) in pathology

The Quality Assurance in Pathology Committee (QAPC) is a multidisciplinary group accountable to the Royal College of Pathologists (RCPath) for:

• oversight of performance in EQA schemes
• monitoring the EQA performance of clinical laboratories in the United Kingdom (UK)

The QAPC has discipline specific panels that report to it. It works with failing laboratories and reports persistent substandard performance to the Care Quality Commission (CQC).

National quality assurance advisory panel (NQAAP) in cellular pathology

The cellular pathology NQAAP oversees:

• the running of EQA schemes in histopathology and cytopathology
• the annual activities of each scheme registered with the college, including the investigation of persistent substandard performance by individual pathologists or laboratories

The panel promotes EQA to make sure the work of cellular pathology laboratories across the country and their contribution to clinical services is of a uniformly high standard.

1.2 Aims

The scheme aims to provide an acceptable degree of reliability and consistency by educating, advising and supporting all participants.

The scheme is committed to:

• upholding professional standards, good professional practice and conduct
• providing direction, leadership and guidance for the overall operation and management of the scheme
• complying with all data security legislation including the General Data Protection Regulations (GDPR) and Caldicott principles
• complying with all relevant health and safety legislation to warrant the safety and welfare of all staff on site and visitors
• providing equal access, within the limits of reasonable adjustment, to all eligible participants, taking into consideration their needs and requirements to promote continual improvement of the scheme
• meeting the needs of all stakeholders by delivering an acceptable degree of reliability and consistency between laboratories through education, advice and support for all participants
• enabling inter-laboratory monitoring and review of ongoing participation and performance
• providing a service to the highest levels of quality, safety and efficiency
• making sure the scheme is run by high calibre staff who are competent in their EQA roles on the basis of appropriate education, experience, skills and expertise
• treating participants fairly, sensitively, without prejudice and maintain confidentiality within the limits of the scheme
• providing an established and maintained framework for the QMS to integrate the organisation, procedures, processes and resources needed to meet (and exceed) the needs of participants
• setting measurable quality objectives which are taken into account as part of the reviews undertaken by management

1.3 EQA enquiries

For any scheme related enquiries, please contact the Screening Quality Service (SQAS) (Midlands and East) at PHE.csp-geqa-admin@nhs.net

Support is available during normal office hours (9am to 5pm).

2. Scheme roles and responsibilities

2.1 EQA facilitator

An EQA facilitator and administrative team are responsible for the day-to-day running of the scheme.

The EQA facilitator is responsible for:

• development and maintenance of local administrative and quality systems in line with published protocols and quality procedures
• undertaking the role of scheme quality manager
• overseeing arrangements for providing laboratories and participants with appropriate notice of EQA requirements
• identifying and highlighting actual or potential issues in relation to the scheme’s operation and bringing these to the attention of the appropriate manager
• maintaining all aspects of confidentiality and for the safe storage, recording and tracking of samples submitted for use in EQA
• working closely with the local support team to identify areas for development in relation to the EQA Online reporting system
• preparing the audit strategy and internal audit programme and making sure findings are recorded and completed appropriately
• preparing for external audits and maintenance of routine activities in line with United Kingdom Accreditation Service (UKAS) requirements
• producing accurate reports as requested for management, steering group and operational review meetings
• taking action in the event of substandard performance as documented in this protocol, and acting as the scheme secretary as required

2.2 Scheme organiser and deputy scheme organiser

NHS England subcontracts the scheme organiser role.

The scheme organiser must:

• be a consultant cytopathologist, a consultant histopathologist with an interest in cervical cytopathology, a consultant biomedical scientist who holds the advanced specialist diploma in cervical cytology or a biomedical scientist in cervical cytology working in a senior position
• be professionally regulated through their registration body
• be a participant in the scheme and have experience of selecting and reviewing slides for EQA
• hold a senior post in a laboratory providing gynaecological cytopathology for the UK cervical screening programmes
• participate in an appropriate continuing professional development (CPD) scheme
• fulfil NHS England cervical screening guidance and pre-existing professional body requirements for CPD
• have some knowledge of quality management systems and accreditation requirements
• be an active member of the EQA management board and contribute to setting annual objectives for the scheme and its strategic direction
• chair the EQA steering group and receive advice from its members on the practical aspects of scheme design and operation, and feed back to the EQA management board
• take responsibility for the oversight of substandard performance and communicate with the cellular pathology NQAAP chair in the event of persistent substandard performance
• oversee difficult and sensitive situations associated with participant or laboratory substandard performance, complaints and appeals
• actively contribute to the scheme’s regular review of its documentation, including the annual reports for service users and management
• promote the value of the scheme and associated activities at relevant professional meetings, and chair the annual EQA participant workshop
• provide advice on the educational elements of the scheme
• respond urgently to any situation during EQA rounds which has the potential to adversely affect a patient’s management
• provide advice and contribute as necessary to UKAS visits and the scheme’s continuing accreditation
• proactively highlight areas where improvements can be made
• provide ongoing, ad hoc clinical and professional advice to the national EQA team

The deputy scheme organiser must:

• be a consultant cytopathologist, a consultant histopathologist with an interest in cervical cytopathology, a consultant biomedical scientist and holder of the advanced specialist diploma in cervical cytology or biomedical scientist working in a senior position
• be professionally regulated through their registration body
• be a participant in the scheme and have experience of selecting and reviewing slides for EQA
• hold a senior post in a laboratory providing gynaecological cytopathology for the UK cervical screening programmes
• participate in an appropriate continuing medical education (CME) or CPD scheme
• fulfil NHS CSP and pre-existing professional body requirements for CPD
• have some knowledge of quality management systems and accreditation requirements
• be an active member of the EQA steering group and contribute to setting annual objectives for the scheme and its strategic direction
• promote the value of the scheme and associated activities at relevant professional meetings, and contribute to the annual EQA participant workshop
• proactively highlight areas where improvements can be made

The deputy will provide operational cover in the scheme organiser’s absence. This includes:

• chairing the EQA steering group and feeding back to the EQA management board
• taking responsibility for the oversight of substandard performance and communicating with the cellular pathology NQAAP chair in the event of persistent substandard performance
• overseeing difficult and sensitive situations associated with participant or laboratory substandard performance, complaints and appeals
• promoting and providing advice on the educational elements of the scheme
• responding urgently to any situation during EQA rounds which has the potential to adversely affect a person’s management
• providing advice and contributing as necessary to UKAS visits to the scheme’s operations centre and the scheme’s continuing accreditation
• providing ongoing, ad hoc clinical and professional advice to the national EQA team

2.3 National EQA team

The national cervical QA lead has national responsibility for EQA as part of their role for cervical screening. This includes:

• day-to-day support and advice to the EQA operations team
• line management for the EQA facilitator and administrative function
• responsibility for EQA Online through the line management of the SQAS staff who develop and maintain the system
• joint authorisation, with the EQA facilitator, of individual end of round reports
• review and authorisation of all operational work instructions and associated documentation
• chairing the EQA operational group meetings

The national EQA team is comprised of staff with defined responsibilities to support the EQA facilitator as well as responsibilities for management, IT and data support. The cytology education manager is responsible for maintaining the national EQA scheme protocol. The national team provides appropriate cover in the absence of these staff.

3. Governance

3.1 Operational group

The operational group:

• is chaired by the national cervical QA lead
• includes the national EQA administration team, staff responsible for the maintenance of EQA Online and a representative from the national programme team responsible for protocol
• may be attended by the scheme organiser or deputy organiser
• discusses all operational activity
• meets in accordance with the requirements and frequency expressed in the terms of reference
• is administered by the national EQA team

3.2 EQA steering group

The EQA steering group:

• is chaired by the scheme organiser
• provides a supporting and advisory role
• meets in accordance with the requirements and frequency described in the terms of reference
• includes individuals from laboratories to make sure the requirements and interests of participants are represented
• provides technical advice and assistance on scheme design and operational issues (its role does not extend to involvement in performance issues unless these relate to the scheme’s design and performance monitoring systems)
• is administered by the national EQA team

3.3 EQA management board

The EQA management board:

• is chaired by the national cervical screening programme manager
• includes the national cervical screening programme manager, scheme organiser, deputy organiser, EQA facilitator, national cervical QA lead and cytology education manager (the national cervical screening programme manager and cytology education manager do not have a role in operational provision)
• meets in accordance with the requirements and frequency described in the terms of reference
• is an executive team which provides a review and governance mechanism for the scheme
• is responsible for the strategic direction, planning and development of the scheme
• is administered by the national EQA team
• can escalate any major concerns about the scheme to the NHS England Quality Assurance Executive Group

3.4 National oversight

The EQA management board is accountable to the NHS England Quality Assurance Executive Group chaired by the national QA lead. The national QA lead is accountable to the NHS England Director of Vaccination and Screening.

EQA is a standing agenda item at the Clinical Professional Group for Laboratories (Laboratory CPG).

3.5 EQA online

EQA Online is a secure website accessed via an N3 or Health and Social Care Network (HSCN) connection. It provides a secure facility for existing participants to enter their results for the current round and to view, download and print their individual reports.

Laboratories without an N3 or HSCN connection can scan and email their EQA response forms securely.

4. Laboratory participation

An agreement to participate in the EQA scheme exists between SQAS, NHS cervical screening laboratories and their staff who report gynaecological cytopathology for the NHS CSP.

The agreement also applies to private laboratories and those from outside the UK who participate in the scheme. The agreement describes the general terms and conditions of participation for all parties. This is subject to periodic review.

4.1 General terms and conditions of participation

All staff who report gynaecological cytology for the NHS CSP are required to participate in the scheme. All staff whether employed on a full-time, part-time, permanent or temporary basis are required to participate in the scheme.

This applies to:

• cytology screeners
• biomedical scientists
• consultant biomedical scientists
• pathologists

Participation in relevant accredited EQA schemes is a prerequisite for laboratories maintaining their UKAS accreditation against ISO 15189.

Laboratories are responsible for notifying the national EQA team of any changes in service provision, and changes in staffing which may affect scheme arrangements.

Participants are responsible for making sure they access only the most recent version of the scheme documents.

Please note that:

• the scheme protocol is maintained on GOV.UK
• downloaded, printed or electronically transmitted versions of the files are not controlled documents
• page numbering is not applicable to HTML documents

Where possible, the EQA team will make reasonable adjustments to facilitate access to the scheme for all potential participants. Laboratories are responsible for notifying the national EQA team if a member of staff has any needs that may require an adjustment as to how they participate in EQA.

The scheme conducts assessments using ThinPrep™ liquid based cytology (LBC) slides exclusively. Participant documents (for example, final results reports and certificates of participation) relating to historical SurePath™ assessments can still be provided upon request.

Laboratories must accept all terms and conditions described in the EQA agreement to facilitate staff enrolment on the scheme. There is no requirement for laboratories to re-enrol their staff on an annual basis.

Laboratories are responsible for the safekeeping of EQA slides during each round and preserving their quality for subsequent participants. Laboratories must report damage or breakages to the EQA facilitator immediately via email. The EQA facilitator advises the laboratory how to proceed in these circumstances.

The national EQA team manages the administration of all data entry and result reports via EQA Online.

The scheme retains all scheme documents, electronic and paper, in accordance with The retention and storage of pathological records and specimens, fifth edition April 2015.

4.2 Labelling, packaging and transport

The EQA facilitator gives laboratories detailed instructions to make sure EQA slides are correctly and safely packaged and labelled in line with Royal Mail requirements. Slides transport must be by Royal Mail special delivery. This method tracks packages during transit and upon receipt.

Laboratories may use other methods of transport subject to approval by the scheme’s management. Laboratories must not use unauthorised methods of slide transport.

4.3 Sanctions for non-compliance with the EQA agreement

Laboratories that do not conform to the protocol requirements may be subject to the scheme’s escalation policy. Removal from the scheme presents a risk to a laboratory maintaining its ISO 15189 accreditation.

4.4 Eligibility requirements

Students and trainees

Students and trainees who intend to pursue a career that includes cervical cytology are eligible to participate in the scheme for educational purposes if they are working in an NHS contracted laboratory or a private laboratory which is providing cervical screening services to the NHS.

The performance analysis excludes student and trainee scores. Laboratory managers must make sure this staff group discusses their results with the relevant training officer or supervisor.

Locum staff

Locum staff can participate in the national scheme and comply fully with all conditions and arrangements if they work:

• in an NHS contracted cervical screening laboratory
• in a private laboratory which is providing cervical screening services to the NHS

Note that the production of a satisfactory report or evidence of participation in EQA does not guarantee eligibility for employment in an NHS-contracted cervical screening laboratory or a private laboratory providing cervical screening services to the NHS.

Private laboratories under contract to the NHS

Staff working in private laboratories providing cervical screening services for the NHS must comply fully with its conditions and arrangements.

Private cervical screening laboratories in the UK

The EQA agreement also describes the conditions of participation for staff working in private laboratories (not contracted to the NHS) within the UK, and staff working in cervical screening laboratories outside the UK.

Staff working in private laboratories providing non-NHS CSP screening may wish to subscribe to the scheme. Participation is not mandatory for these individuals.

Individuals working in private laboratories who choose to participate in the scheme must note that their responses are not included in the consensus calculations to determine slide classifications. In addition, their overall scores are not included in the statistical analysis used to determine individual performance.

Results and feedback to these participants are determined by comparison against participants who carry out cervical screening work for the NHS.

Cervical screening laboratories outside the UK

Subject to operational capability and resources, the scheme is open to laboratories from outside the UK subject to their meeting specified terms and conditions of the EQA agreement.

Participants working in laboratories outside the UK must note that their responses are not included in the consensus calculations to determine slide classifications. In addition, their overall scores are not included in the statistical analysis used to determine individual performance.

Non-UK laboratories must declare the pathway for professional accountability within their employing organisation and regulatory body under the terms and conditions of the EQA agreement. The scheme reports persistent substandard performance to the relevant regulatory body.

Ineligible staff

Individuals are not eligible to participate in the EQA scheme if they are not currently screening or reporting.

4.5 Enrolment

Upon initial enrolment, the EQA facilitator directs participants to EQA Online, the secure website for general guidance and instructions on how to set up and manage their EQA account.

Participants must use an employer issued email address to set up and operate an EQA account. The EQA facilitator will advise participants unable to access EQA Online using an N3 or HSCN connection of the alternative arrangements for participation using established manual systems.

EQA Online prompts participants to confirm they have read and understood this document.

Participants who are unsure about any EQA Online content or a scheme operational matter must contact the EQA facilitator for guidance.

4.6 Participant responsibilities

Late participation

Participants must be available to complete the EQA during the time the slides are present in the laboratory.

The EQA facilitator can make provision for staff unavailable on the agreed EQA dates for a legitimate reason (for example due to pre-arranged annual leave or a short-term authorised absence). The EQA facilitator can make the slides available on an alternative date at the end of the round or at another suitable point in the cycle (offering only one further date). Local management are responsible for making sure staff are allocated time to participate in EQA as agreed.

Non-participation

Participants need to be aware of the consequences of non-participation in EQA.

An individual who is absent on the agreed EQA date, or who does not participate without a legitimate reason, is classed as a non-participant without a valid reason. This may lead to a result of substandard performance.

If an individual fails to participate in 2 out of 3 rounds without legitimate reasons, the EQA facilitator passes the information to the:

• scheme organiser
• national cervical QA lead
• regional head of screening QA responsible for the relevant laboratory (who alerts the commissioners of the service)
• cellular pathology NQAAP chair

Legitimate reasons for non-participation include:

• short-term or long-term authorised absence
• maternity leave
• pre-arranged annual leave
• an extended period of sabbatical leave

4.7 Confidentiality

The EQA facilitator communicates information disclosure liabilities, responsibilities and procedures to all EQA staff, including those employed directly by and under contract to NHS England.

All participant data is confidential. All electronic data is stored in password protected files or in areas accessible only to authorised EQA staff. Paper based data is stored in a secure, locked cabinet and only authorised EQA staff have access to it.

The scheme is confidential under the conditions of participation in EQA schemes determined by the professional bodies through the QAPC.

Participants may choose to waive confidentiality for the purposes of discussion and mutual assistance. They can also choose to waive confidentiality for regulatory or recognition purposes.

The scheme does not make a participant’s identity available to any party except under the following circumstances:

• when a participant has requested a reminder of their code number either by email or via the facility provided in their EQA Online account
• when the chair of the QAPC is notified of a member of medical staff or a consultant biomedical scientist who has reached the second action point and the chair of the QAPC has subsequently requested additional information (including the identity of the participant)

Laboratory managers must have systems in place to monitor and review ongoing staff participation and performance in EQA, and to monitor trends in results as appropriate. The scheme recommends that the laboratory records this information within the departmental appraisal process.

At present, it is extremely unlikely that any government regulatory authorities will make a direct request to the scheme for participants’ results.

If the scheme receives a third party request from a regulatory authority requiring EQA results, then the national EQA team will investigate and verify the source and validity of the request. The scheme will comply with a valid and verified request, and notify the affected participants of this action in writing within 5 working days.

4.8 Subcontracted services

NHS England subcontracts with the NHS cervical screening laboratories that provide EQA material, and the staff who review it. The provision of suitable material and qualified staff who perform the slide reviews is critical to the success and integrity of the scheme.

NHS cervical screening laboratories or those laboratories under contract to provide cervical screening services to the NHS, which are accredited by or registered for accreditation with UKAS, are eligible to provide EQA materials.

The scheme deems staff to be competent on the basis of:

• qualification
• experience
• satisfactory participation in the scheme
• employment by a laboratory which is contracted to provide a cervical screening service to the NHS

Cervical screening laboratories not under contract to the NHS (including those outside the UK) should, in the first instance, contact the EQA facilitator to establish their eligibility to submit slide material and, or provide staff to undertake slide reviews.

4.9 Sourcing slides for EQA

All NHS cervical screening laboratories (or private laboratories providing cervical screening services to the NHS) and those considered eligible (private UK, or from outside the UK) are required to submit slides under the terms and conditions of the EQA agreement.

Controls are in place to minimise loss or damage to slides during EQA activity and the EQA facilitator monitors this annually. The EQA facilitator records and reports any loss or damage to stakeholder property which renders it unsuitable for use in the scheme. Stakeholder property is physical property including glass slides, participant information and scheme data.

The total number of slides requested for any particular round is at least 3 times the number needed. This is because up to two-thirds of slides submitted may be unsuitable due to poor technical quality or a lack of agreement on the classification of the slide.

Laboratories must keep a record of the slides and supporting documents submitted for EQA purposes.

The national EQA team returns all slides to the submitting laboratory at the end of the EQA year (or sooner) by arrangement.

The national EQA team provides feedback to submitting laboratories on panel consensus and laboratories are required to consider this information during their review and EQA performance monitoring processes.

Copy slides

The EQA facilitator requests copy slides (repeat preparations) produced from a single LBC sample from each laboratory. Laboratories can only submit copy slides to the scheme for inclusion in EQA rounds. If the laboratory produces more than one copy from the same sample then the scheme considers each as an entirely separate case for EQA purposes. Laboratories must not assume identical consensus for each of the copy slides.

Slides selected for EQA

Laboratories must provide good (not necessarily easy) examples for potential use in EQA.

The slides selected for EQA range across the classifications recognised by the British Association for Cytopathology (BAC). This covers:

• inadequate
• negative
• borderline change in squamous cells
• borderline change in endocervical cells
• low-grade dyskaryosis
• high-grade dyskaryosis (moderate)
• high-grade dyskaryosis (severe)
• ?invasive squamous carcinoma
• ?glandular neoplasia of endocervical type
• ?glandular neoplasia (non-cervical)

For further explanation see guidance for laboratories providing HPV testing and cytology services in the NHS CSP.

Slides containing infections may be included, although there is no requirement to record these on the response form. Interesting cases may be included to add educational value to the scheme. Rarities are not included, nor are slides from unusual or problem cases of educational value.

Slides must have histological confirmation of diagnosis if they show:

• high-grade dyskaryosis (moderate)
• high-grade dyskaryosis (severe)
• ?invasive squamous carcinoma
• ?glandular neoplasia of endocervical type
• ?glandular neoplasia (non-cervical)

Histological confirmation is required for slides that are high-grade (moderate) or high-grade (severe) as either CIN2 or CIN3 (an exact match is unnecessary).

Slides agreed for submission

Submitted slides must have achieved an in-house consensus opinion among staff, including a cytology screener, a checker and a pathologist or consultant biomedical scientist.

Staff who can select slides include cytology screeners, checkers, consultant biomedical scientists and pathologists. These individuals are qualified and trained to report cervical cytology, and are currently reporting cervical cytology on behalf of the NHS CSP. These individuals are also responsible for assessing the suitability and technical quality of slides for potential use in EQA.

Supporting information for submitted slides

With each slide submitted the laboratory provides:

• the consensus opinion on the classification of each slide, based on agreement between at least one cytology screener, one checker, and one pathologist or consultant biomedical scientist
• clinical information derived from the HMR101 form or its electronic equivalent (age of individual and last menstrual period (LMP) or cycle stage)
• histological confirmation of diagnosis for any cases with a consensus opinion of high-grade dyskaryosis (moderate) or worse

Cytology samples showing low-grade dyskaryosis or borderline change in squamous or endocervical cells may not have had further follow-up, due to the individual returning to routine recall after colposcopic assessment. In such cases, it may not be possible to obtain histological confirmation, and slides falling into these reporting categories do not need further slides or histology.

Negative or inadequate slides do not require a second slide confirming the classification in order to be included in the EQA scheme.

If a laboratory provides copy reports only, they must use the corresponding slide number identification. The laboratory must remove any patient identifiable information beforehand.

Slide confidentiality

The laboratory is responsible for concealing any patient identifiable information on the slides before submission. The laboratory must obscure the names of the patient and the submitting laboratory for the purposes of the EQA round but not remove them from the slide. The laboratory must label the slides sufficiently to allow the EQA facilitator to match slides to the relevant accompanying information.

Dotted templates

EQA does not permit the preparation of dotted templates for staff on ‘catch-up’ prior to their participation.

5. Review of submitted slides

Slides submitted by laboratories undergo a further review before they are included in the scheme. The EQA facilitator is responsible for organising the slide review.

5.1 Reviewers

Laboratories are responsible for providing qualified and competent staff to undertake the slide review. A cytology screener, a checker and a pathologist/consultant biomedical scientist undertake the review.

Slide reviewers can report slides that they may have examined as part of the review process. To reduce the risk of recognition, the EQA facilitator anonymises the slides for the review process, and again if used in the EQA round. Slide sets generated for EQA, as far as possible, minimise the number of slides included from a single laboratory.

5.2 Review method

The reviewers agree the classification of a slide that is consistent with the submitting laboratory opinion, patient history, or histology where appropriate (for example a cytological report of high-grade dyskaryosis (moderate) matching a histological diagnosis of CIN II).

The panel reaches its consensus without prior knowledge of the submitting laboratory’s classification. The reviewers assess slides accepted for EQA for their technical quality, including stain quality, mounting and any cracks. The reviewers reject any slides that are not technically adequate.

5.3 Discordant opinions

If the panel opinion differs from the submitted opinion and would result in different management of the individual, the EQA facilitator notifies the originating laboratory and returns the slide(s) immediately. For example, there may be disagreement on whether to report the slide as negative, inadequate, or abnormal. Alternatively, the agreed panel opinion may be high-grade (moderate) or worse, whereas the laboratory may have reported the slide as borderline change in squamous or endocervical cells, or low-grade dyskaryosis.

The medical head of the submitting laboratory is responsible for taking any necessary follow-up action in the event of such notification.

In the rare event that a participant discovers any unexpected clinical findings (for example, psammoma bodies) during an EQA round, they must bring this to the attention of the EQA facilitator in the first instance. The EQA facilitator must speak with the scheme organiser to confirm the findings and ascertain the required action. The scheme organiser must contact the submitting laboratory immediately to agree the appropriate action. The submitting trust is responsible for following up any clinical implications for the individual concerned.

5.4 Professional code of conduct

The scheme relies on each participant’s professional honesty to make sure that they do not discuss cases until all staff in the laboratory (excluding valid non-participants) have completed and submitted their EQA responses.

Participants must not pass on information during the EQA round to allow another person to submit their opinions as their own. The scheme defines such behaviour as collusion.

At the beginning of every EQA round, laboratory managers must highlight the potential consequences of collusion. These are that:

• any evaluation of personal performance is meaningless
• it allows an individual(s) to gain unfair advantage over their colleagues
• the risk of submitting false responses is increased and this could adversely affect individual performance
• whether proven or unproven, collusion seriously diminishes an individual’s professional integrity
• the value and integrity of the EQA scheme is undermined

Laboratory managers must be sensitive when dealing with substandard performance and provide support for underperforming staff. Underperforming staff must avail themselves of the support provided by management.

These measures are in the interests of all participants with the aim of avoiding adverse effects on individual performance.

An individual must notify the EQA facilitator in the first instance if they have evidence of, or a strong suspicion of, collusion. The scheme organiser must collaborate with the relevant laboratory. The laboratory is responsible for investigating the allegation at local level and reporting to the scheme organiser. The scheme organiser must escalate proven or admitted collusion immediately. The scheme organiser must notify the cellular pathology NQAAP chair, the individual’s employer, SQAS, and the relevant regulatory organisation.

See the procedure for reporting and investigating alleged collusion.

5.5 Financial aspects

The EQA scheme is free of charge for laboratories in England providing a cervical screening service to the NHS CSP. There is a charge for NHS laboratories outside England and private laboratories not contracted to the NHS. The scheme funding arrangements are set out in the EQA agreement.

6. Preparation for EQA

The EQA facilitator is responsible for detailed operational planning both directly and through their support team. The EQA facilitator provides participants with detailed information about the scheme, including their rights and responsibilities, in advance of the EQA round. This information is available to participants via EQA Online or by email if the system is not available to them.

6.1 Organising the transfer of slides

The EQA facilitator is responsible for working with laboratories to agree dates for slide receipt, onward transmission or return of slides, and a closing date and time for submission of responses.

6.2 Tracking the location of slides

The EQA facilitator is responsible for knowing the location of every slide during an EQA round and documenting this information.

6.3 Slide bank

The EQA facilitator selects slides used in the EQA rounds. The EQA facilitator is responsible for establishing a comprehensive slide bank record and its maintenance.

Obscuring laboratory details

The EQA facilitator must obscure any laboratory details that would disclose the origin of a slide for the duration of a round. This information is not removed from the slide, but covered by a label that only shows the unique identifying number of the slide and set.

Removal of dots and markers

The EQA facilitator must check that the laboratory has removed all dots or markers from submitted slides.

Preservation of staining quality

Within each new round, the EQA facilitator must vary the order in which they circulate slide sets between laboratories. This makes the distribution of pale slides more equitable if staining deteriorates because of repeated microscopy during the EQA round. The EQA facilitator provides participants with instructions on handling slides, their preservation during EQA and storage before their return.

Slides used in more than one round

If EQA slides are to be used for a third round, then they must be reassessed beforehand to make sure their technical quality has not diminished. Slides retained for future use, or as a contingency measure, must be stored in a suitable light-impervious container within a locked drawer or cabinet.

6.4 Slide breakage, damage or loss

The laboratory lead must notify the EQA facilitator immediately via email to report any slides that are broken, damaged or missing either in transit or during the EQA round. The EQA facilitator will advise the laboratory lead regarding next steps.

The nominated QA biomedical scientist must assess the slide and advise the EQA facilitator that the laboratory can repair the slide, if the damage is not sufficient to disadvantage any participants, or to remove the slide from the circulation. A laboratory must not attempt repairs without the written consent of the submitting laboratory.

The scheme considers lost or broken (irreparable) slides to be non-consensus and they are not included in performance calculations. It is impossible to replace damaged or broken slides with others from the same category, as no 2 slides are identical. In such instances, the EQA facilitator notifies the submitting laboratory of the breakage.

At the end of the round, the EQA facilitator returns broken or damaged slides to the submitting laboratory with the rest of the batch. If any slides returned to the EQA facilitator are broken upon receipt then the QA biomedical scientist only attempts repairs with the written consent of the submitting laboratory.

6.5 Assembly of slide sets

The EQA facilitator is responsible for assembling and numbering the slide sets.

Number of sets required for cytology screeners and checkers

The EQA facilitator determines how many slide sets are required to complete the round. Where possible a minimum of 45 participants (excluding trainees and participants working in laboratories that only screen private work) will see each slide within a set. The scheme has sought advice and confirmed there is no minimum number of participants necessary to conduct a statistically valid assessment.

Composition of slide sets

Each slide set contains 10 EQA slides and any additional slides included for educational interest. These additional slides are clearly labelled as such, and do not form part of the assessment. The EQA facilitator determines the final composition of the slide sets using the following criteria:

The EQA facilitator selects slides from all the categories recognised by the British Association for Cytopathology (BAC). Slide sets must comprise a greater proportion of abnormal cases than is seen in routine practice.

Each slide set should contain between 2 and 6 negative or inadequate slides. The EQA facilitator must vary the ratio of negative or inadequate slides to slides of other types for each round.

Where the EQA facilitator uses more than one slide for a peer group, each must contain the same number of slides from each classification in each round.

The EQA facilitator must use all classifications in any one EQA year (within 2 consecutive rounds). This includes:

• negative
• inadequate
• borderline change (of either type)
• low-grade dyskaryosis
• high-grade dyskaryosis (moderate)
• high-grade dyskaryosis (severe)
• ?invasive squamous carcinoma and ?glandular neoplasia (wherever possible)

Pathologists’ slide set

The EQA facilitator selects the slides at random from the available sets. The EQA facilitator labels the set(s) allocated for the sole use of pathologists and consultant biomedical scientists.

6.6 Information and instructions for participants

Participants receive supporting information and instructions for use before the start of an EQA round and the circulation of slide sets. The EQA facilitator instructs participants on where and how to access the relevant information online.

Supporting information includes:

• a response form
• an abnormal referrals sheet
• slide set details
• participant instructions for recording and submitting responses
• relevant information for special interest slides
• a complaints procedure
• an appeals procedure

The EQA facilitator will send documentation via email to participants using manual systems before the EQA round starts.

Response form

The manual version of the response form replicates the content of the EQA Online version.

For manual systems, the EQA facilitator emails response forms to laboratory contacts, for distribution to local participants.

Referral sheets for reporting abnormal slides

In each round, one or more cytology screeners from each laboratory must mark the pathologists’ slide set in addition to their EQA. These individuals are required to complete the referral sheet. In addition to completing their own EQA, one or more checkers must assess each slide that the cytology screener considers potentially abnormal and to complete the standard referral sheet.

The laboratory manager or their nominated deputy circulates the completed referral sheet to all participants viewing the pathologists’ slide set. The laboratory retains the original referral sheet. There is no requirement to submit this form as part of the assessment.

Slide set details

Slide information accompanies each slide set. For each slide within the set, the following information is given:

• the slide number
• the age of the individual
• LMP

The classifications of slides included for educational interest, with any further comments, accompany the slide set.

6.7 Other documentation

The EQA facilitator provides guidance to participants in either written or electronic format, or provides a reference to the location of controlled documents. This includes instructions or action to take regarding:

• receipt of broken or damaged slides
• slides damaged or broken during the EQA round
• handling, cleaning and storing slides during the EQA round
• unexpected findings during a live round
• safe packaging and dispatch of slides

7. Examining EQA slides

Participants examine EQA slides under conditions that resemble those used in their routine practice as closely as possible. They will not undertake the assessment in an examination setting. Participants are required to evaluate their own diagnostic opinion against those of their peers. Participants must not discuss the slides in any way before all participants (excluding valid non-participants) have viewed them and recorded and submitted their results.

In each round there are 2 peer groups: cytology screeners and checkers, and pathologists and consultant biomedical scientists. Where a peer group uses more than one slide set, the EQA facilitator allocates each participant a set. The EQA facilitator makes sure that the number of participants viewing each set is sufficient for a consensus calculation to be valid.

The slide categories are representative of the categories of slide result found in routine workload. The nature and numbers of slides circulated and the results analysis are key features of the scheme.

7.1 Non-medical staff

Typically, there are 2 clearly identifiable tiers of activity undertaken by non-medical staff: primary screening and checking. At present, staff (other than consultant biomedical scientists) who are not medically qualified do not report abnormal slides. They sort slides into those they report as negative or inadequate, and those that they pass to senior colleagues for further review. The scheme assesses non-medical staff based on their ability to distinguish between negative, inadequate and abnormal slides. Many laboratories routinely encourage their checkers to suggest a classification. The scoring scheme covers all staff in order to enhance the educational benefit of the scheme.

Individuals (usually cytology screeners and biomedical scientists) who undertake primary screening decide whether a slide is negative, inadequate, or potentially abnormal. They refer abnormal slides on to senior staff for more detailed reporting. Individuals who routinely carry out primary screening undertake EQA with unscreened and unmarked slides.

There is no limit set on the maximum time that cytology screeners can look at a particular EQA slide. As assessment conditions resemble those of normal working practice, the expectation is that cytology screeners are able to screen the 10 EQA slides in approximately 2 hours.

The scheme assesses a cytology screener on only one set of slides and measures performance based on their ability to differentiate between negative, inadequate, and potentially abnormal slides, and on any instances of missed dyskaryosis. The scheme requires cytology screeners to suggest a grading classification for abnormal slides, and provides quantitative and qualitative feedback based on these opinions for personal educational purposes. This feedback is not included in the performance analysis.

In addition, cytology screeners can view and report slides used in the EQA scheme for pathologists and consultant biomedical scientists. The scheme treats their results as purely educational and does not use them in performance analysis.

Checking

Experienced cytology screeners and biomedical scientists have varied duties. Checkers usually undertake some primary screening, rapid review, and checking of cervical slides that appear potentially abnormal. Checkers, like cytology screeners, report one set of 10 unmarked slides.

There may be some checkers who only look at marked slides and do not perform any primary screening. The scheme allows these individuals to perform their EQA on marked slides. They report all 10 slides within the slide set, not only those that have been marked. Cytology screeners within the laboratory must mark the slides as they would in routine practice before the checkers perform their EQA.

Checkers who receive marked slides do not receive a copy of cytology screener opinions. The scheme calculates the performance of those checkers who do not engage in primary screening in the same way as that of cytology screeners and checkers who do.

Checkers receive quantitative and qualitative feedback for personal educational purposes on their grading of abnormalities. The scheme assesses performance on their ability to distinguish between negative, inadequate, and potentially abnormal cervical samples and on any instances of missed dyskaryosis.

Checkers can view and report slides used in the EQA scheme for pathologists and, or consultant biomedical scientists. The scheme treats their results as purely educational and does not use them in performance analysis.

7.2 Consultant biomedical scientists in cervical cytology

Consultant biomedical scientists sign out abnormal slides and provide management recommendations. They may also undertake checking duties and report unchecked slides, effectively acting as their own checker. The scheme uses slides screened by cytology screeners and other checkers, in line with routine practice in the laboratory.

While consultant biomedical scientists act under the direction of the consultant pathologist, they are not under his or her direct supervision. From an EQA point of view, it is appropriate that they are included alongside medical practitioners.

There may be some consultant biomedical scientists who perform primary screening. The scheme does not expect these individuals to participate in EQA as a cytology screener.

7.3 Medical staff

The 2 major activities undertaken by most pathologists who routinely report for the NHS cervical cytology screening programme are as follows:

1. Reporting of slides referred from cytology screeners and checkers as potentially abnormal.
2. Review of slides previously reported by cytology screeners or checkers as negative or inadequate, and later identified as requiring medical review.

The scheme assesses a pathologist’s performance in providing an opinion on slides identified as potentially abnormal, and in reviewing negative and inadequate slides that have been through primary screening. Cytology screeners and checkers may wish to participate in this type of EQA in addition to their regular EQA. The scheme does not use their responses for performance assessment and does not include them in the final results analysis of the slide set.

For medical staff, the scheme uses slides that cytology screeners and checkers have screened in line with routine practice in the laboratory. Practice varies between laboratories, but cytology screeners and checkers often mark slides. Where this is normal practice, the EQA slides should be marked for assessment purposes. Pathologists and consultant biomedical scientists should know which slides cytology screeners and checkers have referred, but not the suggested classifications offered by either group.

The response form used by medical staff when reporting slides is in line with recommendations from the BAC, in particular their Code of Practice for Cytopathology Laboratories. This states that the pathologist should see all abnormal material and a proportion of negative material to make sure that laboratory is maintaining accuracy and quality.

A pathologist has experience of screening unmarked slides (and, in particular, of rescreening negative slides when abnormalities have subsequently been found), and of rescreening an entire slide when equivocal cell groups have been identified as requiring an opinion. Pathologists who undertake EQA assessment examine the whole of every slide, including slides that have already been marked by cytology screeners and checkers.

The cytology screeners selected to screen slides in the pathologists’ set mark these as they would in routine practice. They indicate (with an ‘R’ on the referral sheet), any slides that they consider potentially abnormal. The checkers then look at the slides marked with an ‘R’ and indicate (with an ‘A’ on the referral sheet) any slides that they would refer to the pathologist or consultant biomedical scientist as potentially abnormal. The checkers also mark the slides as they would in routine practice.

The pathologists and consultant biomedical scientists then perform their EQA on the marked slide set using only the referral sheet for reference (they remain unaware of cytology screener or checker opinions concerning the classification of the slides). Participants examine all EQA slides within the set, and pathologists or consultant biomedical scientists check the entire slide and not just areas marked as potentially abnormal.

All 10 slides count towards the assessment. The scheme assesses pathologists and consultant biomedical scientists on their ability to distinguish between negative, inadequate, and abnormal slides, the grading of each abnormal slide, and any instances of missed dyskaryosis.

Pathologists and consultant biomedical scientists may also choose to examine the non-medical slides for educational purposes, and these scores are not included in the performance analysis.

7.4 Trainee staff

Trainee cytology screeners, trainee biomedical scientists and trainee medical staff who intend to work in the field of cervical cytology are encouraged to participate in EQA.

The scheme is of educational value for trainee staff. The scheme assesses them in the same way as qualified staff within their peer group and excludes their results from any consensus calculation or performance analysis of the slide set.

7.5 Additional slide sets

If participants choose to view extra slide sets in addition to their allocated EQA slide set, results from these additional sets are not included in the performance analysis and not recorded on EQA Online.

8. The assessment

8.1 Recording and submitting responses

All participants in the EQA scheme record their opinion of the cytological pattern on an EQA slide in a format consistent with both standard BAC reporting and the standard request form (HMR 101).

The response form

The primary method for recording and submitting responses is by EQA Online.

Where this is not possible, the response form is available in manual format for those participants unable to access EQA Online.

The response form includes mandatory fields for:

• personal identity code (only on paper or electronic version, not online)
• laboratory code
• date of EQA
• test set identity

Special interest slides are not part of the assessment.

The EQA facilitator provides participants with instruction on completing whichever format form they use. The EQA facilitator makes participants aware of the penalties for transcription errors.

Participants must note that:

• only one box must be marked in the result column of the cytological pattern section for each slide
• responses must be in the form of a tick, cross or strike (paper/electronic version), and any other marks are not acceptable and will be recorded as a ‘no response’
• a mark (tick, cross or strike) which spans more than one box is recorded as a ‘transcription error’
• if a box is marked in error, it must be blocked out completely using ink or correction fluid (paper version)
• failure to mark one box or marking more than one box in any column is recorded as a ‘transcription error’

Recording and submitting responses using the manual form

Participants are responsible for recording their responses on the form and submitting them as instructed by the EQA facilitator. Participants cannot change their responses after submitting them.

The EQA facilitator will enter the submitted responses into EQA Online, on behalf of the participant, exactly as they are recorded on the manual form.

Recording and submitting responses using EQA Online

Participants using EQA Online must upload their responses immediately after screening their slides, and while the slides are still in the laboratory. EQA Online prompts participants to check and confirm their responses before submitting them.

Participants using EQA Online are responsible for submitting their own responses. The EQA scheme staff will not assist participants entering their responses into EQA Online.

Participants who do not enter their responses via the online system will be unauthorised non-participants and subject to the conditions of sub-standard performance.

All participants must keep a paper or electronic record of their responses after submitting them and until they receive the final results. This includes participants who record their results using EQA Online, to mitigate the potential loss of EQA data in the event of IT system failure.

8.2 Following assessment

Release of submitted responses

Once all participants (or one of the peer groups) within the laboratory have viewed the EQA slides and recorded and submitted their results, the opinions that have been provided by the submitting laboratory are released to those participants while the EQA slides are still in the laboratory.

Interim feedback

EQA Online makes interim feedback available to view in the form of submitted opinions. Paper or electronic format interim feedback is available for participants unable to access EQA Online. This allows individuals to compare their own responses with the submitted opinions for the EQA slides at an early point.

Interim feedback is provisional and participants must not assume this will necessarily match their final result. The interim report form clearly states that the results are confidential to the individual participant.

The scheme bases final EQA results on consensus opinions that are only available at the end of the round.

Final feedback

The scheme provides confidential final feedback to each participant at the end of the EQA round. This is in a report (or series of reports) and includes an overall score of performance in the scheme. Verbal feedback on individual performance is not available.

8.3 Participant reports

The national cervical QA lead and the EQA facilitator have joint authorisation to issue reports to individuals at the end of each completed round. The scheme issues participant reports within the 6-month period allocated for each round (no later than 30 September for the first round, and no later than 31 March for the second round wherever possible). Participants must keep their own records of performance in proficiency testing, including the outcomes of investigations of any unsatisfactory results.

Performance results are a useful record for participants, although there are limitations on using these to determine competence. Successful performance may represent evidence of competency for a particular round but may not reflect an individual’s ongoing competence. Similarly, unsatisfactory performance in a particular round may reflect a random departure from a participant’s normal competence.

8.4 Certificate of participation

All eligible individuals will receive a certificate of participation following the validation and issue of the final results reports. All eligible laboratories will also receive a certificate of participation.

9. Slide consensus

9.1 Consensus opinion

The scheme uses consensus agreement for the final analysis of participants’ performance. The scheme bases the ‘correct’ classification of a slide on consensus opinion determined at the end of the round, and is on 80% consensus of participant results for each peer group.

Within the appropriate peer group that has submitted a valid response for an EQA slide, at least 80% of the participants will have agreed in their classification of that slide. Such calculations only take into account EQA responses from participants who participate in primary screening or the issuing of reports for the NHS CSP. The scheme excludes responses from participants working in laboratories that perform only private screening from the consensus calculations.

If a participant fails to provide a response for a slide, or provides multiple responses for a slide, the scheme excludes them from the consensus calculations for that slide and scores them according to the guidance.

9.2 Data handling and calculations

To simplify data handling and calculations, the EQA facilitator keeps a record of responses by participant number (not name) for each EQA slide.

Automatic electronic calculation of consensus scoring is acceptable, provided that these calculations have been previously audited and validated.

Although the calculation of consensus results in a delay in the provision of final feedback to participants, the difficulties inherent in reaching an agreement on cytological classification mean that this approach is necessary.

9.3 Slides with consensus different to the submitted classification

If the consensus opinion differs significantly from that of the submitting laboratory and review panel then the EQA facilitator informs the laboratory and returns the slide immediately.

This includes any slides where there is a mismatch between the submitted opinion and the consensus opinion as to whether the slide is negative, inadequate, or abnormal. It also applies to slides in the pathologists’ set with 80% consensus as high-grade dyskaryosis (moderate) or worse, submitted as showing:

• borderline change in squamous cells
• borderline change in endocervical cells
• low-grade dyskaryosis

The scheme scores participants on the consensus opinion.

It is the responsibility of the medical head of the submitting laboratory to take any necessary action following mismatch between submitted and consensus opinion.

9.4 Consensus on negative, inadequate and normal slides

The scheme uses consensus agreement on all valid participant responses in the peer group for negative, inadequate and abnormal slides.

Consensus calculations for the distinction between negative, inadequate and abnormal

To calculate the consensus agreement for the distinction between negative, inadequate, and abnormal slides, codes on the response form deemed abnormal are:

• 8 (borderline change in squamous cells)
• 9 (borderline change in endocervical cells)
• 3 (low-grade dyskaryosis)
• 7 (high-grade dyskaryosis (moderate))
• 4 (high-grade dyskaryosis (severe))
• 5 (?invasive squamous carcinoma)
• 6 (?glandular neoplasia of endocervical type)
• 0 (?glandular neoplasia (non-cervical)

If all participants have submitted the same response for each slide classified as negative, inadequate, or abnormal then the consensus agreement for the slide is 100%. For slides where there is more than one classification suggested, the scheme calculates the consensus agreement using the classification that has the highest number of responses, as follows:

Consensus % = highest number of participants giving the same classification ÷ total number of participants giving a valid response for the slide x 100

9.5 Identifying missed dyskaryosis

Missed dyskaryosis is defined as a participant reporting a slide as negative or inadequate which is classified by consensus as dyskaryotic.

Consensus for identifying missed dyskaryosis

This is calculated by assessing whether all responses of low-grade dyskaryosis or worse within the group total 80% or greater.

9.6 Grading consensus for abnormal slides

The scheme uses consensus agreement on the grading of abnormal slides for all valid opinions reported on the slide by each peer group. The scheme provides grading consensus for cytology screeners and checkers for educational feedback only.

The scheme uses consensus opinion and scoring of pathologists and consultant biomedical scientists for the grading of abnormal slides on EQA responses from pathologists and consultant biomedical scientists only.

If the consensus agreement (calculated in section 6.4) was 80% or greater for an abnormal classification of a particular slide, then the scheme calculates grading consensus for that slide as follows:

Consensus % = highest number of participants giving the same grade ÷ total number of participants giving a valid response for the slide x 100

The scheme permits the amalgamation of adjacent grades to reach 80% consensus. The scheme combines the grade with the highest level of consensus with the grade with the next highest consensus, provided that the 2 grades in question conform to the following groupings:

• borderline change in squamous cells and borderline change in endocervical cells
• borderline change (squamous and endocervical cells) and low-grade dyskaryosis
• low-grade dyskaryosis and high-grade dyskaryosis (moderate)
• high-grade dyskaryosis (moderate) and high-grade dyskaryosis (severe)
• high-grade dyskaryosis (severe) and ?invasive squamous carcinoma
• high-grade dyskaryosis (severe) and ?glandular neoplasia of endocervical type, ?glandular neoplasia (non-cervical), or both ?glandular neoplasia categories
• ?invasive squamous carcinoma and either ?glandular neoplasia of endocervical type, or ?glandular neoplasia (non-cervical), or both ?glandular neoplasia categories
• ?glandular neoplasia of endocervical type and ?glandular neoplasia (non-cervical)

If combining the different categories of borderline change still does not achieve 80% consensus, the scheme permits amalgamation with a third grade (low-grade dyskaryosis). Similarly, the scheme allows the combination of ?glandular neoplasia categories with each other or another adjacent grade to reach consensus. The scheme also permits amalgamation of both ?glandular neoplasia categories plus one other adjacent grade (which has the third highest level of consensus).

In the unlikely event that the categories on either side of the grade with the highest level of consensus have the same percentage consensus, follow the guidance given in the amalgamation of grades table.

These amalgamations exist for scoring purposes only and may not be compatible with any specific management recommendation.

9.7 Slides not reaching consensus

The scheme removes slides not reported with 80% consensus as negative, inadequate, or abnormal from any performance analysis. This means scoring participants only against the remaining slides in the set that reach consensus. To achieve statistical validity, a minimum of 6 slides within any set has to reach consensus.

9.8 Slides with abnormal consensus but no grade consensus

If a slide has at least 80% consensus for an abnormal classification, but does not reach 80% consensus for the grading of that slide (even when amalgamating adjacent grades), there is no score allocated for grading.

See table illustrating the amalgamation of grades when both adjacent grades have equal consensus.

10. Scoring

10.1 Consensus slides

The scheme only scores participants on slides in their EQA slide set that reach 80% consensus.

The scheme scores participants from laboratories carrying out only private screening work against the consensus for each slide calculated using responses from participants who work in the NHS CSP.

The basis of EQA schemes generally is an 80% consensus agreement, and that consensus may or may not be correct.

10.2 Cytology screeners and checkers

NIA score

Cytology screeners and checkers are marked on their ability to distinguish between negative, inadequate, and abnormal slides (an NIA score) as follows:

Slide correctly interpreted as negative or inadequate or abnormal = 2 marks.

Slide incorrectly interpreted = 0 marks.

Incorrect interpretations can be because the slide has:

• a negative consensus and the participant has reported it as inadequate or abnormal
• an inadequate consensus and the participant has reported it as negative or abnormal
• an abnormal consensus and the participant has reported it as negative or inadequate

See the scoring matrix for cytology screeners and checkers.

Missed dyskaryosis

Where cytology screeners and checkers have reached consensus that a slide shows dyskaryosis (80% or more of participants have classified it as low-grade dyskaryosis or worse), the scheme records a participant who reports the slide as negative or inadequate as having missed dyskaryosis. This counts as substandard performance.

Educational value

To enhance the educational value of the scheme, qualitative and quantitative cytology screeners and checkers receive feedback on their grading of abnormal slides.

10.3 Pathologists and consultant biomedical scientists

NIA score

The scheme scores pathologists and consultant biomedical scientists on their ability to distinguish between negative, inadequate, and abnormal slides (an NIA score).

In addition to the NIA score, the scheme allocates extra marks for grading of consensus abnormal slides:

Correct grading of slide = 2 marks.

Grading of slide within one category either side = 1 mark.

Grading of slide wider than one category either side = 0 marks.

Participants do not lose a mark for reporting a slide that has a consensus report of borderline change in squamous cells as borderline change in endocervical cells, and vice versa.

See the scoring matrix for pathologists and consultant biomedical scientists.

Missed dyskaryosis

If a slide has reached 80% consensus among pathologists and consultant biomedical scientists as showing dyskaryosis (80% or more of participants have classified it as low-grade dyskaryosis or worse), the scheme records a participant who grades the slide as negative or inadequate as having missed dyskaryosis. This counts as substandard performance.

10.4 Transcription errors

If a participant fails to tick a box on the response form for a slide, or ticks more than one box for a single slide, this is penalised as an overcall or undercall as follows.

No box ticked

If a participant ticks no box then no mark is given. If the consensus report is dyskaryosis, this counts as missed dyskaryosis, and thus as substandard performance.

Two boxes ticked (both abnormal)

If the consensus is abnormal, 2 marks are given for correctly classifying the slide as abnormal (NIA score), then marked for grading using the ticked box that is furthest away in grade from the consensus grade. For example, if a participant ticks both low-grade dyskaryosis and high-grade dyskaryosis (severe), and the consensus is ?invasive squamous carcinoma, the mark is as if the participant has only ticked low-grade dyskaryosis. If the consensus is negative or inadequate, the participant receives zero marks.

Negative or abnormal ticked, and abnormal ticked

If the participant ticks negative or abnormal and abnormal then no mark is given. If the consensus is dyskaryosis, this counts as missed dyskaryosis, and thus substandard performance.

One box is marked inadequate and one is marked negative

If one box is marked inadequate and one is marked negative then no mark is given. Where consensus is dyskaryosis, this is missed dyskaryosis and thus substandard performance.

Two or more responses given for a single slide

Where a participant has given 2 or more responses for a single slide the consensus calculation excludes those responses for that slide.

10.5 Final scores

To make sure that scores are comparable within peer groups (cytology screeners and checkers or pathologists and consultant biomedical scientists), the total score for each participant is converted into a percentage of the possible score for the particular slide set that was used (after any non-consensus slides have been removed).

% score = participant’s score ÷ maximum possible score for the slide set x 100

The scheme produces separate lists of scores for cytology screeners and checkers, and pathologists and consultant biomedical scientists. The scheme lists scores by participant code and ranks them in percentage score order. Reported scores do not include trainee staff or staff from laboratories that do not participate in the NHS CSP.

The scheme expresses percentage scores to 2 decimal places, and rounds down values below 4 in the third decimal place and rounds up values of 5 or above.

11. Assessment of performance

The assessment of personal performance differs for the 2 peer groups of participants.

11.1 Cytology screeners and checkers

For cytology screeners and checkers, the scheme assesses personal performance on the individual’s ability to distinguish between negative, inadequate and abnormal slides. It also counts any instance of missed dyskaryosis.

11.2 Pathologists and consultant biomedical scientists

For pathologists and consultant biomedical scientists, the scheme assesses personal performance on an individual’s ability to distinguish between negative, inadequate and abnormal slides, the grading of cytological patterns for abnormal slides, and any instance of missed dyskaryosis.

11.3 Identification of substandard performance

The scheme specifies substandard performance as:

• a score that falls below the bottom 2.5 percentile point when percentage scores are placed in rank order for the slide circulation
• an instance of missed dyskaryosis

Calculation of the bottom 2.5 percentile point

It is possible to calculate the bottom 2.5 percentile point in Excel by using the inbuilt function key (fx) on the tool bar. The percentile function is in the statistical function category. The ‘array’ of data is all percentage scores in the rank and the ‘K’ value is the percentile value sought, expressed as a decimal (such as 0.025). The value obtained is the bottom 2.5 percentile point of all the results. The scheme identifies results falling beneath this value as substandard for the purposes of EQA performance assessment.

Automatic electronic calculation of the bottom 2.5 percentile point is acceptable, provided that the scheme has previously audited and validated these calculations.

It is also possible to calculate the score at the 2.5 percentile manually, as shown in the following example:

The calculation is based on an EQA round consisting of 68 percentage scores that range from 100% at the upper level to 60% at the lower level. The score of the second lowest participant (R2) was 66%; that of the third lowest participant (R3) was 72%.

The formula used is:

1 + P(N – 1) = a + b

where a is the integer of the result indicating the ranked position of the score, b is the fraction indicating the proportion of the difference between scores at (a) and (a + 1), P is the percentile (0.025), and N is the number of participants (68).

1 + 0.025(68 – 1) = a + b

1 + (0.025 x 67) = a + b

1 + 1.675 = a + b

a + b = 2.675

a = 2

b = 0.675

Since a = 2, the score of the second lowest participant is taken as the baseline; if the calculation resulted in a = 1 then the score of the lowest participant would be used as the baseline, or the third lowest participant if the result was a = 3 and so on. With a = 2, the value of b, 0.675, relates to the proportion of the difference between the scores of the second and third lowest participants (if the calculation resulted in a = 1 then the value of b would relate to the difference between the scores of the lowest and second lowest participant and so on).

Calculate the proportional difference between the 2 figures as follows:

(R3 – R2) x 0.675

(72 – 66) x 0.675

6 x 0.675 = 4.05

Calculate the desired percentile by adding this difference to the score of the second lowest participant (as a = 2 in the first part of the calculation) as follows:

66 + 4.05 = 70.05%

Any scores below 70.05 are below the 2.5 percentile point and considered to be substandard performance.

Owing to the nature of the scoring system, on some occasions there may be individuals whose scores equal the 2.5 percentile point with no scores falling below the value. In such cases, the scheme does not consider these individuals to have shown substandard performance.

Participants from private (non-NHS contracted) laboratories or overseas laboratories have their percentage scores compared to the 2.5 percentile point after calculations are completed for the other participants. Their individual percentage scores are therefore not included in the calculations to determine the 2.5 percentile.

The scheme identifies substandard performance by placing the scores for each peer group in rank order for each circulation. The scheme classes individuals with scores below the 2.5 percentile point as having substandard performance for that EQA round.

Missed dyskaryosis

The scheme identifies missed dyskaryosis as a participant report of negative or inadequate for a slide that has a consensus classification from the relevant peer group of dyskaryosis. This is reached by combining all valid responses of low-grade dyskaryosis, high-grade dyskaryosis (moderate), high-grade dyskaryosis (severe), ?invasive squamous carcinoma, ?glandular neoplasia of endocervical type and ?glandular neoplasia (non-cervical).

Notification of substandard performance

If a participant scores below the 2.5 percentile and misses dyskaryosis, this is deemed to be a single instance of substandard performance for that round.

Any participant with substandard performance receives a confidential letter notifying them of this, and offering support from the scheme organiser. Although this does not constitute persistent substandard performance, the scheme encourages staff to discuss a result of this nature with their training officer or line manager.

11.4 Persistent substandard performance

The scheme defines persistent substandard performance as the occurrence in 2 out of 3 consecutive EQA rounds of:

• scoring below the 2.5 percentile point
• missed dyskaryosis

After each round, results are analysed to identify persistent substandard performance based on the participant’s EQA results for the 3 most recent rounds. The EQA facilitator keeps a record of participants’ previous scores complete with comments to identify persistent substandard performance easily.

The scheme does not classify a participant as showing persistent substandard performance on the results of a single round. For example, if a participant misses dyskaryosis on 2 or more occasions in one round, or misses dyskaryosis on one or more occasions and also falls below the 2.5 percentile point in that round, his or her performance is graded as substandard for that round, but not as persistent substandard performance.

11.5 Action points for persistent substandard performance

NQAAP (histopathology and cytopathology) defines when a participant reaches action points and the remedial measures taken subsequently.

First action point

The participant reaches the first action point when the scheme identifies their persistent substandard performance.

Second action point

The second action point is reached when a participant’s performance continues to be substandard in 1 out of 2 consecutive rounds. Persistent substandard performance is determined on a ‘rolling’ basis (using the results of the last 3 consecutive rounds). Substandard performance in either of the next 2 rounds after triggering the first action point means that the participant has reached the second action point.

See examples of the various combinations of results that can lead to the first and second action points, depending on an individual’s pattern of persistent substandard performance.

The specific action a participant takes when reaching action points is different for each peer group.

Actions taken when the first action point is reached

Medical staff and consultant biomedical scientists:

1. The scheme writes to the participant explaining they have reached the first action point and requests evidence of a remedial action plan be submitted within 4 weeks of the correspondence date.

2. The participant confirms receipt of correspondence and provides evidence to the scheme that they have implemented a remedial action plan within 4 weeks of correspondence date.

3. If there is no response from the participant after 4 weeks, the scheme sends them a reminder.

4. If there is no reply from the participant within 3 weeks of the reminder, the scheme informs the cellular pathology NQAAP chair.

Non-medical staff

1. The scheme writes to the participant explaining they have reached the first action point and requests evidence of a remedial action plan be submitted within 4 weeks of the correspondence date.

2. The scheme sends a copy of the correspondence to the participant’s scientific head of department and their medical head of department.

3. The scientific head confirms receipt of correspondence and provides evidence that the participant has implemented a remedial action plan within 4 weeks of the correspondence date.

4. If there is no response from the scientific head after 4 weeks to confirm the participant has implemented a remedial action plan, the scheme sends them a reminder.

5. If there is no reply from the scientific head within 3 weeks of the reminder, the scheme organiser informs the cellular pathology NQAAP chair.

Actions taken when the second action point is reached

Medical staff and consultant biomedical scientists

1. The scheme informs the cellular pathology NQAAP chair.

2. The cellular pathology NQAAP chair convenes an investigation panel.

3. The investigation panel seeks possible explanations for the persistent substandard performance.

4. The cellular pathology NQAAP chair corresponds with the participant via the EQA facilitator.

5. If there is no response, a lack of co-operation or no reasonable explanation from the participant after 4 weeks, the cellular pathology NQAAP chair will inform the QAPC chair.

6. The QAPC chair will inform the chair of the RCPath professional performance committee, the employing trust medical director and the Health & Care Professions Council.

7. The chair of the RCPath professional performance committee will convene a review panel.

Non-medical staff

1. The scheme informs the cellular pathology NQAAP chair and writes to the participant confirming the course of action.

2. The scheme sends a copy of the correspondence to the participant’s scientific head of department and their medical head of department.

3. The cellular pathology NQAAP chair and the scheme agree further action.

Developing a remedial action plan

Participants identified as persistent substandard performers are required to produce and implement a remedial action plan. The participant (jointly with the laboratory manager when necessary) will provide evidence that the plan has been implemented and appropriate action taken.

Suggested components of a remedial action plan are:

• explanation of the reason for substandard performance
• review of workload figures for cervical cytology
• review of screener and checker performance data for the period of EQA substandard performance consisting of abnormal sensitivities, high grade sensitivities and reporting profiles
• review of pathologist and consultant biomedical scientist performance data for the period of EQA substandard performance consisting of positive predictive value (PPV), abnormal predictive value (APV) and referral value (RV)
• proposals for a review of in-house teaching sets or the scheduled attendance of suitable training
• reflective statement taking into account the wider perspective of the working environment and current performance

Participants from outside the UK

Non-UK laboratories declare their professional lines of accountability within their employing organisation and regulatory body under the terms and conditions of the EQA agreement. The EQA scheme reports persistent substandard performance to the relevant regulatory body.

12. Communication

12.1 Feedback

The scheme considers any concern or negative comment relating to any aspect of its operation or management to be a complaint. Scheme participants are welcome to express concerns or suggest improvements via email, telephone or letter. The national EQA team actively seeks feedback (both positive and negative) through a national annual survey and a participants’ forum.

The scheme defines the term ‘complaint’ as dissatisfaction about or criticism of the EQA operation or management. If a communication displays the criteria indicated, the scheme will treat it as a complaint. The EQA facilitator logs all communications, takes action as needed, and uses them to help identify system improvements.

12.2 Complaints

Complaints go directly to the EQA facilitator PHE.csp-admin-geqa@nhs.net in the first instance. In the event of failure to resolve a complaint, the individual in question can complain directly to the scheme organiser. If the issue is still unresolved, the participant can then refer their complaint to the chair of the EQA management board. The management board decision is final.

The complaints procedure describes how participants can lodge a complaint regarding any aspect of scheme operation or management.

12.3 Appeals

If an EQA participant wishes to appeal regarding their performance assessment or results, they must contact the EQA facilitator at PHE.csp-admin-geqa@nhs.net in the first instance. In the event of escalation, the participant can ask to have their appeal escalated to the scheme organiser.

If the issue is still unresolved, the participant can then refer their complaint to the chair of the EQA management board. The management board decision is final.

The appeals procedure describes how participants can lodge a appeal against an evaluation of their EQA performance.

12.4 Protocol changes

The protocol may undergo changes when improvement opportunities are identified, or when non-conformity within a process has been identified and requires corrective and, or preventive action.

All participants are required to confirm, via EQA Online or through the submission of a written declaration that they have read and accept the terms specified in the protocol.

Changes may also arise as a consequence of participant feedback or direct request from the operational group, steering group or management board.

The scheme uses a change control procedure to prioritise and manage all changes. The EQA facilitator notifies participants of agreed changes to the scheme and the date when they are to come into effect.

12.5 Participants’ annual meeting

The participants’ meetings are a forum for staff to view scheme results comparisons, discuss operational aspects of the scheme and review any challenging slides and educational cases. Meetings can be held either face-to-face or by virtual means.

The EQA operational group manages feedback from the participants’ annual meeting and provides a summary in the annual report.

The annual meetings are open to all staff. Laboratories are encouraged to send at least one member of staff who can then cascade information to colleagues.

12.6 Scheme audit

The EQA operational group sets the scope of the audit strategy and prepares the internal audit programme. The EQA management board is responsible for authorising the audit strategy and audit programme, and can advise adjustments if necessary.

The scheme organiser is responsible for notifying participants if audit findings indicate that non-conformities have affected participant EQA results.

The national cervical QA lead is responsible for all UKAS related communications and for meeting the overall requirements of external assessments.

A summary of audit activity is included in the annual report.

12.7 Surveys

The scheme uses short surveys to obtain feedback or views from participants on actual or proposed scheme activity.

There is an annual survey that all participants are encouraged to complete. They can register their satisfaction with the EQA service and provide constructive criticism, comments and improvement suggestions.

The EQA operational group manages findings from the participant surveys, and a summary is included in the annual report.

12.8 Annual report

The scheme produces an annual report for:

• participating laboratories and participants
• members of the EQA management board, EQA steering group and EQA operational group
• clinical professional groups (laboratories, education and training)
• directors of pathology screening education and training centres
• national screening QA service lead
• regional cervical screening QA portfolio leads (for cascade to regional cervical screening QA staff)
• RCPath
• UKAS

The EQA facilitator distributes the annual report widely within the programme by the end of the second quarter of each financial year.

13. Alleged collusion

13.1 Purpose

To provide a procedure for reporting and investigating collusion or any behaviour witnessed or strongly suspected, whether deliberate or not, which could compromise the validity of EQA results and consequently damage the integrity of the scheme.

13.2 Scope

This procedure applies to all scheme participants and EQA staff.

13.3 Policy

Participants must not pass on information during the EQA round to allow another person to submit their opinions as their own. The scheme defines such behaviour as collusion. The scheme takes seriously collusion or any behaviour that threatens the integrity of the scheme and escalates it to the highest authorities.

13.4 Responsibilities and authorities

All individuals have a responsibility for reporting collusion or any behaviour that could potentially undermine the scheme’s integrity.

The scheme organiser is responsible for responding to a reported allegation of collusion. The scheme organiser is responsible for collaborating with the individuals involved and the appropriate laboratory lead.

The appropriate laboratory leads are responsible for any local investigation and notifying the outcome to the scheme organiser.

If the investigation proves collusion, the appropriate laboratory lead must refer the matter to the relevant professional regulatory body and notify the scheme organiser.

13.5 Procedure

In the first instance, the individual must notify the EQA facilitator in writing if they witness or strongly suspect collusion. The EQA facilitator refers the information to the scheme organiser.

Medical staff

1. The scheme organiser writes to individuals separately via the EQA facilitator within 7 days of receiving notification.
2. The recipients must acknowledge receipt of the letter and respond to the scheme organiser within 14 days. The recipients should identify themselves to the clinical head of the laboratory and offer an explanation for the allegation of collusion.
3. The scheme organiser writes to the clinical head of the laboratory setting out the concern, and explains that the individuals are required to identify themselves to them. The email does not identify the individuals concerned. The clinical head of the laboratory should acknowledge the letter and respond to the scheme organiser within 14 days of receipt.
4. The clinical head of the laboratory provides appropriate feedback and guidance to the individuals concerned. The clinical head of the laboratory writes to the scheme organiser to confirm the outcome of any local investigation within a further 14 days.
5. If no acknowledgement is forthcoming by 20 days then the scheme organiser sends a reminder. If the scheme organiser does not receive a reply after a further 7 days, they notify the cellular pathology NQAAP chair. The cellular pathology NQAAP chair speaks with the national cervical QA lead about further action.
6. If the investigation proves collusion, the scheme organiser invalidates the EQA round of the individuals concerned and counts it as substandard performance. The clinical laboratory lead reports the matter to the organisation’s human resources (HR) department, Screening Quality Assurance Service (SQAS) and relevant regulatory body. The clinical laboratory lead notifies the scheme organiser who in turn notifies the cellular pathology NQAAP chair.

Non-medical staff

1. The scheme organiser writes to individuals separately via the EQA facilitator within 7 days of receiving notification about the allegation.
2. The recipients should acknowledge the letter and respond to the scheme organiser within 14 days of receipt. The recipients should identify themselves to the scientific head of the laboratory and offer an explanation for the allegation of collusion.
3. The scheme organiser writes to the laboratory scientific head for cervical cytology setting out the concern, and confirms they have advised the individuals to identify themselves. The email does not identify the individuals concerned.
4. The scientific head of laboratory should acknowledge the letter and respond to the scheme organiser within 14 days of receipt.
5. The scientific head of laboratory provides appropriate feedback and guidance to the individuals concerned. The scientific head of the laboratory writes to the scheme organiser to confirm the outcome of any local investigation within a further 14 days.
6. The laboratory’s scientific head informs the lead consultant for the cytology service and the medical head of department. If it is the scientific head that is under investigation, then a copy letter goes to the consultant pathologist responsible for reporting cervical cytology.
7. If no acknowledgement is forthcoming by 20 days then the scheme organiser sends a reminder. If there is still no response after a further 7 days, then the scheme organiser notifies the cellular pathology NQAAP chair. The cellular pathology NQAAP chair speaks with the national cervical QA lead about further action.
8. If the investigation proves collusion, the scheme organiser invalidates the EQA round of the individuals concerned and counts it as substandard performance. The scientific laboratory lead reports the matter to the organisation’s HR department, SQAS and the relevant regulatory body. The scientific laboratory lead notifies the scheme organiser who in turn notifies the cellular pathology NQAAP chair.

14. Complaints

14.1 Purpose

This guidance provides a procedure for dealing with complaints from any individual in the event of dissatisfaction with any aspect of scheme operation or management.

14.2 Scope

This procedure applies to all participants and EQA staff.

14.3 Responsibilities and authorities

Individuals are responsible for following the procedure if making a complaint. EQA staff are responsible for taking appropriate action and offering a solution within the required timeframes.

Individuals and EQA staff are responsible for keeping complaints confidential between the complainant and the scheme. The scheme will handle all concerns that could infer a complaint as an official complaint and treat these as confidential.

14.4 Procedure

If an individual wishes to make a complaint, then they must do so by writing to the EQA facilitator.

1. The EQA facilitator sends a holding response within 3 working days of receiving the complaint.
2. The EQA facilitator logs the complaint together with a summary of the communication and subsequent discussions that may involve other EQA staff including the scheme organiser.
3. The complainant can expect to receive a response offering a solution within 20 working days.
4. If the matter is not resolved to the satisfaction of the complainant then the EQA facilitator will refer the matter to the scheme organiser. The scheme organiser will provide a response within 28 working days.
5. If the matter is still not resolved then the scheme organiser will escalate it to the EQA management board. The EQA management board decision is final. The complainant can expect to receive a final response within 28 working days.

15. Appeals

15.1 Purpose

This guidance provides a procedure for dealing with participants in the event of an appeal against an evaluation of their EQA performance.

15.2 Scope

This procedure applies to all participants.

15.3 Responsibilities and authorities

Participants are responsible for making a timely appeal against an evaluation of their EQA performance. EQA staff are responsible for taking appropriate action and offering a solution within the required timeframes.

15.4 Procedure

If a participant wishes to appeal against an evaluation of their EQA performance, then they may do so by writing to the EQA facilitator within 28 working days upon receipt of their final results report.

The EQA facilitator sends a holding response within 3 working days of receiving the appeal.

The EQA facilitator logs the appeal together with a summary of the communication and subsequent discussions that may involve other EQA staff including the scheme organiser.

The appellant can expect to receive a response offering a solution within 20 working days.

If the matter is not resolved to the satisfaction of the appellant then the EQA facilitator refers the matter to the scheme organiser. The scheme organiser provides a response within 28 working days.

If the matter is still not resolved to the appellant’s satisfaction then the scheme organiser escalates it to the EQA management board. The EQA management board decision is final. The appellant can expect a final response within 28 working days.