Guidance

Guidance notes for equipment evaluation and protocol for user evaluation of imaging equipment for mammographic screening and assessment

Updated 15 September 2025

Introduction

This document provides a framework for practical evaluations of new equipment and significant upgrades for use in the NHS Breast Screening Programme (NHS BSP).  These are required, in combination with a technical evaluation, before equipment is used in the NHS BSP.

The outcomes from the evaluation will be published as a stand-alone report following a successful technical evaluation on the National Coordinating Centre for the Physics of Mammography (NCCPM) website.  In cases where a small practical evaluation is undertaken following a modification, an addendum to the report can be added on completion.

Mammographic X-ray equipment used in the NHS BSP is evaluated in centres where staff routinely perform screening and assessment examinations of women and eligible people.  Evaluations are undertaken to assess the use of equipment in a practical, clinical setting and are not intended to be clinical trials.

The practical evaluation of equipment using protocols provided by the NHS BSP are undertaken by breast screening centres in England.  These evaluations are staged processes, starting with a technical evaluation by the NCCPM.  If the technical evaluation is satisfactory, following approval by the central portfolio team within NHS England (NHSE), a subsequent practical evaluation is conducted, with the support and agreement of the local physics service. This is to provide additional information on performance of the equipment within a screening environment. This includes the quality of diagnostic images and the usability and reliability of equipment in the clinical setting. Following successful completion of the  technical and practical evaluations the full evaluation reports are available on the NCCPM website. The list of current reports is also  shown online.

A successful evaluation and subsequent purchase of any new system within the NHS BSP should not be considered the end of the process. The manufacturer or supplier needs to inform the central portfolio team within NHSE and NCCPM of any significant upgrades e.g. new features such as biopsy facilities; and any proposed changes to the system that can affect the image quality, radiation dose or safety including relevant software upgrades.

Guidance on equipment requiring evaluation

Criteria for evaluation

The general criteria for equipment evaluation are:

• new model of x-ray system
• new technology
• significant modification to existing equipment especially any significant upgrades e.g. new features such as biopsy facilities; and any proposed changes to the system that can affect the image quality, radiation dose or safety including relevant software upgrades.

NHSE may consider other technologies on a case-by-case basis.

Application process

Manufacturers and suppliers apply for evaluation of new equipment before any agreements to install equipment. They will complete the standard evaluation form in appendices 8 and 9. The applicant must provide sufficient technical information and clinical evidence to support their application.

Decision process

The central portfolio team within NHSE, regional commissioning team and Screening Quality Assurance Service (SQAS) will make the decision on the outcome of the evaluation request. This may be undertaken in conjunction with other parties such as the NCCPM.  Any potential changes to image quality and dose will be considered. There are three potential outcomes:  

• full or partial technical and/or practical evaluation approval
• request for further information
• sufficient information for the equipment to be used without further testing

Pre-evaluation of clinical images provided by the supplier

Prior to undertaking the technical or practical evaluations, the supplier/manufacturer may be asked to supply NCCPM with a range of anonymised clinical images including cancers, benign lesions and normal cases for review by both the NCCPM and image readers at the service who will be undertaking the practical evaluation. The minimum number of cases should be 50. The cases must contain the typical range and proportion of cancers for each imaging modality being tested. At least 30 of the cases should contain cancers. If more than one modality are to be evaluated, e.g. 2D and tomosynthesis, then data sets for each must be sent. The diagnostic quality of the images will determine whether the technical evaluation should proceed. The images should be jointly reviewed by a team of at least one radiologist, one radiographer and one physicist with substantial experience of reviewing clinical images (preferably from NCCPM).

Guidance for practical evaluation centres

Process for choosing a site

Manufacturers and suppliers are encouraged to suggest potential sites to the NHS BSP and NCCPM for practical equipment evaluations via completion of the standard evaluation form (appendices 8 and 9) but the final decision lies with the central portfolio within NHSE and regional commissioning team following review of their eligibility.  This will include assessment of the following:

• appropriate service size
• robust physics support
• named lead with capacity to co-ordinate and undertake the evaluation
• achievement of KPIs involving screening round length and waits to results and assessment

Site suitability for undertaking a practical evaluation

Breast screening services taking part in the practical evaluation of equipment must fulfil the following criteria to ensure that they are able to provide the appropriate level of expertise and sufficient throughput of women for screening mammography: 

• it must be possible to X-ray specimens before the evaluation moves on to the next stage of imaging women
• the equipment must be capable of receiving a worklist from the National Breast Screening System (NBSS)
• the breast screening centre selected for the evaluation must have reader training for the technology type being evaluated
• the screening unit should have sufficient throughput for the period of the evaluation
• a robust Quality Assurance (QA) system must be in place at the centre, so that it meets all relevant NHSBSP objectives and guidelines
• there must be a robust system in place for the unprocessed images to be stored on the Picture Archiving and Communication System (PACS).  These images are preferable in case there are issues with image processing and must be retrievable when required for comparison with new examinations
• the local mammography physics service, and all radiographers and radiologists involved in the evaluation, must comply with the relevant NHS BSP professional guidelines.  Compliance can be confirmed via a recent QA visit report
• if the evaluation of equipment at a static site indicates any potential issues with equipment robustness or potential practical issues with use in the confined space of a trailer, then an evaluation of the equipment on a trailer may be required
• the local physics service should confirm their willingness and ability to support the equipment evaluation at an early stage
• the service must be able to complete a breast dose survey and have the ability to easily extract client dose data from images or the supplier must ensure that they can easily be removed.

3.1 Service Level Agreement

A Service Level Agreement (SLA) between NHS England and the evaluation centre is required to cover timescales and responsibilities.

3.2 Project management

The screening centre will appoint a project leader to coordinate the evaluation and make sure that all the required areas are covered while ensuring effective communication between the screening centre, the supplier/manufacturer/installer and the NHSBSP representatives. Timescales should be established and agreed with the central portfolio team within NHSE as soon as the availability of the equipment is confirmed.

Adequate arrangements must be made for delivering applications training to all staff involved in the evaluation. A staged approach should be taken, starting with a technical evaluation completed by NCCPM. The technical evaluation could be carried out offsite, in which case NCCPM should report whether the system meets the required standards before a practical evaluation commences.

The equipment should only be used in a clinical setting once the technical evaluation has been successfully completed and any outstanding actions resolved. The manufacturer should confirm that the same model and software version is used for the technical and practical elements of the evaluation.

3.3 Organisation/Trust awareness of the evaluation

If the equipment has been borrowed from the supplier for the period of the evaluation, it is important to have a local, formal agreement in place between the supplier and the host Organisation/Trust to cover costs and liability. For example, some Organisation/Trusts may require the evaluation project to obtain approval from the ethics committee or the novel procedures committee. The supplier must provide indemnity cover for the period of the evaluation.

3.4 Early termination of the evaluation

If the equipment is unreliable or there are concerns about the consistency of dose or image quality, it may be necessary to terminate the evaluation before it is complete. The decision to do so would be taken by the central portfolio team in consultation with NCCPM, the screening director, and medical physics staff involved in the evaluation. The service can pause the evaluation if it felt that there is a risk to the service, pending an agreed resolution or agreement that evaluation must be terminated. If a terminated evaluation needs to be fully or partially repeated due to equipment fault or design, then the supplier may be charged an amount commensurate with the additional work.

Outline of equipment evaluation report and evaluation methods

The project lead (or nominated person) at the breast screening service performing the evaluation will prepare the practical evaluation report. The report should be submitted in draft for review by the central portfolio screening team after completion of the practical evaluation.

The following section can be used both as a template for the report and description of the methods for the evaluation.

Report Template:

Executive summary

This section should be written after the rest of the report is complete. It should briefly summarise the main points and include the following:

• purpose and duration of evaluation
• main outcomes on performance, ease of use, image quality
• indicate whether the results of the dose survey were below the national diagnostic reference level (DRL)
• conclusions

1. Introduction

This section should provide the following information about the evaluation:

• site/location
• population served
• period of evaluation
• installation date and other updates, if any

2.  Equipment evaluated

Objectives

To include the assessment of

• reliability
• ergonomics related to operator
• ergonomics related to client
• image quality
• dose assessment
• interfaces with other equipment/systems as required

 Very brief description of the following should be included:

• acceptance testing and commissioning
• x-ray set and workstation
• other equipment such as biopsy equipment, mag tables
• integration with NBSS and PACS
• any specific practical issues relating to the equipment

3.  Routine Quality Control

This section reports on the daily, weekly and monthly tests undertaken during the evaluation period including reference to the appropriate NHSBSP guidelines:

• Routine quality control tests for full-field digital mammography systems Equipment report 1303
• Routine quality control tests for breast tomosynthesis (Radiographers) NHSBSP Equipment Report 1406

If there are any other tests specifically required by the manufacturer, these should be identified in the report.

A brief summary must be provided to state that all tests were performed for the duration of the evaluation and met the specified requirements.  The summary should include anything which was identified as problematic or where performance was demonstrated to be an outlier.  Graphs and results do not need to be included as evidence of tests undertaken but the team must keep them as a record for evidence of the evaluation.

4. Data on images evaluated and interventional procedures performed

4.1 Dose survey

The service must be able to complete a patient dose survey during the period of the evaluation, which must include a minimum of 200 women. For technologies for which small numbers are imaged routinely, then fewer cases may be acceptable to prevent the evaluation being unnecessarily long, this would need wider discussion with NHS BSP and NCCPM.

Note that the use of dose management software or structured dose reports can make this process of extracting the data easier. Also NCCPM supply DicomTagBrowser to aid and simplify the extraction of the relevant data from images.  

Paste results into table below and outline a very brief summary, including the ease with which dose data could be extracted. (delete columns that are not evaluated).

4.2 Clinic workflow

Summarise in 2 sentences the impact on service delivery within routine mammography setting and assessment (e.g., comment if any delays inherent in use or otherwise) and comment on image reconstruction time.

All evaluations should pay particular attention to any novel design features or modes of operating the equipment that may affect throughput.

4.3 Ergonomics

Equipment design is important in helping to reduce repetitive strain injury in practitioners, as different functions and workflows all play their part in either contributing to or limiting these risks. See guidance on ergonomics good practice.

5. Reader assessment of images

Mammographers working on the system under evaluation are required to keep details of all images taken. Records should be stored in such a way that they can be retrieved and reviewed at any point in time. PACS storage and archiving should be implemented. For the purpose of the evaluation, all unprocessed images should be kept, this may be on the PACS or on the system (until the evaluation and evaluation report is complete).

The evaluation centre are strongly encouraged to read European guidelines for quality assurance in breast cancer screening and diagnosis (Section S2, Part 2, 3.2) for further information on evaluation of clinical images.

Standard recording for images are described in the following sections:

5.1 Reader assessment of diagnostic value of routine 2D images

Include a brief summary of findings (3 or 4 sentences) from Appendix 1 - form 1. Include form 1 as an appendix.

5.2 Reader assessment of diagnostic value of magnification images

Include a brief summary of findings (3 or 4 sentences) from Appendix 2 - form 2.  Include form 2 as an appendix.

5.3 Reader assessment of diagnostic value of biopsy examinations

Include a brief summary of findings (3 or 4 sentences) from Appendix 3 - form 3.  Include form 3 as an appendix.

5.4 Reader assessment of diagnostic value of tomosynthesis images

Include a brief summary of findings (3 or 4 sentences) from Appendix 4 - form 4.  Include form 4 as an appendix.

5.5 Reader assessment of diagnostic value of contrast enhanced digital mammography

Include a brief summary of findings (3 or 4 sentences) from Appendix 5 - form 5. Include form 5 as an appendix. 

5.6 Reliability of equipment

Centres must log any faults, downtime, numbers of women affected by the fault, and resolution of fault. These must also be recorded on the NCCPM equipment database for faults.

Include a brief summary of findings (3 or 4 sentences) from Appendix 6 - form 6.

In addition, include a brief summary of overall comments (3 - 4 sentences) from Appendix 7 - form 7. 

6. Confidentiality

This should include a reference to the NHS Cancer Screening Programme Confidentiality & Disclosure Policy.

7. Training

Summarise the training provided to ensure staff are proficient to use the:

• equipment
• reporting workstation
• additional equipment or software

8. Conclusion & recommendations

This section includes the conclusions and recommendations that have been drawn from the evaluation. It should address all the topics of the objectives as stated in the introduction.

Appendix 1

NHSBSP equipment evaluation form 1: Exposure and image quality record routine 2D

Screening service:

Evaluation centre (mobile/static):

Start of evaluation:

End of evaluation:                                                                                               

Images from 100 women minimum to ensure as much natural variation is captured within the 6-week evaluation period.

Appendix 2

NHSBSP equipment evaluation form 2: Exposure and image quality record - magnification mammograms in assessment

Screening service:

Evaluation centre:

Start of evaluation:

End of evaluation:

All images should be captured within the 6-week evaluation period

• Physical magnification (with mag platform) or high resolution mode ** Grade as excellent (E), good (G), satisfactory (S), poor (P). Note: you may wish to collect further exposure data with different settings such as mA value. Images should also be viewed in optical magnification mode and compared with physical magnification.

Appendix 3

NHSBSP equipment evaluation form 3: Exposure and image quality record - biopsy examinations (use one line for each exposure) for assessment for a period of 6 weeks

Screening service:

Evaluation centre:

Start of evaluation:

End of evaluation:

Type of biopsy (Stereo/Tomo/MRI/CEDM):

All images should be captured within the 6- to 12-week evaluation period

• This should include a check of the measurement tool ** Grade as excellent (E), good (G), satisfactory (S), poor (P)

NHSBSP equipment evaluation form 3: Exposure and image quality record - (use one line for each exposure) for assessment for a period of 6-12 weeks

Present a bar chart of comparative performance by radiological abnormality

Appendix 4

NHSBSP equipment evaluation form 4: Exposure and image quality record digital breast tomosynthesis

Screening service:

Evaluation centre (mobile/static):

Start of evaluation:

End of evaluation:

Images from 100 women minimum to ensure as much natural variation is captured within the 6-week evaluation period

Appendix 5

NHSBSP equipment evaluation form 5: Exposure and image quality record Contrast Enhanced Digital Mammography (CEDM)

Screening service:

Evaluation centre (mobile/static):

Start of evaluation:

End of evaluation:                                                                                               

100 images minimum should be captured within the 6-week evaluation period

Appendix 6

NHSBSP equipment evaluation form 4: Reliability of equipment evaluated

Screening service:

Evaluation centre:

Start of evaluation:

End of evaluation:

Appendix 7

NHSBSP equipment evaluation form 7: Overall comments

Appendix 8: Proforma for suppliers requesting an equipment evaluation

This proforma should be completed by the supplier before a technical evaluation and practical is arranged.

Please give full details as the information enables us to decide on the appropriateness of a NHSBSP technical/practical evaluation at this time. It also helps us to plan the technical evaluation and work more efficiently.

Type of evaluation requested: (delete as appropriate)	Technical and practical (Note that a satisfactory NHSBSP technical evaluation is generally a prerequisite for a NHSBSP practical evaluation).
Supplier
Manufacturer (if different)
Model
Software version
Contact details for company representative: (name, title, email, phone)
Possible site for technical evaluation? (to be agreed) When?
Is the equipment CE marked?
Is it market-ready, that is, no further modification / development expected in the near future?*
Approx. number in use in Europe, rest of world?
Has it been trialled in a busy screening environment for example, 10-12 women per hour?
Clinical trials in progress / completed?  Published?
Technical evaluation already done by EUREF or another European country’s testing body?
Type of equipment: (delete as appropriate)	FFDM (2-D) / tomosynthesis / other (specify)
Type of detector, manufacturer
Pixel size (µm)
Detector size (width and depth (mm x mm)
Is the image flat-fielded?
Target and filter(s)
Number of AEC modes. “Smart AEC”? (which detects dense areas)
AEC mode provided where user selects kV, T/F and system determines mAs?
Available field size(s). Shifting paddle?
Fine focus, mag table available?
Biopsy unit available? (now or later?)
“For processing” DICOM images readily available to physicists? (that is, without engineer assistance)
Radiation Dose Structured Report exported as standard?
Size of images (MB)
Time from start of exposure to image appearing on AWS?
Time from start of exposure to when next exposure can start?
Please provide any other useful information and / or describe any special features not covered above (for example, tomosynthesis capability)
Contact details for company representative: (name, title, email, phone)
Signature and date

*Note: If significant changes are made after the technical evaluation, and before the system is marketed, NCCPM will have to be reimbursed for the cost of any subsequent re-evaluation that is needed to ensure NHSBSP users have up-to-date information.

Appendix 9: Proforma for suppliers requesting an equipment re-evaluation

This proforma should be completed by the supplier before a technical re-evaluation of equipment not yet in use is arranged.

Please give full details, as the information enables us to decide on whether a NHSBSP technical re-evaluation is justified.

Supplier
Manufacturer (if different)
Model
Software version
Date of original technical evaluation
Proposed date and site of re-evaluation
Reason(s) for needing to repeat the technical evaluation (please explain in detail)
Is the equipment NOW market-ready, that is, no further modification / development expected in the near future?
Has the cost of the re-evaluation been agreed with NCCPM?  If so, state cost agreed.*
Any further comments
Contact details for company representative: (name, title, email, phone)
Signature and date

*Note: If significant changes are made after the initial technical evaluation, before the system is marketed, NCCPM must be reimbursed for the cost of any subsequent re-evaluation that is carried out.

Appendix 10: Manufacturer’s comments

Any comments from the manufacturer, after they have reviewed the report, are included in this appendix.