Guidance

Oversight and monitoring activities

Published 28 January 2022

Oversight and monitoring activities can include a broad range of activities, for example:

• the use of committees to manage the trial or review the emerging safety data
• central review of clinical trial data
• documents and reports
• feedback from questionnaires sent to investigators
• data management processes
• statistical review of the data from the trial
• pharmacovigilance signal detection
• regular review meetings (for example, sponsor with contract research organisation (CRO)/chief investigator)

We consider audits and visits to the investigator site by a trial monitor and auditors to assess the conduct of the trial to be part of oversight.

Oversight

Oversight is important where the sponsor has delegated functions to other parties, either within the same organisation (for example, to a chief investigator in the trust) or to an external CRO. The sponsor’s project management or governance should have sufficient processes in place to verify that the functions are being conducted appropriately. The sponsor should be approving documents and processes in place to carry out the delegated functions such as (not exhaustive):

• protocols
• case report forms (CRFs)
• standard operating procedures (SOPs)
• analysis plans
• data management plans

The delegated parties would typically be implementing some or all of the detailed processes decided from the monitoring strategy. Oversight would usually involve:

• assessment of the processes to be used by those delegated parties
• regular review meetings with personnel
• review and approval of specific documentation (such as site visit report, regulatory green light documentation) or perhaps visits or co-visits to investigator sites and audit

Monitoring

Monitoring is one of the main mechanisms where the sponsor can be assured that it is in compliance with the legislation and the trial protocol/procedures. Effective monitoring may also provide an opportunity for feedback to the sponsor for process improvement.

Traditionally, “monitoring” has tended to focus on monitors visiting sites as part of quality control. Commercial sponsors have used this approach extensively, consisting of regular on-site visits covering the activities outlined in ICH GCP guidance. Non commercial trials have primarily taken a more centralised approach to monitoring activities with less reliance on on-site visits.

Audit

Audit activities are sample based and may occur during or after the trial is completed, or for cause in response to issues detected by monitoring activities. The auditor would also assess the effectiveness of the monitoring activities and compliance with the processes outlined in the protocol/monitoring plan/SOPs.

Important factors in determining the oversight and monitoring strategy

Monitoring may aim to assess compliance with every detail of the protocol and trial procedures and conduct checks of every data point for consistency with source documents and validity. Such an approach has been used traditionally and is extremely resource intensive. The main aim is to ensure the rights, safety and wellbeing of the trial participants are protected and that the final results of the trial are reliable.

We recommend that a proportionate approach to the management and monitoring of the trial is based on the trial risk assessment that identifies the areas that matter to achieving the above key aim, that is, those activities/data that, if incorrect, would have a negative impact on participant safety and trial results.

A thorough risk assessment of the trial should take place to identify the risks and then determine the strategies and procedures to mitigate them. This risk-based approach is consistent with ICH GCP where the “sponsor should determine the extent and nature of the monitoring” .

It is not possible to provide an exhaustive list, but the strategy, as considered by risk assessment, is likely to depend on:

• trial categorisation following the MRC/DHSC (Department of Health and Social Care)/MHRA risk adapted approach (A, B or C)
• complexity of the trial protocol and procedures, for example some trials can have complicated protocols involving several different objectives and collection of extensive data
• novelty of the trial, for example, the use of new methods/equipment
• endpoint measurements, for example, could be directly collected/transferred electronically, for example if laboratory based
• size, multicentre, multinational, number of participants, data quantity
• use of vulnerable participants, for example consent process involving parents, carers, legal representatives
• type of investigator site and experience of site staff
• whether you are using electronic data capture – an eCRF allows additional information about who and when data were entered (from the audit trail)
• the use of commercial or academic contract research organisations, which would involve their assessment
• complexity and ease of use of the case report form (CRF)
• training needs of monitoring staff, investigators and research teams
• type and effectiveness of central monitoring approaches available (for example, if it is a single-site trial there will not be the opportunity to compare data across a number of sites)

Documentation of the oversight and monitoring strategy

The risk assessment should document the necessary oversight/monitoring actions put in place to mitigate risks, and further documentation to implement these would usually be prepared. The sponsor decides how this is done.

This may be contained in one document, for example the protocol, an SOP, or a monitoring plan which clearly outlines for a particular trial how the trial would be overseen, managed and monitored to ensure compliance with the regulations.

In many organisations, the oversight and monitoring strategy is contained in a range of documents, for example:

• SOPs
• protocol
• monitoring plan
• committee charters
• data management plan
• data validation plan
• medical monitoring plan
• pharmacovigilance plan

We recommend that there is clarity on how these link together to carry out the oversight and monitoring strategy, and an overriding document to explain this. Such a document would be particularly useful where there is delegation of functions from the sponsor, for example, to CROs in different countries, as it will facilitate consistency. Where these types of documents or plans are being used extensively for trials in an organisation, there should be formal procedures to control the content, version control and maintenance of them.

We recommend that where the sponsor is undertaking many trials, there is a procedure in place for the development of the documentation that implements the oversight and monitoring strategy. For example, templates to ensure all necessary content is addressed for all. Items for consideration for inclusion in documentation could include the following examples (not intended as an exhaustive list):

• a clear link to the risk assessment to identify the areas that matter to the results reliability, participant safety and rights, and compliance with the legislation
• how sponsor oversight of (chief) investigator (non-commercial) and vendors will be carried out
• type of monitoring to be used (central, remote and/or on-site)
• the standards and any SOPs to be followed and any document templates to be used
• cross-references to other relevant documents in the suite of documentation implementing the oversight and monitoring strategy (for example, medical monitoring plan, safety plan, data management plan, committee charters)
• site assessment and initiation procedures
• training of site staff plus training of sponsor and CRO personnel involved in monitoring, audit, data management and so on
• frequency of monitoring activities such as site visits, database interrogation reports
• communication activities with vendors and site(s)
• monitor site capacity (that is: maximum number of sites per monitor in relation to resource/skills)
• where site visits take place, which departments will be visited (in addition to the investigator, any ancillary departments such as pharmacy, laboratories, imaging)
• what data will be reviewed at site (percentage of SDV needed and on what data)
• data validation specifications
• data collection and transfer methods
• computer systems validation/study build validation (for example, for data transfers, eCRF, IRT use)
• procedures for use and monitoring of central laboratories, specialist services such as imaging, ECGs
• considerations for unblinded monitors and reviewers (if necessary)
• expectations for availability of the principal investigator (PI) during monitoring visits
• how non-compliance will be recorded and resolved
• oversight of the investigator and ancillary department site files
• escalation process:
  • for central monitoring, what triggers you will use for escalating to on-site monitoring visits, or triggers from site visits for audit, and so on
  • for both central and on-site monitoring, how unresolved issues or serious non-compliances are handled that the monitor cannot address
• supplies management and monitoring processes (for example, IMP and ancillary materials, subcontractors, QP certification)
• SAE reporting processes and associated monitoring responsibilities
• AE and SAE review, signal detection, medical monitoring
• query management:
  • data queries from data management activities and found by monitoring at site
  • protocol queries to medical monitor
  • IMP (including temperature excursion management)
• documentation and review of monitoring activities:
  • types of reports (for example, monitoring visit report, central monitoring metrics, statistical monitoring)
  • format of the reports
  • responsibility for reviewing the reports
  • timelines for preparing and reviewing reports
• role and procedures of adjudication, data monitoring, management and steering committees
• processes for interim analysis (data gathering, analysis and reporting control)
• sponsor trial master file management
• regulatory and research ethic committee (REC) approval maintenance (process for amendments, urgent safety measures, serious breach evaluation and reporting)
• contract and insurance review and maintenance
• audit plans
• checks on randomisation processes

Appropriately qualified individuals in the sponsor organisation should carry out the review of the documentation. Where there are several documents prepared by different areas, we recommend a cross-functional review to ensure consistency and compliance with the overall risk-based strategy.

There is no requirement to send additional documentation outlining the oversight and monitoring strategy to the MHRA or the REC as part of the approval process, as there is no requirement for this to be approved by the MHRA or REC, and there are currently no plans for this to be introduced in the future. However, the protocol must contain the monitoring policy according to the principles of GCP that must be adhered to, though it may not contain the level of detail as other sponsor documentation such as plans or SOPs.

For multi-country trials, global documentation for the trial is acceptable, but it may be necessary to include specific procedures to mitigate any country-specific risks that were identified from the risk assessment – for example, differences in clinical practice or local regulations. It may be necessary to include some site-specific actions such as additional monitoring checks at the chief investigator’s site (for example, if the sponsor has delegated numerous functions) or the site may be responsible for undertaking a specific trial activity or where a site-specific risk may have been identified.

The monitoring strategy must be followed, together with any procedures described in the protocol (for example, setting up of data monitoring committee) and/or standard operating procedures. Any deviations should be documented, justified, and any potential impact on the trial quality identified.

There must be documented evidence of the activities outlined in the monitoring strategy which would provide evidence to support compliance. While site visit reports are well established monitoring evidence, and take a similar format across organisations, activity reports from central and statistical monitoring are not established in the same way, so all the documentation and checks undertaken to demonstrate the planned central monitoring activities must be retained. It is very important for the documentation to show that any non-compliance issues were effectively dealt with in a timely manner, including documentation of any escalation actions.

Adaptation of oversight and monitoring strategies

The adaptation of aspects of the conduct of the trial based on a trial categorisation to Type A/B/C and trial specific risk assessment is a broad approach that affects all areas of the trial conduct, but beneath this, the activities outlined as the monitoring strategy could include a further flexible or adaptive approach. This ensures that the monitoring tasks undertaken are also risk based and reflect accumulating information about the compliance of the investigator site(s) or from data/information from outside the trial (for example, new legislation).

The document(s) comprising the monitoring strategy (for example, protocol, SOPs, data management plan, medical monitoring plan, safety plan, monitoring plan) should be based on the vulnerabilities identified in the risk assessment, but we also recommend that they contain further risk-based flexibility, for example:

• the intensity of monitoring/oversight and/or visits is high initially and then decreases when compliance is acceptable
• high-intensity (visits, contact) for unknown or new sites with little trial experience compared with sites used many times by the sponsor
• triggering an on-site visit if central monitoring reveals concerns at a particular site (for example, a serious breach has occurred)

There should always be a process and resources available to escalate the monitoring activities when necessary. Such decisions could be delegated to the monitor to decide the appropriate level and frequency of monitoring based on guidance in the monitoring strategy together with the outcome of monitoring activities and actively overseen by the project manager/chief investigator/trial steering group. The monitoring strategy should define how adaptations will be decided and the rationale/justification for the change should be documented. In some cases, the monitoring strategy documents may need to be amended if the planned adaptation is beyond the scope of flexibility built into the original strategy.

We also recommend that you review the strategy when other important documents are amended such as the protocol, the risk assessment or the investigator’s brochure or when significant trial events occur (for example, the addition of extra sites).

The sponsor must ensure that the trial is conducted in accordance with the legislation and principles of GCP. In some trials categorised as Type A, it may be possible to reduce sponsor oversight substantially, but some activity will still be needed. However, the risk assessment of the trial may show that central or on-site monitoring activities may not be necessary. This would be very rare and the trial would need to be very low risk for no such monitoring to be justified. The rationale for the oversight and monitoring strategy would be contained in the risk assessment. For example, this may be limited to sponsor oversight of the initiation process and then a detailed questionnaire returned from the investigator as a self-assessment process, perhaps with addition of an audit.

Trials that are very small, investigator-led, and primarily related to scientific investigation of the mode of action or scientific other aspects of a licensed investigational medicinal product (IMP) rather than general clinical efficacy or safety are more likely to lead to a conclusion that no on-site and no extensive central monitoring is necessary when risk assessed. We recommend that there is a contingency plan to implement monitoring if the oversight activities results this is necessary.

Intensity and focus of the oversight and monitoring activities

It is appropriate for the monitoring to focus or have more detailed review (intensity) on different areas in a trial or in different trials. The following examples illustrate this.

Type A (where the IMP is used in line with normal clinical practice) involving vulnerable participants or emergency research

100% source data verification (SDV) of consent may be indicated but no recording of IMP accountability takes place beyond normal clinical practice resulting in limited or no monitoring of this aspect.

In a trial where pharmacokinetic analysis of samples is the primary or an important trial objective

A detailed review of the processing should take place to ensure that it was completed in accordance with the trial protocol/procedures. This ensures their integrity compared with routine samples for haematology/biochemistry or for exploratory analysis, particularly if the routine or exploratory samples were taken in line with the normal hospital practice.

Unlicensed IMP in a trial categorised as Type C

Monitoring of the accountability should take place in more detail than for IMP in a Type A trial (where the IMP is used in line with normal clinical practice).

Additionally, if the IMP needed refrigerated or frozen storage conditions, this would potentially increase the monitoring focus, though still may not necessitate on-site visits. This could be the case for example if there had been a detailed assessment and approval of the facilities and regular provision of remote storage data to demonstrate compliance.

The role of oversight committees in monitoring a clinical trial

In the conduct of a trial, it may be appropriate, if considered as mitigation from the risk assessment, to set up committees as part of the oversight/monitoring strategy. Such committees could be set up on national, regional or global basis dependent on the trial and the remit of the committee.

One type would be a committee that would oversee the trial conduct: a trial management committee (TMC), which may be expanded to also involve members independent of the trial, to then form a trial steering committee (TSC).

These committees would typically oversee that the trial is appropriately managed to ensure compliance and deal with any issues identified during its conduct. One of their functions would be monitoring that appropriate systems and procedures were in place, the trial was adequately resourced, and that progress of the trial was on track. They would be involved in reviewing the monitoring strategy and may make the decisions on any escalation activities identified as being necessary to address non-compliance or any adaptations to the monitoring plan.

The TSC may be acting in a more strategic manner than the TMC, which would be more operationally focused. Such committees may not be necessary in the case of a small trial conducted at one site under the direct control of an investigator, or organisations may have a project manager and project team that is essentially acting as a TMC.

An independent trial data monitoring committee (DMC) may also be set up primarily to review the safety data for a trial at regular intervals to assess whether the trial is to continue or not, or whether amendments need to be made. They may review un-blinded data and also be involved in reviewing efficacy data for sequential trials.

On-site monitoring visits

Investigator site assessment visit

There should be some form of assessment of the trial investigator site and this - together with the decision to proceed with use of the site - should be documented, but this does not necessarily mean conducting an actual visit. An assessment could be either a feasibility or information-gathering exercise (of facilities, resources, staff (CVs), accreditation status, for example, as part of a research network, SOPs, recommendations from CROs) and perhaps by telephone or videoconference to establish the suitability of the site. However, a visit may be necessary after this process, to confirm the suitability for the trial.

The investigator site may be well known to the sponsor and may have been used for similar previous trials. In such situations, there could be a review of existing information about the site, for example, previous monitoring or audit reports to supplement a feasibility review to confirm the continuing suitability of the investigator site.

The complexity of the trial may affect the need for a site assessment visit, for example, a trial categorised as Type A may be no different to normal clinical practice and thus the need for an assessment visit would be reduced. We recommend that the risk assessment for the trial considers the investigator site selection process.

Investigator site initiation visit

The site research team must be suitably trained to undertake the trial activities. There should be a documented process to demonstrate that the sponsor has ensured that the investigator site research team is suitably trained, set up and ready to conduct the trial. While this often takes place with a specific initiation visit and sometimes a meeting of investigators, this may not always be necessary.

Specific training could take place centrally by telephone or videoconferencing or using web-based approaches. You could send training and initiation packages to the site containing manuals/instructions and checklists to complete and return. The level of training needed would depend on the experience of the site staff and the complexity of the trial. The level of instruction given to the trial site staff at this stage and during the initial stages of conducting the trial is likely to influence the compliance level of the site.

Remote training may have risks as it will be reduced in length and may be harder for the trainer to identify any lack of understanding, particularly if it takes place without any interaction (for example, a web presentation). The sponsor should retain evidence of the training given and ideally, evidence of the effectiveness (the understanding of the site staff of the training provided). Again, we recommend that the risk assessment considers the need for site initiation and training. We also recommend that there is the means to train new staff or re-train existing staff if the need arises.

On-site monitoring visits

The UK legislation does not require regular on-site visits. The MHRA GCP Inspectorate believes that ICH GCP underestimates the role that central monitoring can take as it states that monitoring without on-site visits should only be in “exceptional circumstances” (although ICH E6 R(2) does revise this to state that the sponsor may choose centralised monitoring only, where justified).

If the risk assessment has not identified the need for on-site visits to mitigate vulnerabilities and assure that the trial would follow the requirements of the legislation and GCP, participants are protected and the results would be reliable, then this would provide suitable justification for an “exceptional circumstance”.

We do not recommend that you interpret the ICH GCP statement “in general there is a need for on-site monitoring before, during and after the trial” to mean that on-site visits for assessment, initiation and on-going monitoring and closeout must always take place. Reports from such activities, however, must be retained. A need for on-site visit(s) may arise from information gained during the trial conduct.

Where routine on-site visits are not part of the monitoring strategy, we recommend that provision to carry out a triggered or “for cause” on-site visit is in place if central monitoring activities show the need for such an escalation activity to take corrective and preventative actions to maintain compliance. If only 1 planned monitoring visit is to take place per site, post initiation, we recommend that this happens soon after the trial commences so that preventative actions can be implemented at an early stage.

Centralised or remote monitoring techniques may cover many of the tasks that a monitor would undertake at site provided the appropriate and genuine documentation has been provided by the investigator.

There are, however, some important benefits of an on-site visit, including:

• reviewing patient medical records (for example, when central remote electronic review is not possible by the sponsor) to conduct SDV, verify the existence of a participant verifying the existence and quality of source documents and detect unreported adverse events
• meeting, training and motivating the research team and the investigator
• interviewing staff face to face, which builds rapport and can determine the actual processes used at the site (and any issues associated with that) rather than only conducting a review of the outcome (documentation) that may not present the whole picture
• reviewing facilities and equipment
• establishing the role of the investigator in the trial conduct (for example, monitoring the delegation of duties and the investigator’s involvement and oversight of the trial)
• providing an overall impression of the quality of the conduct of the trial at the site, which may enable action to improve quality at an earlier stage
• reviewing IMP storage areas and performing any necessary accountability checks on the actual IMP (this may only be a sample initially, if considered appropriate in the risk assessment)
• reviewing investigators’ site file (Investigator’s TMF) and archive facilities
• mentoring new staff/investigators
• witnessing participant visits (for example, consenting process)

If such considerations are identified as vulnerabilities in the trial risk assessment, then this would affect the decision to undertake on-site monitoring and make it more likely to be necessary.

For example, for a trial categorised as a Type C, with little safety information in patients, on-site visits to check for adverse events in the notes is more important than in a trial categorised as a Type A where the IMP is well known and used in accordance with standard clinical practice.

Note that on-site monitoring does not necessarily need frequent regular visits (traditionally once every 4 to 8 weeks). The interval between visits could be set at an appropriate level based on risk assessment, for example, the sites could be visited once, shortly after starting the trial, or on a fairly infrequent basis with a more targeted approach to monitoring activities based on the outcomes seen at previous visits or from central monitoring.

Investigator site close-out visit

The close-out visit by a monitor would primarily be to ensure that:

• all the trial documentation and source data is appropriately filed and stored for future retrieval if necessary
• all trial materials and IMP is accounted for and returned or destroyed
• all the necessary trial data and documentation has been collected
• there are no outstanding monitoring issues or data queries

Some or all of this could take place remotely, with the investigator site staff carrying out tasks under instructions from the sponsor, for example, completing checklists for the contents of the investigator file. If the site has been visited before, then the archiving storage should have been assessed and no further visit needed, or alternatively, the archiving could be arranged at an archiving vendor by the sponsor. Again, we recommend that the risk assessment considers the need for close-down visits and how the investigator site file and source data will be available for the required retention time, as this must be available for at least 5 years.

Central monitoring of a clinical trial

Central monitoring activities, focusing on the areas that matter as identified in the monitoring strategy could be done by numerous roles and across various departments (for example, clinical operations, study management, data management, statistics, medical monitoring) with close co operation where appropriate. We recommend that where the sponsor is carrying out many trials, some generic central monitoring processes are contained in SOPs, rather than trial-specific documents.

Any relevant/important communication or contact relating to the conduct of the trial by the sponsor would be part of monitoring the trial and this oversight - whether telephone calls, emails or letters -should be documented and must be retained. This would also demonstrate regular contact with the investigator(s).

There will be many documents received from the investigator sites in line with the monitoring procedures, these could include:

• consent/eligibility documents
• status reports (for example, recruitment, withdrawals)
• self assessment questionnaires or checklists
• pharmacy records

The sponsor should ensure that processes are in place to ensure blinding of the trial is maintained if documentation that could potentially unblind the trial (for example, records sent from pharmacy) is to be handled in central monitoring activities.

Any generated evidence that receipt and review of such information has taken place must be retained. This could prevent the need to retain the actual copies of the documents received.

Some non-commercial trials use a form of central monitoring that involves sending information from the investigator site to a central data centre. If information that identifies a participant is sent, for example a copy of a consent form, for monitoring purposes, it’s important that the participant has given consent for this and is aware of who will have access to their data. This process should be included in the protocol. A formal system should be in place at the receiving site to restrict access to confidential participant data to comply with the GDPR. Compliance with any local country-specific requirements will also be required.

Additionally, the investigator site will submit the clinical data CRFs either using eCRF or faxing/mailing/emailing paper CRFs. Any generated evidence of central monitoring activities regarding this data must be retained. These could include reports generated from interrogating the database, for example:

• looking at data submission timeliness
• audit trails of the eCRF to examine times of completion
• data query rates and response timeliness
• SAE reporting rate
• comparisons/reconciliations with other databases (for example, IRT, Central Labs) and data from other sites

The investigator must retain the original source document or a certified copy and the sponsor should not have exclusive control of a source document.

The data validation activities, usually done by data management, should be documented and so must be retained. This includes evidence of manual checks of CRFs and outputs of inconsistencies detected from data validation systems. We recommend that the data validation activities are focused on the data that is critical to the reliability of the trial results as identified by the risk assessment rather than excessive resource spent on raising data queries whose resolution makes little or no impact on the quality of the trial, the safety of the participants and reliability of the results. This is similar to the approach taken for proportionate source data verification (SDV).

There should be a formal process for dealing with issues and data queries identified during central monitoring and data management activities, including an escalation process. Any generated evidence of identification of the issue(s), review and discussion and subsequent actions must be retained. The monitoring strategy and procedures must be followed and there should be documentary evidence of this.

The use of central monitoring transfers some additional activities to the investigator site and may affect site resources. However, the investigator and research team must conduct the trial in accordance with the principles of GCP and have some specific responsibilities they must undertake according to the legislation. Some of the additional activities needed for central monitoring may assist in ensuring that this occurs (for example, completing a checklist for review of investigator site file contents).

Statistical monitoring

Statistical monitoring is an aspect of central monitoring. Sometimes this term can refer to a sequential trial design with respect to analysis for stopping rules, but this is not what we are considering here, which relates to monitoring the conduct of the trial. It is where the accumulating data, either clinical data from the CRF (for example, SAE event rates) or performance data (for example, eCRF completion times obtained from audit trail, data query levels/response timelines and CRF return times) is examined using statistical approaches or modelling across the trial.

This allows comparisons of sites to occur and this trending, modelling and process control has the aim of identifying any unusual or extraordinary patterns/variance/distributions in the data and in particular if any sites appear to be “outliers”.

Some statistical monitoring can use multivariate methods using the data from many variables simultaneously to identify outlying sites. The aim of the methodology is to use the information to decide where to target potential increased monitoring such as on-site visits, telephone calls or training. There may be predefined criteria or tolerance limits in the derived parameters which would trigger further monitoring activities or corrective/preventative actions. Outputs from the statistical monitoring usually take the form of graphs.

The use of all the data in this manner can potentially reveal issues that on-site monitoring might not detect, for example potential fraud. Statisticians have traditionally been involved in trial design and data analysis, but an expansion in use of central monitoring is likely to develop new roles in this area for statisticians. It will also increase the necessary interaction between monitoring, data management and others with statistics personnel.

Many organisations are developing quality metrics or major risk/performance indicators to assist in determining which investigator site to visit. Often this is part of the audit function rather than monitoring, but there does not appear to be a list of accepted or validated metrics for sponsors to use. The methodology would include ranking investigator sites with respect to the metrics to identifying outlying sites. Some organisations are also using multivariate statistical methodologies enabling the use of several metrics simultaneously. We recommended that the methods and metrics to be used are documented in the monitoring procedures. Some sponsors have used the following metrics:

• recruitment rate
• screen failure rates
• CRF submission/completion times against actual patient’s progress in the trial
• query rates
• time to queries resolution vs number of active queries (site level)
• SAEs reported
• numbers of missed or late visits/data
• number of subject withdrawals/dropouts
• numbers of protocol/GCP non-compliances recorded/reported
• eCRF – audit trail information on completion times in relation to visits or expected timescales

Example of statistical monitoring

The examples are not definitive approaches. The MHRA does not endorse or recommended them. The MHRA has reviewed them and provided them here to illustrate and share methodologies that are being used to support a risk-based monitoring approach.

Example 1: CluePoints’ SMART™ engine is a statistical software solution that examines if data collected is consistent, and if not, pinpoints those investigator sites that differ substantially from the others involved in a trial. It is powerful in detecting outlying sites, but would be limited in detecting a systematic issue affecting all sites in the trial because no site would be an outlier. It is a multivariate approach, analysing all available data and this generates thousands of p-values as test results (a single p-value being the probability of an observed result arising by chance).

The scoring algorithm summarises the information from the tests into a summary indicator or data inconsistency score (DIS) that pinpoints atypical sites and data subsets. All the calculated individual p-values are used to process a score per site and rank all of them accordingly. The most significant the DIS for a particular site, the most different statistically the site is.

As the chance to identify outliers increases with the number of sites in the study, the SMART™ engine also offers users an adjustable FDR (false discovery rate %) that takes multiplicity into account. The FDR sets an upper limit to the probability of detecting a centre as an outlier when it is not one. The smaller the FDR, the more confident we can be that a centre is truly an outlier. The output is displayed on charts to identify the outlying investigator sites and this can then be investigated further, perhaps triggering audit or site visit.

View an Example where centralised statistical monitoring detected mis-calibrated thermometers in a clinical trial.

References:

L. Desmet, D. Venet, E. Doffagne, T. Burzykowski, C. Legrand and M. Buyse Signal detection and power in central statistical monitoring of multicenter trials. Statist. Med. 2010; 00: 1-16.

D. Venet, E.Doffagne, T. Burzykowski, F. Beckers, Y. Tellier, E. Genevois-Marlin, U. Becker, V. Bee, V. Wilson, C. Legrand and M. Buyse A statistical approach to central monitoring of data quality in clinical trials. Clinical Trials 2012; 9: 705-713.

Dealing with non-compliance

There should be a formal process to identify, assess and document any non-compliance (failure to comply with the protocol, GCP or the legislation) identified through the monitoring activities. This is to protect the rights and wellbeing of the trial participants and the integrity of the trial results. Ability to do this is also necessary to comply with the requirements of serious breach reporting, as some non-compliances must to be reported to the MHRA.

Actions could include:

• increasing site monitoring intensity and/or frequency
• visits to site by sponsor senior management, conducting an audit
• holding or terminating recruitment at the site until the non-compliance issue is resolved, for example by retraining or using additional resources (such temporary halting of recruitment at a site may constitute an urgent safety measure)

The corrective and preventative actions to deal with the non-compliance should be documented and followed up to ensure that they are completed following a formalised escalation process and in a timely manner. In many MHRA GCP inspections where the monitor found non-compliance issues at investigator sites, there has been a failure to address them in a timely manner and to an appropriate resolution.

There should be a mechanism to ensure that all non-compliance is documented so that it can be reviewed as part of the analysis of the data for impact on the trial results and details provided in the clinical study report and/or publication.

The importance of accuracy of the clinical trial data

It is not the accuracy of the individual trial data that is important, but the reliability and robustness of the trial results. The aim of the management, monitoring and data management activities is to focus on the data and activities that are critical to the reliability of the trial results, for example, the endpoint for the primary objective of the trial or main design aspects (for example, randomisation). These would be identified during a risk assessment of the trial. You should aim for a high level of accuracy in these areas identified and potentially accept some degree of error in other areas. We recommend that you define such acceptability in terms of tolerance limits.

Sponsors may raise concerns about the reliability of the data where no SDV is conducted, for example if:

• the primary endpoint has not been verified
• there has been no review of patient notes to detect unreported SAEs
• the data submitted in the CRF is found to be incorrect

These concerns arise from the traditional monitoring approach that all the data has to be accurate and that any error is unacceptable. The design of the trial can assist in reducing or mitigating the impact of missing or incorrect data. For example, the results of large blinded, randomised trials with high power are unlikely to be affected by increased variability/omissions of the data, particularly as the errors/omissions would not be differential on a treatment basis (biased).

Small blinded and randomised trials may suffer from reduced power with increased data variability/omissions and there is potential to increase the risk of a false negative result.

Open trials are more at risk from bias, as errors and omissions could be potentially differential for the treatment groups. We recommend that you evaluate this issue as part of the risk assessment to determine what level of SDV (and other monitoring checks) is needed to mitigate any concerns about the reliability of the trial results.

The monitoring plan may take a conservative approach initially, then reduce the monitoring intensity if the concerns are not realised. The data accuracy and proper conduct of the trial can be influenced not only by the monitor detecting and the investigator correcting errors retrospectively (where possible), but by prevention of such errors in the first place, for example, by appropriate trial design, training, communication and systems that facilitate the conduct of the trial.

Source data verification (SDV)

We recommend that any SDV is focused on the data that matters to the reliability of the trial results. For example, focus may be on consent, eligibility, data for outcome measures related to the primary objective, safety reporting, non-compliance, or IMP accountability rather than 100% SDV of all the data. Also, if the monitor is doing 100% SDV then they generally have little time for anything else, and important issues can be missed if the monitoring focus is this narrow. The most important data could be identified during the risk assessment and the trial protocol. We also recommended that you define what the source data is at a particular investigator site.

For trials categorised as Type C, due to the exploratory nature of the trial and the uncertainty, it is likely that a higher level of SDV would be needed compared to a trial categorised as Type A, where the IMP is well known and used in line with normal clinical practice. This illustrates the risk-based approach. There may be situations where no or very limited SDV is acceptable by the risk assessment. The necessary SDV checks should to be contained in the protocol, SOPs or monitoring strategy documents for the trial.

The sponsor may wish to share these planned SDV checks with the investigator, because if the investigator is informed of which data is of importance for the trial results, this may improve the quality of this data. Where there is not 100% SDV of the identified critical data, but a sampling mechanism is used, then there should be some procedure for escalation if the sample verified reveals problems with data quality.

GCP inspectors may perform some SDV at investigator site inspections, but we do not recommend organisations carry out 100% SDV using this as a rationale. The inspectors will always consider the risk assessment (where available) and the plans and procedures which will define the sponsor’s plans for the SDV of the trial data.

Inspectors accept that discrepancies may be found if they examine data where SDV is not planned to take place (or the planned SDV had not yet taken place at the time of the inspection, for example data collected since the previous monitoring SDV). Also, if the SDV plan is risk based, focusing on potential discrepancies that would affect the trial results, then similarly, as the inspectors will use a risk-based approach to selecting data to check, then inspection examination of the data that is not planned to have SDV is unlikely to occur anyway.

If discrepancies are found in data that has been documented to have been verified by the monitor, and the selection of this data for SDV by the monitor was risk based, then this may result in a significant inspection finding as there is obviously a potential to affect the trial results (which is why it was selected to be verified).

It is likely that trials will continue to be increasingly electronically based, with the potential for data to be copied from source data electronically and transferred to the sponsor. This occurs already for laboratory data and we have seen examples where patient electronic records have been accessed to obtain data. Such an increase in this activity is likely to reduce the need for source data verification, with more emphasis on computer system validation to ensure that the systems used function correctly and obtain the correct data.

Finally, some source data can be accessed remotely and verified, for example, there are registries to confirm the date of and reason for death that could be used in long-term survival trials. Access to such information may be restricted to sponsors who are authorised to do this, where participants have explicitly consented to this and where the relevant participant’s personal data (for example, NHS number) necessary for such access has been provided and is kept confidential (complying with GDPR), for example, sponsors who are NHS trusts.