Research and analysis

Automation of COVID-19 test processing: evaluation of the Automated Workcell

Published 27 February 2023

Applies to England

COVID-19 challenge to testing capacity and capability

The emergence of coronavirus (COVID-19) highlighted the need for increased testing capacity. This saw the growth and expansion of COVID-19 testing services both nationally and globally. However, scaling up laboratory-based testing systems is challenging due to the time-intensive and manual processes involved which often require qualified and specialised technical staff (1).

The unprecedented demand for testing in this pandemic showed that testing must be easily accessible, reliable and deliver a quick turnaround on results. To meet these challenges, the UK Health Security Agency (UKHSA) has continuously sought new solutions and innovated at every opportunity. This report describes and presents the evaluation of a purpose-built automated laboratory that was designed to handle a high throughput of clinical samples with maximal automation and minimal requirement for trained technicians. The key piece of equipment in this lab is called the Automated Workcell and was set up to perform a reverse-transcription loop-mediated isothermal amplification (RT-LAMP) assay using saliva samples for the rapid detection of SARS-CoV-2 (2). In addition, this automated platform can be adapted for other diagnostic or screening technologies using swab or saliva samples to test a wide range of healthcare-related illnesses.

Benefits of lab automation

The ability to determine a positive or negative test result quickly and accurately is crucial during global outbreaks (3). High throughput automated testing enables a large number of the population to be tested quickly, and allows testing to be scaled at pace in case of a sudden outbreak. During the current COVID-19 pandemic, innovation in laboratory automation of testing has accelerated the ability to monitor and limit viral transmission as part of a test-trace-isolate strategy (3).

In the case of the automated lab to be described in this report, innovation has occurred at the intersection of the 3 disciplines of science, engineering and healthcare provision (see Figure 1).

Figure 1. Merging disciplines for innovation

Pilot testing and validation

The Workcell was conceived as an integrated end-to-end highly automated platform for use with the OptiGene COVID-19 Direct RT-LAMP assay. It was designed and built by PA Consulting (PA) in collaboration with Peak Analysis and Automation (PAA), a lab automation supplier. The University of Leicester (UoL) COVID-19 Screening Laboratory piloted the Workcell, with the aim of providing independent verification that it was fit for purpose as a COVID-19 screening tool using a saliva-based LAMP assay.

The Workcell consisted of 4 sequential operational modules (see Figure 2).

Figure 2. The 4 operational modules of the Automated LAMP Workcell. Illustration credit: image adapted from Peak Analysis and Automation Ltd

Figure 2 is a representation of the entire Automated Workcell. The pre-analytical phase makes use of module OP10 where samples are received, de-bagged and sorted. The analytical phase comprises use of modules OP20 and OP30 which involves samples processing and template addition respectively. The post-analytical phase employs OP40 making use of Genie®HT Analysers.

Evaluation

The evaluation of the Automated Workcell was carried out against the following 3 criteria.

1. Technical evaluation: The Workcell was intended to demonstrate the capacity to automate an end-to-end clinical process effectively. Furthermore, it was evaluated to determine if automation could lead to improvements in availability, quality and performance when compared with the manual alternative.

2. Clinical evaluation: As a minimum, the automation of a clinical assay should meet the expected and observed outcomes in comparison to the manual assay, in particular when detecting standardised viral concentrations and in meeting published limits of detection.

3. Business benefits: The introduction of automation should demonstrate a cost-effective investment, particularly by reducing the size of the workforce and required skill levels, and through more efficient use of laboratory space.

Technical evaluation

The full technical evaluation is not presented in this report. However, the most relevant sections are summarised below.

Operational qualification (OQ)

Equipment was checked and worked as expected in terms of times, temperatures and volumes. This included the tip detection system, barcode reader, sample tube de-cappers, liquid handling and coldblocks.

Sample tracking was confirmed and intentional errors, such as barcode errors or tip aspiration failures, were introduced. The system was able to correctly identify these forced errors and sort these samples, which were then correctly sent back to the OP10 module for re-processing.

Performance qualification (PQ)

This was a measure of the entire automated platform to ensure the workflow could perform the COVID-19 test as expected.

Clinical sample ‘run-at-rate’ trials were conducted using negative and positive saliva samples provided by University Hospital of Southampton and OptiGene COVID-19 reagents were used on each run.

The Automated Workcell met the target output of 2,304 tests, equivalent to 24 x 96-well plates, over an 8-hour shift. The Workcell was run continuously over two 8-hour shifts across 2 consecutive days without any major issues.

Automated platform digital integration

‘LANTERN’ was used as the laboratory information management system (LIMS) for the UoL pilot. This software was developed by the Royal Free London Innovation Team in partnership with Conjure and enables NHS trusts to manage lateral flow device (LFD) and LAMP tests for staff and patients nationally. The Automated Workcell was designed to interface with UKHSA’s test reporting system via a scheduling software system. LANTERN in conjunction with the scheduling software allowed UoL lab staff to operate the Automated Workcell, manage instruments, track samples and reagents, as well as process and report results (see Figure 3).

Figure 3. Automated Workcell lab digital integration

Figure 3 is a flow diagram of the entire testing process split into 11 different stages.

Stage 1

Clinical saliva samples were sourced and re-purposed from other testing laboratories (15,000 negative and 130 positive saliva samples).

Stage 2

The samples were then dropped off to the lab, re-labelled with a fresh LANTERN barcode (as part of the validation trial).

Stage 3

Each test was registered on the NHS-digital COVID-19 portal as part of the ‘light registration’ trial system.

Stage 4

The sample entered the Automated Workcell in the OP10 step where the sample was booked into the LANTERN system, which synchronised with the scheduling software.

Stage 5

Key metrics (such as sample ID) were recorded in the OP20 sample processing step. Any errors at this stage were logged.

Stage 6

Key metrics were recorded as part of the OP30 sample processing step. Any errors at this stage were logged.

Stage 7

The test result from the Genie®HT analyser in OP40 was collected by the LAMP workcell software and sent to the LANTERN service by an application programming interface (API).

Stage 8

The test results were then matched to the identity of the original sample. Any positive tests and sample errors at this stage were put back through the process at the OP10 step to reconfirm results.

Stage 9

Test results were recorded and could be checked on the SQL database.

Stage 10

The lab’s biomedical scientist (BMS) then reviewed and approved the result.

Stage 11

The test results were emailed or texted to the recipient.

LANTERN live testing was carried out on 74 unique barcoded samples. These were registered to the live LANTERN environment and then introduced to the Automated Workcell for processing. Of these, 66 were successfully processed and uploaded to LANTERN for BMS approval. The 8 tests that failed to upload were not due to an error in the digital end-to-end reporting, but rather due to a de-capping error (failure to remove the sample tube cap lids) which was logged on the system. The programme team, including NHS Digital, was satisfied that the Automated Workcell can register, process and report tests accurately.

Clinical evaluation

Pre-clinical validation

Sample tracking was verified from placement of primary saliva tubes at OP10 (24-well rack), loading samples for processing in OP20 (96-well plate) to finally template addition in OP30 (8-well polymerase chain reaction (PCR) strips).

Control panels supplied by the National Institute for Biological Standards and Control (NIBSC), containing SARS-CoV-2 samples of varying viral concentrations were used in the pre-clinical validation. The analytical sensitivity of the automated platform was 2.1 x 10⁶ copies/ml based on the limit of detection. This is what would be expected from a manual LAMP lab using the OptiGene Direct LAMP assay.

Pooled NIBSC samples of a high viral concentration were used in a checkerboard arrangement on the 96-well plate inside OP20. Results showed no cross-contamination of samples throughout the automated platform.

The analytical reagents used on the automated platform were proven to remain stable throughout the entire process. Sample reagents (such as the OptiGene Positive Control) stored on cold blocks in OP30 was consistent over an 8-hour shift across 3 days.

The time taken to process a full batch of samples on the automated platform and the effect on the sensitivity of the COVID-19 detection assay was also investigated. The time taken to process the first and last sample on the automated workflow did not impact the sensitivity of the COVID-19 assay.

Other variables such as time of day, number of days, analysers, systems (manual or automated) were also investigated and showed comparable and reproducible results between the automated and manual workflows.

Clinical validation

Clinical samples (saliva) were sourced and re-purposed from other laboratories. Known positive or negative saliva samples were used on the Automated Workcell and compared against a manual workflow.

There were 15,000 negative saliva samples processed on the automated platform, of which all gave a ‘negative’ COVID-19 result giving 100% concordance. Incorrect sample type accounted for 0.03% (4 out of 15,000) and were excluded. Repeat testing was required to confirm a negative result for 0.2% (30 out of 15,000).

The automated platform processed 130 known positive saliva samples which were confirmed manually in the reference lab. Due to insufficient sample volume, 8 out of 130 were excluded as inconclusive. Expected concordance was shown in 90% (110 out of 122) of the positive samples.

A root cause analysis determined that the 10% discordance was likely due to deterioration in the saliva samples. Samples with a late ‘time to positivity’ in conjunction with transportation, storage and repeated freeze-thawing cycles are less than ideal conditions and could have contributed to RNA degradatation. Fresh saliva samples (24 to 48 hours old) are ideal for testing using this particular assay.

Overall, from the extensive pre-clinical validation experiments using the NIBSC panels and clinical experiments using confirmed saliva samples, the automated platform has proven to perform the COVID-19 assay with reproducible results. Equivalent sensitivity to the manual reference laboratory with a 99.7% specificity for SARS-CoV-2 was demonstrated.

Void rates for the Automated Workcell could not be determined as it was not tested in a live environment. The overall void rate for the LAMP programme was 1 to 2%. Should the penultimate mechanical errors (for example, de-capping failures) and other minor engineering snags be resolved, the Automated Workcell would not be expected to exceed this range. The primary reasons for voids were due to external factors to the lab such as unassigned samples, incorrect sample type, samples leaked in transit, insufficient volume or sample expiration.

Business benefits

The third criteria of evaluation was to quantify the business and commercial benefits of the Automated Workcell. We have been able to prove the main business benefits associated with workforce reduction (with associated labour cost reduction), space reduction (with associated infrastructure cost avoidance), increased tests per operator and increased sample turnaround time. We have used manual lab comparators to quantify these benefits.

Lab comparison: manual versus automated

Operating under real lab conditions, with trained staff using saliva samples and clinical integration, the sample turnaround time (TAT) and throughput were calculated (see Table 1).

Table 1. Automation versus manual lab comparison

Comparable	Manual	Automation
Sample turnaround-time (TAT)	5.3hr (317min)	1.2hr (71min)
Sample throughput (samples/hr)	88	288
Space requirements for workflow	30m2	56m2

The manual sample TAT from 2 established and verified labs was calculated on a monthly average across several months and gave a value of 317 minutes. In comparison, the sample TAT for the automated platform for 24 plates was 71 minutes, nearly 4.5 times faster than a manual workflow.

Lab space requirements for one manual workflow was calculated at approximately 30m² with a sample throughput of 88 samples per hour. Although the Automated Workcell occupies a slightly larger footprint of 56m² (14 x 4m), it can process 288 samples per hour, an increase in throughput by 3 times compared to a manual workflow.

Additional assumed business benefits identified by the evaluation team, but for which we don’t have comparative data, are:

• reduced reagent wastage from reducing human error
• reduction in operator repetitive strain injury (RSI)
• public health benefits associated with increased sample throughput
• flexibility and accelerated scale-up

We would consider these as cost avoidance rather than cost savings.

Project status and summary

The pilot at the UoL was completed at the end of March 2022. The lab has since been decommissioned and both Automated Workcells have been dismantled and stored with the lab automation supplier.

Overall, the automated platform has proved operational and functional as a COVID-19 screening platform. However, with changes to the government testing policy and time restrictions on the LAMP programme, the Automated Workcell did not progress to the next stage: testing in a live environment. Nevertheless, the workcell is suitable not only for RT-LAMP testing of saliva samples, but could also be adapted for use in the future for PCR testing of swab or saliva samples.

This evaluation has demonstrated that the Automated Workcell is an innovation that can be deployed safely and at scale for future pandemics or in other cases of urgent public health need. Having shown their capacity and use, Automated Workcells could be set up within a current healthcare laboratory infrastructure. They would be available to utilise if a rapid ramp-up and scalability of testing capacity should be required.

Standing up an Automated Workcell and return on the investment

The Automated Workcell, which at the time of writing is in storage, could be used to satisfy a future public health need. The first step would involve identifying a suitable testing site. Ideally, the automated platforms should be installed and commissioned by the lab automation supplier, including site acceptance testing (SAT), within a month. In the case of this pilot, this took 6 months due to unforeseen mechanical challenges which then impacted the timeline of the clinical validation. To avoid such delays in future, there should be a contractual requirement on the lab automation supplier to confirm their installation timelines in writing and a financial penalty should be applied if the operation is time-critical. The installation timeline does not include the time required for quality assurance (QA) approvals but in a surge situation where rapid deployment is needed, given the correct resources and expertise, the QA processes could be expedited.

The capital expenditure for each Automated Workcell is approximately £800,000 to £1,000,000. Evaluation of return on investment (ROI) was estimated based on situating the workcells at the Rosalind Franklin Laboratory (RFL), recognising the very specific characteristics of this laboratory such as its environment, workflow design and operating model. Value for money analysis indicates that, on a like-for-like equivalence, ROI is achieved at 1.9 years after operationalisation of 4 Automated Workcells. This is based on a reduction of staffing headcount from 326 full-time equivalents (FTE) down to 118 FTE, to deliver 40,000 tests under the current operating model (though the workcell can of course be run at reduced volumes if required).

However, ‘value’ of the Automated Workcells should not be determined solely by financial measures for reasons including suitability in the environment, effectiveness of automation versus manual workflows and prudence. The primary non-financial value benefits, already mentioned above but presented here in more detail are:

Resilience and associated additional costs

Maximum operating capacity of a manual line is stated at 24,000 tests processed per 24-hour period. In contrast, maximum operating capacity of the 4 Automated Workcells is 40,000, a 40% increase in productivity.

The industry standard of 80% utilisation results in 20,000 tests for a manual line and 32,000 for Automated Workcells. Depending on policy and requirements, the Automated Workcells can be used responsively and have the design capability to flex up or down on throughput of samples and could operate up to 40,000 tests per day. This provides significant and rapid capacity resilience, particularly during surge or testing for variants of concern, where additional capacity is required.

Previously, if additional capacity was required within laboratories, additional manual lines had to be opened in the lighthouse laboratories or at RFL. This significantly increased costs and had a long lead-in time while new lab staff were recruited and trained. Increasing capacity on the Automated Workcells (to 90% to 100%) provides a comparative reduction in the FTE cost per test, as no additional resources are required.

The Automated Workcells also provide workforce resilience. With a lower staffing requirement, capacity is less likely to be impacted with volatility in workforce issues (for example, sickness, variability in manual operating pace and competency). Furthermore, in times when capacity drops, an automated workflow reduces the risk of workforce being under-utilised and the associated unnecessary high workforce cost base.

The dynamic nature of scalability means that, for example, the Automated Workcells could run at 60% in the morning, 80% between 12pm to 3pm, surging to 90% until 9pm before returning to 80% overnight.

Staff training time and costs are also reduced due to the greater non-clinical operating model.

Standardisation

An automated workflow provides a standardised methodology, pace and predictability of output. The workflow will reduce unwarranted clinical and performance related variation typically experienced in a manual line. Standardisation and automation also increase system reliability and reproducibility of the processing line, ultimately improving the quality of the results.

Technology agnostic

The Automated Workcells provide a degree of testing flexibility and increased options whereby the extracted RNA from saliva or swab samples can be used for multiple technologies, for example, ePCR, qPCR or genomic sequencing. Furthermore, with further research funding the automation could be adapted in the future for use with other molecular biology techniques (such as DNA extraction or genotyping) and provide additional diversification of laboratory processing capabilities and revenue generating opportunities.

References

1. Gao A, Murphy RR, Chen W, Dagnino G, Fischer P, Guiterrez MG and others. ‘Progress in robotics for combating infectious diseases’. Science Robotics 2021: volume 6, issue 52, page eabf1462
2. Fowler VL, Armson B, Gonzales JL, Wise EL, Howson EL, Vincent-Mistiaen Z, and others. ‘A highly effective reverse-transcription loop-mediated isothermal amplification (RT-LAMP) assay for the rapid detection of SARS-CoV-2 infection’. Journal of infection 2021: volume 82, issue 1, page 117 to 125
3. Courtney P, Royall PG. ‘Using Robotics in Laboratories During the COVID-19 Outbreak: A Review’. IEEE Robotics and Automation Magazine 2021: volume 28, issue 1, page 28 to 39

---

This report was prepared by:

UK Health Security Agency:

• Paul Summerville
• Michael Barker
• Saba Khan
• Sarah Tunkel
• Tom Fowler

PA Consulting:

• Mark Gilligan
• Ross Hafenden

University of Leicester:

• Jacqui Shaw
• Caroline Cowley