Research and analysis

Rapid evaluation of Oxford Nanopore Technologies’ LamPORE assay

Published 28 January 2021

Applies to England

Executive summary

The LamPORE assay was evaluated according to the published Technical Validation Group protocol.

This was a collaboration across 5 NHS trusts and 2 university partners who evaluated LAMP molecular testing technologies (including Oxford Nanopore LamPORE), in 22,941 samples, in an NHS asymptomatic staff pilot study.

As part of this, we also investigated the accuracy of LamPORE on 848 symptomatic patients who had undergone previous respiratory pathogen testing in a ‘real-world’ situation using RNA from samples sent to Public Health West Midlands for testing of patients with influenza-like illness (ILI).

The LamPORE assay was assessed on saliva and swab samples from asymptomatic participants in the setting of RNA extraction when the underlying disease incidence was 1.99%.

The LamPORE assay on swabs in asymptomatic patients, with RNA extraction, returned a sensitivity of 99.57% (95% confidence interval (CI) 98.46% to 99.99%) and the positive predictive value was 78.91% (95% CI 75.84% to 81.69%). Specificity was 99.40% (95% CI 99.28% to 99.50%) and negative predictive value was 99.99% (95% CI 99.96% to 100.0%). For comparison, previous studies have estimated the PPV of RT-qPCR in low-prevalence settings to be between 20% and 74%.

The LamPORE assay on saliva in asymptomatic patients returned a sensitivity of 98.94% (95% CI 96.23% to 99.87%) and the positive predictive value was 62.54% (95% CI 58.11% to 66.77%). Specificity was 99.39% (95% CI 99.26% to 99.49%) and negative predictive value was 99.99% (95% CI 99.96% to 100.0%).

The LamPORE assay on swab samples in symptomatic patients, with RNA extraction returned a sensitivity of 100% (95% CI 85.18% to 100%) and a specificity of 100%.

Participating NHS and academic laboratories developed an SOP during the study that eliminated contamination related false positives.

LamPORE demonstrates equivalent sensitivity and specificity to ‘gold standard’ RT-qPCR with the advantage of enhanced multiplexing (up to 768 samples per flow cell), which provides additional capacity for winter respiratory syndrome testing with significantly higher throughput than some existing technologies (96 samples per plate).

1. Background

1.1. The importance of testing, isolation and contact tracing for control of SARS-CoV-2 transmission has been highlighted by the World Health Organisation (WHO) as a critical intervention to prevent the spread of infection and ensuing morbidity and mortality from COVID-19. International efforts have largely focused on the detection of infection in symptomatic individuals as well as evolving clinical testing systems to detect those with asymptomatic or pre-symptomatic (subsequently referred to collectively as asymptomatic) infection that may still be infectious to others.

1.2. LamPORE combines loop-mediated isothermal amplification (LAMP) and nanopore sequencing to provide a highly scalable, multi-gene assay for the detection of SARS-CoV-2. LAMP is a single-tube technique for the amplification of DNA and as a standalone method it may be considered as a low-cost, rapid alternative to RT-PCR. Reverse transcription loop-mediated isothermal amplification (RT-LAMP) combines LAMP with a reverse transcription step to allow the detection of RNA. Target sequence is amplified at a constant temperature. Typically, 4 different primers are used to amplify 6 distinct regions on the target gene, which increases specificity. Additional pairs of ‘loop primers’ can further accelerate the reaction. The amount of amplified product produced in LAMP is considerably higher than PCR-based amplification, due to the use of multiple primer sets. LamPORE builds on the LAMP method to add sequencing as a process, for high-sensitivity, high-specificity analysis.

1.3. The technology of choice for this provision to date has been reverse transcription polymerase chain reaction testing (RT-qPCR), typically provided in central laboratory facilities, with a sample receipt to result time of more than 24 hours and reliance on international supply chains for a singular testing system. Considering these challenges, alternative technology approaches are being evaluated, including other forms of nucleic acid amplification (that is, LAMP or LamPORE) and protein-based methods (that is, lateral flow devices) for viral antigen detection, potentially providing diversification to the overall supply chain and options for an integrated testing strategy. These additional testing assets have also been considered in the context of other components of the wider end to end testing system, including the source and ease of sample collection and frequency of provision (saliva versus swab), RNA extraction requirements, and efficiency of returning results from a testing system into a clinical system for full maximal clinical utility, public health reporting and intervention requirements.

1.4. LAMP is an isothermal nucleic acid amplification technique, in contrast to the polymerase chain reaction (PCR) technology, in which the reaction is carried out with a series of alternating temperature steps or cycles, isothermal amplification is carried out at a constant temperature, and does not require a thermal cycler. In the case of LamPORE the LAMP process is combined with a sequencing process for high-sensitivity, high-specificity analysis.

1.5. This report outlines the technical validation undertaken in conjunction with the NHS Test and Trace Technical Validation Group and, given the need to prevent nosocomial spread and maintain NHS capacity and capability, it was conducted in collaboration with NHS trusts and university partners as a real-world evidence study.

2. Technical validation

2.1. The Technical and Validation function was established under NHS Test and Trace, inclusive of NHS and PHE experts and working closely with the Medicines and Healthcare Products Regulatory Agency (MHRA) and research bodies. The Technical and Validation function considers manufacturers of SARS-CoV-2 (COVID-19) tests for viral detection (including LAMP technologies) and registers their interest in the national procurement process if their test meets, or are intended to meet, the requirements of a relevant MHRA Target Product Profile (TPP). The Technical and Validation function reviews product information, undertakes technical and clinical validation, and establishes and/or works with service evaluation projects and pilots.

2.2. The asymptomatic pilot study recruited 1,200 healthcare workers across 4 hospital sites (University Hospitals Birmingham NHS Foundation Trust, University Hospitals Southampton NHS Foundation Trust, Hampshire Hospitals NHS Foundation Trust and Manchester Hospitals NHS Foundation Trust). Participants underwent self-administered nasophayngeal swabbing at days 1, 7, 14 and 21 and saliva sampling on a daily basis. A total of 3,966 swab and 18,435 saliva samples were collected as part of this study.

Assay description and intended purpose

2.3 LamPORE test is CE marked for use on the Oxford Nanopore GridION device. Each GridION device can run between 8 and 768 samples using up to 5 individually addressable flow cells (consumable units on which the sequencing is performed). Each individual flow cell can used to test up to 96 samples, with recalibration of the flow cell allowing for multiple uses for batch sizes of less than 96 samples. LamPORE is an assay produced by Oxford Nanopore Technologies consisting of 3 key stages:

• the assay uses reverse-transcription coupled to loop-mediated isothermal amplification (RT-LAMP) to amplify 3 highly conserved genes in SARS-CoV-2 positive samples (E1 gene, N2 gene, ORF1ab) and a negative control gene ACTB (human beta actin)
• samples are given a molecular barcode so that multiple samples can be analysed together
• these are then sequenced on the GridION Mk1 device, and analysed

2.4 In order for RT-LAMP to be combined with Nanopore sequencing technologies, several modifications to the RT-LAMP protocol have been made. Each sample is uniquely identified in a 96 well plate by the addition of 2 sets of molecular barcodes. The first set is attached to the forward inner primer (FIP). The second set is attached during sequencing library preparation allowing for between 96 and 768 samples to be pooled onto each flow cell.

2.5 Samples undergo Nanopore sequencing for one hour, and then sequence data is aligned via a custom algorithm to the SARS-CoV-2 genome. Each sample in the pool is identified by its molecular barcodes and absolute read counts per sample for the ORF1ab, N2 and E1 genes, as well as an ACTB (human beta actin) internal control are obtained. A positive is defined as any of the target genes having greater than 50 reads per sample, indeterminate being between 30 and 50 reads and negative anything below this. Samples are called as invalid if there are less than 50 ACTB reads detected.

2.6 The validation was performed on nasal/oropharyngeal swabs and saliva. Nasal and oropharyngeal swabs were collected in viral transport medium (Sigma Virocult). Saliva was collected neat in universal plastic tubes. The samples were all collected in clinical settings from asymptomatic healthcare workers across 4 hospital sites (Birmingham, Southampton, Basingstoke and Manchester).

2.7 All the equipment required to perform the test is supplied by the manufacturer except for a plate centrifuge, thermal cycler, pipettes, magnetic stand, laboratory consumables, saliva collection containers and nasopharyngeal/oropharyngeal swabs.

Performance characteristics

2.8 Analytical Sensitivity of SARS CoV-2 targets

The analytical sensitivity (ASe) for the LamPORE assay was evaluated using a blinded panel of guanidine isothiocynate inactivated SARS-CoV-2 virus grown on Vero E6 cells and quantified by droplet digital PCR to 20,000 copies/ml. The CerTest ViaSure SARS-CoV-2 multiplex RT-qPCR assay was used as a direct comparator. LamPORE successfully detected SARS-CoV-2 to a detection threshold of 20 copies/ml (Tables 1a and 1b). All assays were performed in triplicate.

Table 1.a: LamPORE analytical sensitivity – limit of detection of LamPORE

Concentration (copies/ml)	Detected	ORF1ab median reads	E1 median reads	N2 median reads
20,000	Positive	6,429	808	2,288
2,000	Positive	1,385	18	602
200	Positive	27	4	82
20	Positive	67	6	979
2	Negative	0	0	0
0.2	Negative	16	0	0

Table 1.b: LamPORE analytical sensitivity – reference standard (ViaSure qPCR)

Concentration (copies/ml)	Detected	ORF1ab median CT	N1 median CT
20,000	Positive	19.4	23.0
2,000	Positive	21.8	22.7
200	Positive	30.2	27.3
20	Positive	33.3	28.7
2	Equivocal	N/A	29.8
0.2	Negative	N/A	N/A

2.9 Precision and robustness

Intra-assay repeatability and inter-assay reproducibility was carried out using quantified inactivated live virus (as previously quantified by droplet digital PCR). A series of 5 replicates of the same sample undertaken on the same day is shown in Table 2. For intra-assay precision the Standard Deviation (SD) was 50 reads with a coefficient of variation (CV) of +/- 2.3% (a satisfactory assay has a CV less than 10%) (Table 2). As LamPORE is designed as a qualitative assay rather than quantitative, the applicability of precision metrics is uncertain in this context.

Table 2: intra-assay precision

Sample	Inhibition control value (Unmapped reads)	ORF1ab	E1 gene	N2 gene
Replicate 1	658	2,207	970	11
Replicate 2	788	2,176	112	306
Replicate 3	1,261	2,286	637	782
Replicate 4	3,126	2,161	159	312
Replicate 5	565	2,237	11	121

For inter-assay precision the same sample was analysed on multiple days and is seen in Table 3. The SD was 178 reads with a CV of +/- 7.8%. Satisfactory assay performance has a CV less than 15%.

Table 3: inter-assay precision

Sample	Inhibition control value (unmapped reads)	ORF1ab	E1 gene	N2 gene
Sample 1 (day 1)	796	2,269	821	207
Sample 1 (day 2)	1,248	2,359	93	93
Sample 1 (day 3)	564	2,007	124	219
Sample 1 (day 4)	3,747	2,292	1,541	510
Sample 1 (day 5)	1,380	2,495	1,687	1,547

Quantified SARS-CoV-2 virus (via droplet digital PCR) was used to input 20,000 copies/ml of extracted SARS-CoV-2 RNA into the LamPORE assay for 24 replicates in order to understand reproducibility. Standard deviation was 128 reads with a CV of +/- 3.9% (Table 4).

Table 4: reproducibility assessment for LamPORE

Replicate	Concentration	ORF1ab	E1	N2
1	20,000 copies/ml	3,334	1,384	1,342
2	20,000 copies/ml	3,084	445	0
3	20,000 copies/ml	3,500	20	177
4	20,000 copies/ml	3,174	11	60
5	20,000 copies/ml	3,367	60	400
6	20,000 copies/ml	3,013	1	0
7	20,000 copies/ml	3,337	22	352
8	20,000 copies/ml	3,375	14	151
9	20,000 copies/ml	3,183	11	87
10	20,000 copies/ml	3,173	1,045	22
11	20,000 copies/ml	3,190	128	0
12	20,000 copies/ml	3,344	1,293	1,837
13	20,000 copies/ml	3,191	416	664
14	20,000 copies/ml	3,233	821	1,823
15	20,000 copies/ml	3,256	1,792	1,867
16	20,000 copies/ml	3,237	1,779	1,640
17	20,000 copies/ml	3,017	377	142
18	20,000 copies/ml	3,144	1,517	311
19	20,000 copies/ml	3,171	230	365
20	20,000 copies/ml	3,241	574	2,774
21	20,000 copies/ml	3,386	60	1,518
22	20,000 copies/ml	3,493	1,902	16
23	20,000 copies/ml	3,211	82	476
24	20,000 copies/ml	3,206	1,042	1,201

2.11 Analytical specificity (interferences and cross-reactions)

Analytical specificity (ASp) was determined using the NATtrol™ Respiratory Verification Panel 2 (ZeptoMetrix Corporation, New York, United States) containing pathogens causing indistinguishable clinical signs to COVID-19 (n=22). No cross reactivity was observed in LamPORE to any respiratory pathogen.

Table 5: specificity assessment for LamPORE

Target	Result
Parainfluenza2	Negative
Negative control	Negative
Influenza B Florida 02 06	Negative
Coronavirus NL63	Negative
B.parapertussis A747	Negative
Parainfluenza4	Negative
Influenza A H1 a/newcal/20/99	Negative
Parainfluenza 3	Negative
Coronavirus HKU-1	Negative
B.parapertussis A639	Negative
Metapneumovirus 9 - peru6-2003	Negative
Rhinovirus type 1a	Negative
Adenovirus 31	Negative
Parainfluenza 1	Negative
Adenovirus 1	Negative
M.pneumoniae M129	Negative
Coronavirus 229E	Negative
RSV-A2	Negative
Influenza A H1N1pdm	Negative
Coronavirus OC43	Negative
Influenza AH3	Negative
C.pneumioniae CWL-029	Negative
Adenovirus3	Negative
NATtrolMERS-CoV stock	Negative

3. Diagnostic sensitivity and specificity

Clinical validation with confirmed positives and negatives

3.1 Samples included in the validation were appropriate to the intended use case scenarios, including, low medium and high viral load samples as part of the asymptomatic pilot and the symptomatic samples. This dynamic range permits a rigorous evaluation of diagnostic sensitivity (Table 6).

Table 6: range of viral loads for validation samples

Gene	Mean (CT)	Min (CT)	Max (CT)
ORF1ab	17.1	16.2	37.2
N1	14.3	11.0	37.2
IC	19.8	22.8	25.5

3.2 Diagnostic sensitivity: confirmed clinical samples from all symptomatic and asymptomatic participants (positive RT-qPCR result) were compared. The CT (cycle threshold) values or equivalent for both the assessed and comparator assays were included in the submitted validation data:

• 34 positive prospective asymptomatic swabs analysed by LamPORE
• 299 positive prospective asymptomatic saliva samples analysed by LamPORE
• 116 positive retrospective respiratory swab samples analysed by LamPORE

3.3 Diagnostic specificity: confirmed clinical samples from patients (negative RT-qPCR result) were used. The CT values or equivalent for both the assessed and comparator assays were included in the submitted validation data:

• 3,932 negative prospective asymptomatic swabs analysed by LamPORE
• 18,136 negative prospective asymptomatic saliva samples analysed by LamPORE
• 752 negative retrospective respiratory samples analysed by LamPORE

3.4 The diagnostic sensitivity (DSe) and specificity (DSp) (diagnostic precision) derived from the above samples are as follows:

3.4.1 All samples

• For LamPORE, diagnostic sensitivity (DSe) was calculated as 99.39% (95% CI 97.81% to 99.93%) and positive predictive value was 72.61% (95% CI 68.96% to 75.97%). Underlying disease incidence was 1.41% (Table 7)
• For LamPORE, diagnostic specificity (DSp) was calculated as 99.46% (95% CI 99.36% to 99.55%) and negative predictive value was 99.99% (95% CI 99.97% to 100.0%)

Table 7: diagnostic precision on all samples

—	Comparator assay result: positive	Comparator assay result: negative	Total
Assessed assay: positive	326	123	449
Assessed assay: negative	2	22,818	22,820
Total	328	22,941	23,269

3.4.2 Swab (asymptomatic) samples

• For LamPORE, diagnostic sensitivity (DSe) was calculated as 100% (95% CI 85.18% to 100%) and positive predictive value was 67.65% (95% CI 53.68% to 79.05%). Underlying disease prevalence was 0.58% (Table 8).
• For LamPORE, diagnostic specificity (DSp) was calculated as 99.72% (95% CI 99.50% to 99.86%) and negative predictive value was 100%.

Table 8: diagnostic precision on swab samples

—	Comparator assay result: positive	Comparator assay result: negative	Total
Assessed assay: positive	23	11	34
Assessed assay: negative	0	3,932	3,932
Total	23	3,943	3,966

3.4.3 Saliva (asymptomatic) samples

• For LamPORE, diagnostic sensitivity (DSe) was calculated as 98.94% (95% CI 96.23% to 99.87%) and positive predictive value was 62.54% (95% CI 58.11% to 66.77%). Underlying disease prevalence was 1.03% (Table 9).
• For LamPORE, diagnostic specificity (DSp) was calculated as 99.39% (95% CI 99.26% to 99.49%) and negative predictive value was 99.99% (95% CI 99.96% to 100%).

Table 9: diagnostic precision on saliva samples

—	Comparator assay result: positive	Comparator assay result: negative	Total
Assessed assay: positive	187	112	299
Assessed assay: negative	2	18,134	18,136
Total	189	18,246	18,435

3.4.4 Retrospective (symptomatic) samples

• The symptomatic sample set was collected from patients undergoing diagnostic testing in the laboratories of Public Health West Midlands (n=848) who underwent swabbing for an influenza like illness (ILI) in the period March 2020 to June 2020 or in the Milton Keynes Lighthouse Laboratory where both swabs and saliva was taken (n=20)
• For LamPORE, diagnostic sensitivity (DSe) was calculated as 100% (95% CI 96.48% to 100%) and positive predictive value was 100%. Underlying disease prevalence was 12% (Table 10)
• For LamPORE, diagnostic specificity (DSp) was calculated as 100% and negative predictive value was 100% (95% CI 99.51% to 100%)
• A significant increase in accuracy was observed in these samples as they were analysed at the end of project where the maximum experience with LamPORE had been gained

Table 10: diagnostic precision on retrospective samples

—	Comparator assay result: positive	Comparator assay result: negative	Total
Assessed assay: positive	116	0	116
Assessed assay: negative	0	752	752
Total	116	752	868

4. Conclusions

4.1 The technical performance of the LamPORE assay reveals a sensitivity of 99.57% (95% CI 98.46% to 99.99%) and specificity of 99.40% (95% CI 99.28% to 99.50%). This provides a comparable analysis to current standard of care RT-qPCR tests on RNA extracted swabs (in asymptomatic settings). This supports the introduction of this alterative nucleic acid amplification technology into the testing repertoire as a diagnostic tool.

4.2 When the technology is evaluated in the setting of saliva samples, using RNA extraction the sensitivity is 98.94% (95% CI 96.23% to 99.87) and specificity 99.39% (95% CI 99.26% to 99.49%) across all samples tested. The performance of LamPORE demonstrated excellent specificity and sufficient sensitivity to be used as part of the infection control strategy for SARS-CoV-2.

4.3 In a low-prevalence setting, previous studies (Skitrall et al., CMI 2020) have shown that the positive predictive value (PPV) of RT-qPCR is between 20.1% and 73.8%. In this context, the PPV of LamPORE is line with what has been observed in studies using RT-qPCR as the low prevalence means estimates of PPV can be changed significantly by even very low numbers of false positives.

4.4 In an attempt to understand the accuracy of LamPORE in a high prevalence setting, using retrospective samples from Public Health England ILI plates, the sensitivity was shown to be 100% with a specificity of 100%. This section of the project was carried out at the end when the testing laboratory had the maximum experience with LamPORE and the high accuracy was probably as a consequence of the experience that the participating laboratories had gained in the technique.

4.5 Early in the study we noted issues with contamination causing spurious false positives in saliva samples, which we found were due to LAMP contamination and the viscosity of saliva making automated liquid handling challenging. By modifying the provided SOP from Oxford Nanopore, we found these contamination issues were largely remedied. The SOP was modified by adding additional steps to decontaminate plasticware and work surfaces of LAMP products before each step and the use of separate laboratory areas for each step of the process. The pipetting speed of automated saliva transfer via liquid handling robots was also changed to compensate for saliva viscosity. The technique is vulnerable to contamination and so very careful attention must be paid to the SOP.

4.6 LamPORE is designed to be a qualitative rather than quantitative technique, due to the non-linear amplification properties of LAMP. Therefore, the reads per sample output of LamPORE must not be used to estimate viral load as it is not suitable for this.

4.7 Operation of LamPORE (and any LAMP technique) requires a dedicated lab set up according to a standardised SOP in order to prevent problems with contamination. Additionally, data exchange and integration between the GridION instrument and specific Laboratory Information Management Systems (LIMS) is required.

4.8 The key advantage of LamPORE, as part of a testing regime for both saliva and swabs, is its capability to offer high throughput testing of SARS-CoV-2 for both asymptomatic and symptomatic testing. The recommended use case is as a mass testing tool for large asymptomatic populations, either as a static or mobile deployment. It is also suitable as a direct replacement for current NHS testing methodologies.

4.9 The aim of an asymptomatic healthcare worker testing service is to detect unsuspected COVID-19 and thereafter to reduce the risk of SARS-CoV-2 transmission to other workers, to patients and to workers’ families. The vulnerability of patients is a key issue here, and such testing offers the opportunity to support infection control measures that are already in place. An early detection system could also support staff workforce resilience and so maintain NHS capacity.

4.10 Within a use case scenario of regular, frequent, testing of asymptomatic individuals within a population (for example, healthcare worker staff group) for SARS-CoV-2, the ability to rapidly and accurately identify individuals with high levels of infective virus to facilitate their prompt isolation will be an important contribution to reducing transmission of infection. The LamPORE assay in the RNA mode has demonstrated sufficiency accuracy in this use case to be utilised in populations of healthcare workers with the aim of curtailing transmission between staff and to patients within healthcare settings. LamPORE could form part of additional capacity for the national Lighthouse laboratories due to its scale up abilities.

4.11 LamPORE has comparable sensitivity and specificity to the current gold standard assay but with a considerably higher multiplexing capacity, with the ability to incorporate other gene targets such as Influenza and Respiratory Syncytial Virus (RSV) which can form part of a winter respiratory illness screening programme.

4.12 Further development of the LamPORE technology should consider applicability to raw, inactivated samples, challenges of scaling across differently sized organisations, the adoption of a new technology in clinical laboratories and systems that are unaccustomed to this nucleic acid amplification approach and the information technology integration that is needed to support a quality assured clinical service.

5. Additional data

Local verification reports

5.1 The LamPORE assay has been locally verified in 4 sites, with the activities of each site listed below.

Site	RT-LAMP evaluations
Institute of Cancer & Genomic Science University of Birmingham (samples from University Hospitals Birmingham NHS Foundation Trust)	- Optimisation of LamPORE assay and generation of new standard operating procedure (SOP) - Limit of detection and analytical sensitivity (ASe) and specificity (ASp) - RNA LamPORE as a screening tool in asymptomatic patients using swabs - RNA LamPORE as a screening tool in asymptomatic patients using saliva - RNA LamPORE in a high prevalence symptomatic cohort
Hampshire Hospital Foundation Trust (Basingstoke and North Hampshire Hospitals site)	- RNA LamPORE as a screening tool in asymptomatic patients using swabs - RNA LamPORE as a screening tool in asymptomatic patients using saliva
University Hospital Southampton (RNA extraction) / University of Southampton (Testing)	- RNA LamPORE as a screening tool in asymptomatic patients using saliva
Manchester University Foundation Trust	- RNA LamPORE as a screening tool in asymptomatic patients using swabs - RNA LamPORE as a screening tool in asymptomatic patients using saliva

6. Sample provenance

6.1 Data tables showing sample provenance by site and by methodology are available on request from paul.chambers@dhsc.gov.uk.