Cancer mutation screening: Comparison of high-resolution melt analysis between two platforms
Henry O Ebili1, 2 and Mohammad Ilyas3
1 Morbid Anatomy and Histopathology Department, Olabisi Onabanjo University, Ago-Iwoye, Nigeria
2 University College Hospital, Ibadan, Nigeria
3 Division of Pathology, University of Nottingham, Queen’s Medical Centre Campus, Nottingham, UK
Correspondence to: Henry O Ebili. Email: bakabakaxl@yahoo.com and ebili.henry@oouagoiwoye.edu.ng
Abstract
High-resolution melt analysis (HRMA) is a cheap and reliable post-polymerase chain reaction (PCR) cancer mutation screening technique, which is fast gaining clinical relevance. The HRMA capabilities of the LightScanner (Idaho Technology) have been severally studied. However, the ABI 7500 HRM has not been tested against the purpose-built HRM instrument such as the LightScanner.
DNA from formalin-fixed, paraffin-embedded gastric cancer, colorectal cancer, and normal tissue as well as from colorectal cancer cell lines were amplified at exons 2, 3, and 4 of KRAS, and at exons 11 and 15 of BRAF in the ABI 7500 fast real-time PCR machine and subjected to melting both on the ABI and on the LightScanner. HRMA data were analysed with the ABI HRM software v2.0.1 and the LightScanner Call-IT 2.5. We tested the ABI 7500 HRM for internal precision, accuracy, sensitivity, and specificity at mutation screening relative to the LightScanner, using crude percentage concordance, kappa statistics, and the area under the receiver operator characteristics (AUROC) curve on SPSS version 19.
The results show that the ABI 7500 HRMA has a high internal precision, and excellent concordance, sensitivity, and specificity at mutation screening compared with the LightScanner. However, in contrast to the LightScanner HRM software analysis, the ABI HRM software v.2.0.1, cannot distinguish real from certain pseudovariations in PCR amplicons that are sometimes brought about by the artefacts of the melting process.
In conclusion, the ABI HRM has a comparable performance level with the LightScanner, although in certain respects mentioned previously, the LightScanner has an edge over the ABI.
Keywords: cancer mutation screening performance, HRMA platforms, ABI 7500, LightScanner
Copyright: © the authors; licensee ecancermedicalscience. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Background
High-resolution melt analysis is a highly reliable PCR-based technique which has found use in genotyping and mutation scanning and is fast gaining clinical relevance [1, 2].
There are two classes of instruments utilized for HRMA, namely real-time PCR instruments with HRM modules and HRM instruments without PCR capabilities [1, 3]. In the HRM instruments such as the LightScanner, PCR is carried out in a thermocycler/qPCR instruments, and amplicon melting is done in the HRM machines [4]. The captured melting data are thereafter analysed with instrument-specific software. In the real-time instruments such as the ABI 7500 fast real-time PCR machine, PCR and amplicon melting are carried out as a closed tube procedure. The data obtained are analysed with HRM software specific for the instruments [5].
A few studies have compared the performance of HRM across different platforms. Generally, the HRM instruments have a higher discriminating power than the real-time PCR instruments which incorporate HRM modules [1, 3, 6]. In particular, the LightScanner has been shown to have similar sensitivity and specificity as different sequencing methods [7–10]. However, the HRM capabilities of the ABI 7500 real-time PCR has little been tested against the HRM instruments.
The aims of this study are to determine the degree of concordance between the ABI 7500 real-time PCR and the LightScanner in mutation screening and to compare the sensitivity and specificity of the ABI 7500 fast real-time PCR HRMA relative to that of the LightScanner. Our hypotheses are that there is a high rate of concordance between the HRMA results of the ABI 7500 real-time PCR and the LightScanner, and the ABI fast real-time instrument has an excellent sensitivity and specificity relative to the LightScanner. To test these hypotheses, the HRM results obtained with these instruments were analysed with the appropriate statistical tests of agreement. Furthermore, the sensitivity and specificity of the ABI instrument were compared using the area under the receiver operator characteristics (AUROC) curve and manually calculated from the results of HRMA. The results are presented in this article.
Materials and Methods
This study was carried out at the Division of Pathology, University of Nottingham, Queen’s Medical Centre Campus, Nottingham, United Kingdom.
We tested intra-assay precision or reproducibility of each of the HRM modules, and then, the inter-assay agreements among the modules.
DNA samples
DNA from clinical gastric cancer, colorectal cancer and normal (all formalin-fixed, paraffin-embedded) tissues, and colorectal cancer cell lines (RKO, DLD1, and HCT116) were included in this study.
Polymerase chain reaction and amplicon melting
Different genomic regions were amplified using the recently described two-stage PCR protocol (manuscript submitted) in the ABI 7500 fast real-time instrument. PCR was carried out in a 10 μL volume made up of 5 μL of Diamond Hotshot master mix, 1 μL of LC Green Plus dye, 0.5% of Tween, 300 nm of primers, and 2 μL of amplicon-enriched DNA template.
The samples were amplified in singles (for inter-assay testing) and duplicates (for intra-assay testing), and PCR was followed by melting in the ABI 7500 and then in the LightScanner. The primer sequences, cycling and melting parameters used in this study are shown in Tables 1 and 2.
The melting data were analysed with the HRM software v.2.0.1 (ABI 7500) and the LightScanner software Call-IT 2.5. The LightScanner data were analysed with the Amplicon Scanning (LSAS) and the Amplicon Genotyping (LSAG) programmes of the software. In some instances, the sensitivity levels of the LSAS were adjusted until the curves of all the wild-type samples (normal DNA), and other samples tightly clustering with them were identified as a single variant.
Table 1. Sequences of primer pairs used in the study.
Table 2. Cycling parameters for utilized for amplification of genes in this study.
Statistical analyses
Intra-assay and inter-assay agreements were tested for each HRM operation using the crude percentage concordance and kappa statistics in the SPSS version 19. The sensitivity and specificity of the ABI HRMA in detecting mutations relative to the LightScanner were derived from the coordinates of the area under the receiver operating characteristic (ROC) curve and confirmed by manual calculations from the cross tabulations of the HRM results.
Strategy
DNA samples from cancer and normal tissue were amplified in five gene regions: exons 15 and 11 of BRAF, and exons 2, 3, and 4 of KRAS. For the purpose of comparison of assays, each amplified gene region was treated as a sample (a gene region sample), melted with ABI and then, the LightScanner, and analysed with the ABI HRM software, LSAS and LSAG. For example, when a single DNA sample is experimentally amplified at BRAF 15, BRAF 11, KRAS 2, and KRAS 3, it is regarded as four (gene region) samples and statistically analysed as such. Different experiments were performed for the intra-assay testing than for the inter-assay tests. For the intra-assay agreement, DNA samples were amplified in duplicates at KRAS 2 and KRAS 4 up to a total of 53 gene region samples and melted on the ABI and LightScanner. The degrees of agreements between the duplicates were compared for ABI, LSAS, and LSAG using the statistical tests mentioned earlier. For the inter-assay agreement, DNA samples were experimentally amplified in singles at BRAF 15, BRAF 11, KRAS 2, and KRAS 3 up to a total of 292 gene region samples. The degrees of agreement among the ABI, LSA, and LSAG were also compared using the same statistical tests mentioned.
Results
Test of reproducibility of the ABI 7500 real-time fast PCR and the LightScanner
Exons 2 and 4 of KRAS were amplified in some of the above-mentioned samples to a total of 53 duplicate gene regions and subsequently melted on both instruments. Up to eight different amplicon variants were identified by each assay per gene region amplified. For each gene region sample, the duplicate HRMA data were input into SPSS as two different variables (first and second variant calls) for each assay. Statistical analyses with crude percentage concordance and kappa statistics were performed to determine the precision (reproducibility) of each of the assays. The results showed that while each of the assays had excellent intra-assay agreement (ABI: 90.6%, К = 0.811 [P < 0.001]; LSAS: 96.2%, К = 0.920 [P < 0.001]; LSAG: 94.3%, К = 0.884 [P < 0.001]), the LSAS and LSAG had better intra-assay precision or reproducibility.
Inter-assay agreement between the ABI and the LightScanner
About 292 gene region samples were amplified in singles at exons 11 and 15 of BRAF, and exon 3 of KRAS. The HRMA data of each sample were entered into SPSS as either wild type or mutant for each assay, irrespective of the number of variants identified by each assay. Then, crude percentage concordance and kappa statistics were utilised to test for agreement among ABI HRM, LSAG, and LSAG. The results showed that the agreements among the three assays were outstanding (ABI versus LSAS: 97.6%, К = 0.864 [P < 0.001]; LSAG versus ABI: 97.6%, К = 0.850 [P < 0.001]; LSAS versus LSAG: 97.9%, К = 0.878 [P < 0.001]) (Figure 1).
Figure 1. Inter-assay agreement. HRMA of exon 15 of BRAF using the LightScanner (A & C) and the ABI 7500 fast real-time (B & D) platforms showing highly reproducible results between both platforms seen here as identical Normalised Melt Curves (A & B) and mirror-image difference plots (C & D). The non-grey curves in A and C, and the non-blue curves in B and D are the same aberrantly melting samples, while the grey curves in the LightScanner and blue curves in the ABI output represent the same normally melting samples. The LightScanner normalized difference curves output, by default, is inverted relative to the ABI difference plots output.
Performance of the ABI and the LSAG relative to LSAS
The area under the receiver operating characteristics (AUROC) was derived for ABI (relative to LSAS and LSAG) and LSAG (relative to LSAS). Table 3 shows the areas and coordinates of the curve for the different HRM assays. Both the ABI HRMA and the LSAG show excellent sensitivity and specificity relative to each other and to the LSAS.
The above results were confirmed by manual calculations of sensitivity and specificity from the cross-tabulations of the results from the three HRMA assays (Tables 4 and 5).
Table 3. AUROC analysis for HRMA results of ABI and LightScanner.
Table 4. Cross tabulation of results of mutation screening by ABI and LightScanner.
Table 5. Manual calculation of performance measures of the ABI and LightScanner.
Discussion
Although the intra-assay agreement for all three assays were outstanding, the crude percentage concordance, and kappa values show that the LightScanner assays have more internal consistency than the ABI 7500 HRM assay. This better internal consistency of the LightScanner is in agreement with a previous study which showed that HRM instruments have a better precision than non-HRM instruments [1, 3].
In this study, we have used the LightScanner as the standard to compare the inter-assay reproducibility and performance of the ABI because the LightScanner has been severally shown to have similar performance levels for mutation detection as the Sequencing methods [7–10]. The three assays show excellent inter-assay agreement, although both LightScanner HRM modules have better inter-assay agreement between them than either has with the ABI. This outstanding inter-assay reproducibility across HRM platforms have similarly been shown among HRM instruments and qPCR instruments with HRM capacity [9, 11]. This simply attests to the robustness of HRMA as a mutation screening method.
The sensitivity of 83.3% of the ABI HRM module at mutation detection found in this study, although excellent, is lower than the 92–100% sensitivity obtained by ABI HRM relative to sequencing in the detection of micro-organisms [12]. However, the specificity of 98.1% is comparable to what was obtained (98.36–100%) in the aforementioned study.
The 80% sensitivity obtained with the LSAG relative to the LSAS analysis in mutation scanning was expectedly low, considering that LSAG is designed for genotyping, not scanning [4]. We have utilized LSAG module in the analysis of the melting data of amplicons that are 105–145 bp long and that have skewed mutation spots (Figure 2). This is different from the recommendation of the LightScanner manufacturer that LSAG be used for analysing melting data of amplicons designed to be 50–100 bp long with the mutation spot at the middle of the amplicon length. However, the degree of inter-assay agreement between the LSAS and LSAG analyses and sensitivity level of 80% obtained in this study shows that LSAG may just as well be suitable for mutation scanning as LSAS. This is more so when the specificity of the LSAG at mutation detection as found in this study is 100%. Furthermore, the high degree of overlap of the sensitivity and specificity of the LSAG and the ABI 7500 as determined by the AUROC curve lends support to the suggestion that LSAG may also be used for mutation scanning.
Figure 2. Sequences amplified in this study with mutation hotspots. All the bases in orange and red colours represent mutation hotspots in the gene regions amplified. The red bases shown in KRAS exon 2 represent codon 12 of KRAS, while the orange bases represent codon 13. The two colours used here are intended to differentiate codons 12 and 13 of KRAS.
Accession numbers of DNA sequences shown in Figure 2
• BRAF exon 11 [NCBI Reference Sequence NC_018918.2]
• BRAF exon 15 [NCBI Reference Sequence NC_018918.2]
• KRAS exon 2 [NCBI Reference Sequence NC_018923.2]
• KRAS exon 3 [NCBI Reference Sequence NC_018923.2]
• KRAS exon 4 [NCBI Reference Sequence NC_018923.2]
Our study did not include any sequencing methods as the aim is to show the extent to which the ABI HRMA replicates the LightScanner HRMA at mutation screening. This is more so as HRMA has severally been shown to have similar sensitivity and specificity as sequencing at mutation detection [7–10]. We have applied the stringent measures for adequate PCR amplification, that is, cycle threshold value of 30 or less and a normal amplification curve in order to ensure robust HRMA results [13].
Finally, we observed that the LightScanner software analysis is superior to that of the ABI 7500 HRMA in two respects. The first is that the variant calls by the LightScanner software are ‘stable’. That is, they are not affected by the addition or subtraction of a sample with a deviant melting pattern to a set of samples. In contrast, analysis by the ABI HRM software is more ‘delicate’ and its variant calls are significantly affected by the addition or subtraction of one or more samples with different melting patterns (Figure 3). The second respect in which the LightScanner software analysis is more robust than that of the ABI 7500 is in the capacity of the LightScanner analysis to accommodate certain subtle artificial variations in amplicons that are brought about by the vagaries of the melting process. For example, instruments used for HRMA are known to show some variations in melting temperature, Tm, during amplicon melting. These variations are worse with the qPCR instruments incorporating HRM (including some of those manufactured by ABI) than the purpose-built HRM instruments. Variations in Tm can result in false variant calls by the instruments [2]. While the LSAG module can accommodate such artificial variants by default, the LSAS module can be adjusted to accommodate them using the sensitivity level of the software. This capability is lacking with the ABI HRM v2.0.1 software (Figure 4).
Figure 3. ‘Unstable’ variant calls by the ABI HRM software v2.0.1. A, B & C. Sequential exclusion of aberrant variants from analysis with the ABI results in variant call changes of the remaining samples. D, E & F. Sequential exclusion of samples with aberrant melting patterns from analysis with the LightScanner Call-IT 2.5, however, did not change the original calls of the remaining samples.
Figure 4. Spurious variant calls by the ABI HRM software. A. In this set of experiment, the ABI HRM software identified two large clusters of samples as belonging to two different melting groups (red and green curves) even though both clusters merge snugly and contain wild-type controls. Furthermore, the following assignment of normal samples from both clusters as wild-type controls, the software still recognised them as different. B & C. The amplicon scanning programme of the LightScanner Call-IT 2.5 (LSAS) also recognized some samples from same cluster as different. However, when the sensitivity level of the programme was adjusted, the software correctly classified the samples as one variant (grey). D. The amplicon genotyping programme (LSAG) correctly classified the samples as the same variant (grey), even without adjustment of the sensitivity.
Conclusion
The ABI 7500 HRMA shows a high internal precision, and excellent concordance, sensitivity, and specificity at mutation screening compared with the LightScanner. In addition, the LightScanner HRM software analysis, but not the ABI HRM software v.2.0.1, can nullify certain pseudovariations in PCR amplicons brought about by the artefacts of the melting processes.
List of Abbreviations
HRMA: High-resolution melt analysis
LSAS: LightScanner amplicon scanning programme
LSAG: LightScanner amplicon genotyping programme
Declaration of Interest
The authors declare no conflict of interest.
Author’s Contributions
HOE: Experiment development, execution and analysis, manuscript drafting, editing and preparation of the final manuscript version. MI: Conception, experimental analysis, manuscript editing, and preparation of final version for publication.
Acknowledgements
We thank the Department of Pathology, University College Hospital, Ibadan, for providing some of the samples for the study. We wish to acknowledge the Pathological Society of Great Britain and Ireland which provided some of the funding for this study, as well as the Union for International Control of Cancer for the Travel and Maintenance award provided HOE for the study.
References
1. Reed GH et al (2007) High-resolution melting analysis for simple and efficient molecular diagnostics Pharmacogenomics 8 597–608 DOI: 10.2217/14622416.8.6.597 PMID: 17559349
2. Erali M et al (2008) High resolution melting applications for clinical laboratory medicine Exp Mol Pathol 85 50–58 DOI:10.1016/j.yexmp.2008.03.012 PMID: 18502416 PMCID: 2606052
3. Hermann MG et al (2006) Amplicon DNA melting analysis for mutation scanning and genotyping: cross-platform comparison of instruments and dyes Clin Chem 52 494–503 DOI: 10.1373/clinchem.2005.063438
4. LightScanner Operator’s Manual for software version 2.0 (Rev 04, 11/07)
5. Applied biosystems high resolution melting getting started guide Available at www.appliedbiosystems/hrm. Accessed on: 5/8/2014
6. Hermann MG et al (2007) Expanded instrument comparison of amplicon DNA melting analysis for mutation scanning and genotyping Clin Chem 53 1544–1548 DOI: 10.1373/clinchem.2007.088120
7. van der Stoep N et al (2009) Diagnostic guidelines for high-resolution melting curve (HRM) analysis: an interlaboratory validation of BRCA1 mutation scanning using the 96-well LightScannerTM Hum Mutat 30 899–909 DOI: 10.1002/humu.21004 PMID: 19370767
8. Akiyoshi K et al (2013) KRAS mutations in patients with colorectal cancer as detected by high-resolution melting analysis and direct sequencing Anticancer Res 33 2129–2134 PMID: 23645765
9. De Leeneer K et al (2008) Rapid and sensitive detection of BRCA1/2 mutations in a diagnostic setting: comparison of two high-resolution melting platforms Clin Chem 54 82–989 DOI: 10.1373/clinchem.2007.098764
10. Cho MH et al (2008) High-resolution melting curve analysis of genomic and whole-genome amplified DNA Clin Chem 54 2055–2058 DOI: 10.1373/clinchem.2008.109744 PMID: 19042988 PMCID: 2755063
11. Laurie AD et al (2007) Detection of factor VIII gene mutations by high-resolution melting analysis Clin Chem 53 2211–2214 DOI: 10.1373/clinchem.2007.093781 PMID: 17901109
12. Singh PR et al (2013) Identification of clinically relevant mycobacterium other than tuberculosis (MOTT) species by real-time PCR coupled with a high-resolution melting system JBPR 2 30–34
13. White H et al (2006) Mutation scanning by high resolution melt analysis. Evaluation of rotor-gene 6000 (Corbett Life Science), HR-1 and 384-well lightscanner (Idaho Technology) National Genetics Reference Laboratory 1–45
