Display options
Cite
Hayeems RZ, Dimmock D, Bick D, et al. Clinical utility of genomic sequencing: a measurement toolkit. NPJ Genom Med. 2020;5(1):56doi: 10.1038/s41525-020-00164-7.
Hayeems, R. Z., Dimmock, D., Bick, D., Belmont, J. W., Green, R. C., Lanpher, B., Jobanputra, V., Mendoza, R., Kulkarni, S., Grove, M. E., Taylor, S. L., Ashley, E. (2020). Clinical utility of genomic sequencing: a measurement toolkit. NPJ genomic medicine, 5(1), 56. https://doi.org/10.1038/s41525-020-00164-7
Hayeems, Robin Z, et al. "Clinical utility of genomic sequencing: a measurement toolkit." NPJ genomic medicine vol. 5,1 (2020): 56. doi: https://doi.org/10.1038/s41525-020-00164-7
Hayeems RZ, Dimmock D, Bick D, Belmont JW, Green RC, Lanpher B, Jobanputra V, Mendoza R, Kulkarni S, Grove ME, Taylor SL, Ashley E. Clinical utility of genomic sequencing: a measurement toolkit. NPJ Genom Med. 2020 Dec 15;5(1):56. doi: 10.1038/s41525-020-00164-7. PMID: 33319814; PMCID: PMC7738524.
Copy Download .nbib Format: NLM
Share it on

NPJ Genom Med. 2020 Dec 15;5(1):56. doi: 10.1038/s41525-020-00164-7.

Clinical utility of genomic sequencing: a measurement toolkit.

NPJ genomic medicine

Robin Z Hayeems, David Dimmock, David Bick, John W Belmont, Robert C Green, Brendan Lanpher, Vaidehi Jobanputra, Roberto Mendoza, Shashi Kulkarni, Megan E Grove, Stacie L Taylor, Euan Ashley,

Affiliations

  1. Program in Child Health Evaluative Sciences, The Hospital for Sick Children and the Institute of Health Policy Management and Evaluation, University of Toronto, Toronto, ON, Canada. [email protected].
  2. Rady Children's Hospital Institute for Genomic Medicine, San Diego, CA, USA.
  3. HudsonAlpha Institute for Biotechnology, Huntsville, AL, USA.
  4. Illumina Inc., San Diego, CA, USA.
  5. Brigham and Women's Hospital Broad Institute and Harvard Medical School, Boston, MA, USA.
  6. Mayo Clinic, Rochester, MN, USA.
  7. New York Genome Center, New York, NY, USA.
  8. Department of Pathology and Cell Biology Columbia University Medical Center, New York, NY, USA.
  9. The Division of Clinical and Metabolic Genetics, The Hospital for Sick Children, Toronto, ON, Canada.
  10. Baylor Genetics and Baylor College of Medicine, Houston, TX, USA.
  11. Department of Molecular and Human Genetics, Baylor College of Medicine, Houston, TX, USA.
  12. Stanford Medicine, Stanford, CA, USA.

PMID: 33319814 PMCID: PMC7738524 DOI: 10.1038/s41525-020-00164-7

Abstract

Whole-genome sequencing (WGS) is positioned to become one of the most robust strategies for achieving timely diagnosis of rare genomic diseases. Despite its favorable diagnostic performance compared to conventional testing strategies, routine use and reimbursement of WGS are hampered by inconsistencies in the definition and measurement of clinical utility. For example, what constitutes clinical utility for WGS varies by stakeholder's perspective (physicians, patients, families, insurance companies, health-care organizations, and society), clinical context (prenatal, pediatric, critical care, adult medicine), and test purpose (diagnosis, screening, treatment selection). A rapidly evolving technology landscape and challenges associated with robust comparative study design in the context of rare disease further impede progress in this area of empiric research. To address this challenge, an expert working group of the Medical Genome Initiative was formed. Following a consensus-based process, we align with a broad definition of clinical utility and propose a conceptually-grounded and empirically-guided measurement toolkit focused on four domains of utility: diagnostic thinking efficacy, therapeutic efficacy, patient outcome efficacy, and societal efficacy. For each domain of utility, we offer specific indicators and measurement strategies. While we focus on diagnostic applications of WGS for rare germline diseases, this toolkit offers a flexible framework for best practices around measuring clinical utility for a range of WGS applications. While we expect this toolkit to evolve over time, it provides a resource for laboratories, clinicians, and researchers looking to characterize the value of WGS beyond the laboratory.

References

  1. Clark, M. M. et al. Meta-analysis of the diagnostic and clinical utility of genome and exome sequencing and chromosomal microarray in children with suspected genetic diseases. NPJ Genom. Med. 3, 16 (2018). - PubMed
  2. Petrikin, J. E. et al. The NSIGHT1-randomized controlled trial: rapid whole-genome sequencing for accelerated etiologic diagnosis in critically ill infants. NPJ Genom. Med. 3, 6 (2018). - PubMed
  3. Lionel, A. C. et al. Improved diagnostic yield compared with targeted gene sequencing panels suggests a role for whole-genome sequencing as a first-tier genetic test. Genet. Med. https://doi.org/10.1038/gim.2017.119 (2017). - PubMed
  4. Farnaes, L. et al. Rapid whole-genome sequencing decreases infant morbidity and cost of hospitalization. NPJ Genom. Med. 3, 10 (2018). - PubMed
  5. Bick, D., Jones, M., Taylor, S. L., Taft, R. J. & Belmont, J. Case for genome sequencing in infants and children with rare, undiagnosed or genetic diseases. J. Med. Genet. 56, 783–791 (2019). - PubMed
  6. Trosman, J. R. et al. Perspectives of US private payers on insurance coverage for pediatric and prenatal exome sequencing: Results of a study from the Program in Prenatal and Pediatric Genomic Sequencing (P3EGS). Genet. Med. 22, 283–291 (2019). - PubMed
  7. Smith, H. S. et al. Clinical application of genome and exome sequencing as a diagnostic tool for pediatric patients: a scoping review of the literature. Genet. Med. 21, 3–16 (2019). - PubMed
  8. Grosse, S. D. & Farnaes, L. Genomic sequencing in acutely ill infants: what will it take to demonstrate clinical value? Genet. Med. 21, 269–271 (2019). - PubMed
  9. Botkin, J. R. et al. Outcomes of interest in evidence-based evaluations of genetic tests. Genet. Med. 12, 228–235 (2010). - PubMed
  10. Grosse, S. D. & Khoury, M. J. What is the clinical utility of genetic testing? Genet. Med. 8, 448–450 (2006). - PubMed
  11. Joseph, L. et al. The spectrum of clinical utilities in molecular pathology testing procedures for inherited conditions and cancer: a report of the Association for Molecular Pathology. J. Mol. Diagn. 18, 605–619 (2016). - PubMed
  12. CDC. ACCE Model Process for Evaluating Genetic Tests. https://www.cdc.gov/genomics/gtesting/acce/index.htm (2010). - PubMed
  13. CDC. ACCE Model List of 44 Targeted Questions. https://www.cdc.gov/genomics/gtesting/acce/acce_proj.htm (2010). - PubMed
  14. Tatsioni, A. et al. Challenges in systematic reviews of diagnostic technologies. Ann. Intern. Med. 142, 1048–1055 (2005). - PubMed
  15. Fryback, D. G. & Thornbury, J. R. The efficacy of diagnostic imaging. Med. Decis. Mak. 11, 88–94 (1991). - PubMed
  16. ACMG. Clinical utility of genetic and genomic services: a position statement of the American College of Medical Genetics and Genomics. Genet. Med. 17, 505–507 (2015). - PubMed
  17. Bossuyt, P. M., Reitsma, J. B., Linnet, K. & Moons, K. G. Beyond diagnostic accuracy: the clinical utility of diagnostic tests. Clin. Chem. 58, 1636–1643 (2012). - PubMed
  18. Williams, J. L. et al. Harmonizing outcomes for genomic medicine: comparison of eMERGE outcomes to ClinGen outcome/intervention pairs. Healthcare 6, (2018). - PubMed
  19. Williams, M. S. Early lessons from the implementation of genomic medicine programs. Annu. Rev. Genomics Hum. Genet. 20, 389–411 (2019). - PubMed
  20. ClinGen. Actionability: Aims to Identify Those Human Genes That, When Significantly Altered, Confer A High Risk of Serious Disease That Could Be Prevented Or Mitigated If the Risk Were Known. https://clinicalgenome.org/working-groups/actionability/ (2019). - PubMed
  21. Lu, J. T. et al. Evaluation for genetic disorders in the absence of a clinical indication for testing: elective genomic testing. J. Mol. Diagn. 21, 3–12 (2019). - PubMed
  22. Baudhuin, L. M., Biesecker, L. G., Burke, W., Green, E. D. & Green, R. C. Predictive and precision medicine with genomic data. Clin. Chem. 66, 33-41 (2019). - PubMed
  23. Green, R. C. et al. ACMG recommendations for reporting of incidental findings in clinical exome and genome sequencing. Genet. Me.d 15, 565–574 (2013). - PubMed
  24. Burke, W. et al. Recommendations for returning genomic incidental findings? We need to talk! Genet. Med. 15, 854–859 (2013). - PubMed
  25. Brook, R. H. & Lohr, K. N. Efficacy, effectiveness, variations, and quality: boundary-crossing research. Med. Care 23, 710–722 (1985). - PubMed
  26. Sun, F., Bruening, W., Erinoff, E. & Schoelles, K. M. in Addressing Challenges in Genetic Test Evaluation: Evaluation Frameworks and Assessment of Analytic Validity (Agency for Healthcare Research and Quality (US), 2011). - PubMed
  27. Marshall, C. R. et al. Best practices for the analytical validation of clinical whole-genome sequencing intended for the diagnosis of germline disease. npj Genom. Med. 5, 47 (2020). - PubMed
  28. Liu, P. et al. Reanalysis of clinical exome sequencing data. N. Engl. J. Med. 380, 2478–2480 (2019). - PubMed
  29. Posey, J. E. et al. Resolution of disease phenotypes resulting from multilocus genomic variation. N. Engl. J. Med. 376, 21–31 (2017). - PubMed
  30. French, C. E. et al. Whole genome sequencing reveals that genetic conditions are frequent in intensively ill children. Intensive Care Med. 45, 627–636 (2019). - PubMed
  31. Hart, M. R. et al. Secondary findings from clinical genomic sequencing: prevalence, patient perspectives, family history assessment, and health-care costs from a multisite study. Genet. Med. 21, 1100–1110 (2019). - PubMed
  32. Scocchia, A. et al. Clinical whole genome sequencing as a first-tier test at a resource-limited dysmorphology clinic in Mexico. NPJ Genom. Med. 4, 5 (2019). - PubMed
  33. Lenassi, E. et al. Clinical utility of genetic testing in 201 preschool children with inherited eye disorders. Genet. Med. 22, 745–751 (2019). - PubMed
  34. Oei, K., Hayeems, R. Z., Ungar, W. J., Cohn, R. D. & Cohen, E. Genetic testing among children in a complex care program. Children 4, 42 (2017). - PubMed
  35. Kingsmore, S. F. et al. A randomized, controlled trial of the analytic and diagnostic performance of singleton and trio, rapid genome and exome sequencing in Ill infants. Am. J. Hum. Genet. 105, 719–733 (2019). - PubMed
  36. Wang, H. et al. Optimized trio genome sequencing (OTGS) as a first-tier genetic test in critically ill infants: practice in China. Hum. Genet. 139, 473–482 (2020). - PubMed
  37. Stark, Z. et al. Does genomic sequencing early in the diagnostic trajectory make a difference? A follow-up study of clinical outcomes and cost-effectiveness. Genet. Med. 21, 173-180 (2018). - PubMed
  38. Rehm, H. L. et al. ACMG clinical laboratory standards for next-generation sequencing. Genet. Med. 15, 733–747 (2013). - PubMed
  39. Mackay, Z. P. et al. Quantifying downstream healthcare utilization in studies of genomic testing. Value Health 23, 559–565 (2020). - PubMed
  40. Hayeems, R. Z. et al. The development of the Clinician-reported Genetic testing Utility InDEx (C-GUIDE): a novel strategy for measuring the clinical utility of genetic testing. Genet. Med. 22, 95–101 (2019). - PubMed
  41. Buchanan, J., Wordsworth, S. & Schuh, A. Issues surrounding the health economic evaluation of genomic technologies. Pharmacogenomics 14, 1833–1847 (2013). - PubMed
  42. Regier, D. A., Weymann, D., Buchanan, J., Marshall, D. A. & Wordsworth, S. Valuation of health and nonhealth outcomes from next-generation sequencing: approaches, challenges, and solutions. Value Health 21, 1043–1047 (2018). - PubMed
  43. WHO. How to use the ICF: a practical manual for using the international classification of functioning, disability and health (ICF). https://www.who.int/classifications/drafticfpracticalmanual2.pdf?ua=1 (2013). - PubMed
  44. Stein, M. T. & Lukasik, M. K. in Developmental-Behavioral Pediatrics (ed. H. Feldman) Ch. 79, 1060 (Saunders, 2019). - PubMed
  45. Lord, C. et al. The autism diagnostic observation schedule-generic: a standard measure of social and communication deficits associated with the spectrum of autism. J. Autism Dev. Disord. 30, 205–223 (2000). - PubMed
  46. Sparrow, S. S., Cicchetti, D. V. & Balla, D. A. Vineland Adaptive Behavior Scales (Pearson Assessments, 2005). - PubMed
  47. Mayer, A. N. et al. A timely arrival for genomic medicine. Genet. Med. 13, 195–196 (2011). - PubMed
  48. Worthey, E. A. et al. Making a definitive diagnosis: successful clinical application of whole exome sequencing in a child with intractable inflammatory bowel disease. Genet. Med. 13, 255–262 (2011). - PubMed
  49. Vassy, J. L. et al. The impact of whole-genome sequencing on the primary care and outcomes of healthy adult patients: a pilot randomized trial. Ann. Intern. Med. 167, 159–169 (2017). - PubMed
  50. Denny, J. C. et al. The “All of Us” research program. N. Engl. J. Med. 381, 668–676 (2019). - PubMed
  51. Sanford, E. F. et al. Rapid whole genome sequencing has clinical utility in children in the PICU. Pediatr. Crit. Care Med., 20, 1007–1020 (2019). - PubMed
  52. Kohler, J. N. et al. Defining personal utility in genomics: a Delphi study. Clin. Genet. 92, 290–297 (2017). - PubMed
  53. Grant, P. E., Pampaka, M., Payne, K., Clarke, A. & McAllister, M. Developing a short-form of the Genetic Counselling Outcome Scale: The Genomics Outcome Scale. Eur. J. Med. Genet. 62, 324–334 (2019). - PubMed
  54. Kaphingst, K. A. et al. Effects of informed consent for individual genome sequencing on relevant knowledge. Clin. Genet. 82, 408–415 (2012). - PubMed
  55. Lupo, P. J. et al. Patients’ perceived utility of whole-genome sequencing for their healthcare: findings from the MedSeq project. Per Med. 13, 13–20 (2016). - PubMed
  56. McAllister, M., Wood, A. M., Dunn, G., Shiloh, S. & Todd, C. The Genetic Counseling Outcome Scale: a new patient-reported outcome measure for clinical genetics services. Clin. Genet. 79, 413–424 (2011). - PubMed
  57. Berkenstadt, M., Shiloh, S., Barkai, G., Katznelson, M. B. & Goldman, B. Perceived personal control (PPC): a new concept in measuring outcome of genetic counseling. Am. J. Med. Genet. 82, 53–59 (1999). - PubMed
  58. Hamilton, J. G., Lobel, M. & Moyer, A. Emotional distress following genetic testing for hereditary breast and ovarian cancer: a meta-analytic review. Health Psychol. 28, 510–518 (2009). - PubMed
  59. Creamer, M., Bell, R. & Failla, S. Psychometric properties of the Impact of Event Scale - Revised. Behav. Res. Ther. 41, 1489–1496 (2003). - PubMed
  60. Robinson, J. O. et al. Psychological outcomes related to exome and genome sequencing result disclosure: a meta-analysis of seven Clinical Sequencing Exploratory Research (CSER) Consortium studies. Genet. Med. 21, 2781–2790 (2019). - PubMed
  61. Zigmond, A. S. & Snaith, R. P. The hospital anxiety and depression scale. Acta Psychiatr. Scand. 67, 361–370 (1983). - PubMed
  62. Kroenke, K., Spitzer, R. L. & Williams, J. B. The PHQ-9: validity of a brief depression severity measure. J. Gen. Intern Med. 16, 606–613 (2001). - PubMed
  63. Spitzer, R. L., Kroenke, K., Williams, J. B. & Lowe, B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch. Intern. Med. 166, 1092–1097 (2006). - PubMed
  64. Cella, D. et al. A brief assessment of concerns associated with genetic testing for cancer: the multidimensional Impact of Cancer Risk Assessment (MICRA) questionnaire. Health Psychol. 21, 564–572 (2002). - PubMed
  65. Li, M. et al. The Feelings About genomiC Testing Results (FACToR) Questionnaire: development and preliminary validation. J. Genet. Couns. 28, 477–490 (2019). - PubMed
  66. Cernat, A. et al. Cascade genetic testing and health service use in families of children with cardiomyopathy: implications for health technology assessment (Oral presentation). 2020 Canadian Agency for Drugs and Technology in Health Symposium (2020). - PubMed
  67. Marshall, D. A. et al. The value of diagnostic testing for parents of children with rare genetic diseases. Genet. Med. 21, 2789–2806 (2019). - PubMed
  68. Kulchak Rahm, A. et al. Parental attitudes and expectations towards receiving genomic test results in healthy children. Transl. Behav. Med. 8, 44–53 (2018). - PubMed
  69. Mitchell, P. B. et al. Enhancing autonomy in biobank decisions: too much of a good thing? J. Empir. Res. Hum. Res. Ethics 13, 125–138 (2018). - PubMed
  70. Chassagne, A. et al. Exome sequencing in clinical settings: preferences and experiences of parents of children with rare diseases (SEQUAPRE study). Eur. J. Hum. Genet. 27, 701–710 (2019). - PubMed
  71. Lewis, C. et al. Parents’ motivations, concerns and understanding of genome sequencing: a qualitative interview study. Eur. J. Hum. Genet. 28, 874–884 (2020). - PubMed
  72. Buchanan, J. & Wordsworth, S. Evaluating the outcomes associated with genomic sequencing: a roadmap for future research. Pharmacoecon Open 3, 129–132 (2019). - PubMed
  73. Phillips, K. A. et al. Methodological issues in assessing the economic value of next-generation sequencing tests: many challenges and not enough solutions. Value Health 21, 1033–1042 (2018). - PubMed
  74. Sanders, G. D. et al. Recommendations for conduct, methodological practices, and reporting of cost-effectiveness analyses: second panel on cost-effectiveness in health and medicine. Jama 316, 1093–1103 (2016). - PubMed
  75. Russell, L. B., Gold, M. R., Siegel, J. E., Daniels, N. & Weinstein, M. C. The role of cost-effectiveness analysis in health and medicine. Jama 276, 1172–1177 (1996). - PubMed
  76. Schwarze, K., Buchanan, J., Taylor, J. C. & Wordsworth, S. Are whole-exome and whole-genome sequencing approaches cost-effective? A systematic review of the literature. Genet. Med. 20, 1122–1130 (2018). - PubMed
  77. Payne, K., Eden, M., Davison, N. & Bakker, E. Toward health technology assessment of whole-genome sequencing diagnostic tests: challenges and solutions. Per Med. 14, 235–247 (2017). - PubMed
  78. Christensen, K. D., Dukhovny, D., Siebert, U. & Green, R. C. Assessing the costs and cost-effectiveness of genomic sequencing. J. Pers. Med. 5, 470–486 (2015). - PubMed
  79. NHMRC. NHMRC Levels of Evidence and Grades for Recommendations for Developers of Guidelines. http://citeseerx.ist.psu.edu/viewdoc/download;jsessionid=AEFFDA62A5245D6D07F060B56789ED5A?doi=10.1.1.177.4984&rep=rep1&type=pdf (2009). - PubMed
  80. Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M. & Stetler, C. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med. Care 50, 217–226 (2012). - PubMed
  81. FDA. Considerations for Design, Developmment, and Analytical Validation of Next Generation Sequencing (NGS) - Based In Vitro Diagnostics (IVDs) Intended to Aid in the Diagnosis of Suspected Germline Diseases. (2018). - PubMed
  82. Strande, N. T. et al. Evaluating the Clinical Validity of Gene-Disease Associations: An Evidence-Based Framework Developed by the Clinical Genome Resource. American journal of human genetics 100, 895–906 (2017). - PubMed
  83. Richards, S. et al. Standards and guidelines for the interpretation of sequence variants: a joint consensus recommendation of the American College of Medical Genetics and Genomics and the Association for Molecular Pathology. Genet Med 17, 405–424 (2015). - PubMed
  84. Niguidula, N. et al. Clinical whole-exome sequencing results impact medical management. Molecular genetics & genomic medicine 6, (2018). - PubMed
  85. Schofield, D. et al. Cost-effectiveness of massively parallel sequencing for diagnosis of paediatric muscle diseases. NPJ Genom Med 2, (2017). - PubMed
  86. Furlong, W. J., Feeny, D. H., Torrance, G. W. & Barr, R. D. The Health Utilities Index (HUI) system for assessing health-related quality of life in clinical studies. Ann Med (2001). - PubMed
  87. Varni, J. W., Seid, M. & Rode, C. A. The PedsQL: measurement model for the pediatric quality of life inventory. Med Care 37, 126–139 (1999). - PubMed
  88. Rabin, R. & de Charro, F. EQ-5D: a measure of health status from the EuroQol Group. Ann Med 33, 337–343 (2001). - PubMed
  89. Stark, Z. et al. Meeting the challenges of implementing rapid genomic testing in acute pediatric care. Genet Med 20, 1554–1563 (2018). - PubMed
  90. Marshall, D. A. et al. Direct health-care costs for children diagnosed with genetic diseases are significantly higher than for children with other chronic diseases. Genet Med 21, 1049–1057 (2018). - PubMed
  91. Dragojlovic, N. et al. The cost trajectory of the diagnostic care pathway for children with suspected genetic disorders. Genet Med, 22, 292–300 (2019). - PubMed
  92. Tan, T. Y. et al. Diagnostic Impact and Cost-effectiveness of Whole-Exome Sequencing for Ambulant Children With Suspected Monogenic Conditions. JAMA Pediatr 171, 855–862 (2017). - PubMed
  93. Tsiplova, K. et al. A microcosting and cost-consequence analysis of clinical genomic testing strategies in autism spectrum disorder. Genet Med, 19, 1268–1275 (2017). - PubMed
  94. Yuen, R. et al. Whole genome sequencing resource identifies 18 new candidate genes for autism spectrum disorder. Nat Neurosci 20, 602–611 (2017). - PubMed
  95. Stark, Z. et al. Prospective comparison of the cost-effectiveness of clinical whole-exome sequencing with that of usual care overwhelmingly supports early use and reimbursement. Genet Med 19, 867–874 (2017). - PubMed

Publication Types