Display options
Cite
Schurink NW, van Kranen SR, Roberti S, et al. Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility. Eur Radiol. 2021;doi: 10.1007/s00330-021-08251-8.
Schurink, N. W., van Kranen, S. R., Roberti, S., van Griethuysen, J. J. M., Bogveradze, N., Castagnoli, F., Khababi, N. E., Bakers, F. C. H., de Bie, S. H., Bosma, G. P. T., Cappendijk, V. C., Geenen, R. W. F., Neijenhuis, P. A., Peterson, G. M., Veeken, C. J., Vliegen, R. F. A., Beets-Tan, R. G. H., & Lambregts, D. M. J. (2021). Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility. European radiology, . https://doi.org/10.1007/s00330-021-08251-8
Schurink, Niels W, et al. "Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility." European radiology vol. (2021). doi: https://doi.org/10.1007/s00330-021-08251-8
Schurink NW, van Kranen SR, Roberti S, van Griethuysen JJM, Bogveradze N, Castagnoli F, Khababi NE, Bakers FCH, de Bie SH, Bosma GPT, Cappendijk VC, Geenen RWF, Neijenhuis PA, Peterson GM, Veeken CJ, Vliegen RFA, Beets-Tan RGH, Lambregts DMJ. Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility. Eur Radiol. 2021 Oct 16; doi: 10.1007/s00330-021-08251-8. Epub 2021 Oct 16. PMID: 34655313.
Copy Download .nbib Format: NLM
Share it on

Eur Radiol. 2021 Oct 16; doi: 10.1007/s00330-021-08251-8. Epub 2021 Oct 16.

Sources of variation in multicenter rectal MRI data and their effect on radiomics feature reproducibility.

European radiology

Niels W Schurink, Simon R van Kranen, Sander Roberti, Joost J M van Griethuysen, Nino Bogveradze, Francesca Castagnoli, Najim El Khababi, Frans C H Bakers, Shira H de Bie, Gerlof P T Bosma, Vincent C Cappendijk, Remy W F Geenen, Peter A Neijenhuis, Gerald M Peterson, Cornelis J Veeken, Roy F A Vliegen, Regina G H Beets-Tan, Doenja M J Lambregts

Affiliations

  1. Department of Radiology, The Netherlands Cancer Institute, POB 90203, 1006 BE, Amsterdam, The Netherlands.
  2. GROW School for Oncology & Developmental Biology, University of Maastricht, Maastricht, The Netherlands.
  3. Department of Radiation Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands.
  4. Department of Epidemiology and Biostatistics, The Netherlands Cancer Institute, Amsterdam, The Netherlands.
  5. Department of Radiology, Acad. F. Todua Medical Center, Research Institute of Clinical Medicine, Tbilisi, Georgia.
  6. Department of Radiology, Maastricht University Medical Centre, Maastricht, The Netherlands.
  7. Department of Radiology, Deventer Ziekenhuis, Deventer, The Netherlands.
  8. Department of Interventional Radiology, Elisabeth Tweesteden Hospital, Tilburg, The Netherlands.
  9. Department of Radiology, Jeroen Bosch Hospital, 's-Hertogenbosch, The Netherlands.
  10. Department of Radiology, Northwest Clinics, Alkmaar, The Netherlands.
  11. Department of Surgery, Alrijne Hospital, Leiderdorp, The Netherlands.
  12. Department of Radiology, Spaarne Gasthuis, Haarlem, The Netherlands.
  13. Department of Radiology, IJsselland Hospital, Capelle Aan Den IJssel, The Netherlands.
  14. Department of Radiology, Zuyderland Medical Center, Heerlen, The Netherlands.
  15. Department of Radiology, The Netherlands Cancer Institute, POB 90203, 1006 BE, Amsterdam, The Netherlands. [email protected].
  16. GROW School for Oncology & Developmental Biology, University of Maastricht, Maastricht, The Netherlands. [email protected].
  17. Department of Radiology, The Netherlands Cancer Institute, POB 90203, 1006 BE, Amsterdam, The Netherlands. [email protected].

PMID: 34655313 DOI: 10.1007/s00330-021-08251-8

Abstract

OBJECTIVES: To investigate sources of variation in a multicenter rectal cancer MRI dataset focusing on hardware and image acquisition, segmentation methodology, and radiomics feature extraction software.

METHODS: T2W and DWI/ADC MRIs from 649 rectal cancer patients were retrospectively acquired in 9 centers. Fifty-two imaging features (14 first-order/6 shape/32 higher-order) were extracted from each scan using whole-volume (expert/non-expert) and single-slice segmentations using two different software packages (PyRadiomics/CapTk). Influence of hardware, acquisition, and patient-intrinsic factors (age/gender/cTN-stage) on ADC was assessed using linear regression. Feature reproducibility was assessed between segmentation methods and software packages using the intraclass correlation coefficient.

RESULTS: Image features differed significantly (p < 0.001) between centers with more substantial variations in ADC compared to T2W-MRI. In total, 64.3% of the variation in mean ADC was explained by differences in hardware and acquisition, compared to 0.4% by patient-intrinsic factors. Feature reproducibility between expert and non-expert segmentations was good to excellent (median ICC 0.89-0.90). Reproducibility for single-slice versus whole-volume segmentations was substantially poorer (median ICC 0.40-0.58). Between software packages, reproducibility was good to excellent (median ICC 0.99) for most features (first-order/shape/GLCM/GLRLM) but poor for higher-order (GLSZM/NGTDM) features (median ICC 0.00-0.41).

CONCLUSIONS: Significant variations are present in multicenter MRI data, particularly related to differences in hardware and acquisition, which will likely negatively influence subsequent analysis if not corrected for. Segmentation variations had a minor impact when using whole volume segmentations. Between software packages, higher-order features were less reproducible and caution is warranted when implementing these in prediction models.

KEY POINTS: • Features derived from T2W-MRI and in particular ADC differ significantly between centers when performing multicenter data analysis. • Variations in ADC are mainly (> 60%) caused by hardware and image acquisition differences and less so (< 1%) by patient- or tumor-intrinsic variations. • Features derived using different image segmentations (expert/non-expert) were reproducible, provided that whole-volume segmentations were used. When using different feature extraction software packages with similar settings, higher-order features were less reproducible.

© 2021. The Author(s).

Keywords: Image processing, Computer-assisted; Magnetic resonance imaging; Multicenter study; Rectal neoplasms; Reproducibility of results

References

  1. Schurink NW, Lambregts DMJ, Beets-Tan RGH (2019) Diffusion-weighted imaging in rectal cancer: current applications and future perspectives. Br J Radiol 92:20180655 - PubMed
  2. Di Re AM, Sun Y, Sundaresan P et al (2021) MRI radiomics in the prediction of therapeutic response to neoadjuvant therapy for locoregionally advanced rectal cancer: a systematic review. Expert Rev Anticancer Ther 21:425–449 - PubMed
  3. Staal FCR, van der Reijd DJ, Taghavi M et al (2021) Radiomics for the prediction of treatment outcome and survival in patients with colorectal cancer: a systematic review. Clin Colorectal Cancer 20:52–71 - PubMed
  4. Pham TT, Liney GP, Wong K, Barton MB (2017) Functional MRI for quantitative treatment response prediction in locally advanced rectal cancer. Br J Radiol 90:20151078 - PubMed
  5. Lambin P, Rios-Velazquez E, Leijenaar R et al (2012) Radiomics: extracting more information from medical images using advanced feature analysis. Eur J Cancer 48:441–446 - PubMed
  6. Wright BD, Vo N, Nolan J et al (2020) An analysis of key indicators of reproducibility in radiology. Insights Imaging 11:65 - PubMed
  7. Song J, Yin Y, Wang H et al (2020) A review of original articles published in the emerging field of radiomics. Eur J Radiol 127:108991 - PubMed
  8. Aerts HJWL (2018) Data science in radiology: a path forward. Clin Cancer Res 24:532–534 - PubMed
  9. Traverso A, Wee L, Dekker A, Gillies R (2018) Repeatability and reproducibility of radiomic features: a systematic review. Int J Radiat Oncol 102:1143–1158 - PubMed
  10. Kahan BC (2014) Accounting for centre-effects in multicentre trials with a binary outcome – when, why, and how? BMC Med Res Methodol 14:20 - PubMed
  11. Luo J, Schumacher M, Scherer A et al (2010) A comparison of batch effect removal methods for enhancement of prediction performance using MAQC-II microarray gene expression data. Pharmacogenomics J 10:278–291 - PubMed
  12. Da-Ano R, Visvikis D, Hatt M (2020) Harmonization strategies for multicenter radiomics investigations. Phys Med Biol 65:24TR02 - PubMed
  13. Mi H, Yuan M, Suo S et al (2020) Impact of different scanners and acquisition parameters on robustness of MR radiomics features based on women’s cervix. Sci Rep 10:20407 - PubMed
  14. Baeßler B, Weiss K, Pinto dos Santos D (2019) Robustness and reproducibility of radiomics in magnetic resonance imaging. Invest Radiol 54:221–228 - PubMed
  15. Ammari S, Pitre-Champagnat S, Dercle L et al (2021) Influence of magnetic field strength on magnetic resonance imaging radiomics features in brain imaging, an in vitro and in vivo study. Front Oncol 10:1–11 - PubMed
  16. Yuan J, Xue C, Lo G et al (2021) Quantitative assessment of acquisition imaging parameters on MRI radiomics features: a prospective anthropomorphic phantom study using a 3D–T2W-TSE sequence for MR-guided-radiotherapy. Quant Imaging Med Surg 11:1870–1887 - PubMed
  17. Gourtsoyianni S, Doumou G, Prezzi D et al (2017) Primary rectal cancer: repeatability of global and local-regional mr imaging texture features. Radiology 284:552–561 - PubMed
  18. Hoebel KV, Patel JB, Beers AL et al (2021) Radiomics repeatability pitfalls in a scan-rescan MRI study of glioblastoma. Radiol Artif Intell 3:e190199 - PubMed
  19. Traverso A, Kazmierski M, Shi Z et al (2019) Stability of radiomic features of apparent diffusion coefficient (ADC) maps for locally advanced rectal cancer in response to image pre-processing. Phys Med 61:44–51 - PubMed
  20. Attenberger UI, Pilz LR, Morelli JN et al (2014) Multi-parametric MRI of rectal cancer - do quantitative functional MR measurements correlate with radiologic and pathologic tumor stages? Eur J Radiol 83:1036–1043 - PubMed
  21. Perinetti G (2018) StaTips Part IV: Selection, interpretation and reporting of the intraclass correlation coefficient. South Eur J Orthod Dentofac Res 5:3–5 - PubMed
  22. Fiset S, Welch ML, Weiss J et al (2019) Repeatability and reproducibility of MRI-based radiomic features in cervical cancer. Radiother Oncol 135:107–114 - PubMed
  23. Michoux NF, Ceranka JW, Vandemeulebroucke J et al (2021) Repeatability and reproducibility of ADC measurements: a prospective multicenter whole-body-MRI study. Eur Radiol 31:4514–4527 - PubMed
  24. Donati OF, Chong D, Nanz D et al (2014) Diffusion-weighted MR imaging of upper abdominal organs: field Strength and intervendor variability of apparent diffusion coefficients. Radiology 270:454–463 - PubMed
  25. Orlhac F, Lecler A, Savatovski J et al (2021) How can we combat multicenter variability in MR radiomics? Validation of a correction procedure. Eur Radiol 31:2272–2280 - PubMed
  26. Haarburger C, Schock J, Truhn D, et al (2019) Radiomic feature stability analysis based on probabilistic segmentations - PubMed
  27. Lee J, Steinmann A, Ding Y et al (2021) Radiomics feature robustness as measured using an MRI phantom. Sci Rep 11:3973 - PubMed
  28. Lambregts DMJ, Beets GL, Maas M et al (2011) Tumour ADC measurements in rectal cancer: effect of ROI methods on ADC values and interobserver variability. Eur Radiol 21:2567–2574 - PubMed
  29. Nougaret S, Vargas HA, Lakhman Y et al (2016) Intravoxel incoherent motion–derived histogram metrics for assessment of response after combined chemotherapy and radiation therapy in rectal cancer: initial experience and comparison between single-section and volumetric analyses. Radiology 280:446–454 - PubMed
  30. van Griethuysen JJM, Lambregts DMJ, Trebeschi S et al (2020) Radiomics performs comparable to morphologic assessment by expert radiologists for prediction of response to neoadjuvant chemoradiotherapy on baseline staging MRI in rectal cancer. Abdom Radiol (NY) 45:632–643 - PubMed
  31. Trebeschi S, van Griethuysen JJM, Lambregts DMJ et al (2017) Deep learning for fully-automated localization and segmentation of rectal cancer on multiparametric MR. Sci Rep 7:5301 - PubMed
  32. Van Heeswijk MM, Lambregts DMJ, van Griethuysen JJM et al (2016) Automated and semiautomated segmentation of rectal tumor volumes on diffusion-weighted MRI: can it replace manual volumetry? Int J Radiat Oncol Biol Phys 94:824–831 - PubMed
  33. Van Griethuysen J, Schurink N, Lahaye MJ, et al (2020) Deep learning for fully automated segmentation of rectal tumours on MRI in a multicentre setting. In: ESGAR 2020 Book of Abstracts. Insights Imaging, 11:64 - PubMed
  34. Li Q, Bai H, Chen Y et al (2017) A fully-automatic multiparametric radiomics model: towards reproducible and prognostic imaging signature for prediction of overall survival in glioblastoma multiforme. Sci Rep 7:14331 - PubMed
  35. Duron L, Balvay D, Vande Perre S et al (2019) Gray-level discretization impacts reproducible MRI radiomics texture features. PLoS One 14:e0213459 - PubMed
  36. Isaksson LJ, Raimondi S, Botta F et al (2020) Effects of MRI image normalization techniques in prostate cancer radiomics. Phys Med 71:7–13 - PubMed

Publication Types

Grant support