Display options
Cite
Blanche P, Dartigues JF, Riou J. A closed max-t test for multiple comparisons of areas under the ROC curve. Biometrics. 2020;doi: 10.1111/biom.13401.
Blanche, P., Dartigues, J. F., & Riou, J. (2020). A closed max-t test for multiple comparisons of areas under the ROC curve. Biometrics, . https://doi.org/10.1111/biom.13401
Blanche, Paul, et al. "A closed max-t test for multiple comparisons of areas under the ROC curve." Biometrics vol. (2020). doi: https://doi.org/10.1111/biom.13401
Blanche P, Dartigues JF, Riou J. A closed max-t test for multiple comparisons of areas under the ROC curve. Biometrics. 2020 Nov 18; doi: 10.1111/biom.13401. Epub 2020 Nov 18. PMID: 33207001.
Copy Download .nbib Format: NLM
Share it on

Biometrics. 2020 Nov 18; doi: 10.1111/biom.13401. Epub 2020 Nov 18.

A closed max-t test for multiple comparisons of areas under the ROC curve.

Biometrics

Paul Blanche, Jean-François Dartigues, Jérémie Riou

Affiliations

  1. Section of Biostatistics, University of Copenhagen, Øster Farimagsgade, Copenhagen K, Denmark.
  2. Department of Cardiology, Copenhagen University Hospital Herlev and Gentofte, Hellerup, Denmark.
  3. Department of Cardiology, The Heart Centre, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark.
  4. Inserm, Bordeaux Population Health Research Center, UMR 1219 Univ. Bordeaux, Bordeaux, France.
  5. Memory Consultation, CMRR, Bordeaux University Hospital, Bordeaux, France.
  6. MINT UMR INSERM 1066, CNRS 6021, Université d'Angers, France.

PMID: 33207001 DOI: 10.1111/biom.13401

Abstract

Comparing areas under the ROC curve (AUCs) is a popular approach to compare prognostic biomarkers. The aim of this paper is to present an efficient method to control the family-wise error rate when multiple comparisons are performed. We suggest to combine the max-t test and the closed testing procedures. We build on previous work on asymptotic results for ROC curves and on general multiple testing methods to efficiently take into account both the correlations between the test statistics and the logical constraints between the null hypotheses. The proposed method results in an uniformly more powerful procedure than both the single-step max-t test procedure and popular stepwise extensions of the Bonferroni procedure, such as Bonferroni-Holm. As demonstrated in this paper, the method can be applied in most usual contexts, including the time-dependent context with right censored data. We show how the method works in practice through a motivating example where we compare several psychometric scores to predict the t-year risk of Alzheimer's disease. The example illustrates several multiple testing settings and demonstrates the advantage of using the proposed methods over common alternatives. R code has been made available to facilitate the use of the methods by others.

© 2020 The International Biometric Society.

Keywords: ROC curve; biomarker; closed testing; max-t test; multiple testing; survival analysis

References

  1. Aisen, P.S., Andrieu, S., Sampaio, C., Carrillo, M., Khachaturian, Z.S., Dubois, B., et al. (2011) Report of the task force on designing clinical trials in early (predementia) AD. Neurology, 76, 280-286. - PubMed
  2. Amieva, H., Le Goff, M., Millet, X., Orgogozo, J.M., Pérès, K., Barberger-Gateau, P., et al. (2008) Prodromal Alzheimer's disease: successive emergence of the clinical symptoms. Annals of Neurology, 64, 492-498. - PubMed
  3. Blanche, P., Dartigues, J.-F. and Jacqmin-Gadda, H. (2013) Estimating and comparing time-dependent areas under receiver operating characteristic curves for censored event times with competing risks. Statistics in Medicine, 32, 5381-5397. - PubMed
  4. Bretz, F., Hothorn, T. and Westfall, P. (2016) Multiple Comparisons Using R. BOca Raton, FL: CRC Press. - PubMed
  5. Dartigues, J.F., Gagnon, M., Barberger-Gateau, P., Letenneur, L., Commenges, D., Sauvel, C., et al. (1992) The Paquid epidemiological program on brain ageing. Neuroepidemiology, 11, 14-18. - PubMed
  6. DeLong, E.R., DeLong, D.M. and Clarke-Pearson, D.L. (1988) Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics, 44, 837-845. - PubMed
  7. Genz, A. and Bretz, F. (2009) Computation of Multivariate Normal and t Probabilities. New York, NY: Springer Science & Business Media. - PubMed
  8. Hothorn, T., Bretz, F. and Westfall, P. (2008) Simultaneous inference in general parametric models. Biometrical Journal, 50, 346-63. - PubMed
  9. Hsu, J.C., Qiu, P., Hin, L.Y., Mutti, D.O. and Zadnik, K. (2004) Multiple comparisons with the best ROC curve. In Recent Developments in Multiple Comparison Procedures. Benjamini, Y. and Bretz, F. and Sarkar, S. (eds.). Beachwood, OH: Institute of Mathematical Statistics, pp. 65-75. - PubMed
  10. Hung, H. and Chiang, C. (2010) Estimation methods for time-dependent AUC models with survival data. Canadian Journal of Statistics, 38, 8-26. - PubMed
  11. Marcus, R., Peritz, E. and Gabriel, K.E. (1976) On closed testing procedures with special reference to ordered analysis of variance. Biometrika, 63, 655-660. - PubMed
  12. Pepe, M. (2003) The Statistical Evaluation of Medical Tests for Classification and Prediction. Oxford, MA: Oxford University Press. - PubMed
  13. Pipper, C.B., Ritz, C. and Bisgaard, H. (2012) A versatile method for confirmatory evaluation of the effects of a covariate in multiple models. Journal of the Royal Statistical Society: Series C (Applied Statistics), 61, 315-326. - PubMed
  14. Proust-Lima, C., Amieva, H., Dartigues, J.-F. and Jacqmin-Gadda, H. (2007) Sensitivity of four psychometric tests to measure cognitive changes in brain aging-population-based studies. American Journal of Epidemiology, 165, 344-350. - PubMed
  15. Shaffer, J.P. (1986) Modified sequentially rejective multiple test procedures. Journal of the American Statistical Association, 81, 826-831. - PubMed
  16. Tsiatis, A. (2006) Semiparametric theory and missing data. New York, NY: Springer Verlag. - PubMed
  17. Uno, H., Cai, T., Tian, L. and Wei, L. (2007) Evaluating prediction rules for t-year survivors with censored regression models. Journal of the American Statistical Association, 102, 527-537. - PubMed
  18. van derVaart, A. (1998) Asymptotic Statistics. Cambridge, MA: Cambridge University Press. - PubMed
  19. Westfall, P. and Young, S. (1993) Resampling-Based Multiple Testing: Examples and Methods for P-Value Adjustment. New York, NY: Wiley. - PubMed
  20. Westfall, P.H. (1997) Multiple testing of general contrasts using logical constraints and correlations. Journal of the American Statistical Association, 92, 299-306. - PubMed
  21. Westfall, P.H. and Tobias, R.D. (2007) Multiple testing of general contrasts: truncated closure and the extended Shaffer-Royen method. Journal of the American Statistical Association, 102, 487-494. - PubMed
  22. Westfall, P.H., Tobias, R.D. and Wolfinger, R.D. (2011) Multiple Comparisons and Multiple Tests Using SAS, 2nd edition. Cary, NC: SAS Institute. - PubMed
  23. Zou, G. and Yue, L. (2013) Using confidence intervals to compare several correlated areas under the receiver operating characteristic curves. Statistics in Medicine, 32, 5077-5090. - PubMed

Publication Types

Grant support