Display options
Cite
Unberath M, Gao C, Hu Y, et al. The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective. Front Robot AI. 2021;8:716007doi: 10.3389/frobt.2021.716007.
Unberath, M., Gao, C., Hu, Y., Judish, M., Taylor, R. H., Armand, M., & Grupp, R. (2021). The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective. Frontiers in robotics and AI, 8716007. https://doi.org/10.3389/frobt.2021.716007
Unberath, Mathias, et al. "The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective." Frontiers in robotics and AI vol. 8 (2021): 716007. doi: https://doi.org/10.3389/frobt.2021.716007
Unberath M, Gao C, Hu Y, Judish M, Taylor RH, Armand M, Grupp R. The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective. Front Robot AI. 2021 Aug 30;8:716007. doi: 10.3389/frobt.2021.716007. eCollection 2021. PMID: 34527706; PMCID: PMC8436154.
Copy Download .nbib Format: NLM
Share it on

Front Robot AI. 2021 Aug 30;8:716007. doi: 10.3389/frobt.2021.716007. eCollection 2021.

The Impact of Machine Learning on 2D/3D Registration for Image-Guided Interventions: A Systematic Review and Perspective.

Frontiers in robotics and AI

Mathias Unberath, Cong Gao, Yicheng Hu, Max Judish, Russell H Taylor, Mehran Armand, Robert Grupp

Affiliations

  1. Advanced Robotics and Computationally Augmented Environments (ARCADE) Lab, Department of Computer Science, Johns Hopkins University, Baltimore, MD, United States.

PMID: 34527706 PMCID: PMC8436154 DOI: 10.3389/frobt.2021.716007

Abstract

Image-based navigation is widely considered the next frontier of minimally invasive surgery. It is believed that image-based navigation will increase the access to reproducible, safe, and high-precision surgery as it may then be performed at acceptable costs and effort. This is because image-based techniques avoid the need of specialized equipment and seamlessly integrate with contemporary workflows. Furthermore, it is expected that image-based navigation techniques will play a major role in enabling mixed reality environments, as well as autonomous and robot-assisted workflows. A critical component of image guidance is 2D/3D registration, a technique to estimate the spatial relationships between 3D structures, e.g., preoperative volumetric imagery or models of surgical instruments, and 2D images thereof, such as intraoperative X-ray fluoroscopy or endoscopy. While image-based 2D/3D registration is a mature technique, its transition from the bench to the bedside has been restrained by well-known challenges, including brittleness with respect to optimization objective, hyperparameter selection, and initialization, difficulties in dealing with inconsistencies or multiple objects, and limited single-view performance. One reason these challenges persist today is that analytical solutions are likely inadequate considering the complexity, variability, and high-dimensionality of generic 2D/3D registration problems. The recent advent of machine learning-based approaches to imaging problems that, rather than specifying the desired functional mapping, approximate it using highly expressive parametric models holds promise for solving some of the notorious challenges in 2D/3D registration. In this manuscript, we review the impact of machine learning on 2D/3D registration to systematically summarize the recent advances made by introduction of this novel technology. Grounded in these insights, we then offer our perspective on the most pressing needs, significant open problems, and possible next steps.

Copyright © 2021 Unberath, Gao, Hu, Judish, Taylor, Armand and Grupp.

Keywords: artificial intelligence; augmented reality; computer-assisted interventions; deep learning; image registration; robotic surgery; surgical data science

References

  1. IEEE Trans Robot. 2020 Feb;36(1):222-239 - PubMed
  2. IEEE J Biomed Health Inform. 2021 Sep;25(9):3300-3309 - PubMed
  3. Phys Med Biol. 2020 Oct 22;65(20):20TR01 - PubMed
  4. Langenbecks Arch Surg. 2013 Apr;398(4):501-14 - PubMed
  5. J Med Imaging (Bellingham). 2018 Apr;5(2):021212 - PubMed
  6. IEEE Trans Med Robot Bionics. 2020 Aug;2(3):437-446 - PubMed
  7. Eur Spine J. 2000 Feb;9 Suppl 1:S78-88 - PubMed
  8. Comput Vis Image Underst. 2013 Sep 1;117(9):1095-1106 - PubMed
  9. Int J Comput Assist Radiol Surg. 2020 Jul;15(7):1177-1186 - PubMed
  10. Int J Med Robot. 2021 Apr;17(2):e2228 - PubMed
  11. Med Phys. 2016 Mar;43(3):1235-48 - PubMed
  12. Adv Exp Med Biol. 2018;1093:47-63 - PubMed
  13. Phys Med Biol. 2013 Dec 7;58(23):8535-53 - PubMed
  14. J Med Imaging (Bellingham). 2015 Apr;2(2):024007 - PubMed
  15. Knee Surg Sports Traumatol Arthrosc. 2016 Nov;24(11):3482-3495 - PubMed
  16. IEEE Trans Med Imaging. 1997 Apr;16(2):187-98 - PubMed
  17. Annu Rev Biomed Eng. 2011 Aug 15;13:297-319 - PubMed
  18. Spine (Phila Pa 1976). 2019 Dec 1;44(23):E1396-E1400 - PubMed
  19. Phys Med Biol. 2014 Jan 20;59(2):271-87 - PubMed
  20. Mach Learn Med Imaging. 2020 Oct;12436:281-291 - PubMed
  21. Med Image Anal. 2013 Jul;17(5):538-55 - PubMed
  22. IEEE Trans Med Imaging. 2014 Aug;33(8):1592-600 - PubMed
  23. Med Image Anal. 2019 Jan;51:144-156 - PubMed
  24. J Biomech. 2019 May 24;89:128-133 - PubMed
  25. Int J Comput Assist Radiol Surg. 2018 Aug;13(8):1141-1149 - PubMed
  26. Int J Comput Assist Radiol Surg. 2019 Sep;14(9):1517-1528 - PubMed
  27. IEEE Trans Med Imaging. 2020 Oct;39(10):3159-3174 - PubMed
  28. J Appl Clin Med Phys. 2016 Jan 08;17(1):22-33 - PubMed
  29. Med Image Anal. 2019 Oct;57:226-236 - PubMed
  30. Int J Comput Assist Radiol Surg. 2020 May;15(5):759-769 - PubMed
  31. N Engl J Med. 2013 Oct 10;369(15):1434-42 - PubMed
  32. Med Image Anal. 2021 Feb;68:101917 - PubMed
  33. Ann Surg. 2021 Apr 1;273(4):684-693 - PubMed
  34. IEEE Trans Med Imaging. 2012 Apr;31(4):870-81 - PubMed
  35. IEEE Trans Med Imaging. 2017 Feb;36(2):607-617 - PubMed
  36. Med Image Comput Comput Assist Interv. 2013;16(Pt 2):190-7 - PubMed
  37. J Med Imaging (Bellingham). 2018 Apr;5(2):021204 - PubMed
  38. Evol Comput. 2003 Spring;11(1):1-18 - PubMed
  39. Med Image Comput Comput Assist Interv. 2020 Oct;12263:329-339 - PubMed
  40. Int J Comput Assist Radiol Surg. 2019 Sep;14(9):1495-1506 - PubMed
  41. Int J Comput Assist Radiol Surg. 2021 May;16(5):779-787 - PubMed
  42. Phys Med Biol. 2017 Apr 21;62(8):3330-3351 - PubMed
  43. Comput Methods Programs Biomed. 2021 Feb;199:105901 - PubMed
  44. Phys Med Biol. 2008 Aug 21;53(16):4303-16 - PubMed
  45. Phys Med Biol. 2015 May 7;60(9):3617-37 - PubMed
  46. BMJ. 2009 Jul 21;339:b2535 - PubMed
  47. Proc SPIE Int Soc Opt Eng. 2016 Feb-Mar;9784: - PubMed
  48. IEEE Trans Med Imaging. 2016 May;35(5):1352-1363 - PubMed
  49. Proc IEEE Inst Electr Electron Eng. 2020 Jan;108(1):198-214 - PubMed
  50. Phys Med Biol. 2020 Dec 12;65(24):245019 - PubMed
  51. Med Image Anal. 2012 Apr;16(3):642-61 - PubMed
  52. Med Image Anal. 2015 Dec;26(1):108-19 - PubMed
  53. IEEE Trans Med Imaging. 2013 Feb;32(2):317-28 - PubMed
  54. Int J Comput Assist Radiol Surg. 2019 Sep;14(9):1463-1473 - PubMed
  55. J Neurointerv Surg. 2021 May;13(5):483-491 - PubMed
  56. Adv Vis Comput. 2016 Dec;10072:3-12 - PubMed
  57. IEEE Trans Biomed Eng. 2020 Feb;67(2):441-452 - PubMed
  58. J Orthop Sci. 2003;8(3):442-8 - PubMed
  59. Neuroimage. 2017 Sep;158:378-396 - PubMed
  60. Int J Comput Assist Radiol Surg. 2020 Nov;15(11):1787-1796 - PubMed
  61. Med Phys. 2015 Nov;42(11):6433-47 - PubMed

Publication Types