Display options
Cite
Khajouei R, Zahiri Esfahani M, Jahani Y. Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems. J Am Med Inform Assoc. 2017;24:e55-e60doi: 10.1093/jamia/ocw100.
Khajouei, R., Zahiri Esfahani, M., & Jahani, Y. (2017). Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems. Journal of the American Medical Informatics Association : JAMIA, 24e55-e60. https://doi.org/10.1093/jamia/ocw100
Khajouei, Reza, et al. "Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems." Journal of the American Medical Informatics Association : JAMIA vol. 24 (2017): e55-e60. doi: https://doi.org/10.1093/jamia/ocw100
Khajouei R, Zahiri Esfahani M, Jahani Y. Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems. J Am Med Inform Assoc. 2017 Apr 01;24:e55-e60. doi: 10.1093/jamia/ocw100. PMID: 27497799; PMCID: PMC7651936.
Copy Download .nbib Format: NLM
Share it on

J Am Med Inform Assoc. 2017 Apr 01;24:e55-e60. doi: 10.1093/jamia/ocw100.

Comparison of heuristic and cognitive walkthrough usability evaluation methods for evaluating health information systems.

Journal of the American Medical Informatics Association : JAMIA

Reza Khajouei, Misagh Zahiri Esfahani, Yunes Jahani

Affiliations

  1. Medical Informatics Research Center, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran.
  2. Department of Health Information Management and Technology, Faculty of Management and Medical Information Sciences, Kerman University of Medical Sciences, Kerman, Iran.
  3. Regional Knowledge Hub and World Health Organization Collaborating Centre for HIV Surveillance, Institute for Futures Studies in Health, Kerman University of Medical Sciences, Kerman, Iran.
  4. Department of Biostatistics and Epidemiology, School of Public Health, Kerman University of Medical Sciences, Kerman, Iran.

PMID: 27497799 PMCID: PMC7651936 DOI: 10.1093/jamia/ocw100

Abstract

OBJECTIVE: There are several user-based and expert-based usability evaluation methods that may perform differently according to the context in which they are used. The objective of this study was to compare 2 expert-based methods, heuristic evaluation (HE) and cognitive walkthrough (CW), for evaluating usability of health care information systems.

MATERIALS AND METHODS: Five evaluators independently evaluated a medical office management system using HE and CW. We compared the 2 methods in terms of the number of identified usability problems, their severity, and the coverage of each method.

RESULTS: In total, 156 problems were identified using the 2 methods. HE identified a significantly higher number of problems related to the "satisfaction" attribute ( P  = .002). The number of problems identified using CW concerning the "learnability" attribute was significantly higher than those identified using HE ( P  = .005). There was no significant difference between the number of problems identified by HE, based on different usability attributes ( P  = .232). Results of CW showed a significant difference between the number of problems related to usability attributes ( P  < .0001). The average severity of problems identified using CW was significantly higher than that of HE ( P  < .0001).

CONCLUSION: This study showed that HE and CW do not differ significantly in terms of the number of usability problems identified, but they differ based on the severity of problems and the coverage of some usability attributes. The results suggest that CW would be the preferred method for evaluating systems intended for novice users and HE for users who have experience with similar systems. However, more studies are needed to support this finding.

© The Author 2016. Published by Oxford University Press on behalf of the American Medical Informatics Association. All rights reserved. For Permissions, please email: [email protected]

Keywords: cognitive walkthrough; comparison; health information systems; heuristic evaluation; user-computer interface

References

  1. J Am Med Inform Assoc. 2002 Jan-Feb;9(1):1-15 - PubMed
  2. J Healthc Inf Manag. 2003 Fall;17(4):51-7 - PubMed
  3. Pediatrics. 2015 Jan;135(1):e7-15 - PubMed
  4. Ann Fam Med. 2013 Jan-Feb;11(1):14-9 - PubMed
  5. Stud Health Technol Inform. 2015;208:221-5 - PubMed
  6. Int J Med Inform. 2011 May;80(5):341-50 - PubMed
  7. Int J Med Inform. 2010 Oct;79(10):690-8 - PubMed
  8. Int J Med Inform. 2009 May;78(5):340-53 - PubMed
  9. Int J Med Inform. 2010 Aug;79(8):565-75 - PubMed
  10. Stud Health Technol Inform. 2009;143:322-7 - PubMed
  11. J Biomed Inform. 2010 Apr;43(2):224-32 - PubMed
  12. J Biomed Inform. 2011 Aug;44(4):700-1; discussion 702-3 - PubMed
  13. AMIA Annu Symp Proc. 2009 Nov 14;2009:714-8 - PubMed
  14. Stud Health Technol Inform. 2009;150:532-6 - PubMed
  15. J Am Med Inform Assoc. 2007 Jan-Feb;14(1):110-7 - PubMed

MeSH terms

Publication Types