Display options
Cite
Norcini J, Reshetar R, Lipner R. The answer key as a source of error in performance assessment. Adv Health Sci Educ Theory Pract. 1996;1(2):103-10doi: 10.1007/BF00159274.
Norcini, J., Reshetar, R., & Lipner, R. (1996). The answer key as a source of error in performance assessment. Advances in health sciences education : theory and practice, 1(2), 103-10. https://doi.org/10.1007/BF00159274
Norcini, J, et al. "The answer key as a source of error in performance assessment." Advances in health sciences education : theory and practice vol. 1,2 (1996): 103-10. doi: https://doi.org/10.1007/BF00159274
Norcini J, Reshetar R, Lipner R. The answer key as a source of error in performance assessment. Adv Health Sci Educ Theory Pract. 1996 Jan;1(2):103-10. doi: 10.1007/BF00159274. PMID: 24178999.
Copy Download .nbib Format: NLM
Share it on

Adv Health Sci Educ Theory Pract. 1996 Jan;1(2):103-10. doi: 10.1007/BF00159274.

The answer key as a source of error in performance assessment.

Advances in health sciences education : theory and practice

J Norcini, R Reshetar, R Lipner

Affiliations

  1. American Board of Internal Medicine, 3624 Market Street, 19104, Philadelphia, PA, USA, [email protected].

PMID: 24178999 DOI: 10.1007/BF00159274

Abstract

OBJECTIVE: To determine the impact of variability in answer key construction (i.e., option weighting) on total errors of measurement and to compare several designs for reducing this effect.

METHODS: A video-based format that assesses ability to interpret arteriograms is studied because it reproduces with high fidelity an important task faced by cardiologists. Responses given by highly qualified examinees are used to develop answer keys by the aggregate scoring method, and generalizability theory is applied to estimate the error variance in answer key construction.

SAMPLE: Two hundred and two examinees who volunteered to participate in the study after taking their certifying examination in cardiovascular diseases.

RESULTS: RESULTS show less scorer variability than case variability; however, several scorers make a sizeable reduction in measurement error.

CONCLUSION: Although there is some error in the answer key construction process, its size is relatively small and it should not be a major impediment to the use of performance-based item formats.

References

  1. Eval Health Prof. 1982 Jun;5(2):210-7 - PubMed
  2. Acad Med. 1990 Sep;65(9 Suppl):S15-6 - PubMed
  3. Med Educ. 1985 Jan;19(1):43-7 - PubMed
  4. J Med Educ. 1972 Apr;47(4):263-6 - PubMed

Publication Types