J Appl Meas. 2016;17(1):54-78.

The Impact of Item Parameter Drift in Computer Adaptive Testing (CAT).

Journal of applied measurement

Nicole Risk

PMID: 26784378

Abstract

This study looked at numerous aspects of item parameter drift (IPD) and its impact on measurement in computer adaptive testing (CAT). A series of CAT simulations were conducted, varying the amount and magnitude of IPD, as well as the size of the item pool. The effects of IPD on measurement precision, classification, and test efficiency, were evaluated using a number of criteria. These included bias, root mean square error (RMSE), absolute average difference (AAD), total percentages of misclassifcation, the number of false positives and false negatives, the total test lengths, and item exposure rates. The results revealed negligible differences when comparing the IPD conditions to the baseline condition for all measures of precision, classification accuracy, and test efficiency. The most relevant finding indicates that magnitude of drift has a larger impact on measurement precision than the number of items with drift.

MeSH terms

• Computer Simulation
• Computer-Assisted Instruction / methods
• Computer-Assisted Instruction / statistics & numerical data
• Data Interpretation, Statistical*
• Educational Measurement / methods
• Educational Measurement / statistics & numerical data
• Models, Statistical*
• Psychometrics / methods
• Reproducibility of Results
• Sensitivity and Specificity

Publication Types

• Journal Article