VALIDITY IN COMPUTER-BASED TESTING: A LITERATURE REVIEW OF COMPARABILITY ISSUES AND EXAMINEE PERSPECTIVES
DOI:
https://doi.org/10.22373/ej.v2i2.345Keywords:
Computer-Based Tests, Paper-and-Pencil Based Tests, comparability, examinee perspectives, validity, reliabilityAbstract
These past years have seen the growing popularity of the Computer-Based Tests (CBTs) in various disciplines, for various purposes, although the Paper-and Pencil Based Tests (P&Ps) are still in use. However, many question on whether the use of CBTs outperform the effectiveness of the P&Ps or if the CBTs can become a valid measuring tool compared to the PBTs. This paper tries to present the comparison on both the CBTs and the P&Ps and their respective examinee perspectives in order to figure out if doubts should arise to the emergence of the CBTs over the classic P&Ps. Findings showed that the CBTs are advantageous in that they are both efficient (reducing testing time) and effective (maintaining the test reliability) over the P&P versions. Nevertheless, the CBTs still need to have their variables well-designed (e.g., study design, computer algorithm) in order for the scores to be comparable to those in the P&P tests since the score equivalence is one of the validity evidences needed in a CBT.References
American Educational Research Association (AERA), American Psychological Association (APA), & National Council on Measurement in Education (NCME). (1999). Standards for educational and psychological testing. Washington, DC: Author.
Bennett, R. E., & Rock, D. A. (1995). Generalizability, validity, and examinee perceptions of a computer-delivered formulating hypothesis test. Journal of Educational Measurement, 32(1), 19–36.
Gallagher, A., Bennett, R. A., Cahalan, C., & Rock, D. A. (2002). Validity and fairness in technology-based assessment: detecting construct-irrelevant variance in an open-ended, computerized mathematics task. Educational Assessment, 8(1), 27–41.
Green, B. F., Bock, R. D., Humphreys, L. G., Linn, R. L., & Reckase, M. D. (1984). Technical guidelines for assessing computerized adaptive tests. Journal of Educational Measurement, 21(4), 347-360.
Mead, A. D., & Drasgow, F. (1993). Equivalence of computerized and paper-and-pencil cognitive ability tests: a meta-analysis. Psychological Bulletin, 114(3), 449-458.
Neuman, G., & Baydoun, R. (1998). Computerization of paper-and-pencil tests: when are they equivalent? Applied Psychological Measurement, 22(1), 71-83.
Olsen, J. B. (2000). Guidelines for computer-based testing. Retrieved May 17, 2008 from http://www.isoc.org/oti/articles/0500/olsen.html.
Pomplun, M., Frey, S., & Becker, D. (2002). The score equivalence of paper-and-pencil and computerized versions of a speeded test of reading comprehension. Educational and Psychological Measurement, 62(2), 337-35.
Russell, M., Goldberg, A., & O’Connor, K. (2003). Computer based-testing and validity: a look back and into the future. Retrieved May 2, 2008 from http://escholarship.bc.edu/cgi/viewcontent.cgi?article=1003&context=intasc.
Wang, T, & Kolen, M. J. (2001). Evaluating comparability in computerized adaptive testing: issues, criteria and an example. Journal of Educational Measurement, 38 (1), 19-49.
Wang, S., Jiao, H., Young, M. J., Brooks, T, & Olson, J. (2008). Comparability of computer-based and paper-and-pencil testing in K 12 reading assessments: a meta-analysis of testing mode effects. Educational and Psychological Measurement, 28(1), 5-24.
Wise, S. L., & Kingsbury, G. G. (2000). Practical issues in developing and maintaining a computerized adaptive testing program. Psicológica, 21,135-155.
Wise, S. L., & Kong, X. (2005). Response time effort: a new measure of examinee motivation in computer-based tests. Applied Measurement in Education, 18(2), 163–183.
Zenisky, A. L., & Sireci, S. G. (2002). Technological innovations in large-scale assessment. Applied Measurement in Education, 15(4), 337-362.
Downloads
Published
Issue
Section
License
Proposed Policy for Journals That Offer Open Access
Authors who publish with Englisia journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
