Comparability and Integrity of Online Remote Vs. Onsite Proctored Credentialing Exams

Comparability and Integrity of Online Remote Vs. Onsite Proctored Credentialing Exams

Authors

  • Gregory M. Hurtz
     Professor, Department of Psychology, California State University, Sacramento 6000, J St. Sacramento, CA 95819-6007
  • John A. Weiner
     Chief Science Officer, PSI Services LLC 611 N. Brand Blvd, 10th Floor Glendale, CA 91203

Keywords:

Remote Proctoring, Remote Online Proctoring, Remote Invigilation, Equivalence of Proctoring Conditions, Test Security, Preventing Online Test Fraud, Data Forensics, High-Stakes Testing, Computer-based Testing, Internet-based Testing, Test-taker Perceptions

Abstract

Since the onset of the pandemic in 2020, many credentialing organizations have incorporated online remote administration of their examinations to enable continuity of their programs. This paper describes a research study examining several high stakes credentialing examination programs that utilized mixed delivery modes, including online remote testing at home, as well as testing in test centers. Candidates were monitored in real time by a test proctor, either remotely by video camera, or in person. The study examined the comparability of test scores, instances of irregular candidate testing behavior (potential cheating), and candidate test taking experience ratings across modalities. Overall, results of the study indicated that test scores were psychometrically sound and comparable across modes; rates of suspect test taking behavior were low and not significantly different across modes; and candidate experience ratings were favorable and unrelated to testing modality. Implications for future practice and research are discussed.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

Downloads

Published

2022-03-19

How to Cite

Hurtz, G. M., & Weiner, J. A. (2022). Comparability and Integrity of Online Remote Vs. Onsite Proctored Credentialing Exams. Journal of Applied Testing Technology, 23, 36–45. Retrieved from https://www.jattjournal.net/index.php/atp/article/view/164535

Issue

Volume 23, Special Issue, 2022

Section

Articles

References

AERA, APA, NCME (2014). Standards for educational and psychological testing. Washington DC: American Educational Research Association.

Bonett, D. G. (2003). Sample size requirements for comparing two alpha coefficients. Applied Psychological Measurement, 27, 72-74. https://doi.org/10.1177/0146621602239477

Harnisch, D. L., & Linn, R. L. (1981). Analysis of item response patterns: Questionable test data and dissimilar curriculum practices. Journal of Educational Measurement, 18, 133−146. https://doi.org/10.1111/j.1745-3984.1981.tb00848.x

Hurtz, G. M., & Weiner, J. A. (2019). Analysis of test-taker profiles across a suite of statistical indices for detecting the presence and impact of cheating. Journal of Applied Testing Technology, 20(1), 1-15.

Karabatsos, G. (2003). Comparing the aberrant response detection performance of thirty-six person-fit statistics. Applied Measurement in Education, 16, 277−298. https://doi.org/10.1207/S15324818AME1604_2

Spence, D., Ward, R., Wooden, S., Browne, M., Song, H., Hawkins, R., & Wojnakowski, M. (2019). Use of resources and method of proctoring during the NBCRNA continued professional certification assessment: Analysis of outcomes. Journal of Nursing Regulation, 10(3), 37-46. https://doi.org/10.1016/S2155-8256(19)30147-4

van der Linden, W. J. (2006). A lognormal model for response times on test items. Journal of Educational and Behavioral Statistics, 31(2), 181-204. https://doi.org/10.3102/10769986031002181

Weiner, J., Saiar, A., & Granger, E. (2013). An empirical method for the detection of potential test fraud. Presented at the 2nd annual meeting of the Society for the Detection of Potential Test Fraud in Madison, Wisconsin.

Weiner, J. A., & Hurtz, G. M. (2017). A comparative study of online remote proctored versus onsite proctored highstakes exams. Journal of Applied Testing Technology, 18, 13-20.