Educational Test Approaches: The Suitability of Computer-Based Test Types for Assessment and Evaluation in Formative and Summative Contexts

Educational Test Approaches: The Suitability of Computer-Based Test Types for Assessment and Evaluation in Formative and Summative Contexts

Authors

  • Maaike M. van Groen
     Cito, Amsterdamseweg 13, 6814 CM Arnhem, The Netherlands
  • Theo J. H. M. Eggen
     Department of Research Methodology, Measurement and Data Analysis, University of Twente, P.O. Box 217, 7500 AE Enschede, The Netherlands

Keywords:

Computer-based Testing, Formative Testing and Summative Testing

Abstract

When developing a digital test, one of the first decisions that need to be made is which type of Computer-Based Test (CBT) to develop. Six different CBT types are considered here: linear tests, automatically generated tests, computerized adaptive tests, adaptive learning environments, educational simulations, and educational games. The selection of a CBT type needs to be guided by the intended purposes of the test. The test approach determines which purposes can be achieved by using a particular test. Four different test approaches are discussed here: formative assessment, formative evaluation, summative assessment, and summative evaluation. The suitability of each CBT type to measure performance for the different test approaches is evaluated based on four test characteristics: test purpose, test length, level of interest for measurement (student, class, school, system), and test report. This article aims to provide some guidance in the selection of the most appropriate type of CBT.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

Downloads

Published

2019-09-05

How to Cite

Groen, M. M. van, & Eggen, T. J. H. M. (2019). Educational Test Approaches: The Suitability of Computer-Based Test Types for Assessment and Evaluation in Formative and Summative Contexts. Journal of Applied Testing Technology, 21(1), 12–24. Retrieved from http://www.jattjournal.net/index.php/atp/article/view/146484

Issue

Volume 21, Issue 1, 2020

Section

Articles

References

Becker, K. A., & Bergstrom, B. A. (2013). Test administration models. Practical Assessment, Research & Evaluation, 18(14). Retrieved from http://pareonline.net/getvn.asp?v=18&n=14.

Bennett, R. E. (2011). Formative assessment: A critical review. Assessment in Education: Principles, Policy & Practice, 18, 5-25. doi:10.1080/0969594X.2010.513678 https://doi.org/10.1080/0969594X.2010.513678.

Brusilovsky, P. (1999). Adaptive and intelligent technologies for web-based education. Künstliche Intelligenz [Artificial Intelligence], 13(4), 19-25.

Crisp, G. T. (2012). Integrative assessment: Reframing assessment practice for current and future learning. Assessment & Evaluation in Higher Education. doi:10.1080/02602938.2010.494234 https://doi.org/10.1080/02602938.2010.494234.

Davey, T. (2011). Practical considerations in computer-based testing. Princeton, NJ: Educational Testing Service.

De Klerk, S., Van Dijk, P., & Van den Berg, L. (2015). Voordelen en uitdagingen voor toetsing in computer simulaties [Advantages and challenges of assessment in computersimulations]. Examens, 12(1), 11-17.

Eggen, T. J. H. M. (2018). Multi-segment computerized adaptive testing for educational testing purposes. Frontiers in Education. doi: 10.3389/feduc.2018.00111 https://doi.org/10.3389/feduc.2018.00111.

Haertel, E. (2013). How is testing supposed to improve schooling? Measurement: Interdisciplinary Research and Perspectives, 11, 1-18. doi: 10.1080/15366367.2013.783752 https://doi.org/10.1080/15366367.2013.783752.

Hambleton, R. K., Swaminathan, H., & Rogers, H. J. (1991). Fundamentals of item response theory. Newbury Park, CA: Sage.

Harlen, W. (2007).The quality of learning: Assessment alternatives for primary education (Primary Review Research Survey 3/4). Cambridge, England: University of Cambridge.

Harlen, W., & James, M. (1997). Assessment and learning: Differences and relationships between formative and summative assessment. Assessment in Education: Principles, Policy&Practice, 4, 365-379. doi: 10.1080/ 0969594970040304 https://doi.org/10.1080/0969594970040304.

Kato, P. M., & De Klerk, S. (2017). Serious games for assessment: Welcome to the jungle. Journal of Applied Testing Technology, 1, 1-6.

Levy, R. (2013). Psychometric and evidentiary advances, opportunities, and challenges for simulationbased assessment. Educational Assessment, 18,182-207. doi:10.1080/10627197.2013.814517 https://doi.org/10.1080/10627197.2013.814517.

Leighton, J. P., & Gierl, M. J. (2007). Defining and evaluating models of cognition used in educational measurement to make inferences about examinees’ thinking processes. Educational Measurement: Issues and Practice, 26, 3-16. doi:10.1111/j.1745-3992.2007.00090.x https://doi.org/10.1111/j.1745-3992.2007.00090.x.

Mellenbergh, G. J. (2011). A conceptual introduction to psychometrics. Den Haag, the Netherlands: Eleven International.

Mislevy, R. J. (2011).Evidence-centered design for simulationbased assessment (CRESST Report 800). Los Angeles, CA: University of California, National Center for Research on Evaluation, Standards, and Student Testing (CRESST).

Mislevy, R. J., Oranje, A., Bauer, M. I., von Davier, A. A., Hao, J., Corrigan, S., . . .John, M. (2014). Psychometric considerations in game-based assessment. Redwood City, CA: GlassLab Research, Institute of Play.

Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). On structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1, 3-62. doi:10.1207/S15366359MEA0101_02 https://doi.org/10.1207/S15366359MEA0101_02.

Novak, E., Johnson, T. E., Tenenbaum, G., & Shute, V. J. (2014). Effects of an instructional gaming characteristic on learning effectiveness, efficiency, and engagement: Using a storyline for teaching basic statistical skills. Interactive Learning Environments, 24(3), 523-538. doi:10.1080/10494820.2014.881393 https://doi.org/10.1080/10494820.2014.881393.

Oakes, J. (1989). What educational indicators? The case for assessing the school context. Educational Evaluation and Policy Analysis, 11, 181-199. doi: 10.3102/01623737011002181.https://doi.org/10.3102/01623737011002181.

Parshall, C. G., Spray, J. A., Kalohn, J. C., & Davey, T. (2002). Practical considerations in computerbased testing. New York, NY: Springer. https://doi.org/10.1007/978-1-4613-0083-0.

Rossi, P. H., Lipsey, M. W., & Freeman, H. E. (2004). Evaluation: A systematic approach (7th ed.). Thousand Oaks, CA: Sage.

Sanders, P. (2013). Het doel van toetsen [The purpose of testing]. In P. Sanders (Ed.), Toetsen op school [Testing at schools] (pp. 15-23). Arnhem, the Netherlands: Stichting Cito Instituut voor Toetsontwikkeling.

Scheerens, J., Glas, C. A. W., & Thomas, S. M. (2003). Educational evaluation, assessment, and monitoring. London, England: Taylor & Francis.

Schildkamp, K., & Kuiper, W. (2010). Data-informed curriculum reform: Which data, what purposes, and promoting and hindering factors. Teaching and Teacher Education, 26, 482-496. doi:10.1016/j.tate.2009.06.007https://doi.org/10.1016/j.tate.2009.06.007.

Scriven, M. (1967).The methodology of evaluation. In R. E. Stake (Ed.), Curriculum evaluation (pp. 39-83). Chicago, IL: Rand McNally.

Shepard, L. A. (2005, October). Formative assessment: Caveat emptor. Paper presented at the ETS Invitational Conference, The Future of Assessment: Shaping Teaching and Learning, New York, NY.

Shute, V. J., & Ke, F. (2012). Games, learning, and assessment. In D. Ifenthaler, D. Eseryel, & X. Ge (Eds.), Assessment in game-based learning (pp.43-58). New York, NY: Springer Science+Business. https://doi.org/10.1007/978-1-4614-3546-4_4.

Shute, V. J., & Ventura, M. (2014). Stealth assessment: Measuring and supporting learning in video games. Cambridge, MA: Massachusetts Institute of Technology.https://doi.org/10.7551/mitpress/9589.001.0001.

Stobart, G. (2008). Testing times: The uses and abuses of testing. London, England: Routledge. https://doi.org/10.4324/9780203930502.

Tolboom, J., & Kuiper, W. (2014). Quantifying correspondence between the intended and the implemented intervention in educational design research. Studies in Educational Evaluation, 43, 160-168. doi:10.1016/j.stueduc.2014.09.001 https://doi.org/10.1016/j.stueduc.2014.09.001.

Van der Kleij, F. M., Vermeulen, J. A., Schildkamp, K., & Eggen, T. J. H. M. (2013). Integrating data-based decision making, assessment for learning and diagnostic testing in formative assessment. Assessment in Education: Principles, Policy & Practice. doi: 10.1080/0969594X.2014.999024https://doi.org/10.1080/0969594X.2014.999024.

Van der Linden, W. J. (2005). Linear models for optimal test design. New York, NY: Springer. doi:10.1007/0.387.29054.0 https://doi.org/10.1007/0-387-29054-0.

Van der Linden, W. J., & Glas, C. A. W. (2010). Preface.In W. J. van der Linden, & C. A. W. Glas (Eds.), Elements of adaptive testing (pp. v-vii). New York,NY: Springer. doi:10.1007/978-0-387-85461-8 https://doi.org/10.1007/978-0-387-85461-8.

Van Groen, M. M., Eggen, T. J. H. M., & Veldkamp, B. P. (2014). Item selection methods based on multiple objective approaches for classification of respondents into multiple levels. Applied Psychological Measurement, doi: 10.1177/0146621613509723 https://doi.org/10.1177/0146621613509723.

Wauters, K. (2012). Adaptive item sequencing in item-based learning environments (Unpublished doctoral dissertation).KU Leuven, Belgium.

Wiliam, D. (2013). How is testing supposed to improve schooling? Some reflections. Measurement: Interdisciplinary Research and Perspectives, 11, 55-59. doi:10.1080/15366367.2013.784165 https://doi.org/10.1080/15366367.2013.784165.

Yan, D., Lewis, C., & von Davier, A. (2014). Overview of computerized multistage tests.In D. Yan, A. A. von Davier, & C. Lewis (Eds.), Computerized multistage testing: Theory and applications. Boca Raton, FL: CRC Press.

Zenisky, A., Hambleton, R. K., & Luecht, R. M. (2010).Multistage testing: Issues, designs, and research. In W. J. van der Linden & C. A. W. Glas (Eds.), Elements of adaptive testing (pp. 355-372). New York, NY: Springer. doi:10.1007/978-0-387-85461-8_18 https://doi.org/10.1007/978-0-387-85461-8_18.