Designing and Validating a Stealth Assessment for Calculus Competencies

Designing and Validating a Stealth Assessment for Calculus Competencies

Authors

  • Ginny Smith
     Doctoral Candidate at Florida State University
  • Valerie Shute
     Mack and Effie Campbell Tyner Endowed Professor of Education, Florida State University, 1114 W. Call Street, Tallahassee, Florida 32306-4453
  • Amber Muenzenberger
     Chief Learning Officer at Triseum

Keywords:

Assessment, Calculus, Game-Based Learning, Innovative Instruction, Limits, Stealth Assessment

Abstract

The following research focuses on Game-Based Learning (GBL) for assessment through the lens of designing and validating a stealth assessment for the calculus game Variant: LimitsTM. The process of using Evidence-Centered Design (ECD) to create a valid assessment is highlighted through the development of a competency model and scoring rubrics. A sample of college students enrolled in a Calculus 1 course played the game outside of class during a 2-week period. Comparing the results of students’ in-game and external assessments, researchers examined the validity of the stealth assessment measure. The stealth assessment was significantly correlated with the external measures and found to be a valid assessment of students’ overall calculus knowledge as defined by the competency model.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

Downloads

Published

2019-04-24

How to Cite

Smith, G., Shute, V., & Muenzenberger, A. (2019). Designing and Validating a Stealth Assessment for Calculus Competencies. Journal of Applied Testing Technology, 20(S1), 52–59. Retrieved from http://www.jattjournal.net/index.php/atp/article/view/142702

Issue

Volume 20, Special Issue 1[Game Based Assessment], 2019

Section

Articles

References

Bressoud, D., Mesa, V., & Rasmussen, C. (2015). Insights and Recommendations from the MAA National Study of College Calculus. Washington, DC: The Mathematical Association of America.

Blair, R. M., Kirkman, E. E., & Maxwell, J. M. (2013). Statistical abstract of undergraduate programs in the mathematical sciences in the United States: Fall 2010 CBMS survey. Providence RI: American Mathematical Society. Retrieved from: www. ams.org/profession/data/cbms-survey/cbms2010.

Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital games, design, and learning: A systematic review and meta-analysis. Review of Educational Research, 86(1), 79-122. doi:10.3102/0034654315582065

College Board. (2016). 2016 AP Exam score distributions.Retrieved from: https://www.totalregistration.net/ AP-Exam-Registration-Service/2016-AP-Exam-ScoreDistributions.php.

Entertainment Software Association. (2015). Essential facts about the computer and video game industry: 2015 sales,demographic and usage data. Retrieved from: http://www.theesa.com/wpcontent/ uploads/2015/04/ESA-Essential-Facts-2015.pdf

Federation of American Scientists. (2006). Summit on educational games: Harnessing the power of video games for learning. Washington D.C. Retrieved from: https:// fas.org/programs/ltp/policy_and_publications/summit/ Summit%20on%20Educational%20Games.pdf

Kim, Y. J., Almond, R. G., & Shute, V. J. (2016). Applying Evidence-Centered Design for the development of gamebased assessments in Physics Playground. International Journal of Testing, 16(2), 142-163.doi:10.1080/15305058.2 015.1108322

Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2003). Focus article: On the structure of educational assessments. Measurement: Interdisciplinary Research and Perspectives, 1(1), 3-62.

National Academies of Sciences, Engineering, and Medicine.(2018). Indicators for Monitoring Undergraduate STEM Education. Washington, DC: The National Academies Press. https://doi.org/10.17226/24943

President’s Council of Advisors on Science and Technology. (2012, February). Report to the President: Engage to excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics.

Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153-189.

doi:10.3102/0034654307313795

Shute, V. J. (2011). Stealth assessment in computer-based games to support learning. In S. Tobias & J. D. Fletcher (Eds.), Computer games and instruction (pp. 503-524). Charlotte, NC: Information Age Publishers.

Shute, V. J., Rieber, L., & Van Eck, R. (2011). Games . . . and . . . Learning. In R. Reiser & J. Dempsey (Eds.), Trends and issues in instructional design and technology (3rd ed.) (pp. 321-332). Upper Saddle River, NJ: Pearson Education, Inc.

Shute, V. J., Wang, L., Greiff, S., Zhao, W., & Moore, G. (2016). Measuring problem solving skills via stealth assessment in an engaging video game. Computers in Human Behavior, 63, 106-117. doi:10.1016/j.chb.2016.05.047

Variant: Limits [computer software]. (2017). Bryan, TX: Triseum.

Vogel, J. J., Vogel, D. S., Cannon-Bowers, J., Bowers, C. A., Muse, K., & Wright, M. (2006). Computer gaming and interactive simulations for learning: A meta-analysis. Journal of Educational Computing Research, 34(3), 229-243.

Wang, L., Shute, V. J., & Moore, G. (2015). Lessons learned and best practices of stealth assessment. International Journal of Gaming and Computer Mediated Simulations, 74(4), 66-87. doi:10.4018/IJGCMS.2015100104

Wenglinsky, H. (1998). Does it compute? The relationship between educational technology and student achievement in mathematics. Educational Testing Service. Policy Information Center. Retrieved from: www.ets.org/research/pic

Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249-265. doi:10.1037/a0031311