Artificial Intelligence: Transforming the Future of Feedback in Education

Artificial Intelligence: Transforming the Future of Feedback in Education

Authors

  • Department of Educational Psychology, University of Alberta
  • Department of Educational Psychology, University of Alberta
  • Centre for Research in Applied Measurement and Evaluation, University of Alberta
  • Department of Human Centred Computing, Monash University
  • Department of Human Centred Computing, Monash University

Keywords:

Artificial Intelligence, Educational Data Mining, Educational Feedback, Learning Analytics, Natural Language Processing

Abstract

Feedback is a crucial component of student learning. As advancements in technology have enabled the adoption of digital learning environments with assessment capabilities, the frequency, delivery format, and timeliness of feedback derived from educational assessments have also changed progressively. Advanced technologies powered by Artificial Intelligence (AI) enable teachers to generate different types of feedback supporting student learning. Despite the rapid uptake of digital technologies in education, previous studies on educational feedback primarily focused on the theoretical underpinnings of feedback practices, which are limited in terms of their coverage of AI-based technologies. This paper aims to inform both researchers and practitioners about the present and future of AI applications in feedback practices, identify and organize potential areas for the use of AI for feedback purposes, and establish venues for AI research and practice in educational feedback. Furthermore, the role of the three branches of AI (i.e., natural language processing, educational data mining, and learning analytics) in feedback practices and potential areas for their future development are discussed.

Downloads

Download data is not yet available.

Metrics

Metrics Loading ...

Downloads

Published

2022-10-18

How to Cite

Wongvorachan, T., Lai, K. W., Bulut, O., Tsai, Y.-S., & Chen, G. (2022). Artificial Intelligence: Transforming the Future of Feedback in Education. Journal of Applied Testing Technology, 23, 95–116. Retrieved from https://www.jattjournal.net/index.php/atp/article/view/170387

Issue

Section

Articles

References

Ai, H. (2017). Providing graduated corrective feedback in an intelligent computer-assisted language learning environment. ReCALL, 29(3), 313–334. https://doi.org/10.1017/ S095834401700012X

Anjewierden, A., Kolloffel, B., & Hulshof, C. (2007). Towards educational data mining: Using data mining methods for automated chat analysis to understand and support inquiry learning processes. International Workshop on Applying Data Mining in E-Learning (ADML 2007), 27–36.

Araka, E., Maina, E., Gitonga, R., & Oboko, R. (2019). A conceptual model for measuring and supporting selfregulated learning using educational data mining on learning management systems. 2019 IST-Africa Week Conference (IST-Africa), 1 –11. https://doi.org/10.23919/ ISTAFRICA.2019.8764852

Barana, A., Conte, A., Fissore, C., Marchisio, M., & Rabellino, S. (2019). Learning analytics to improve formative assessment strategies. Journal of E-Learning and Knowledge Society, 15(3), 75–88. https://doi.org/10.20368/1971- 8829/1135057

Bengfort, B., Bilbro, R., & Ojeda, T. (2018). Applied text analysis with Python: Enabling language-aware data products with machine learning (First edition). O’Reilly Media, Inc.

Bethany, F., Foy, P., & Yin, L. (2021). TIMSS 2019 user guide for the international database (2nd ed.). TIMSS & PIRLS International Study Center.

Boud, D. (2020). Challenges for reforming assessment: The next decade. International Virtual Meeting: Teaching, Learning & Assessment in Higher Education, Universidad del Desarrollo. https://educacionsuperior2020. udd.cl/

Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: The challenge of design. Assessment & Evaluation in Higher Education, 38(6), 698–712. https:// doi.org/10.1080/02602938.2012.691462

Brookhart, S. M. (2008). How to give effective feedback to your students (1st ed.). Association for Supervision and Curriculum Development.

Bulut, O., Cutumisu, M., Aquilina, A. M., & Singh, D. (2019). Effects of digital score reporting and feedback on students’ learning in higher education. Frontiers in Education, 4(65). https://doi.org/10.3389/feduc.2019.00065

Bulut, O., Cutumisu, M., Singh, D., & Aquilina, A. M. (2020). Guidelines for generating effective feedback from e-assessments. Hacettepe University Journal of Education, 1–13. https://doi.org/10.16986/HUJE.2020063705

Carless, D. (2019). Feedback loops and the longer-term: Towards feedback spirals. Assessment & Evaluation in Higher Education, 44(5), 705–714. https://doi.org/10.10 80/02602938.2018.1531108

Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315–1325. https://doi.org/10.1080/02602938.2018.146 3354

Carless, D., & Winstone, N. (2020). Teacher feedback literacy and its interplay with student feedback literacy. Teaching in Higher Education, 1–14. https://doi.org/10.1080/1356 2517.2020.1782372

Cavalcanti, A. P., Diego, A., Mello, R. F., Mangaroska, K., Nascimento, A., Freitas, F., & Gašević, D. (2020). How good is my feedback?: A content analysis of written feedback. Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 428–437. https:// doi.org/10.1145/3375462.3375477

Chan, K. S., & Zary, N. (2019). Applications and challenges of implementing artificial intelligence in medical education: Integrative review. JMIR Medical Education, 5(1), e13930. https://doi.org/10.2196/13930

Chen, G., Rolim, V., Mello, R. F., & Gašević, D. (2020). Let’s shine together!: A comparative study between learning analytics and educational data mining. Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 544–553. https://doi. org/10.1145/3375462.3375500

Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. IEEE Access, 8, 75264–75278. https://doi.org/10.1109/ACCESS.2020.2988510

Crossley, S., McNamara, D. S., Baker, R., Wang, Y., Paquette, L., Barnes, T., & Bergner, Y. (2015, June 26). Language to completion: Success in an educational data mining massive open online class. International Conference on Educational Data Mining (EDM). International Conference on Educational Data Mining (EDM), Spain.

de Laat, M., Joksimovic, S., & Ifenthaler, D. (2020). Artificial intelligence, real-time feedback and workplace learning analytics to support in situ complex problem-solving: A commentary. The International Journal of Information and Learning Technology, 37(5), 267–277. https://doi. org/10.1108/IJILT-03-2020-0026

Desai, S., & Chin, J. (2020). An explorative analysis of the feasibility of implementing metacognitive strategies in self-regulated learning with the conversational agents. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 64(1), 495–499. https://doi. org/10.1177/1071181320641112

Dzikovska, M., Steinhauser, N., Farrow, E., Moore, J., & Campbell, G. (2014). BEETLE II: Deep natural language understanding and automatic feedback generation for intelligent tutoring in basic electricity and electronics. International Journal of Artificial Intelligence in Education, 24(3), 284–332. https://doi.org/10.1007/ s40593-014-0017-9

Economides, A. A. (2005). Personalized feedback in CAT. WSEAS Transactions on Advances in Engineering Education, 2(3), 174–181.

Elatia, S., Ippercieli, D., & Zaiane, O. (Eds.). (2016). Data mining and learning analytics: Applications in educational research. John Wiley & Sons.

Elliott, S. W. (2017). Computers and the future of skill demand. OECD. https://doi.org/10.1787/9789264284395-en

Flodén, J. (2017). The impact of student feedback on teaching in higher education. Assessment & Evaluation in Higher Education, 42(7), 1054–1068. https://doi.org/10. 1080/02602938.2016.1224997

Forsythe, A., & Johnson, S. (2017). Thanks, but no-thanks for the feedback. Assessment & Evaluation in Higher Education, 42(6), 850–859. https://doi.org/10.1080/026 02938.2016.1202190

Gamlem, S. M., & Smith, K. (2013). Student perceptions of classroom feedback. Assessment in Education: Principles, Policy & Practice, 20(2), 150–169. https://doi.org/10.108 0/0969594X.2012.749212

Gardner, J., O’Leary, M., & Yuan, L. (2021). Artificial intelligence in educational assessment: “Breakthrough? or buncombe and ballyhoo?” Journal of Computer Assisted Learning, 37(5), 1207–1216. https://doi.org/10.1111/ jcal.12577

Goddard, W. (2021, September 7). Natural language processing in education. IT Chronicles. https://itchronicles.com/ natural-language-processing-nlp/natural-language-processing- in-education/

Guo, B., Zhang, R., Xu, G., Shi, C., & Yang, L. (2015). Predicting students performance in educational data mining. 2015 International Symposium on Educational Technology (ISET), 125–128. https://doi.org/10.1109/ ISET.2015.33

Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https:// doi.org/10.3102/003465430298487

Holmes, W., Bialik, M., & Fadel, C. (2019). Artificial intelligence in education: Promises and implications for teaching and learning. The Center for Curriculum Redesign.

Hounsell, D. (2007). Chapter 8: Towards more sustainable feedback to students. In D. Boud & N. Falchikov (Eds.), Rethinking assessment in higher education: Learning for the longer term (pp. 101–113). Routledge.

Hussain, S., Atallah, R., Kamsin, A., & Hazarika, J. (2018). Classification, clustering and association rule mining in educational datasets using data mining tools: A case study. In R. Silhavy (Ed.), Cybernetics and algorithms in intelligent systems, (Vol. 765, pp. 196–211). Springer International Publishing. https://doi.org/10.1007/978-3- 319-91192-2_21

Jimenez, L., & Boser, U. (2021, September). Future of testing in education: Artificial intelligence. Center for American Progress. https://www.americanprogress.org/article/ future-testing-education-artificial-intelligence

Jurs, P., & Špehte, E. (2021). The role of feedback in the distance learning process. Journal of Teacher Education for Sustainability, 23(2), 91–105. https://doi.org/10.2478/ jtes-2021-0019

Karaoglan Yilmaz, F. G., & Yilmaz, R. (2021). Learning analytics as a metacognitive tool to influence learner transactional distance and motivation in online learning environments. Innovations in Education and Teaching International, 58(5), 575–585. https://doi.org/10.1080/1 4703297.2020.1794928

Ketchum, C., LaFave, D. S., Yeats, C., Phompheng, E., & Hardy, J. H. (2020). Video-based feedback on student work: An investigation into the instructor experience, workload, and student evaluations. Online Learning, 24(3). https://doi.org/10.24059/olj.v24i3.2194

Knight, S., Shibani, A., Abel, S., Gibson, A., Ryan, P., Sutton, N., Wight, R., Lucas, C., Sándor, Á., Kitto, K., Liu, M., Mogarkar, R. V., & Buckingham Shum, S. (2020). AcaWriter: A learning analytics tool for formative feedback on academic writing. Journal of Writing Research, 12(1), 141–186. https://doi.org/10.17239/ jowr-2020.12.01.06

Kochmar, E., Vu, D. D., Belfer, R., Gupta, V., Serban, I. V., & Pineau, J. (2020). Automated personalized feedback improves learning gains in an intelligent tutoring system. In I. I. Bittencourt, M. Cukurova, K. Muldner, R. Luckin, & E. Millán (Eds.), Artificial Intelligence in Education (Vol. 12164, pp. 140–146). Springer International Publishing. https://doi.org/10.1007/978-3-030-52240- 7_26

Kulik, J. A., & Fletcher, J. D. (2016). Effectiveness of intelligent tutoring systems: A meta-analytic review. Review of Educational Research, 86(1), 42–78. https://doi. org/10.3102/0034654315581420

Lan, A. S., Vats, D., Waters, A. E., & Baraniuk, R. G. (2015). Mathematical language processing: Automatic grading and feedback for open response mathematical questions. Proceedings of the Second (2015) ACM Conference on Learning @ Scale, 167–176. https://doi. org/10.1145/2724660.2724664

Larusson, J. A., & White, B. (Eds.). (2014). Chapter 1: Introduction. In Learning analytics from research to practice (pp. 1–12). Springer.

Lemay, D. J., Baek, C., & Doleck, T. (2021). Comparison of learning analytics and educational data mining: A topic modeling approach. Computers and Education: Artificial Intelligence, 2, 100016. https://doi.org/10.1016/j. caeai.2021.100016

Li, C., & Xing, W. (2021). Natural language generation using deep learning to support MOOC learners. International Journal of Artificial Intelligence in Education, 31(2), 186– 214. https://doi.org/10.1007/s40593-020-00235-x

Loyola-Gonzalez, O. (2019). Black-box vs. white-box: Understanding their advantages and weaknesses from a practical point of view. IEEE Access, 7, 154096–154113. https://doi.org/10.1109/ACCESS.2019.2949286

Magis, D., Yan, D., & von Davier, A. A. (2017). Chapter 1: Overview of adaptive testing. In R. Gentleman, K. Hornik, & G. Parmigiani (Eds.), Computerized adaptive and multistage testing with R (pp. 1–5). Springer International Publishing.

Marriott, P., & Teoh, L. K. (2012). Using screencasts to enhance assessment feedback: Students’ perceptions and preferences. Accounting Education, 21(6), 583–598. https://doi.org/10.1080/09639284.2012.725637

Matcha, W., Uzir, N. A., Gasevic, D., & Pardo, A. (2020). A systematic review of empirical studies on learning analytics dashboards: A self-regulated learning perspective. IEEE Transactions on Learning Technologies, 13(2), 226– 245. https://doi.org/10.1109/TLT.2019.2916802

Mbunge, E., Fashoto, S. G., & Olaomi, J. (2021). COVID-19 and online learning: Factors influencing students’ academic performance in first-year computer programming courses in higher education. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3757988

Merceron, A., & Yacef, K. (2005). Educational data mining: A case study. In Artificial intelligence in education: Supporting learning through intelligent and socially informed technology (Vol. 125, pp. 467–474). IOS Press.

Molloy, E., Boud, D., & Henderson, M. (2020). Developing a learning-centred framework for feedback literacy. Assessment & Evaluation in Higher Education, 45(4), 527–540. https://doi.org/10.1080/02602938.2019.16679 55

Moreno, A., & Redondo, T. (2016). Text analytics: The convergence of big data and artificial intelligence. International Journal of Interactive Multimedia and Artificial Intelligence, 3(6), 57. https://doi.org/10.9781/ ijimai.2016.369

National Center for Educational Statistics [NCES]. (2022). Program for the international assessment of adult competencies (PIAAC). Organisation for Economic Co-Operation and Development. https://nces.ed.gov/ surveys/piaac/about.asp

Organisation for Economic Co-Operation and Development [OECD]. (2019). PISA 2018 results (volume I): What students know and can do. OECD Publishing.

Ötleş, E., Kendrick, D. E., Solano, Q. P., Schuller, M., Ahle, S. L., Eskender, M. H., Carnes, E., & George, B. C. (2021). Using natural language processing to automatically assess feedback quality: Findings from 3 surgical residencies. Academic Medicine, 96(10), 1457–1460. https://doi.org/10.1097/ACM.0000000000004153

Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2019). Using learning analytics to scale the provision of personalised feedback: Learning analytics to scale personalised feedback. British Journal of Educational Technology, 50(1), 128–138. https://doi.org/10.1111/ bjet.12592

Pardo, A., Poquet, O., Martinez-Maldonado, R., & Dawson, S. (2017). Provision of data-driven student feedback in la & edm. In C. Lang, G. Siemens, A. Wise, & D. Gasevic (Eds.), Handbook of learning analytics (First, pp. 163– 174). Society for Learning Analytics Research (SoLAR). https://doi.org/10.18608/hla17.014

Pask, G. (1982). SAKI: Twenty-five years of adaptive training into the microprocessor era. International Journal of Man-Machine Studies, 17(1), 69–74. https://doi. org/10.1016/S0020-7373(82)80009-6

Pechenizkiy, M., Calders, T., Vasilyeva, E., & De Bra, P. (2008). Mining the student assessment data: Lessons drawn from a small scale case study. Educational Data Mining 2008.

Pengel, N., Martin, A., Meissner, R., Arndt, T., Neumann, A. T., de Lange, P., & Wollersheim, H.-W. (2021). TecCoBot: Technology-aided support for self-regulated learning. ArXiv Preprint ArXiv:2111.11881. http://arxiv. org/abs/2111.11881

Perikos, I., Grivokostopoulou, F., & Hatzilygeroudis, I. (2017). Assistance and feedback mechanism in an intelligent tutoring system for teaching conversion of natural language into logic. International Journal of Artificial Intelligence in Education, 27(3), 475–514. https://doi. org/10.1007/s40593-017-0139-y

Piotrkowicz, A., Dimitrova, V., Treasure-Jones, T., Smithies, A., Harkin, P., Kirby, J., & Roberts, T. (2017). Quantified self analytics tools for self-regulated learning with myPAL. Proceedings of the 7th Workshop on Awareness and Reflection in Technology Enhanced Learning Co-Located with the 12th European Conference on Technology Enhanced Learning (EC-TEL 2017), 1997, 16.

Qazdar, A., Er-Raha, B., Cherkaoui, C., & Mammass, D. (2019). A machine learning algorithm framework for predicting students performance: A case study of baccalaureate students in Morocco. Education and Information Technologies, 24(6), 3577–3589. https://doi.org/10.1007/ s10639-019-09946-8

Ramachandran, L., Gehringer, E. F., & Yadav, R. K. (2017). Automated assessment of the quality of peer reviews using natural language processing techniques. International Journal of Artificial Intelligence in Education, 27(3), 534– 581. https://doi.org/10.1007/s40593-016-0132-x

Ramaswami, M., & Bhaskaran, R. (2010). A CHAID based performance prediction model in educational data mining. IJCSI International Journal of Computer Science. http://arxiv.org/abs/1002.1144

Ray, S., & Saeed, M. (2018). Applications of educational data mining and learning analytics tools in handling big data in higher education. In M. M. Alani, H. Tawfik, M. Saeed, & O. Anya (Eds.), Applications of Big Data Analytics (pp. 135–160). Springer International Publishing. https://doi. org/10.1007/978-3-319-76472-6_7

Roberts, T. (2019). 5 Natural language processing examples: How NLP is used. Bloomreach. https://www.bloomreach. com/en/blog/2019/09/natural-language-processing. html

Rodway-Dyer, S., Knight, J., & Dunne, E. (2011). A case study on audio feedback with geography undergraduates. Journal of Geography in Higher Education, 35(2), 217–231. https://doi.org/10.1080/03098265.2010.5241 97

Romero, C., & Ventura, S. (2010). Educational data mining: A review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(6), 601–618. https://doi.org/10.1109/ TSMCC.2010.2053532

Ryan, T., Gašević, D., & Henderson, M. (2019). Identifying the impact of feedback over time and at scale: Opportunities for learning analytics. In M. Henderson, R. Ajjawi, D. Boud, & E. Molloy (Eds.), The Impact of Feedback in Higher Education (pp. 207–223). Springer International Publishing. https://doi.org/10.1007/978-3- 030-25112-3_12

Sedrakyan, G., Malmberg, J., Verbert, K., Järvelä, S., & Kirschner, P. A. (2020). Linking learning behavior analytics and learning science concepts: Designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior, 107, 105512. https://doi.org/10.1016/j.chb.2018.05.004

Siemens, G. (2012). Learning analytics and educational data mining: Towards communication and collaboration. 3.

Sipes, S. (2017). Using learning analytics in SoTL. Center for Innovative Teaching and Learning @IUB. https://blogs. iu.edu/citl/2017/12/13/using-learning-analytics-in-sotl

Solano, Q. P., Hayward, L., Chopra, Z., Quanstrom, K., Kendrick, D., Abbott, K. L., Kunzmann, M., Ahle, S., Schuller, M., Ötleş, E., & George, B. C. (2021). Natural language processing and assessment of resident feedback quality. Journal of Surgical Education, 78(6), e72–e77. https://doi.org/10.1016/j.jsurg.2021.05.012

Tempelaar, D. (2020). Supporting the less-adaptive student: The role of learning analytics, formative assessment and blended learning. Assessment & Evaluation in Higher Education, 45(4), 579–593. https://doi.org/10.1080/026 02938.2019.1677855

Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: Learning analytics in a data-rich context. Computers in Human Behavior, 47, 157–167. https://doi.org/10.1016/j. chb.2014.05.038

Tsai, Y. S., Mello, R. F., Jovanović, J., & Gašević, D. (2021). Student appreciation of data-driven feedback: A pilot study on Ontask. LAK21: 11th International Learning Analytics and Knowledge Conference, 511–517. https:// doi.org/10.1145/3448139.3448212

Ubani, S., & Nielsen, R. (2022). Review of collaborative intelligent tutoring systems (CITS) 2009-2021. 2022 11th International Conference on Educational and Information Technology (ICEIT), 67–75. https://doi.org/10.1109/ ICEIT54416.2022.9690733

UNESCO. (2019). Section 2: Preparing learners to thrive in the future with AI. In Artificial intelligence in education: Challenges and opportunities for sustainable development (pp. 17–24). the United Nations Educational, Scientific and Cultural Organization,. https://unesdoc. unesco.org/ark:/48223/pf0000366994/PDF/366994eng. pdf.multi

Viberg, O., Khalil, M., & Baars, M. (2020). Self-regulated learning and learning analytics in online learning environments: A review of empirical research. Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 524–533. https://doi. org/10.1145/3375462.3375483

Wang, D., & Han, H. (2021). Applying learning analytics dashboards based on process‐oriented feedback to improve students’ learning effectiveness. Journal of Computer Assisted Learning, 37(2), 487–499. https://doi. org/10.1111/jcal.12502

Wasson, B., Hansen, C., & Netteland, G. (2016). Data literacy and use for learning when using learning analytics for learners. CEUR Workshop Proceedings. http://hdl.handle. net/11250/2429460

Watling, C. J., & Ginsburg, S. (2019). Assessment, feedback and the alchemy of learning. Medical Education, 53, 76–85. https://doi.org/10.1111/medu.13645

Winne, P. H., & Baker, R. S. J. d. (2013). The potentials of educational data mining for researching metacognition, motivation and self-regulated learning. Journal of Educational Data Mining, 5(1), 8.

Witten, I. H., Frank, E., Hall, M. A., & Pal, C. J. (2017). Data mining: Practical machine learning tools and techniques (4th ed.). Elsevier.

Xiong, W., Litman, D., & Schunn, C. (2012). Natural language processing techniques for researching and improving peer feedback. Journal of Writing Research, 4(2), 155– 176. https://doi.org/10.17239/jowr-2012.04.02.3

Yildirim-Erbasli, S. N., & Bulut, O. (2021). Conversation-based assessments: Real-time assessment and feedback. ELearn, 2021(12). https://doi. org/10.1145/3508017.3495533

Zenisky, A. L., & Hambleton, R. K. (2012). Developing test score reports that work: The process and best practices for effective communication. Educational Measurement: Issues and Practice, 31(2), 21–26. https://doi.org/10.1111/ j.1745-3992.2012.00231.x

Zhang, H., Magooda, A., Litman, D., Correnti, R., Wang, E., Matsmura, L. C., Howe, E., & Quintana, R. (2019). eRevise: Using natural language processing to provide formative feedback on text evidence usage in student writing. Proceedings of the AAAI Conference on Artificial Intelligence, 9619–9625. https://doi.org/10.1609/aaai. v33i01.33019619

Loading...