Artificial Intelligence: Transforming the Future of Feedback in Education
Authors
-
Department of Educational Psychology, University of Alberta
Tarid Wongvorachan -
Department of Educational Psychology, University of Alberta Ka Wing Lai
-
Centre for Research in Applied Measurement and Evaluation, University of Alberta Okan Bulut
-
Department of Human Centred Computing, Monash University Yi-Shan Tsai
-
Department of Human Centred Computing, Monash University Guanliang Chen
Keywords:
Artificial Intelligence, Educational Data Mining, Educational Feedback, Learning Analytics, Natural Language ProcessingAbstract
Feedback is a crucial component of student learning. As advancements in technology have enabled the adoption of digital learning environments with assessment capabilities, the frequency, delivery format, and timeliness of feedback derived from educational assessments have also changed progressively. Advanced technologies powered by Artificial Intelligence (AI) enable teachers to generate different types of feedback supporting student learning. Despite the rapid uptake of digital technologies in education, previous studies on educational feedback primarily focused on the theoretical underpinnings of feedback practices, which are limited in terms of their coverage of AI-based technologies. This paper aims to inform both researchers and practitioners about the present and future of AI applications in feedback practices, identify and organize potential areas for the use of AI for feedback purposes, and establish venues for AI research and practice in educational feedback. Furthermore, the role of the three branches of AI (i.e., natural language processing, educational data mining, and learning analytics) in feedback practices and potential areas for their future development are discussed.
Downloads
Metrics
Downloads
Published
How to Cite
Issue
Section
References
Ai, H. (2017). Providing graduated corrective feedback in an intelligent computer-assisted language learning environment. ReCALL, 29(3), 313–334. https://doi.org/10.1017/ S095834401700012X
Anjewierden, A., Kolloffel, B., & Hulshof, C. (2007). Towards educational data mining: Using data mining methods for automated chat analysis to understand and support inquiry learning processes. International Workshop on Applying Data Mining in E-Learning (ADML 2007), 27–36.
Araka, E., Maina, E., Gitonga, R., & Oboko, R. (2019). A conceptual model for measuring and supporting selfregulated learning using educational data mining on learning management systems. 2019 IST-Africa Week Conference (IST-Africa), 1 –11. https://doi.org/10.23919/ ISTAFRICA.2019.8764852
Barana, A., Conte, A., Fissore, C., Marchisio, M., & Rabellino, S. (2019). Learning analytics to improve formative assessment strategies. Journal of E-Learning and Knowledge Society, 15(3), 75–88. https://doi.org/10.20368/1971- 8829/1135057
Bengfort, B., Bilbro, R., & Ojeda, T. (2018). Applied text analysis with Python: Enabling language-aware data products with machine learning (First edition). O’Reilly Media, Inc.
Bethany, F., Foy, P., & Yin, L. (2021). TIMSS 2019 user guide for the international database (2nd ed.). TIMSS & PIRLS International Study Center.
Boud, D. (2020). Challenges for reforming assessment: The next decade. International Virtual Meeting: Teaching, Learning & Assessment in Higher Education, Universidad del Desarrollo. https://educacionsuperior2020. udd.cl/
Boud, D., & Molloy, E. (2013). Rethinking models of feedback for learning: The challenge of design. Assessment & Evaluation in Higher Education, 38(6), 698–712. https:// doi.org/10.1080/02602938.2012.691462
Brookhart, S. M. (2008). How to give effective feedback to your students (1st ed.). Association for Supervision and Curriculum Development.
Bulut, O., Cutumisu, M., Aquilina, A. M., & Singh, D. (2019). Effects of digital score reporting and feedback on students’ learning in higher education. Frontiers in Education, 4(65). https://doi.org/10.3389/feduc.2019.00065
Bulut, O., Cutumisu, M., Singh, D., & Aquilina, A. M. (2020). Guidelines for generating effective feedback from e-assessments. Hacettepe University Journal of Education, 1–13. https://doi.org/10.16986/HUJE.2020063705
Carless, D. (2019). Feedback loops and the longer-term: Towards feedback spirals. Assessment & Evaluation in Higher Education, 44(5), 705–714. https://doi.org/10.10 80/02602938.2018.1531108
Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315–1325. https://doi.org/10.1080/02602938.2018.146 3354
Carless, D., & Winstone, N. (2020). Teacher feedback literacy and its interplay with student feedback literacy. Teaching in Higher Education, 1–14. https://doi.org/10.1080/1356 2517.2020.1782372
Cavalcanti, A. P., Diego, A., Mello, R. F., Mangaroska, K., Nascimento, A., Freitas, F., & Gašević, D. (2020). How good is my feedback?: A content analysis of written feedback. Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 428–437. https:// doi.org/10.1145/3375462.3375477
Chan, K. S., & Zary, N. (2019). Applications and challenges of implementing artificial intelligence in medical education: Integrative review. JMIR Medical Education, 5(1), e13930. https://doi.org/10.2196/13930
Chen, G., Rolim, V., Mello, R. F., & Gašević, D. (2020). Let’s shine together!: A comparative study between learning analytics and educational data mining. Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 544–553. https://doi. org/10.1145/3375462.3375500
Chen, L., Chen, P., & Lin, Z. (2020). Artificial intelligence in education: A review. IEEE Access, 8, 75264–75278. https://doi.org/10.1109/ACCESS.2020.2988510
Crossley, S., McNamara, D. S., Baker, R., Wang, Y., Paquette, L., Barnes, T., & Bergner, Y. (2015, June 26). Language to completion: Success in an educational data mining massive open online class. International Conference on Educational Data Mining (EDM). International Conference on Educational Data Mining (EDM), Spain.
de Laat, M., Joksimovic, S., & Ifenthaler, D. (2020). Artificial intelligence, real-time feedback and workplace learning analytics to support in situ complex problem-solving: A commentary. The International Journal of Information and Learning Technology, 37(5), 267–277. https://doi. org/10.1108/IJILT-03-2020-0026
Desai, S., & Chin, J. (2020). An explorative analysis of the feasibility of implementing metacognitive strategies in self-regulated learning with the conversational agents. Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 64(1), 495–499. https://doi. org/10.1177/1071181320641112
Dzikovska, M., Steinhauser, N., Farrow, E., Moore, J., & Campbell, G. (2014). BEETLE II: Deep natural language understanding and automatic feedback generation for intelligent tutoring in basic electricity and electronics. International Journal of Artificial Intelligence in Education, 24(3), 284–332. https://doi.org/10.1007/ s40593-014-0017-9
Economides, A. A. (2005). Personalized feedback in CAT. WSEAS Transactions on Advances in Engineering Education, 2(3), 174–181.
Elatia, S., Ippercieli, D., & Zaiane, O. (Eds.). (2016). Data mining and learning analytics: Applications in educational research. John Wiley & Sons.
Elliott, S. W. (2017). Computers and the future of skill demand. OECD. https://doi.org/10.1787/9789264284395-en
Flodén, J. (2017). The impact of student feedback on teaching in higher education. Assessment & Evaluation in Higher Education, 42(7), 1054–1068. https://doi.org/10. 1080/02602938.2016.1224997
Forsythe, A., & Johnson, S. (2017). Thanks, but no-thanks for the feedback. Assessment & Evaluation in Higher Education, 42(6), 850–859. https://doi.org/10.1080/026 02938.2016.1202190
Gamlem, S. M., & Smith, K. (2013). Student perceptions of classroom feedback. Assessment in Education: Principles, Policy & Practice, 20(2), 150–169. https://doi.org/10.108 0/0969594X.2012.749212
Gardner, J., O’Leary, M., & Yuan, L. (2021). Artificial intelligence in educational assessment: “Breakthrough? or buncombe and ballyhoo?” Journal of Computer Assisted Learning, 37(5), 1207–1216. https://doi.org/10.1111/ jcal.12577
Goddard, W. (2021, September 7). Natural language processing in education. IT Chronicles. https://itchronicles.com/ natural-language-processing-nlp/natural-language-processing- in-education/
Guo, B., Zhang, R., Xu, G., Shi, C., & Yang, L. (2015). Predicting students performance in educational data mining. 2015 International Symposium on Educational Technology (ISET), 125–128. https://doi.org/10.1109/ ISET.2015.33
Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112. https:// doi.org/10.3102/003465430298487
Holmes, W., Bialik, M., & Fadel, C. (2019). Artificial intelligence in education: Promises and implications for teaching and learning. The Center for Curriculum Redesign.
Hounsell, D. (2007). Chapter 8: Towards more sustainable feedback to students. In D. Boud & N. Falchikov (Eds.), Rethinking assessment in higher education: Learning for the longer term (pp. 101–113). Routledge.
Hussain, S., Atallah, R., Kamsin, A., & Hazarika, J. (2018). Classification, clustering and association rule mining in educational datasets using data mining tools: A case study. In R. Silhavy (Ed.), Cybernetics and algorithms in intelligent systems, (Vol. 765, pp. 196–211). Springer International Publishing. https://doi.org/10.1007/978-3- 319-91192-2_21
Jimenez, L., & Boser, U. (2021, September). Future of testing in education: Artificial intelligence. Center for American Progress. https://www.americanprogress.org/article/ future-testing-education-artificial-intelligence
Jurs, P., & Špehte, E. (2021). The role of feedback in the distance learning process. Journal of Teacher Education for Sustainability, 23(2), 91–105. https://doi.org/10.2478/ jtes-2021-0019
Karaoglan Yilmaz, F. G., & Yilmaz, R. (2021). Learning analytics as a metacognitive tool to influence learner transactional distance and motivation in online learning environments. Innovations in Education and Teaching International, 58(5), 575–585. https://doi.org/10.1080/1 4703297.2020.1794928
Ketchum, C., LaFave, D. S., Yeats, C., Phompheng, E., & Hardy, J. H. (2020). Video-based feedback on student work: An investigation into the instructor experience, workload, and student evaluations. Online Learning, 24(3). https://doi.org/10.24059/olj.v24i3.2194
Knight, S., Shibani, A., Abel, S., Gibson, A., Ryan, P., Sutton, N., Wight, R., Lucas, C., Sándor, Á., Kitto, K., Liu, M., Mogarkar, R. V., & Buckingham Shum, S. (2020). AcaWriter: A learning analytics tool for formative feedback on academic writing. Journal of Writing Research, 12(1), 141–186. https://doi.org/10.17239/ jowr-2020.12.01.06
Kochmar, E., Vu, D. D., Belfer, R., Gupta, V., Serban, I. V., & Pineau, J. (2020). Automated personalized feedback improves learning gains in an intelligent tutoring system. In I. I. Bittencourt, M. Cukurova, K. Muldner, R. Luckin, & E. Millán (Eds.), Artificial Intelligence in Education (Vol. 12164, pp. 140–146). Springer International Publishing. https://doi.org/10.1007/978-3-030-52240- 7_26
Kulik, J. A., & Fletcher, J. D. (2016). Effectiveness of intelligent tutoring systems: A meta-analytic review. Review of Educational Research, 86(1), 42–78. https://doi. org/10.3102/0034654315581420
Lan, A. S., Vats, D., Waters, A. E., & Baraniuk, R. G. (2015). Mathematical language processing: Automatic grading and feedback for open response mathematical questions. Proceedings of the Second (2015) ACM Conference on Learning @ Scale, 167–176. https://doi. org/10.1145/2724660.2724664
Larusson, J. A., & White, B. (Eds.). (2014). Chapter 1: Introduction. In Learning analytics from research to practice (pp. 1–12). Springer.
Lemay, D. J., Baek, C., & Doleck, T. (2021). Comparison of learning analytics and educational data mining: A topic modeling approach. Computers and Education: Artificial Intelligence, 2, 100016. https://doi.org/10.1016/j. caeai.2021.100016
Li, C., & Xing, W. (2021). Natural language generation using deep learning to support MOOC learners. International Journal of Artificial Intelligence in Education, 31(2), 186– 214. https://doi.org/10.1007/s40593-020-00235-x
Loyola-Gonzalez, O. (2019). Black-box vs. white-box: Understanding their advantages and weaknesses from a practical point of view. IEEE Access, 7, 154096–154113. https://doi.org/10.1109/ACCESS.2019.2949286
Magis, D., Yan, D., & von Davier, A. A. (2017). Chapter 1: Overview of adaptive testing. In R. Gentleman, K. Hornik, & G. Parmigiani (Eds.), Computerized adaptive and multistage testing with R (pp. 1–5). Springer International Publishing.
Marriott, P., & Teoh, L. K. (2012). Using screencasts to enhance assessment feedback: Students’ perceptions and preferences. Accounting Education, 21(6), 583–598. https://doi.org/10.1080/09639284.2012.725637
Matcha, W., Uzir, N. A., Gasevic, D., & Pardo, A. (2020). A systematic review of empirical studies on learning analytics dashboards: A self-regulated learning perspective. IEEE Transactions on Learning Technologies, 13(2), 226– 245. https://doi.org/10.1109/TLT.2019.2916802
Mbunge, E., Fashoto, S. G., & Olaomi, J. (2021). COVID-19 and online learning: Factors influencing students’ academic performance in first-year computer programming courses in higher education. SSRN Electronic Journal. https://doi.org/10.2139/ssrn.3757988
Merceron, A., & Yacef, K. (2005). Educational data mining: A case study. In Artificial intelligence in education: Supporting learning through intelligent and socially informed technology (Vol. 125, pp. 467–474). IOS Press.
Molloy, E., Boud, D., & Henderson, M. (2020). Developing a learning-centred framework for feedback literacy. Assessment & Evaluation in Higher Education, 45(4), 527–540. https://doi.org/10.1080/02602938.2019.16679 55
Moreno, A., & Redondo, T. (2016). Text analytics: The convergence of big data and artificial intelligence. International Journal of Interactive Multimedia and Artificial Intelligence, 3(6), 57. https://doi.org/10.9781/ ijimai.2016.369
National Center for Educational Statistics [NCES]. (2022). Program for the international assessment of adult competencies (PIAAC). Organisation for Economic Co-Operation and Development. https://nces.ed.gov/ surveys/piaac/about.asp
Organisation for Economic Co-Operation and Development [OECD]. (2019). PISA 2018 results (volume I): What students know and can do. OECD Publishing.
Ötleş, E., Kendrick, D. E., Solano, Q. P., Schuller, M., Ahle, S. L., Eskender, M. H., Carnes, E., & George, B. C. (2021). Using natural language processing to automatically assess feedback quality: Findings from 3 surgical residencies. Academic Medicine, 96(10), 1457–1460. https://doi.org/10.1097/ACM.0000000000004153
Pardo, A., Jovanovic, J., Dawson, S., Gašević, D., & Mirriahi, N. (2019). Using learning analytics to scale the provision of personalised feedback: Learning analytics to scale personalised feedback. British Journal of Educational Technology, 50(1), 128–138. https://doi.org/10.1111/ bjet.12592
Pardo, A., Poquet, O., Martinez-Maldonado, R., & Dawson, S. (2017). Provision of data-driven student feedback in la & edm. In C. Lang, G. Siemens, A. Wise, & D. Gasevic (Eds.), Handbook of learning analytics (First, pp. 163– 174). Society for Learning Analytics Research (SoLAR). https://doi.org/10.18608/hla17.014
Pask, G. (1982). SAKI: Twenty-five years of adaptive training into the microprocessor era. International Journal of Man-Machine Studies, 17(1), 69–74. https://doi. org/10.1016/S0020-7373(82)80009-6
Pechenizkiy, M., Calders, T., Vasilyeva, E., & De Bra, P. (2008). Mining the student assessment data: Lessons drawn from a small scale case study. Educational Data Mining 2008.
Pengel, N., Martin, A., Meissner, R., Arndt, T., Neumann, A. T., de Lange, P., & Wollersheim, H.-W. (2021). TecCoBot: Technology-aided support for self-regulated learning. ArXiv Preprint ArXiv:2111.11881. http://arxiv. org/abs/2111.11881
Perikos, I., Grivokostopoulou, F., & Hatzilygeroudis, I. (2017). Assistance and feedback mechanism in an intelligent tutoring system for teaching conversion of natural language into logic. International Journal of Artificial Intelligence in Education, 27(3), 475–514. https://doi. org/10.1007/s40593-017-0139-y
Piotrkowicz, A., Dimitrova, V., Treasure-Jones, T., Smithies, A., Harkin, P., Kirby, J., & Roberts, T. (2017). Quantified self analytics tools for self-regulated learning with myPAL. Proceedings of the 7th Workshop on Awareness and Reflection in Technology Enhanced Learning Co-Located with the 12th European Conference on Technology Enhanced Learning (EC-TEL 2017), 1997, 16.
Qazdar, A., Er-Raha, B., Cherkaoui, C., & Mammass, D. (2019). A machine learning algorithm framework for predicting students performance: A case study of baccalaureate students in Morocco. Education and Information Technologies, 24(6), 3577–3589. https://doi.org/10.1007/ s10639-019-09946-8
Ramachandran, L., Gehringer, E. F., & Yadav, R. K. (2017). Automated assessment of the quality of peer reviews using natural language processing techniques. International Journal of Artificial Intelligence in Education, 27(3), 534– 581. https://doi.org/10.1007/s40593-016-0132-x
Ramaswami, M., & Bhaskaran, R. (2010). A CHAID based performance prediction model in educational data mining. IJCSI International Journal of Computer Science. http://arxiv.org/abs/1002.1144
Ray, S., & Saeed, M. (2018). Applications of educational data mining and learning analytics tools in handling big data in higher education. In M. M. Alani, H. Tawfik, M. Saeed, & O. Anya (Eds.), Applications of Big Data Analytics (pp. 135–160). Springer International Publishing. https://doi. org/10.1007/978-3-319-76472-6_7
Roberts, T. (2019). 5 Natural language processing examples: How NLP is used. Bloomreach. https://www.bloomreach. com/en/blog/2019/09/natural-language-processing. html
Rodway-Dyer, S., Knight, J., & Dunne, E. (2011). A case study on audio feedback with geography undergraduates. Journal of Geography in Higher Education, 35(2), 217–231. https://doi.org/10.1080/03098265.2010.5241 97
Romero, C., & Ventura, S. (2010). Educational data mining: A review of the state of the art. IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews), 40(6), 601–618. https://doi.org/10.1109/ TSMCC.2010.2053532
Ryan, T., Gašević, D., & Henderson, M. (2019). Identifying the impact of feedback over time and at scale: Opportunities for learning analytics. In M. Henderson, R. Ajjawi, D. Boud, & E. Molloy (Eds.), The Impact of Feedback in Higher Education (pp. 207–223). Springer International Publishing. https://doi.org/10.1007/978-3- 030-25112-3_12
Sedrakyan, G., Malmberg, J., Verbert, K., Järvelä, S., & Kirschner, P. A. (2020). Linking learning behavior analytics and learning science concepts: Designing a learning analytics dashboard for feedback to support learning regulation. Computers in Human Behavior, 107, 105512. https://doi.org/10.1016/j.chb.2018.05.004
Siemens, G. (2012). Learning analytics and educational data mining: Towards communication and collaboration. 3.
Sipes, S. (2017). Using learning analytics in SoTL. Center for Innovative Teaching and Learning @IUB. https://blogs. iu.edu/citl/2017/12/13/using-learning-analytics-in-sotl
Solano, Q. P., Hayward, L., Chopra, Z., Quanstrom, K., Kendrick, D., Abbott, K. L., Kunzmann, M., Ahle, S., Schuller, M., Ötleş, E., & George, B. C. (2021). Natural language processing and assessment of resident feedback quality. Journal of Surgical Education, 78(6), e72–e77. https://doi.org/10.1016/j.jsurg.2021.05.012
Tempelaar, D. (2020). Supporting the less-adaptive student: The role of learning analytics, formative assessment and blended learning. Assessment & Evaluation in Higher Education, 45(4), 579–593. https://doi.org/10.1080/026 02938.2019.1677855
Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In search for the most informative data for feedback generation: Learning analytics in a data-rich context. Computers in Human Behavior, 47, 157–167. https://doi.org/10.1016/j. chb.2014.05.038
Tsai, Y. S., Mello, R. F., Jovanović, J., & Gašević, D. (2021). Student appreciation of data-driven feedback: A pilot study on Ontask. LAK21: 11th International Learning Analytics and Knowledge Conference, 511–517. https:// doi.org/10.1145/3448139.3448212
Ubani, S., & Nielsen, R. (2022). Review of collaborative intelligent tutoring systems (CITS) 2009-2021. 2022 11th International Conference on Educational and Information Technology (ICEIT), 67–75. https://doi.org/10.1109/ ICEIT54416.2022.9690733
UNESCO. (2019). Section 2: Preparing learners to thrive in the future with AI. In Artificial intelligence in education: Challenges and opportunities for sustainable development (pp. 17–24). the United Nations Educational, Scientific and Cultural Organization,. https://unesdoc. unesco.org/ark:/48223/pf0000366994/PDF/366994eng. pdf.multi
Viberg, O., Khalil, M., & Baars, M. (2020). Self-regulated learning and learning analytics in online learning environments: A review of empirical research. Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, 524–533. https://doi. org/10.1145/3375462.3375483
Wang, D., & Han, H. (2021). Applying learning analytics dashboards based on process‐oriented feedback to improve students’ learning effectiveness. Journal of Computer Assisted Learning, 37(2), 487–499. https://doi. org/10.1111/jcal.12502
Wasson, B., Hansen, C., & Netteland, G. (2016). Data literacy and use for learning when using learning analytics for learners. CEUR Workshop Proceedings. http://hdl.handle. net/11250/2429460
Watling, C. J., & Ginsburg, S. (2019). Assessment, feedback and the alchemy of learning. Medical Education, 53, 76–85. https://doi.org/10.1111/medu.13645
Winne, P. H., & Baker, R. S. J. d. (2013). The potentials of educational data mining for researching metacognition, motivation and self-regulated learning. Journal of Educational Data Mining, 5(1), 8.
Witten, I. H., Frank, E., Hall, M. A., & Pal, C. J. (2017). Data mining: Practical machine learning tools and techniques (4th ed.). Elsevier.
Xiong, W., Litman, D., & Schunn, C. (2012). Natural language processing techniques for researching and improving peer feedback. Journal of Writing Research, 4(2), 155– 176. https://doi.org/10.17239/jowr-2012.04.02.3
Yildirim-Erbasli, S. N., & Bulut, O. (2021). Conversation-based assessments: Real-time assessment and feedback. ELearn, 2021(12). https://doi. org/10.1145/3508017.3495533
Zenisky, A. L., & Hambleton, R. K. (2012). Developing test score reports that work: The process and best practices for effective communication. Educational Measurement: Issues and Practice, 31(2), 21–26. https://doi.org/10.1111/ j.1745-3992.2012.00231.x
Zhang, H., Magooda, A., Litman, D., Correnti, R., Wang, E., Matsmura, L. C., Howe, E., & Quintana, R. (2019). eRevise: Using natural language processing to provide formative feedback on text evidence usage in student writing. Proceedings of the AAAI Conference on Artificial Intelligence, 9619–9625. https://doi.org/10.1609/aaai. v33i01.33019619