Evaluation of redundancy effect in multimedia learning environment using EEG signals and eye-tracking

Duygu Mutlu Bayraktar; Pınar Özel; Ali Olamat; Fatih Altindis; Bulent Yilmaz

doi:10.14527/edure.2024.06

Authors

Duygu Mutlu Bayraktar Istanbul University-Cerrahpasa Author https://orcid.org/0000-0002-2276-3768
Pınar Özel Istanbul University-Cerrahpasa Author https://orcid.org/0000-0002-9688-6293
Ali Olamat Yildiz Technical University Author https://orcid.org/0000-0002-3544-7916
Fatih Altindis Abdullah Gul University Author https://orcid.org/0000-0002-3891-935X
Bulent Yilmaz Gulf University for Science and Technology Author https://orcid.org/0000-0003-2954-1217

DOI:

https://doi.org/10.14527/edure.2024.06

Keywords:

Redundancy effect, Multimedia learning, Cognitive load, EEG, Eye-tracking

Abstract

In the multimedia learning environment, presenting texts in different formats at the same time revealed the redundancy effect. This study aimed to evaluate cognitive load by recording brain signals and eye movements while texts were presented in different formats. For this purpose, two different multimedia learning environments in which printed text and narration were presented together and solo narration were designed, respectively. An experimental study was conducted by dividing 20 of 40 participants into two different multimedia environments. This study ensured the main perspectives of building up a learning profile structure depending on learners' cognitive profile by providing two different multimedia designs for redundancy effect via participants' brain topographies and eye movements. The findings suggested that the redundancy group demonstrated elevated cognitive load, especially in the frontal and parietal regions, as seen by heightened theta, beta, and gamma wave activity. Conversely, the non-redundancy group exhibited enhanced processing efficiency with less cognitive strain.

Downloads

Download data is not yet available.

References

Adcock, J. E., & Panayiotopoulos, C. P. (2012). Occipital lobe seizures and epilepsies. Journal of Clinical Neurophysiology, 29(5), 397–407. DOI: https://doi.org/10.1097/WNP.0b013e31826c98fe

Akshaya, R. M., Biradar, S. D., & Shastri, R. K. (2015). Mental task classification using empirical mode decomposition. Discovery, 47(217), 77-80.

Albers, F., Trypke, M., Stebner, F., Wirth, J., & Plass, J. L. (2023). Different types of redundancy and their effect on learning and cognitive load. British Journal of Educational Psychology, 93, 339-352. DOI: https://doi.org/10.1111/bjep.12592

Antonenko, P. D., & Niederhauser, D. S. (2010). The influence of leads on cognitive load and learning in a hypertext environment. Computers in Human Behavior, 26(2), 140-150. DOI: https://doi.org/10.1016/j.chb.2009.10.014

Antonenko, P., Paas, F., & Grabner, R. (2010). Using Electroencephalography to measure cognitive load. Educational Psychology, 22, 425-438. DOI: https://doi.org/10.1007/s10648-010-9130-y

Baceviciute, S., Lucas, G., Terkildsen, T., & Makransky, G. (2022). Investigating the redundancy principle in immersive virtual reality environments: An eye‐tracking and EEG study. Journal of Computer Assisted Learning, 38(1), 120-136. DOI: https://doi.org/10.1111/jcal.12595

Baddeley, A. (1992). Working memory. Science, 255(5044), 556-559. DOI: https://doi.org/10.1126/science.1736359

Bayram, S., & Bayraktar, D.M. (2012). Using eye tracking to study on attention and recall in multimedia learning environments: the effects of design in learning. World Journal on Educational Technology, 4(2), 81-98.

Biedert, R., Buscher, G., & Dengel, A. (2010). The eyeBook – Using eye tracking to enhance the reading experience. Informatik-Spektrum, 33(3), 272-281. DOI: https://doi.org/10.1007/s00287-009-0381-2

Calomeni, M. R., Furtado da Silva, V., Velasques, B. B., Feijó, O. G., Bittencourt, J. M., & Ribeiro de Souza e Silva, A. P. (2017). Modulatory effect of association of brain stimulation by light and binaural beats in specific brain waves. clinical pract. Clinical Practice and Epidemiology in Mental Health : CP & EMH., 13, 134–144. DOI: https://doi.org/10.2174/1745017901713010134

Camnalbur, M., Bayraktar, D. M., & Er Amuce, N. (2013). The effect of web-based instruction on achievement: A meta-analysis study. Cypriot Journal of Educational Sciences, 8(3), 292–301.

Chen, O., & Kalyuga, S. (2020). Exploring factors influencing the effectiveness of explicit instruction first and problem-solving first approaches. European Journal of Psychology of Education, 35(3), 607-624. DOI: https://doi.org/10.1007/s10212-019-00445-5

Chik, A. (2013). Becoming English teachers in China: identities at the crossroads. The European Journal of Applied Linguistics and TEFL, 2(1), 135-147.

Chuang, K. -Y. Chen, Y. -H. Balachandran, P., Liang, W -K., & Juan, C. -H. (2019). Revealing the electrophysiological correlates of working memory-load effects in symmetry span task with hht method. Frontiers in Psychology, 10(855), 1-11. DOI: https://doi.org/10.3389/fpsyg.2019.00855

Culham, J. C., & Valyear, K. F. (2006). Human parietal cortex in action. Current Opinion in Neurobiology, 16(2), 205-212. DOI: https://doi.org/10.1016/j.conb.2006.03.005

Dan, A., Reiner, M. (2017). Real-Time EEG based measurements of cognitive load indicates mental states during learning. Journal of Educational Data Mining, 9(2), 31-44.

Doyle, A. E. (2006). Executive functions in attention-deficit/hyperactivity disorder. J. Clin. Psychiatry, 67(8), 21-26.

Dincer, S. (2016). Assessing the computer literacy of university graduates. In The Third International Conference on Open and Flexxible Education (pp. 294-303).

Dutta, S., Singh, M., & Kumar, A. (2018). Automated classification of a non-motor mental task in electroencephalogram based brain-computer interface using a multivariate autoregressive model in the intrinsic mode function domain. Biomedical Signal Processing and Control, 43, 174-182. DOI: https://doi.org/10.1016/j.bspc.2018.02.016

Farkish, A., Bosaghzadeh, A., Amiri, S. H., & Ebrahimpour, R. (2023). Evaluating the effects of educational multimedia design principles on cognitive load using EEG signal analysis. Education and Information Technologies, 28(3), 2827-2843. DOI: https://doi.org/10.1007/s10639-022-11283-2

Gavas, R., Tripathy, S., Chatterjee, D., & Sinha, A. (2018). Cognitive load and metacognitive confidence extraction from pupillary response. Cognitive Systems Research, 52(4), 325-334. DOI: https://doi.org/10.1016/j.cogsys.2018.07.021

Ghaderyan, P., & Abbasi, A. (2017). Dynamic Hilbert warping, a new measure of RR-interval signals evaluated in the cognitive load estimation. Medical Engineering&Physics, 40, 103-109. DOI: https://doi.org/10.1016/j.medengphy.2016.12.008

Grimes, D., Tan, D. S., Hudson, S. E., Shenoy, P., & Rao, R. P. N. (2008). Feasibility and pragmatics of classifying working memory load with an electroencephalograph. In Proc. of CHI (pp. 835-844). ACM Press, New York. DOI: https://doi.org/10.1145/1357054.1357187

Gupta, A., Kumar, D., Chakraborti, A., & Sharma, K. (2017). Performance evaluation of empirical mode decomposition algorithms for mental task classification. Biorxiv, 1-20. DOI: https://doi.org/10.1101/076646

Holm, A., Lukander, K., Korpela, J., Sallinen, M., & Müller, K. M. I. (2009). Estimating brain load from the EEG. The Scientific World Journal, 9, 639-652. DOI: https://doi.org/10.1100/tsw.2009.83

Hossain, G., & Yeasin, M. (2014). Understanding effects of cognitive load from pupillary responses using hilbert analytic phase. IEEE Conference on Computer Vision and Pattern Recognition Workshops, Columbus, OH, USA. DOI: https://doi.org/10.1109/CVPRW.2014.62

Jacob, R. J., & Karn, K. S. (2003). Eye-tracking in human-computer interaction and usability research: Ready to deliver the promises. In The mind's eye (pp.573-605). DOI: https://doi.org/10.1016/B978-044451020-4/50031-1

Jung, J., Kim, D., & Na, C. (2016). Effects of WOE presentation types used in pre-training on the cognitive load and comprehension of content in animation-based learning environments. Journal of Educational Technology & Society, 19(4), 75–86.

Kruger, J. L., Hefer, E., & Matthew, G. (2013). Measuring the impact of subtitles on cognitive load: Eye tracking and dynamic audiovisual texts. In Proceedings of the 2013 Conference on Eye Tracking South Africa (pp.62-66). DOI: https://doi.org/10.1145/2509315.2509331

Lah, S., & Smith, M. L. (2014). Semantic and episodic memory in children with temporal lobe epilepsy: Do they relate to literacy skills? Neuropsychology, 28(1), 113–122. DOI: https://doi.org/10.1037/neu0000029

Lai, M.-L., Tsai, M.-J., Yang, F.-Y., Hsu, C.-Y., Liu, T.-C., Lee, S. W.-Y., Lee, M.-H., Chiou, G.-L., Liang, J.-C., & Tsai, C.-C. (2013). A review of using eye-tracking technology in exploring learning from 2000 to 2012. Educational Research Review, 10, 90-115. DOI: https://doi.org/10.1016/j.edurev.2013.10.001

Lee, H., & Mayer, R. E. (2015). Visual aids to learning in a second language: Adding redundant video to an audio lecture. Applied Cognitive Psychology, 29(3), 445–454. DOI: https://doi.org/10.1002/acp.3123

Liu, H. C., Lai, M. L., & Chuang, H. H. (2011). Using eye-tracking technology to investigate the redundant effect of multimedia web pages on viewers' cognitive processes. Computers in Human Behavior, 27(6), 2410-2417. DOI: https://doi.org/10.1016/j.chb.2011.06.012

Majooni, A., Masood, M., & Akhavan, A. (2016). An eye-tracking experiment on strategies to minimize the redundancy and split attention effects in scientific graphs and diagrams. In Advances in Design for Inclusion. Springer, Cham. (pp. 529-540). DOI: https://doi.org/10.1007/978-3-319-41962-6_47

Makransky, G., Terkildsen, T. S., & Mayer, R. E. (2019). The role of subjective and objective measures of cognitive processing during learning in explaining. Learning and Instruction, 61, 23-34. DOI: https://doi.org/10.1016/j.learninstruc.2018.12.001

Malagoli. A., Corradini, M., Corradini, P., Shuett, T., & Fonda, S. (2017). Towards a method for the objective assessment of cognitive workload. IEEE 3rd International Forum on Research and Technologies for Society and Industry (RTSI), Modena, Italy.

Marzbani, H., Marateb, H. R., & Mansourian, M. (2016). Neurofeedback: A comprehensive review on system design, methodology, and clinical applications. Basic and Clinical Neuroscience, 7(2), 143–158. DOI: https://doi.org/10.15412/J.BCN.03070208

Mayer, R. E. (2014). Handbook of multimedia learning. Cambridge University Press. DOI: https://doi.org/10.1017/CBO9781139547369

Mayer, R. E. (2017). Using multimedia for e-learning. Journal of Computer Assisted Learning, 33(5), 403–423. DOI: https://doi.org/10.1111/jcal.12197

Mayer, R. E., & Johnson, C. I. (20028). Revising the redundancy principle in multimedia learning. Journal of Educational Psychology, 100(2), 380-386. DOI: https://doi.org/10.1037/0022-0663.100.2.380

Mazher, M., Aziz, A. A., Malik, A. S., & Amin, H. U. (2017). An EEG-based cognitive load assessment in multimedia learning using feature extraction and partial directed coherence. IEEE Access, 5, 14819–14829. DOI: https://doi.org/10.1109/ACCESS.2017.2731784

Mutlu Bayraktar, D., & Altun, A. (2014). The effect of multimedia design types on learners' recall performances with varying short term memory spans. Multimedia Tools and Applications, 71(3), 1201–1213. DOI: https://doi.org/10.1007/s11042-012-1257-z

Mutlu-Bayraktar, D., Ozel, P., Altindis, F., & Yilmaz, B. (2023). Relationship between objective and subjective cognitive load measurements in multimedia learning. Interactive Learning Environments, 31(3), 1322-1334. DOI: https://doi.org/10.1080/10494820.2020.1833042

Mutlu-Bayraktar, D., Ozel, P., Altindis, F., & Yilmaz, B. (2022). Split-attention effects in multimedia learning environments: eye-tracking and EEG analysis. Multimedia Tools and Applications, 81(6), 8259-8282. DOI: https://doi.org/10.1007/s11042-022-12296-2

Noshadi, S., Abootalebi, V., Sadeghi, M. T., & Shahvazian, M. S. (2014). Selection of an efficient feature space for EEG-based mental task discrimination. Biocybernetics and Biomedical Engineering, 34(3), 159-168. DOI: https://doi.org/10.1016/j.bbe.2014.03.004

Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. (2003). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38(1), 63-71. DOI: https://doi.org/10.1207/S15326985EP3801_8

Paas, F., Van Gog, T., & Sweller, J. (2010) Cognitive load theory: New conceptualizations, specifications, and integrated research perspectives. Educational Psychology Review, 22(2), 115-121. DOI: https://doi.org/10.1007/s10648-010-9133-8

Park, B., Korbach, A., & Brünken, R. (2015). Do learner characteristics moderate the seductive-details-effect? A cognitive-load-study using eye-tracking. Journal of Educational Technology & Society, 18(4), 24-36.

Pastore, R., Briskin, J., & Asino, T. (2018). The effects of the multimedia, modality, and redundancy principles in a computer-based environment on adult learners. Society for Information Technology & Teacher Education International Conference (pp.1663-1668).

Rahman, M., & Fattah, S. A. (2017). Mental task classification scheme utilizing correlation coefficient extracted from interchannel intrinsic mode function. BioMed Research International, 2017, 1-11. DOI: https://doi.org/10.1155/2017/3720589

Russell, M. (2005). Using eye-tracking data to understand first impressions of a website. Usability News, 7(1), 1-14.

Schüler, A., Scheiter, K., & Gerjets, P. (2013). Is spoken text always better? Investigating the modality and redundancy effect with longer text presentation. Computers in Human Behavior, 29(4), 1590–1601. DOI: https://doi.org/10.1016/j.chb.2013.01.047

Secerbegovic A., Ibric S., Nisic J., Suljanovic N., & Mujcic A. (2017). Mental workload vs. stress differentiation using single-channel EEG. CMBEBIH 2017. IFMBE Proceedings, 62., Springer, Singapore. DOI: https://doi.org/10.1007/978-981-10-4166-2_78

Smith, E. E., & Kosslyn, S. M. (2007). Cognitive psychology: Mind and brain. Upper Saddle River, NJ: Pearson.

Suresh, P., Dash, S.S., & Panigrahi, B.T. (2014). Artificial intelligence and evolutionary algorithms in engineering systems. Proceedings of ICAEES 2014, 2. DOI: https://doi.org/10.1007/978-81-322-2126-5

Sweller, J. (2005). The redundancy principle in multimedia learning. The Cambridge Handbook of Multimedia Learning, 159-167. DOI: https://doi.org/10.1017/CBO9780511816819.011

Sweller, J. (2010). Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational Psychology Review, 22(2), 123-138. DOI: https://doi.org/10.1007/s10648-010-9128-5

Trypke, M., Stebner, F., & Wirth, J. (2023). Two types of redundancy in multimedia learning: a literature review. Frontiers in Psychology, 14, 1-17. DOI: https://doi.org/10.3389/fpsyg.2023.1148035

van Gog, T., Paas, F., & van Merriënboer, J. J. G. (2006). Effects of process-oriented worked examples on troubleshooting transfer performance. Learning and Instruction, 16(2), 154–164. DOI: https://doi.org/10.1016/j.learninstruc.2006.02.003

Young-Min Park, Hee-Jae Che, Chang-Hwan Imc, & Hyung-Tae Jung, S.-M. B. S., (2008). Decreased EEG synchronization and its correlation with symptom severity in Alzheimer' s disease. Elsevier Neuroscience Research, 62, 112-117. DOI: https://doi.org/10.1016/j.neures.2008.06.009

Zarjam, P., Epps, J., Lovell, N.H., & Chen, F. (2012). Characterization of memory load in an arithmetic task using nonlinear analysis of EEG signals. Annual International Conference of the IEEE Engineering in Medicine and Biology Society, San Diego, CA, USA. DOI: https://doi.org/10.1109/EMBC.2012.6346725

Zhuang, N., Zeng, Y., Tong, L., Zhang, C. , Zhang, H. , & Yan, B. (2017). Emotion recognition from EEG signals using multidimensional information in emd domain. BioMed Research International, 2017(1), 1-9. DOI: https://doi.org/10.1155/2017/8317357