Improving Frame-based Engagement Classification in E-Learning Using EfficientNet and Normalized Loss Weighting
Abstract
Engagement can be defined as how individuals are involved in and interact with a task that requires attention and emotional conditions. Engagement is an affective state positively correlated with learning processes. Engagement along with other affective states, such as boredom, confusion, and frustration must be analyzed to identify students’ learning behavior. Implementing proper prevention by measuring student engagement levels could increase students’ learning intake. Such implementation involves building an effective feedback system or rearranging the learning design. Several researchers have proposed deep-learning approaches using the DAiSEE dataset to classify student engagement levels. In addition, previous studies utilized various loss functions equipped with class weighting to assign higher importance to the minor classes, which are low and very low engagement classes. Most of the state-of-the-art models achieved high accuracy, but the f1-score was still low because of the minor class struggle. This research tries to solve engagement level classification on imbalance conditions by proposing a normalized loss function weighting based on the Inverse Class Frequency formula based on each class’ instances to give more importance and focus to the classes and trained on Vanilla EfficientNet model rather than experimenting on more advanced model to keep the efficient and suit the memory constraint on the e-learning implementation. Based on the conducted experiments, the normalized ICF obtained the highest accuracy of 51.64% and weighted f1-score of 50.86%, which is superior to the standard ICF performance, which received 50.32% accuracy and weighted f1-score of 50.49% using the same settings.
Downloads
References
J. Zhang, Z. Yin, P. Chen, dan S. Nichele, “Emotion recognition using multi-modal data and machine learning techniques: A tutorial and review,” Inf. Fusion, vol. 59, hal. 103–126, Jul 2020, doi: 10.1016/j.inffus.2020.01.011.
K. K. Jena et al., “E-Learning Course Recommender System Using Collaborative Filtering Models,” Electronics, vol. 12, no. 1, hal. 157, Des 2022, doi: 10.3390/electronics12010157.
K. A. Laksitowening, A. P. Yanuarifiani, dan Y. F. A. Wibowo, “Enhancing e-learning system to support learning style based personalization,” in 2016 2nd International Conference on Science in Information Technology (ICSITech), Balikpapan, Indonesia: IEEE, Okt 2016, hal. 329–333. doi: 10.1109/ICSITech.2016.7852657.
A. N. R. Paidja dan F. A. Bachtiar, “Engagement Emotion Classification through Facial Landmark Using Convolutional Neural Network,” in 2022 2nd International Conference on Information Technology and Education (ICIT&E), Malang, Indonesia: IEEE, Jan 2022, hal. 234–239. doi: 10.1109/ICITE54466.2022.9759546.
J. J. Appleton, S. L. Christenson, dan M. J. Furlong, “Student engagement with school: Critical conceptual and methodological issues of the construct,” Psychol. Sch., vol. 45, no. 5, hal. 369–386, Mei 2008, doi: 10.1002/pits.20303.
S. Tu, “ENGAGEMENT PREDICTION AND VISUALIZATION IN ONLINELEARNING,” in Jiangsu Annual Conference on Automation (JACA 2020), Zhenjiang, China: Institution of Engineering and Technology, 2021, hal. 57–62. doi: 10.1049/icp.2021.1428.
M. Hu, H. Li, W. Deng, dan H. Guan, “Student Engagement: One of the Necessary Conditions for Online Learning,” in 2016 International Conference on Educational Innovation through Technology (EITT), Tainan, Taiwan: IEEE, Sep 2016, hal. 122–126. doi: 10.1109/EITT.2016.31.
H. A. El-Sabagh, “Adaptive e-learning environment based on learning styles and its impact on development students’ engagement,” Int. J. Educ. Technol. High. Educ., vol. 18, no. 1, 2021, doi: 10.1186/s41239-021-00289-4.
N. K. Mehta, S. S. Prasad, S. Saurav, R. Saini, dan S. Singh, “Three-dimensional DenseNet self-attention neural network for automatic detection of student’s engagement,” Appl. Intell., vol. 52, no. 12, hal. 13803–13823, 2022, doi: 10.1007/s10489-022-03200-4.
S. Mandia, K. Singh, dan R. Mitharwal, “Vision Transformer for Automatic Student Engagement Estimation,” in 2022 IEEE 5th International Conference on Image Processing Applications and Systems (IPAS), Genova, Italy: IEEE, Des 2022, hal. 1–6. doi: 10.1109/IPAS55744.2022.10052945.
Y. Cui, M. Jia, T. Y. Lin, Y. Song, dan S. Belongie, “Class-balanced loss based on effective number of samples,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Long Beach, USA, 2019, hal. 9260–9269. doi: 10.1109/CVPR.2019.00949.
A. Gupta, A. D’Cunha, K. Awasthi, dan V. Balasubramanian, “DAiSEE: Towards User Engagement Recognition in the Wild,” vol. 14, no. 8, hal. 1–12, 2016, [Daring]. Tersedia pada: http://arxiv.org/abs/1609.01885
J. Whitehill, Z. Serpell, Y. C. Lin, A. Foster, dan J. R. Movellan, “The faces of engagement: Automatic recognition of student engagement from facial expressions,” IEEE Trans. Affect. Comput., vol. 5, no. 1, hal. 86–98, 2014, doi: 10.1109/TAFFC.2014.2316163.
L. Geng, M. Xu, Z. Wei, dan X. Zhou, “Learning Deep Spatiotemporal Feature for Engagement Recognition of Online Courses,” in 2019 IEEE Symposium Series on Computational Intelligence, SSCI 2019, Xiamen, China, 2019, hal. 442–447. doi: 10.1109/SSCI44817.2019.9002713.
T. Selim, I. Elkabani, dan M. A. Abdou, “Students Engagement Level Detection in Online e-Learning Using Hybrid EfficientNetB7 Together With TCN, LSTM, and Bi-LSTM,” IEEE Access, vol. 10, hal. 99573–99583, 2022, doi: 10.1109/ACCESS.2022.3206779.
C. Shorten dan T. M. Khoshgoftaar, “A survey on Image Data Augmentation for Deep Learning,” J. Big Data, vol. 6, no. 1, 2019, doi: 10.1186/s40537-019-0197-0.
S. C. Wong, A. Gatt, V. Stamatescu, dan M. D. McDonnell, “Understanding Data Augmentation for Classification: When to Warp?,” 2016 Int. Conf. Digit. Image Comput. Tech. Appl. DICTA 2016, 2016, doi: 10.1109/DICTA.2016.7797091.
J. Zhang dan C. Li, “Adversarial Examples: Opportunities and Challenges,” IEEE Trans. Neural Networks Learn. Syst., vol. 31, no. 7, hal. 2578–2593, 2020, doi: 10.1109/TNNLS.2019.2933524.
R. Rashmi Adyapady dan B. Annappa, “Learning Engagement Assessment in MOOC Scenario,” in 2022 IEEE International Conference on Electronics, Computing and Communication Technologies (CONECCT 2022), Bangalore, India: IEEE, 2022. doi: 10.1109/CONECCT55679.2022.9865699.
A. Abedi dan S. S. Khan, “Improving state-of-the-art in Detecting Student Engagement with Resnet and TCN Hybrid Network,” in 2021 18th Conference on Robots and Vision (CRV), IEEE, 2021, hal. 151–157. doi: 10.1109/CRV52889.2021.00028.
W. Li dan Z. Liu, “A method of SVM with normalization in intrusion detection,” Procedia Environ. Sci., vol. 11, no. PART A, hal. 256–262, 2011, doi: 10.1016/j.proenv.2011.12.040.
M. Tan dan Q. V. Le, “EfficientNet: Rethinking model scaling for convolutional neural networks,” 36th Int. Conf. Mach. Learn. ICML 2019, vol. 2019-June, hal. 10691–10700, 2019.
Copyright (c) 2025 Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright in each article belongs to the author
- The author acknowledges that the RESTI Journal (System Engineering and Information Technology) is the first publisher to publish with a license Creative Commons Attribution 4.0 International License.
- Authors can enter writing separately, arrange the non-exclusive distribution of manuscripts that have been published in this journal into other versions (eg sent to the author's institutional repository, publication in a book, etc.), by acknowledging that the manuscript has been published for the first time in the RESTI (Rekayasa Sistem dan Teknologi Informasi) journal ;
