Explainable AI for Student Dropout Prediction: A Hierarchical LightGBM Approach

Ismail Setiawan; Rahmat Gernowo; Budi Warsito

doi:10.29207/resti.v10i5.7793

Authors

Ismail Setiawan Universitas 'Aisyiyah Surakarta
Rahmat Gernowo Universitas Diponegoro
Budi Warsito Universitas Diponegoro

DOI:

https://doi.org/10.29207/resti.v10i5.7793

Keywords:

dropout prediction, early warning system, educational data mining, LightGBM, SHAP

Abstract

The high dropout rate in health science programs not only affects the efficiency of educational management but also threatens the stability of the professional medical workforce. This research proposes a Hierarchical Early Warning System (HEWS) framework using the Light Gradient Boosting Machine (LightGBM) algorithm to predict student dropout risks at an early stage, utilizing a primary dataset of academic and demographic records from an Indonesian health sciences faculty. While previous studies predominantly utilized full-trajectory academic records, this study compares the model's performance across two temporal scenarios: early detection using historical data up to the second semester and full-trajectory evaluation. The Synthetic Minority Over-sampling Technique (SMOTE) is applied to address the extreme class imbalance inherent in educational datasets. Experimental results demonstrate that the proposed model achieves an F1-Score of 0.95 for the minority class in both internal scenarios and maintains a robust F1-Score of 0.91 when validated against an international dataset. Furthermore, SHapley Additive exPlanations (SHAP) is integrated to dissect the algorithm's black-box nature. The SHAP analysis reveals a critical shift in risk drivers: high school education quality and first-semester grades dominate the early detection phase, whereas fourth-semester grades and sick leave frequencies become primary predictors in the full-trajectory phase. These findings provide actionable insights for university management to design targeted academic and psychological interventions, thereby supporting the achievement of Sustainable Development Goals (SDG) 3 and 4.

Downloads

Download data is not yet available.

References

[1] J. Soerensen, D. S. Nielsen, and G. T. Pihl, “It’s a hard process – Nursing students’ lived experiences leading to dropping out of their education; a qualitative study,” Nurse Educ. Today, vol. 122, no. January, p. 105724, 2023, doi: https://doi.org/10.1016/j.nedt.2023.105724.

[2] M. Arian, A. Kamali, Z. Dalir, F. Hajiabadi, and S. R. Mazloum, “Identifying predictors of nursing dropout and attrition before and after Bachelor’s Graduation based on the IPOD model: A machine learning approach,” Nurse Educ. Pract., vol. 88, no. October, p. 104580, 2025, doi: https://doi.org/10.1016/j.nepr.2025.104580.

[3] M. Shoaib, N. Sayed, J. Singh, J. Shafi, S. Khan, and F. Ali, “AI student success predictor: Enhancing personalized learning in campus management systems,” Comput. Human Behav., vol. 158, no. April, p. 108301, Sep. 2024, doi: https://doi.org/10.1016/j.chb.2024.108301.

[4] R. Ordoñez-Avila, N. Salgado Reyes, J. Meza, and S. Ventura, “Data mining techniques for predicting teacher evaluation in higher education: A systematic literature review,” Heliyon, vol. 9, no. 3, 2023, doi: https://doi.org/10.1016/j.heliyon.2023.e13939.

[5] S. M. Dol and P. M. Jawandhiya, “Classification Technique and its Combination with Clustering and Association Rule Mining in Educational Data Mining — A survey,” Eng. Appl. Artif. Intell., vol. 122, no. February, p. 106071, 2023, doi: https://doi.org/10.1016/j.engappai.2023.106071.

[6] M. Correa-Peralta, J. Vinueza-Martínez, and L. Castillo-Heredia, “Evolution, topics and relevant research methodologies in business intelligence and data analysis in the academic management of higher education institutions. A literature review,” Results Eng., vol. 25, no. December 2024, p. 103782, 2025, doi: https://doi.org/10.1016/j.rineng.2024.103782.

[7] V. Christou et al., “Performance and early drop prediction for higher education students using machine learning,” Expert Syst. Appl., vol. 225, p. 120079, 2023, doi: https://doi.org/10.1016/j.eswa.2023.120079.

[8] C. F. de Oliveira, S. R. Sobral, M. J. Ferreira, and F. Moreira, “Interpretable Success Prediction in a Computer Networks Curricular Unit Using Machine Learning,” Procedia Comput. Sci., vol. 239, no. 2023, pp. 598–605, 2024, doi: https://doi.org/10.1016/j.procs.2024.06.212.

[9] M. Nilashi et al., “Knowledge discovery for course choice decision in Massive Open Online Courses using machine learning approaches,” Expert Syst. Appl., vol. 199, p. 117092, 2022, doi: https://doi.org/10.1016/j.eswa.2022.117092.

[10] L. Hannaford, X. Cheng, and M. Kunes-Connell, “Predicting nursing baccalaureate program graduates using machine learning models: A quantitative research study,” Nurse Educ. Today, vol. 99, p. 104784, 2021, doi: https://doi.org/10.1016/j.nedt.2021.104784.

[11] P. R., K. P., and S. A. A., “Predicting academic performance of learners with the three domains of learning data using neuro-fuzzy model and machine learning algorithms,” J. Eng. Res., vol. 12, no. 3, pp. 397–411, 2024, doi: https://doi.org/10.1016/j.jer.2023.09.006.

[12] J. G. C. Krüger, A. de S. Britto, and J. P. Barddal, “An explainable machine learning approach for student dropout prediction,” Expert Syst. Appl., vol. 233, p. 120933, 2023, doi: https://doi.org/10.1016/j.eswa.2023.120933.

[13] L. Bognár and T. Fauszt, “Factors and conditions that affect the goodness of machine learning models for predicting the success of learning,” Comput. Educ. Artif. Intell., vol. 3, p. 100100, 2022, doi: https://doi.org/10.1016/j.caeai.2022.100100.

[14] B. Prenkaj, D. Distante, S. Faralli, and P. Velardi, “Hidden space deep sequential risk prediction on student trajectories,” Futur. Gener. Comput. Syst., vol. 125, pp. 532–543, Dec. 2021, doi: https://doi.org/10.1016/j.future.2021.07.002.

[15] A. Kukkar, R. Mohana, A. Sharma, and A. Nayyar, “Prediction of student academic performance based on their emotional wellbeing and interaction on various e-learning platforms,” Educ. Inf. Technol., vol. 28, no. 8, pp. 9655–9684, 2023, doi: https://doi.org/10.1007/s10639-022-11573-9.

[16] B. Cheng, Y. Liu, and Y. Jia, “Evaluation of students’ performance during the academic period using the XG-Boost Classifier-Enhanced AEO hybrid model,” Expert Syst. Appl., vol. 238, p. 122136, 2024, doi: https://doi.org/10.1016/j.eswa.2023.122136.

[17] M. Vaarma and H. Li, “Predicting student dropouts with machine learning: An empirical study in Finnish higher education,” Technol. Soc., vol. 76, no. December 2023, p. 102474, 2024, doi: https://doi.org/10.1016/j.techsoc.2024.102474.

[18] H. Sahlaoui, E. A. A. Alaoui, S. Agoujil, and A. Nayyar, An empirical assessment of smote variants techniques and interpretation methods in improving the accuracy and the interpretability of student performance models, vol. 29, no. 5. Springer US, 2024. doi: https://doi.org/10.1007/s10639-023-12007-w.

[19] Q. Huang and J. Chen, “Enhancing academic performance prediction with temporal graph networks for massive open online courses,” J. Big Data, vol. 11, no. 1, 2024, doi: https://doi.org/10.1186/s40537-024-00918-5.

[20] M. Nachouki and M. A. Naaj, “Predicting Student Performance to Improve Academic Advising Using the Random Forest Algorithm,” Int. J. Distance Educ. Technol., vol. 20, no. 1, pp. 1–17, 2022, doi: https://doi.org/10.4018/IJDET.296702.

[21] X. Zheng and C. Li, “Predicting Students’ Academic Performance Through Machine Learning Classifiers: A Study Employing the Naive Bayes Classifier (NBC),” Int. J. Adv. Comput. Sci. Appl., vol. 15, no. 1, pp. 994–1008, 2024, doi: https://doi.org/10.14569/IJACSA.2024.0150199.

[22] A. Almiman and M. T. Ben Othman, “Predictive Analysis of Computer Science Student Performance: An ACM2013 Knowledge Area Approach,” Ing. des Syst. d’Information, vol. 29, no. 1, pp. 169–189, Feb. 2024, doi: https://doi.org/10.18280/isi.290119.

[23] K. Coussement, M. Phan, A. De Caigny, D. F. Benoit, and A. Raes, “Predicting student dropout in subscription-based online learning environments: The beneficial impact of the logit leaf model,” Decis. Support Syst., vol. 135, p. 113325, 2020, doi: https://doi.org/10.1016/j.dss.2020.113325.

[24] A. Asselman, M. Khaldi, and S. Aammou, “Enhancing the prediction of student performance based on the machine learning XGBoost algorithm,” Interact. Learn. Environ., vol. 31, no. 6, pp. 3360–3379, 2023, doi: https://doi.org/10.1080/10494820.2021.1928235.

[25] A. Jokhan, A. A. Chand, V. Singh, and K. A. Mamun, “Increased Digital Resource Consumption in Higher Educational Institutions and the Artificial Intelligence Role in Informing Decisions Related to Student Performance,” Sustain., vol. 14, no. 4, Feb. 2022, doi: https://doi.org/10.3390/su14042377.

[26] Y. Lin, H. Chen, W. Xia, F. Lin, Z. Wang, and Y. Liu, “A Comprehensive Survey on Deep Learning Techniques in Educational Data Mining,” Data Sci. Eng., vol. 10, pp. 564–590, May 2023, doi: https://doi.org/10.1007/s41019-025-00303-z.

[27] T. Hamim, F. Benabbou, and N. Sael, “Student profile modeling using boosting algorithms,” Int. J. Web-Based Learn. Teach. Technol., vol. 17, no. 5, Sep. 2022, doi: https://doi.org/10.4018/IJWLTT.20220901.oa4.

[28] M. Delogu, R. Lagravinese, D. Paolini, and G. Resce, “Predicting dropout from higher education: Evidence from Italy,” Econ. Model., vol. 130, p. 106583, 2024, doi: https://doi.org/10.1016/j.econmod.2023.106583.

[29] L. Hannaford, X. Cheng, and M. Kunes-connell, “Predicting nursing baccalaureate program graduates using machine learning models : A quantitative research study ☆,” Nurse Educ. Today, vol. 99, no. December 2020, p. 104784, 2021, doi: https://doi.org/10.1016/j.nedt.2021.104784.

[30] G. Li et al., “DKT-LCIRT: A Deep Knowledge Tracking Model Integrating Learning Capability and Item Response Theory,” Electron., vol. 11, no. 20, Oct. 2022, doi: https://doi.org/10.3390/electronics11203364.

[31] J. Niyogisubizo, L. Liao, E. Nziyumva, E. Murwanashyaka, and P. C. Nshimyumukiza, “Predicting student’s dropout in university classes using two-layer ensemble machine learning approach: A novel stacked generalization,” Comput. Educ. Artif. Intell., vol. 3, no. November 2021, p. 100066, 2022, doi: https://doi.org/10.1016/j.caeai.2022.100066.

[32] M. Jawthari and V. Stoffová, “Predicting students’ academic performance using a modified kNN algorithm,” Pollack Period., vol. 16, no. 3, pp. 20–26, 2021, doi: https://doi.org/10.1556/606.2021.00374.

[33] M. V Martins, D. Tolledo, J. Machado, L. M. T. Baptista, and V. Realinho, “Early Prediction of student’s Performance in Higher Education: A Case Study BT - Trends and Applications in Information Systems and Technologies,” in Springer, Cham, Á. Rocha, H. Adeli, G. Dzemyda, F. Moreira, and A. M. Ramalho Correia, Eds., Cham: Springer International Publishing, 2021, pp. 166–175. doi: https://doi.org/10.1007/978-3-030-72657-7_16.