Analysis and Classification of Customer Churn Using Machine Learning Models

Muhammad Maulana Sidiq Nurhidayat; Dyah Anggraini

doi:10.29207/resti.v7i6.4933

Muhammad Maulana Sidiq Nurhidayat Universitas Gunadarma
Dyah Anggraini Universitas Gunadarma

DOI: https://doi.org/10.29207/resti.v7i6.4933

Keywords: data mining, machine learning, imbalance, SMOTE, confussion matrix, EDA

Abstract

Analysis studies of customer loss (customer churn) have been used for years to increase profitability and build customer relationships with companies. Customer analysis using exploratory data analysis (EDA) to visualize data and the use of machine learning to classify customer churn are often used by past analysts. This study uses several machine learning models that can be used for customer churn classification, namely Logistic Regression, Random Forest, Support Vector Machine (SVM), Gradient Boosting, AdaBoost, and Extreme Gradient Boosting (XGBoost). However, there is a class imbalance factor in the dataset, which is the biggest challenge that analysts usually face in achieving good results in the classification of machine learning models. The Synthetic Minority Oversampling Technique (SMOTE) method is a popular method applied to deal with class imbalances in datasets. The results of the analysis show that the classification of churn customers using the XGBoost algorithm has the best level of accuracy compared to other algorithms, with an accuracy value of 0.829424, and the oversampling method with SMOTE tends to reduce the accuracy value of each classification algorithm. The Permutation Feature Importance (PFI) technique of the XGBoost model gets the result that tenure, monthly contracts, and TV streaming are the features that affect customer churn the most.

Downloads

Download data is not yet available.

References

N. Suryana, “ Prediksi Churn Dan Segmentasi Pelanggan Tv Berlangganan (Studi Kasus Transvision Jawa Barat),” J. TEDC, vol. 11, no. 2, pp. 185–191, 2019, [Online]. Available: https://ejournal.poltektedc.ac.id/index.php/tedc/article/view/77

J. Pamina et al., “An effective classifier for predicting churn in telecommunication,” J. Adv. Res. Dyn. Control Syst., vol. 11, no. 1 Special Issue, 2019.

K. Matuszelański and K. Kopczewska, “Customer Churn in Retail E-Commerce Business: Spatial and Machine Learning Approach,” J. Theor. Appl. Electron. Commer. Res., vol. 17, no. 1, 2022, doi: 10.3390/jtaer17010009.

B. Chen, Y. Liu, and J. Zheng, “Using Data Mining Approach for Student Satisfaction With Teaching Quality in High Vocation Education,” Front. Psychol., vol. 12, 2022, doi: 10.3389/fpsyg.2021.746558.

Omer Faruk Seymen, Onur Doğan, Orhan Er, Kadir Hızıroğlu, and Emre Ölmez, “Customer Churn Prediction Using Ordinary Artificial Neural Network and Convolutional Neural Network Algorithms: A Comparative Performance Assessment,” J. Sci., vol. 36, no. 2, pp. 720–733, 2021.

N. Bagul, P. Berad, P. Surana, and C. Khachane, “Retail Customer Churn Analysis using RFM Model and K-Means Clustering,” Int. J. Eng. Res. Technol., vol. 10, no. 03, 2021.

N. W. Wardani, G. R. Dantes, and G. Indrawan, “Prediksi Customer Churn dengan Algoritma Decision Tree C4.5 Berdasarkan Segmentasi Pelanggan untuk Mempertahankan Pelanggan pada Perusahaan Retail,” J. Resist. (Rekayasa Sist. Komputer), vol. 1, no. 1, 2018, doi: 10.31598/jurnalresistor.v1i1.219.

R. Venkatraman and R. Ragala, “A survey on churn analysis and prediction in video on demand,” Asian J. Pharm. Clin. Res., vol. 10, 2017, doi: 10.22159/ajpcr.2017.v10s1.19603.

F. Kayaalp, “Review of Customer Churn Analysis Studies in Telecommunications Industry,” Karaelmas Sci. Eng. J., vol. 7, no. 2, 2017.

I. M. Latief, A. Subekti, and W. Gata, “Prediksi Tingkat Pelanggan Churn Pada Perusahaan Telekomunikasi Dengan Algoritma Adaboost,” J. Inform., vol. 21, no. 1, 2021, doi: 10.30873/ji.v21i1.2867.

I. Ullah, B. Raza, A. K. Malik, M. Imran, S. U. Islam, and S. W. Kim, “A Churn Prediction Model Using Random Forest: Analysis of Machine Learning Techniques for Churn Prediction and Factor Identification in Telecom Sector,” IEEE Access, vol. 7, pp. 60134–60149, 2019, doi: 10.1109/ACCESS.2019.2914999.

A. Mahalekshmi and G. H. Chellam, “Analysis of customer churn prediction using machine learning and deep learning algorithms,” Int. J. Health Sci. (Qassim)., May 2022, doi: 10.53730/ijhs.v6ns1.7861.

A. S. Halibas, A. Cherian Matthew, I. G. Pillai, J. Harold Reazol, E. G. Delvo, and L. Bonachita Reazol, “Determining the intervening effects of exploratory data analysis and feature engineering in telecoms customer churn modelling,” in 2019 4th MEC International Conference on Big Data and Smart City, ICBDSC 2019, 2019. doi: 10.1109/ICBDSC.2019.8645578.

A. Amin et al., “Comparing Oversampling Techniques to Handle the Class Imbalance Problem: A Customer Churn Prediction Case Study,” IEEE Access, vol. 4, 2016, doi: 10.1109/ACCESS.2016.2619719.

A. Fernández, S. García, F. Herrera, and N. V. Chawla, “SMOTE for Learning from Imbalanced Data: Progress and Challenges, Marking the 15-year Anniversary,” Journal of Artificial Intelligence Research, vol. 61. 2018. doi: 10.1613/jair.1.11192.

V. Flores, S. Heras, and V. Julian, “Comparison of Predictive Models with Balanced Classes Using the SMOTE Method for the Forecast of Student Dropout in Higher Education,” Electron., vol. 11, no. 3, 2022, doi: 10.3390/electronics11030457.

W. Y. Ayele, “Adapting CRISP-DM for idea mining a data mining process for generating ideas using a textual dataset,” Int. J. Adv. Comput. Sci. Appl., vol. 11, no. 6, 2020, doi: 10.14569/IJACSA.2020.0110603.

V. Plotnikova, M. Dumas, and F. P. Milani, “Applying the CRISP-DM data mining process in the financial services industry: Elicitation of adaptation requirements,” Data Knowl. Eng., vol. 139, 2022, doi: 10.1016/j.datak.2022.102013.

C. Schröer, F. Kruse, and J. M. Gómez, “A systematic literature review on applying CRISP-DM process model,” in Procedia Computer Science, 2021, vol. 181. doi: 10.1016/j.procs.2021.01.199.

J. S. Saltz, “CRISP-DM for Data Science: Strengths, Weaknesses and Potential Next Steps,” in Proceedings - 2021 IEEE International Conference on Big Data, Big Data 2021, 2021. doi: 10.1109/BigData52589.2021.9671634.

F. Schafer, C. Zeiselmair, J. Becker, and H. Otten, “Synthesizing CRISP-DM and Quality Management: A Data Mining Approach for Production Processes,” in 2018 IEEE International Conference on Technology Management, Operations and Decisions, ICTMOD 2018, 2018. doi: 10.1109/ITMC.2018.8691266.

C. P. Lopez, Data Mining. The CRISP-DM Methodology. The CLEM Language and IBM SPSS Modeler, vol. 7, no. 2. 2021.

N. Mohd Nawi, M. Makhtar, M. Salikon, and Z. Afip, “A comparative analysis of classification techniques on predicting flood risk,” Indones. J. Electr. Eng. Comput. Sci., vol. 18, p. 1342, Jun. 2020, doi: 10.11591/ijeecs.v18.i3.pp1342-1350.

K. Sahoo, A. K. Samal, J. Pramanik, and S. K. Pani, “Exploratory data analysis using python,” Int. J. Innov. Technol. Explor. Eng., vol. 8, no. 12, 2019, doi: 10.35940/ijitee.L3591.1081219.

R. S. Oktavian and S. Budi, “Analisis Dataset Google Playstore Menggunakan Metode Exploratory Data Analysis,” J. Strateg. Maranatha, vol. 2, no. 2, 2020.

H. Wen et al., “Multiparametric Quantitative US Examination of Liver Fibrosis: A Feature-Engineering and Machine-Learning Based Analysis,” IEEE J. Biomed. Heal. Informatics, vol. 26, no. 2, 2022, doi: 10.1109/JBHI.2021.3100319.

B. P. Pratiwi, A. S. Handayani, and S. Sarjana, “Pengukuran Kinerja Sistem Kualitas Udara Dengan Teknologi Wsn Menggunakan Confusion Matrix,” J. Inform. Upgris, vol. 6, no. 2, 2021, doi: 10.26877/jiu.v6i2.6552.

W. Rahayu and E. Wahyudi, “Classical Test Theory Of Innapropriate Index Score’s Accuracy Comparison Using Confusion Matrix Accuracy Proportion In Educational Measurement,” IJER - Indones. J. Educ. Rev., vol. 4, no. 1, 2017, doi: 10.21009/ijer.04.01.08.

A. Fisher, C. Rudin, and F. Dominici, “All models are wrong, but many are useful: Learning a variable’s importance by studying an entire class of prediction models simultaneously,” J. Mach. Learn. Res., vol. 20, 2019.

H. Kaneko, “Cross‐validated permutation feature importance considering correlation between features,” Anal. Sci. Adv., vol. 3, no. 9–10, 2022, doi: 10.1002/ansa.202200018.