Enhancing Software Defect Prediction Performance using NR-Clustering SMOTE to Address Class Imbalance

Hairani Hairani; Muhamad Masjun Efendi; Gede Yogi Pratama; Rahayun Amrullah Husaini; M.Khaerul Ihsan

doi:10.29207/resti.v10i7558

Authors

Hairani Hairani Universitas Bumigora
Muhamad Masjun Efendi Universitas Teknologi Mataram
Gede Yogi Pratama Universitas Bumigora
Rahayun Amrullah Husaini Universitas Bumigora
M.Khaerul Ihsan Universitas Bumigora

DOI:

https://doi.org/10.29207/resti.v10i7558

Keywords:

noise reduction, NR-Clustering SMOTE, NR-Modified SMOTE, software defect

Abstract

Software defect detection is important to prevent system failures and increased maintenance costs. However, the complexity of modern software makes manual testing inefficient, so machine learning approaches are used. The main challenge of this approach is data imbalance, where defective cases are far fewer, causing the model to overlook the minority class and reducing detection capability, even though accuracy appears high. This study aims to address class imbalance in software defect detection by applying the NR-Clustering SMOTE method to improve machine learning performance—the classification methods using Random Forest. NR-Clustering SMOTE not only oversamples the minority class but also incorporates a noise-reduction mechanism to remove minority data that may degrade classification performance. The results show that NR-Clustering SMOTE improves the performance of Random Forest compared with the original data, SMOTE, and NR-Modified SMOTE across all evaluation metrics, namely accuracy, recall, and F1-score. These findings indicate that integrating noise reduction and SMOTE-based data balancing using Manhattan distance within each cluster produces a more representative data distribution, thereby improving the model’s ability to classify software defect cases more accurately. Therefore, this study confirms that NR-Clustering SMOTE effectively improves Random Forest performance for software defect detection compared with existing approaches.

Downloads

Download data is not yet available.

References

[1] X. Dong, Y. Liang, S. Miyamoto, and S. Yamaguchi, “Ensemble learning based software defect prediction,” J. Eng. Res., vol. 11, no. 4, pp. 377–391, 2023, doi: https://doi.org/10.1016/j.jer.2023.10.038.

[2] J. Pachouly, S. Ahirrao, K. Kotecha, G. Selvachandran, and A. Abraham, “A systematic literature review on software defect prediction using artificial intelligence: Datasets, Data Validation Methods, Approaches, and Tools,” Eng. Appl. Artif. Intell., vol. 111, p. 104773, 2022, doi: https://doi.org/10.1016/j.engappai.2022.104773.

[3] Q. Zhi, W. Pu, J. Ren, and Z. Zhou, “A Defect Detection Method for the Primary Stage of Software Development,” Comput. Mater. Contin., vol. 74, no. 3, pp. 5141–5155, 2022, doi: https://doi.org/10.32604/cmc.2023.035846.

[4] S. Rehan, B. Al-Bander, and A. Al-Said Ahmad, “Harnessing Large Language Models for Automated Software Testing: A Leap Towards Scalable Test Case Generation,” Electronics, vol. 14, no. 7, pp. 1–25, Apr. 2025, doi: 10.3390/electronics14071463.

[5] P. Dhavakumar and S. Vengadeswaran, “Software defect prediction using graph sample and aggregate-attention network optimized with nomadic people optimizer for enhancing the software reliability,” Comput. Stand. Interfaces, vol. 95, p. 104033, 2026, doi: https://doi.org/10.1016/j.csi.2025.104033.

[6] W. Albattah and M. Alzahrani, “Software Defect Prediction Based on Machine Learning and Deep Learning Techniques: An Empirical Approach,” AI, vol. 5, no. 4, pp. 1743–1758, Sep. 2024, doi: 10.3390/ai5040086.

[7] G. Cauvery, D. Suresh, G. Aswini, P. Jayanthi, and K. Kalaiselvi, “Software Defect Prediction Using Machine Learning Techniques,” in Advances in Artificial and Human Intelligence in the Modern Era, 2023, pp. 180–195. doi: 10.4018/979-8-3693-1301-5.ch010.

[8] Y. Gao, Y. Zhu, and Y. Zhao, “Dealing with imbalanced data for interpretable defect prediction,” Inf. Softw. Technol., vol. 151, p. 107016, 2022, doi: https://doi.org/10.1016/j.infsof.2022.107016.

[9] A. A. Khan, O. Chaudhari, and R. Chandra, “A review of ensemble learning and data augmentation models for class imbalanced problems: Combination, implementation and evaluation,” Expert Syst. Appl., vol. 244, no. November 2023, p. 122778, 2024, doi: 10.1016/j.eswa.2023.122778.

[10] H. Z. Zaidi et al., “Machine Learning Approaches for Software Defect Prediction,” Appl. Comput. Intell. Soft Comput., vol. 2025, no. 1, pp. 1–29, Jan. 2025, doi: 10.1155/acis/7933078.

[11] A. Hakami, “Strategies for overcoming data scarcity, imbalance, and feature selection challenges in machine learning models for predictive maintenance,” Sci. Rep., vol. 14, no. 1, pp. 1–14, 2024, doi: 10.1038/s41598-024-59958-9.

[12] T. Widiyaningtyas, H. Hairani, D. D. Prasetya, U. Pujianto, and W. Caesarendra, “A Modified SMOTE with Noise Filtering and Manhattan Distance Metric Approach to Address Imbalanced Health Datasets,” Eng. Technol. Appl. Sci. Res., vol. 15, no. 4, pp. 25452–25459, Aug. 2025, doi: 10.48084/etasr.11925.

[13] H. Hairani, T. Widiyaningtyas, D. D. Prasetya, and A. Aminuddin, “Addressing Imbalance in Health Datasets: A New Method NR-Clustering SMOTE and Distance Metric Modification,” Comput. Mater. Contin., vol. 82, no. 2, pp. 2931–2949, 2025, doi: 10.32604/cmc.2024.060837.

[14] K. Pin, J. Ho Chang, and Y. Nam, “Software Defect Prediction Harnessing on Multi 1-Dimensional Convolutional Neural Network Structure,” Comput. Mater. Contin., vol. 71, no. 1, pp. 1521–1546, 2022, doi: 10.32604/cmc.2022.022085.

[15] T. Shahzad, S. Khan, T. Mazhar, W. Ahmad, K. Ouahada, and H. Hamam, “Predicting Software Perfection Through Advanced Models to Uncover and Prevent Defects,” IET Softw., vol. 2025, no. 1, pp. 1–17, Jan. 2025, doi: 10.1049/sfw2/8832164.

[16] S. Parija, D. Muduli, M. Shameem, and S. K. Sharma, “Enhancing Software Defect Prediction: A Stacking Ensemble Framework with Advanced Feature Engineering,” in Proceedings of the 2025 29th International Conference on Evaluation and Assessment in Software Engineering Companion, in EASE Companion ’25. New York, NY, USA: Association for Computing Machinery, 2025, pp. 28–34. doi: 10.1145/3727967.3756848.

[17] D. Pradhan and D. Muduli, “Improved prediction of software defect: A particle swarm optimization based ELM approach,” e-Prime - Adv. Electr. Eng. Electron. Energy, vol. 13, no. July, pp. 1–21, Sep. 2025, doi: 10.1016/j.prime.2025.101081.

[18] A. A. P. Ramadhani, R. A. Nugroho, M. R. Faisal, F. Abadi, and R. Herteno, “The impact of software metrics in NASA metric data program dataset modules for software defect prediction,” Telkomnika (Telecommunication Comput. Electron. Control., vol. 22, no. 4, pp. 846–853, 2024, doi: 10.12928/TELKOMNIKA.v22i4.25787.

[19] M. S. Alkhasawneh, “Software Defect Prediction through Neural Network and Feature Selections,” Appl. Comput. Intell. Soft Comput., vol. 2022, pp. 1–16, Sep. 2022, doi: 10.1155/2022/2581832.

[20] A. Priyadarshini, “Software Defect Prediction across Multiple Datasets using Classification Techniques,” J. Inf. Syst. Eng. Manag., vol. 10, no. 41s, pp. 158–167, May 2025, doi: 10.52783/jisem.v10i41s.7807.

[21] M. Ali, T. Mazhar, A. Al-Rasheed, T. Shahzad, Y. Yasin Ghadi, and M. Amir Khan, “Enhancing software defect prediction: a framework with improved feature selection and ensemble machine learning,” PeerJ Comput. Sci., vol. 10, pp. 1–37, Feb. 2024, doi: 10.7717/peerj-cs.1860.

[22] D. Wintana, D. Ismunandar, and E. H. Juningsih, “Improving Software Defect Prediction Performance Using C4 . 5 Based Ensemble Learning with AdaBoost and Bagging Techniques,” J-INTECH (Journal Inf. Technol., no. 10, pp. 411–421, 2025, doi: https://doi.org/10.32664/k7fyc413.

[23] B. Arasteh, K. Arasteh, A. Ghaffari, and R. Ghanbarzadeh, “A new binary chaos-based metaheuristic algorithm for software defect prediction,” Cluster Comput., vol. 27, no. 7, pp. 10093–10123, Oct. 2024, doi: 10.1007/s10586-024-04486-4.

[24] B. M. Shankar, S. A. Sivakumar, D. Dhabliya, P. A. Sundari, M. Asmitha, and S. M. G. Shree, “Software Defect Prediction Using Dagging Meta-Learner-Based Classifier,” in 2023 7th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), IEEE, Oct. 2023, pp. 682–686. doi: 10.1109/I-SMAC58438.2023.10290523.

[25] K. Patel and S. Gupta, “Prediction of Software Defects for Quality Assurance and Improvement Based on Machine Learning Methods,” in 2024 2nd International Conference on Advances in Computation, Communication and Information Technology (ICAICCIT), Nov. 2024, pp. 400–406. doi: 10.1109/ICAICCIT64383.2024.10912382.

[26] A. Iqbal et al., “Performance analysis of machine learning techniques on software defect prediction using NASA datasets,” Int. J. Adv. Comput. Sci. Appl., vol. 10, no. 5, pp. 300–308, 2019, doi: 10.14569/ijacsa.2019.0100538.

[27] I. Mehmood et al., “A Novel Approach to Improve Software Defect Prediction Accuracy Using Machine Learning,” IEEE Access, vol. 11, no. June, pp. 63579–63597, 2023, doi: 10.1109/ACCESS.2023.3287326.

[28] F. Meng, W. Cheng, and J. Wang, “Semi-supervised Software Defect Prediction Model Based on Tri-training,” KSII Trans. Internet Inf. Syst., vol. 15, no. 11, pp. 4028–4042, 2021, doi: 10.3837/TIIS.2021.11.009.

[29] A. S. Mohil, A. Mohil, and M. Mohil, “Stacked Ensemble Learning for Software Defect Prediction : Model Integration and Cross-Dataset Validation,” Int. J. Environ. Sci., vol. 10, no. 4, pp. 392–402, 2024, doi: https://theaspd.com/index.php/ijes/article/view/11187.

[30] S. Goyal, “Handling Class-Imbalance with KNN (Neighbourhood) Under-Sampling for Software Defect Prediction,” Artif. Intell. Rev., vol. 55, no. 3, pp. 2023–2064, Mar. 2022, doi: 10.1007/s10462-021-10044-w.

[31] M. M. Febrian, S. W. Saputro, T. H. Saragih, F. Abadi, and R. Herteno, “Hybrid Feature Selection and Balancing Data Approach for Improved Software Defect Prediction,” Indones. J. Electron. Electromed. Eng. Med. Informatics, vol. 7, no. 2, pp. 232–244, Apr. 2025, doi: 10.35882/ijeeemi.v7i2.67.

[32] M. Mustaqeem and T. Siddiqui, “A Hybrid Software Defects Prediction Model for Imbalance Datasets Using Machine Learning Techniques: (S-SVM Model),” J. Auton. Intell., vol. 6, no. 1, pp. 1–19, Jun. 2023, doi: 10.32629/jai.v6i1.559.

[33] V. Teodorescu and L. Obreja Brașoveanu, “Assessing the Validity of k-Fold Cross-Validation for Model Selection: Evidence from Bankruptcy Prediction Using Random Forest and XGBoost,” Computation, vol. 13, no. 5, pp. 1–22, 2025, doi: 10.3390/computation13050127.

[34] P. Thölke et al., “Class imbalance should not throw you off balance: Choosing the right classifiers and performance metrics for brain decoding with imbalanced data,” Neuroimage, vol. 277, no. August, pp. 1–14, 2023, doi: https://doi.org/10.1016/j.neuroimage.2023.120253.

[35] T. Prexawanprasut and T. Banditwattanawong, “Improving Minority Class Recall through a Novel Cluster-Based Oversampling Technique,” Informatics, vol. 11, no. 2, pp. 1–24, May 2024, doi: 10.3390/informatics11020035.

[36] M. Imani, M. Joudaki, A. Bagheri, and H. R. Arabnia, “Why ROC-AUC Alone Is Insufficient for Highly Imbalanced Data: In-Depth Evaluation of MCC, F2-Score, H-Measure, and AUC-Based Metrics for Rare-Event Binary Classification,” Technologies, vol. 14, no. 1, pp. 1–41, 2025, doi: https://doi.org/10.3390/technologies14010054.

[37] A. A. Khan, O. Chaudhari, and R. Chandra, “A review of ensemble learning and data augmentation models for class imbalanced problems: Combination, implementation and evaluation,” Expert Syst. Appl., vol. 244, no. May, pp. 1–29, 2024, doi: https://doi.org/10.1016/j.eswa.2023.122778.

[38] Z. Sun, J. Zhang, X. Zhu, and D. Xu, “How Far Have We Progressed in the Sampling Methods for Imbalanced Data Classification ? An Empirical Study,” Electronics, vol. 12, no. 20, pp. 1–21, 2023, doi: https://doi.org/10.3390/electronics12204232.

[39] G. Husain et al., “SMOTE vs. SMOTEENN: A Study on the Performance of Resampling Algorithms for Addressing Class Imbalance in Regression Models,” Algorithms, vol. 18, no. 1, pp. 1–16, 2025, doi: 10.3390/a18010037.

[40] Y. Li, Y. Yang, P. Song, L. Duan, and R. Ren, “An improved SMOTE algorithm for enhanced imbalanced data classification by expanding sample generation space,” Sci. Rep., vol. 15, no. July, pp. 1–21, 2025, doi: https://doi.org/10.1038/s41598-025-09506-w.

[41] D. Priyanto, H. Hairani, K. Marzuki, and M. Innuddin, “Optimization of Random Forest for Health Data Classification Using PCA and K-Means SMOTE-ENN,” Eng. Technol. Appl. Sci. Res., vol. 15, no. 5, pp. 27646–27652, 2025, doi: 10.48084/etasr.12976.

[42] F. Kazerouni, A. Bayani, F. Asadi, L. Saeidi, N. Parvizi, and Z. Mansoori, “Type2 diabetes mellitus prediction using data mining algorithms based on the long-noncoding RNAs expression: A comparison of four data mining approaches,” BMC Bioinformatics, vol. 21, no. 1, pp. 1–13, 2020, doi: 10.1186/s12859-020-03719-8.

[43] D. Z. Abidin, L. Afuan, A. N. Toscany, and N. Nurhadi, “A Comprehensive Benchmarking Pipeline for Transformer-Based Sentiment Analysis using Cross-Validated Metrics,” J. Tek. Inform., vol. 6, no. 4, pp. 1797–1810, 2025, doi: 10.52436/1.jutif.2025.6.4.4894.