Comparative Study of Machine Learning Algorithms Using Bagging and XGBoost Techniques for Breast Cancer Classification

Rully  Pramudita; Dwi Ismiyana  Putri; Bambang  Kriswantara; Vina Zahrotun  Nazah; Rahmat  Budiarto

doi:10.29207/resti.v10i4.7216

Authors

Rully Pramudita Universitas Bina Insani
Dwi Ismiyana Putri Universitas Bina Insani
Bambang Kriswantara Universitas Bina Insani
Vina Zahrotun Nazah Universitas Bina Insani
Rahmat Budiarto Al-Baha University

DOI:

https://doi.org/10.29207/resti.v10i4.7216

Keywords:

bagging, breast cancer diagnosis, ensemble learning, : Machine Learning; Agricultural Land Evaluation; Random Forest; Predictive Model; Precision Agriculture, XGBoost

Abstract

Machine learning (ML) has become an important data-driven approach for classification and prediction, including applications in medical diagnosis. Ensemble methods can improve classifier performance by combining complementary learning mechanisms. However, systematic evidence on the sequential use of Bagging and XGBoost across different classifier architectures remains limited. This study develops a staged ensemble framework in which five classifiers—Support Vector Machine (SVM), Neural Network (NN), Logistic Regression (LR), Decision Tree (DT), and K-Nearest Neighbours (KNN)—are first optimized through Bagging and subsequently enhanced using XGBoost. The experiments were conducted on the Breast Cancer Wisconsin (Diagnostic) dataset under a consistent 70:30 train–test protocol. Performance was assessed using accuracy, confusion matrices, ROC curves, and Area Under the Curve (AUC), while repeated experiments were used to examine statistical significance. The results show that the staged Bagging–XGBoost approach improves both predictive accuracy and class discrimination across the evaluated classifier types. Neural Network achieved the largest improvement, with mean accuracy increasing from 93.1% to 97.0% across repeated experiments. The findings demonstrate that the sequential framework can improve non-tree-based as well as tree-based classifiers, providing empirical evidence for broader use of staged ensemble integration in breast cancer classification.

Downloads

Download data is not yet available.

References

[1] Q. H. Hoang, L. M. Duong, P. L. L. Bui, A. V. Tran, T. A. Nguyen, and V. D. Nguyen, “A Comparative Study of Machine Learning Algorithms for Breast Cancer Classification,” in 2023 International Conference on Advanced Technologies for Communications (ATC), 2023, pp. 409–414. doi: 10.1109/ATC58710.2023.10318887.

[2] H. Chen, N. Wang, X. Du, K. Mei, Y. Zhou, and G. Cai, “Classification Prediction of Breast Cancer Based on Machine Learning,” Comput. Intell. Neurosci., vol. 2023, no. 1, 2023, doi: 10.1155/2023/6530719.

[3] Fadhila Tangguh Admojo and B. S. Waluyo Poetro, “Comparative Study on the Performance of the Bagging Algorithm in the Breast Cancer Dataset,” International Journal of Artificial Intelligence in Medical Issues, vol. 1, no. 1, pp. 36–44, 2023, doi: 10.56705/ijaimi.v1i1.87.

[4] M. Dianati-nasab and K. Salimifard, “Machine learning algorithms to uncover risk factors of breast cancer : insights from a large case-control study,” Front Oncol, no. February, pp. 1–13, 2024, doi: 10.3389/fonc.2023.1276232.

[5] R. Pramudita, N. Safitri, and V. Z. Nazah, “Studi Komparatif Algoritma Machine Learning dengan Teknik Bagging dan AdaBoost pada Klasifikasi Kanker Payudara,” Tematik, vol. 12, no. 1, pp. 101–108, 2025, doi: 10.38204/tematik.v12i1.2435.

[6] R. Pramudita, Solikin, and N. Safitri, “Optimization Analysis of Neural Network Algorithms Using Bagging Techniques on Classification of Date Fruit Types,” in Seventh International Conference on Informatics and Computing (ICIC), Denpasar: IEEE, 2022. [Online]. Available: https://ieeexplore.ieee.org/document/10006986

[7] A. Zizaan and A. Idri, “Evaluating and comparing bagging and boosting of hybrid learning for breast cancer screening,” Sci. Afr., vol. 23, no. November 2023, p. e01989, 2024, doi: 10.1016/j.sciaf.2023.e01989.

[8] M. Rahman, K. H. Kobir, S. Akther, and A. H. Kallol, “Ensemble Machine Learning for Enhanced Breast Cancer Prediction : A Comparative Study,” International Journal of Advanced Computer Science and Applications (IJACSA), vol. 15, no. 7, pp. 932–941, 2024, doi: 10.14569/IJACSA.2024.0150792

[9] M. Dutta, K. M. Hasan, A. Akter, and H. Rahman, “An interpretable machine learning-based breast cancer classification using XGBoost , SHAP , and LIME,” Bulletin of Electrical Engineering and Informatics, vol. 13, no. 6, pp. 4306–4315, 2024, doi: 10.11591/eei.v13i6.7866.

[10] M. Nhu, U. Le, J. Zhou, D. Phu, C. Le, and D. Wang, “Ensemble of XGBoost Classifiers Based on LDA Dimensionality Reduction for Predicting Breast Cancer,” Universal Journal of Public Health, vol. 12, no. 3, pp. 434–440, 2024, doi: 10.13189/ujph.2024.120302.

[11] S. Y. Prasetyo, “Synergizing Ensemble Techniques for Precision Breast Cancer Diagnosis: A Comparative Performance Analysis,” Proceeding of the IEEE International Conference on Smart Instrumentation, Measurement and Applications, ICSIMA, no. 2024, pp. 297–302, 2024, doi: 10.1109/ICSIMA62563.2024.10675541.

[12] D. T. Avlani, M. B. Abhijna, G. S. Sai Disha, A. Kodipalli, and T. Rao, “Comprehensive Methodologies for Breast Cancer Classification: Leveraging XAI LIME, SHAP Bagging Boosting, and Diverse Single Classifiers,” 2024 4th Asian Conference on Innovation in Technology, ASIANCON 2024, pp. 1–6, 2024, doi: 10.1109/ASIANCON62057.2024.10837893.

[13] A. Asselman, M. Khaldi, and S. Aammou, “Enhancing the prediction of student performance based on the machine learning XGBoost algorithm,” Interactive Learning Environments, vol. 31, no. 6, pp. 3360–3379, Aug. 2023, doi: 10.1080/10494820.2021.1928235.

[14] X. Deng et al., “Bagging–XGBoost algorithm based extreme weather identification and short-term load forecasting model,” Energy Reports, vol. 8, pp. 8661–8674, 2022, doi: 10.1016/j.egyr.2022.06.072.

[15] O. Vigo, S. Lukas, D. Krisnadi, P. Widjaja, and P. Korespondensi, “Comparison Of Bagging And Adaboost Performance For Multi-Class Data Classification,” Journal Information System Development (ISD), vol. 7, no. 2, pp. 66-69, 2022, doi: 10.19166/isd.v7i2.547

[16] N. H. A. Malek, W. F. W. Yaacob, Y. B. Wah, S. A. Md Nasir, N. Shaadan, and S. W. Indratno, “Comparison of ensemble hybrid sampling with bagging and boosting machine learning approach for imbalanced data,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 29, no. 1, pp. 598–608, 2023, doi: 10.11591/ijeecs.v29.i1.pp598-608.

[17] M. Niazkar et al., “Applications of XGBoost in water resources engineering: A systematic literature review (Dec 2018–May 2023),” Environmental Modelling & Software, vol. 174, p. 105971, 2024, doi: https://doi.org/10.1016/j.envsoft.2024.105971.

[18] J. Li et al., “Application of XGBoost algorithm in the optimization of pollutant concentration,” Atmos. Res., vol. 276, p. 106238, 2022, doi: https://doi.org/10.1016/j.atmosres.2022.106238.

[19] A. Plaia, S. Buscemi, J. Fürnkranz, and E. L. Mencía, “Comparing Boosting and Bagging for Decision Trees of Rankings,” J. Classif., vol. 39, no. 1, pp. 78–99, 2022, doi: 10.1007/s00357-021-09397-2.

[20] C. Zhao, R. Peng, and D. Wu, “Bagging and Boosting Fine-Tuning for Ensemble Learning,” IEEE Transactions on Artificial Intelligence, vol. 5, no. 4, pp. 1728–1742, 2024, doi: 10.1109/TAI.2023.3296685.

[21] S. Demir and E. K. Sahin, “An investigation of feature selection methods for soil liquefaction prediction based on tree-based ensemble algorithms using AdaBoost, gradient boosting, and XGBoost,” Neural Comput. Appl., vol. 35, no. 4, pp. 3173–3190, 2023, doi: 10.1007/s00521-022-07856-4.

[22] T. Srinivas et al., “Novel Based Ensemble Machine Learning Classifiers for Detecting Breast Cancer,” Mathematical Problems in Engineering, Mathematical Problems in Engineering, vol. 2022, no. 1, 2022, doi: 10.1155/2022/9619102.

[23] U. Naseem et al., “An Automatic Detection of Breast Cancer Diagnosis and Prognosis Based on Machine Learning Using Ensemble of Classifiers,” IEEE Access, vol. 10, no. April, pp. 78242–78252, 2022, doi: 10.1109/ACCESS.2022.3174599.

[24] A. K. M. R. H. Rafid, S. Azam, S. Montaha, A. Karim, K. U. Fahim, and Z. Hasan, “An Effective Ensemble Machine Learning Approach to Classify Breast Cancer Based on Feature Selection and Lesion Segmentation Using Preprocessed Mammograms,” Biology, vol. 11, no. 11, pp. 1964, 2022, doi: 10.3390/biology11111654

[25] T. R. Mahesh, V. V. Kumar, V. Muthukumaran, H. K. Shashikala, B. Swapna, and S. Guluwadi, “Performance Analysis of XGBoost Ensemble Methods for Survivability with the Classification of Breast Cancer,” Journal of Sensors, vol. 2022, no. 1, 2022, doi: 10.1155/2022/4649510.

[26] M. Umer et al., “Breast Cancer Detection Using Convoluted Features and Ensemble Machine Learning Algorithm,” Cancers (Basel), vol. 14, no. 23, pp. 1–18, 2022, doi: 10.3390/cancers14236015