Multi-Stage Feature Fusion for Breast Cancer Histopathology using EfficientNetV2 and Vision Transformers
DOI:
https://doi.org/10.29207/resti.v10i5.7769Keywords:
BCH, deep learning, EfficientNetV2, feature fusion, vision transformerAbstract
Early detection of breast cancer is essential for improving patient survival and reducing mortality. Although histopathology-based diagnosis provides definitive clinical evidence, the complex and heterogeneous nature of breast cancer histopathology (BCH) images poses significant challenges for accurate classification. Conventional deep learning models often struggle to capture both local tissue characteristics and global contextual patterns simultaneously. To address this limitation, this study proposes a Multi-Stage Feature Fusion (MSFF) model that integrates EfficientNetV2, a CNN local texture branch, and a Vision Transformer (ViT). EfficientNetV2 extracts high-level semantic features, the CNN branch captures fine-grained local texture information, and the ViT models long-range contextual dependencies through self-attention. These complementary representations are fused through an adaptive multi-stage fusion strategy to enhance discriminative capability for binary BCH classification. An ablation study is conducted to systematically evaluate the contribution of each architectural component. Experimental results on the BreakHis dataset demonstrate that the proposed model achieves the best overall balanced classification performance, with an accuracy of 93.04%, precision of 93.08%, recall of 92.69%, F1-score of 92.89%, and an AUC of 0.9303 at the optimal 200× magnification level. The findings confirm that the proposed fusion strategy provides balanced classification performance, stable optimization behaviour, and strong generalization capability, effectively addressing BCH image heterogeneity and offering strong potential for clinical decision-support applications.Downloads
References
[1] A. Bouziane, S. Boumali, N. Berkane, and F. S. Guendouz, “A Hybrid Approach for Automatic Breast Cancer Detection,” in 2020 International Conference on e-Health and Bioengineering (EHB), Oct. 2020, pp. 1–4. doi: https://doi.org/10.1109/EHB50910.2020.9280280.
[2] A. J. Shephard et al., “An Automated Pipeline for Tumour-Infiltrating Lymphocyte Scoring in Breast Cancer,” in 2024 IEEE International Symposium on Biomedical Imaging (ISBI), May 2024, pp. 1–5. doi: https://doi.org/10.1109/ISBI56570.2024.10635302.
[3] A. J. Suresh and L. S. Nair, “Mitosis Detection In Breast Histopathology Image Using Ensemble Features Fed CNN Model,” in 2023 14th International Conference on Computing Communication and Networking Technologies (ICCCNT), Jul. 2023, pp. 1–5. doi: https://doi.org/10.1109/ICCCNT56998.2023.10307867.
[4] A. Labrada and B. D. Barkana, “Breast Cancer Diagnosis from Histopathology Images using Supervised Algorithms,” in 2022 IEEE 35th International Symposium on Computer-Based Medical Systems (CBMS), Jul. 2022, pp. 102–107. doi: https://doi.org/10.1109/CBMS55023.2022.00025.
[5] Y. Zhang, K. Xia, C. Li, B. Wei, and B. Zhang, “Review of Breast Cancer Pathologigcal Image Processing,” BioMed Research International, vol. 2021, no. 1, p. 1994764, Jan. 2021, doi: https://doi.org/10.1155/2021/1994764.
[6] Y. Feng, L. Zhang, and J. Mo, “Deep Manifold Preserving Autoencoder for Classifying Breast Cancer Histopathological Images,” IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 17, no. 1, pp. 91–101, Feb. 2020, doi: https://doi.org/10.1109/TCBB.2018.2858763.
[7] Y. Benmabrouk, M. Gasmi, H. Bendjenna, and A. Nadjah, “Semantic segmentation of breast cancer histopathology images using deep learning,” in 2022 4th International Conference on Pattern Analysis and Intelligent Systems (PAIS), Oct. 2022, pp. 1–7. doi: https://doi.org/10.1109/PAIS56586.2022.9946874.
[8] M. J. Hasan et al., “Nuclei Segmentation in ER-IHC Stained Histopathology Images using Mask R-CNN,” in 2022 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS), Nov. 2022, pp. 1–4. doi: https://doi.org/10.1109/ISPACS57703.2022.10082832.
[9] S. Iqbal and A. N. Qureshi, “A Heteromorphous Deep CNN Framework for Medical Image Segmentation Using Local Binary Pattern,” IEEE Access, vol. 10, pp. 63466–63480, 2022, doi: https://doi.org/10.1109/ACCESS.2022.3183331.
[10] T. P. T. Armand, S. Bhattacharjee, H. -K. Choi, and H. -C. Kim, “Transformers Effectiveness in Medical Image Segmentation: A Comparative Analysis of UNet-Based Architectures,” in 2024 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), Feb. 2024, pp. 238–242. doi: https://doi.org/10.1109/ICAIIC60209.2024.10463435.
[11] M. A. Ortega-Ruíz, C. Karabağ, E. Roman-Rangel, and C. C. Reyes-Aldasoro, “DRD-UNet, a UNet-Like Architecture for Multi-Class Breast Cancer Semantic Segmentation,” IEEE Access, vol. 12, pp. 40412–40424, 2024, doi: https://doi.org/10.1109/ACCESS.2024.3377428.
[12] L. Zhang, Z. Yang, M. Ju, and H. Liu, “MTC-TransUNet: A Multi-Scale Mixed Convolution TransUNet for Medical Image Segmentation,” in 2023 16th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI), Oct. 2023, pp. 1–7. doi: https://doi.org/10.1109/CISP-BMEI60920.2023.10373226.
[13] J. Chen et al., “TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation,” Feb. 08, 2021, arXiv: 2102.04306. doi: https://doi.org/10.48550/arXiv.2102.04306.
[14] A. M. Alhassan, “Driving Training-Based Optimization- Multitask Fuzzy C-Means (DTBO-MFCM) Image Segmentation and Robust Deep Learning Algorithm for Multicenter Breast Histopathological Images,” IEEE Access, vol. 11, pp. 136350–136360, 2023, doi: https://doi.org/10.1109/ACCESS.2023.3335667.
[15] L. Wang, T. Song, T. Katayama, X. Jiang, T. Shimamoto, and J. -S. Leu, “Deep Regional Metastases Segmentation for Patient-Level Lymph Node Status Classification,” IEEE Access, vol. 9, pp. 129293–129302, 2021, doi: https://doi.org/10.1109/ACCESS.2021.3113036.
[16] Z. Xing, L. Zhu, L. Yu, Z. Xing, and L. Wan, “Hybrid Masked Image Modeling for 3D Medical Image Segmentation,” IEEE Journal of Biomedical and Health Informatics, vol. 28, no. 4, pp. 2115–2125, Apr. 2024, doi: https://doi.org/10.1109/JBHI.2024.3360239.
[17] Y. Ling, Y. Wang, W. Dai, J. Yu, P. Liang, and D. Kong, “MTANet: Multi-Task Attention Network for Automatic Medical Image Segmentation and Classification,” IEEE Transactions on Medical Imaging, vol. 43, no. 2, pp. 674–685, Feb. 2024, doi: https://doi.org/10.1109/TMI.2023.3317088.
[18] Y. Al-Kofahi, W. Lassoued, W. Lee, and B. Roysam, “Improved Automatic Detection and Segmentation of Cell Nuclei in Histopathology Images,” IEEE Transactions on Biomedical Engineering, vol. 57, no. 4, pp. 841–852, Apr. 2010, doi: https://doi.org/10.1109/TBME.2009.2035102.
[19] B. D. Basaran, W. Zhang, M. Qiao, B. Kainz, P. M. Matthews, and W. Bai, “LesionMix: A Lesion-Level Data Augmentation Method for Medical Image Segmentation,” Aug. 17, 2023, arXiv: 2308.09026. doi: https://doi.org/10.48550/arXiv.2308.09026.
[20] D. DECAHNG, “Semi-Supervised Medical Image Segmentation with Multimodal Data Augmentation and Self-Supervised Learning,” in 2024 21st International Computer Conference on Wavelet Active Media Technology and Information Processing (ICCWAMTIP), Dec. 2024, pp. 1–4. doi: https://doi.org/10.1109/ICCWAMTIP64812.2024.10873760.
[21] X. Liu, K. Ono, and R. Bise, “Mixing Data Augmentation with Preserving Foreground Regions in Medical Image Segmentation,” in 2023 IEEE 20th International Symposium on Biomedical Imaging (ISBI), Apr. 2023, pp. 1–5. doi: https://doi.org/10.1109/ISBI53787.2023.10230495.
[22] C. Raghuram and M. Thenmozhi, “Short Review on Contrastive Learning-based Segmentation Techniques for Medical Image Processing,” in 2023 International Conference in Advances in Power, Signal, and Information Technology (APSIT), Jun. 2023, pp. 290–296. doi: https://doi.org/10.1109/APSIT58554.2023.10201707.
[23] H. Irshad, A. Veillard, L. Roux, and D. Racoceanu, “Methods for Nuclei Detection, Segmentation, and Classification in Digital Histopathology: A Review—Current Status and Future Potential,” IEEE Reviews in Biomedical Engineering, vol. 7, pp. 97–114, 2014, doi: https://doi.org/10.1109/RBME.2013.2295804.
[24] H. Fatakdawala et al., “Expectation–Maximization-Driven Geodesic Active Contour With Overlap Resolution (EMaGACOR): Application to Lymphocyte Segmentation on Breast Cancer Histopathology,” IEEE Transactions on Biomedical Engineering, vol. 57, no. 7, pp. 1676–1689, Jul. 2010, doi: https://doi.org/10.1109/TBME.2010.2041232.
[25] J. Albarracin, F. Cano, E. Romero, and A. Cruz-Roa, “A comparative analysis between two convolutional networks architectures for semantic segmentation of histopathology breast cancer images,” in 2023 19th International Symposium on Medical Information Processing and Analysis (SIPAIM), Nov. 2023, pp. 1–5. doi: https://doi.org/10.1109/SIPAIM56729.2023.10373477.
[26] S. Zheng et al., “Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers,” Jul. 25, 2021, arXiv: 2012.15840. doi: https://doi.org/10.48550/arXiv.2012.15840.
[27] E. Xie, W. Wang, Z. Yu, A. Anandkumar, J. M. Alvarez, and P. Luo, “SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers,” Oct. 28, 2021, arXiv: 2105.15203. doi: https://doi.org/10.48550/arXiv.2105.15203.
[28] H. Thisanke, C. Deshan, K. Chamith, S. Seneviratne, R. Vidanaarachchi, and D. Herath, “Semantic Segmentation using Vision Transformers: A survey,” May 05, 2023, arXiv: arXiv:2305.03273. doi: https://doi.org/10.48550/arXiv.2305.03273.
[29] Z. Liang, K. Zhao, G. Liang, S. Li, Y. Wu, and Y. Zhou, “MAXFormer: Enhanced transformer for medical image segmentation with multi-attention and multi-scale features fusion,” Knowledge-Based Systems, vol. 280, p. 110987, Nov. 2023, doi: https://doi.org/10.1016/j.knosys.2023.110987.
[30] J. Yang, L. Jiao, R. Shang, X. Liu, R. Li, and L. Xu, “EPT-Net: Edge Perception Transformer for 3D Medical Image Segmentation,” IEEE Transactions on Medical Imaging, vol. 42, no. 11, pp. 3229–3243, Nov. 2023, doi: https://doi.org/10.1109/TMI.2023.3278461.
[31] Z. Ji, Z. Chen, and X. Ma, “Grouped multi-scale vision transformer for medical image segmentation,” Sci Rep, vol. 15, no. 1, p. 11122, Apr. 2025, doi: https://doi.org/10.1038/s41598-025-95361-8.
[32] E. Rasywir et al., “Robust SegFormer-Based Cervical Neoplastic Lesion Segmentation Using Lesion-Specific Augmentation and Anatomical Label Optimization,” IEEE Access, vol. 13, pp. 205915–205931, 2025, doi: https://doi.org/10.1109/ACCESS.2025.3639572.
[33] E. Rasywir, S. Nurmaini, M. Fachrurrozi, S. Assegaff, M. N. Rachmatullah, and Y. Pratama, “CerviNetFormer: Leveraging Convolutional and Vision Transformer Ensembles for Enhancing Accuracy in Cervical Pre-cancer Lesion Segmentation,” IAENG IJCS, vol. 52, no. 12, pp. 4842–4853, 2025. [Online]. Available: https://www.iaeng.org/IJCS/issues_v52/issue_12/IJCS_52_12_28.pdf.
[34] S. Eskandari, A. Eslamian, and N. Munia, “Evaluating Deep Learning Models for Breast Cancer Classification: A Comparative Study,” presented at the SPIE Medical Imaging 2025, 2025. doi: https://doi.org/10.1117/12.3047441.
[35] M. M. Sani, “Classification of Breast Cancer Histopathology Images using a Modified Supervised Contrastive Learning Method,” 2024, doi: https://doi.org/10.48550/arXiv.2405.03642.
[36] M. Behzadpour, B. L. Ortiz, E. Azizi, and K. Wu, “Breast Tumor Classification Using EfficientNet Deep Learning Model,” arXiv preprint, 2024, doi: https://doi.org/10.48550/arXiv.2411.17870.
[37] M. Shiri, “Supervised Contrastive Vision Transformer for Breast Histopathological Image Classification,” 2024, doi: https://doi.org/10.48550/arXiv.2404.11052.
[38] R. Rawat and F. Ahmed, “Efficient Breast and Ovarian Cancer Classification via ViT-Based Preprocessing and Transfer Learning,” arXiv preprint, 2025, doi: https://doi.org/10.48550/arXiv.2509.18553.
[39] P. Afshin, D. Helminiak, and T. Lu, “Breast Cancer Classification in Deep Ultraviolet Fluorescence Images Using a Patch-Level Vision Transformer Framework,” presented at the IEEE EMBC 2025, 2025. doi: https://doi.org/10.1109/EMBC58623.2025.11253275.
[40] X. Zhou et al., “A Comprehensive Review for Breast Histopathology Image Analysis Using Classical and Deep Neural Networks,” IEEE Access, vol. 8, pp. 90931–90956, 2020, doi: https://doi.org/10.1109/ACCESS.2020.2993788.
[41] E. Egriboz, B. Gokcen, and G. Bilgin, “Classification of Breast Cancer Histopathology Images by Cell-Centered Deep Learning Approach,” in 2020 28th Signal Processing and Communications Applications Conference (SIU), Oct. 2020, pp. 1–4. doi: https://doi.org/10.1109/SIU49456.2020.9302066.
[42] M. V. Deepak Naik, S. Adarsh, S. Vijayakumar, A. P. Nambiar, and L. S. Nair, “Hybrid Feature Set based Mitotic Detection in Breast Histopathology Images,” in 2022 International Conference on Inventive Computation Technologies (ICICT), Jul. 2022, pp. 281–290. doi: https://doi.org/10.1109/ICICT54344.2022.9850552.
[43] Kiran, P. B D, D. R H, K. N. Abdul Sattar, and S. Kumar D S, “Breast Histopathology Image Embedding’s in Invasive Ductal Carcinoma Detection using Self-Supervised Vision Transformers Techniques,” in 2023 International Conference on Evolutionary Algorithms and Soft Computing Techniques (EASCT), Oct. 2023, pp. 1–5. doi: https://doi.org/10.1109/EASCT59475.2023.10393320.
[44] M. Son, W. Jun, and S. Lee, “Autonomous Diagnosis System of Breast Cancer,” in 2024 IEEE International Conference on Consumer Electronics (ICCE), Jan. 2024, pp. 1–6. doi: https://doi.org/10.1109/ICCE59016.2024.10444471.
[45] K. S. Gandle and D. B. Kshirsagar, “Breast Cancer Categories, Analysis, Detection: Systematic Review for Histopathological Images,” in 2024 4th International Conference on Innovative Practices in Technology and Management (ICIPTM), Feb. 2024, pp. 1–6. doi: https://doi.org/10.1109/ICIPTM59628.2024.10563307.
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright in each article belongs to the author
- The author acknowledges that the RESTI Journal (System Engineering and Information Technology) is the first publisher to publish with a license Creative Commons Attribution 4.0 International License.
- Authors can enter writing separately, arrange the non-exclusive distribution of manuscripts that have been published in this journal into other versions (eg sent to the author's institutional repository, publication in a book, etc.), by acknowledging that the manuscript has been published for the first time in the RESTI (Rekayasa Sistem dan Teknologi Informasi) journal ;
