NoduViT: An Attention-Driven DeepLab–CustomViT Hybrid Model for Segmentation-Informed Multiclass Pulmonary Patch Classification

Fachruddin Fachruddin; Yovi Pratama; Errissya Rasywir; Yessi Hartiwi; Jasmir Jasmir; Hastie Audytra; Marrylinteri Istoningtyas

doi:10.29207/resti.v10i5.7124

Authors

Fachruddin Fachruddin Universitas Dinamika Bangsa
Yovi Pratama Universitas Dinamika Bangsa
Errissya Rasywir Universitas Dinamika Bangsa https://orcid.org/0000-0003-1550-4669
Yessi Hartiwi Universitas Dinamika Bangsa
Jasmir Jasmir Universitas Dinamika Bangsa https://orcid.org/0000-0003-0059-2512
Hastie Audytra Universitas Nahdlatul Ulama Sunan Giri https://orcid.org/0000-0002-1803-2911
Marrylinteri Istoningtyas Universitas Dinamika Bangsa https://orcid.org/0009-0005-8285-6519

DOI:

https://doi.org/10.29207/resti.v10i5.7124

Keywords:

classification, CNN, nodules, segmentation, VIT

Abstract

Lung cancer detection using CT imaging requires accurate lung segmentation and nodule classification. This study evaluates SegFormer configurations and hybrid CNN–Transformer models using the LIDC-IDRI dataset. Initial experiments showed that pretrained and from-scratch SegFormer models failed to detect target classes, while fine-tuning achieved IoU scores of 0.6628 and 0.6244 for the right and left lungs, respectively, with corresponding F1-scores of 0.7972 and 0.7688. However, nodule segmentation remained challenging (IoU 0), with limited improvement using a Nodule Loader (IoU 0.1487, F1 0.2589). Compared with recent models reporting Dice scores above 94%, SegFormer demonstrated lower segmentation performance. For classification, standalone ViT, ResNet, and DeepLab achieved accuracies of 72%, 61%, and 82%, respectively, whereas hybrid ResNet+ViT and DeepLab+ViT achieved 98% and 99.54%. Despite these improvements, class imbalance reduced minority-class nodule recall and weighted F1-scores. Overall, fine-tuned SegFormer effectively segments major lung structures but struggles with nodules, while hybrid CNN–Transformer classifiers achieve high accuracy yet require improved class balancing for reliable clinical generalization.

Downloads

Download data is not yet available.

References

[1] N. Agüloğlu, A. Aksu, D. S. Unat, and Ö. Selim Unat, “The value of PET/CT radiomic texture analysis of primary mass and mediastinal lymph node on survival in patients with non-small cell lung cancer,” Revista Española de Medicina Nuclear e Imagen Molecular (English Edition), vol. 43, no. 5, p. 500027, Sep. 2024, doi: https://doi.org/10.1016/j.remnie.2024.500027.

[2] A. Ayeni, O. Evbuomwan, and M.-D.-T. W. Vangu, “The Role of [18F]FDG PET/CT in Monitoring of Therapy Response in Lung Cancer,” Seminars in Nuclear Medicine, vol. 55, no. 2, pp. 175–189, Mar. 2025, doi: https://doi.org/10.1053/j.semnuclmed.2025.02.002.

[3] B. Cangut, R. Akinlusi, A. Mohseny, N. Ghesani, and M. Ghesani, “Evolving Paradigms in Lung Cancer: Latest Trends in Diagnosis, Management, and Radiopharmaceuticals,” Seminars in Nuclear Medicine, vol. 55, no. 2, pp. 264–276, Mar. 2025, doi: https://doi.org/10.1053/j.semnuclmed.2025.02.005.

[4] P. Agustiansyah et al., “Automated Precancerous Lesion Screening Using an Instance Segmentation Technique for Improving Accuracy,” Sensors, vol. 22, no. 15, p. 5489, Jul. 2022, doi: https://doi.org/10.3390/s22155489.

[5] A. W. Arum, S. Nurmaini, D. P. Rini, P. Agustiansyah, and M. N. Rachmatullah, “Segmentation of Squamous Columnar Junction on VIA Images using U-Net Architecture,” ComEngApp, vol. 10, no. 3, pp. 209–219, Oct. 2021, doi: https://doi.org/10.18495/comengapp.v10i3.387.

[6] N. Duklan, S. Kumar, H. Maheshwari, R. Singh, S. D. Sharma, and S. Swami, “CNN Architectures for Image Classification: A Comparative Study Using ResNet50V2, ResNet152V2, InceptionV3, Xception, and MobileNetV2,” SSRG-IJECE, vol. 11, no. 9, pp. 11–21, Sep. 2024, doi: https://doi.org/10.14445/23488549/IJECE-V11I9P102.

[7] H. Irshad, A. Veillard, L. Roux, and D. Racoceanu, “Methods for Nuclei Detection, Segmentation, and Classification in Digital Histopathology: A Review—Current Status and Future Potential,” IEEE Reviews in Biomedical Engineering, vol. 7, pp. 97–114, 2014, doi: https://doi.org/10.1109/RBME.2013.2295804.

[8] G. Geethu Lakshmi and P. Nagaraj, “Lung cancer detection and classification using Optimized CNN features and Squeeze-Inception-ResNeXt model,” Computational Biology and Chemistry, p. 108437, Mar. 2025, doi: https://doi.org/10.1016/j.compbiolchem.2025.108437.

[9] Z. Hou et al., “Overestimation of contralateral hilar lymph node metastasis in non-metastatic non-small cell lung cancer and its predictive model: HAM,” Radiotherapy and Oncology, vol. 201, p. 110575, Dec. 2024, doi: https://doi.org/10.1016/j.radonc.2024.110575.

[10] S. S. Nair, V. N. M. Devi, and S. Bhasi, “Lung Cancer Detection from CT Images: Modified Adaptive Threshold Segmentation with Support Vector Machines and Artificial Neural Network Classifier,” Current Medical Imaging, vol. 20, Jan. 2024, doi: https://doi.org/10.2174/1573405620666230714110914.

[11] J. Park et al., “Development of a multi-modal learning-based lymph node metastasis prediction model for lung cancer,” Clinical Imaging, vol. 114, p. 110254, Oct. 2024, doi: https://doi.org/10.1016/j.clinimag.2024.110254.

[12] S. Vinjamuri and V. Pant, “Demystifying the Role of Immuno PET-CT in Non-Small Cell Lung Cancer: Clinical Value and Research Trends,” Seminars in Nuclear Medicine, vol. 55, no. 2, pp. 212–220, Mar. 2025, doi: https://doi.org/10.1053/j.semnuclmed.2025.02.004.

[13] Q. Wan, L. Bai, Z. Wang, and B. Ji, “Diagnostic performance of FAPI PET/CT for the detection of lymph node metastases in lung cancer patients: a meta-analysis,” Academic Radiology, Oct. 2024, doi: https://doi.org/10.1016/j.acra.2024.09.026.

[14] N. Velmurugan, R. Rajeswari, S. Naganjaneyulu, and A. Anupama, “Rat swarm political optimizer based deep learning approach for lung lobe segmentation and lung cancer detection using CT images,” Biomedical Signal Processing and Control, vol. 105, p. 107612, Jul. 2025, doi: https://doi.org/10.1016/j.bspc.2025.107612.

[15] L. R. Vadala, M. Das, C. Raga Madhuri, and S. Merugula, “SpiLenet based detection and severity level classification of lung cancer using CT images,” Computers and Electrical Engineering, vol. 123, p. 110036, Apr. 2025, doi: https://doi.org/10.1016/j.compeleceng.2024.110036.

[16] E. Rasywir et al., “Robust SegFormer-Based Cervical Neoplastic Lesion Segmentation Using Lesion-Specific Augmentation and Anatomical Label Optimization,” IEEE Access, vol. 13, pp. 205915–205931, 2025, doi: https://doi.org/10.1109/ACCESS.2025.3639572.

[17] F. Öztürk Ergür, A. Öztürk, Ö. Özdağ, E. Tatcı, Ö. Özmen, and A. Yılmaz, “When to Consider Invasive Lymph Node Staging in Non–Small-Cell Lung Cancer? A Novel Scoring System Utilising Metabolic Parameters in 18F-FDG PET/CT,” Archivos de Bronconeumología, vol. 60, pp. S4–S12, Oct. 2024, doi: https://doi.org/10.1016/j.arbres.2024.05.020.

[18] Y. H. Yang, H. E. Kim, B. J. Park, J. G. Lee, D. J. Kim, and C. Y. Lee, “Patterns and preoperative risk factors of occult lymph node metastasis in clinical stage I lung cancer,” Lung Cancer, vol. 202, p. 108461, Apr. 2025, doi: https://doi.org/10.1016/j.lungcan.2025.108461.

[19] P. Shyamala Bharathi and C. Shalini, “Advanced hybrid attention-based deep learning network with heuristic algorithm for adaptive CT and PET image fusion in lung cancer detection,” Medical Engineering & Physics, vol. 126, p. 104138, Apr. 2024, doi: https://doi.org/10.1016/j.medengphy.2024.104138.

[20] F. Li, W. Han, and H. Sun, “Serum miR-155–5p assists the diagnostic sensitivity and accuracy of low-dose spiral CT imaging in early lung cancer,” Molecular and Cellular Probes, vol. 78, p. 101994, Dec. 2024, doi: https://doi.org/10.1016/j.mcp.2024.101994.

[21] C. Richlitzki et al., “Advances of PET/CT in Target Delineation of Lung Cancer Before Radiation Therapy,” Seminars in Nuclear Medicine, vol. 55, no. 2, pp. 190–201, Mar. 2025, doi: https://doi.org/10.1053/j.semnuclmed.2025.02.013.

[22] N. N. Murthy and K. Thippeswamy, “TPOT with SVM hybrid machine learning model for lung cancer classification using CT image,” Biomedical Signal Processing and Control, vol. 104, p. 107465, Jun. 2025, doi: https://doi.org/10.1016/j.bspc.2024.107465.

[23] C. Raghuram and M. Thenmozhi, “Short Review on Contrastive Learning-based Segmentation Techniques for Medical Image Processing,” in 2023 International Conference in Advances in Power, Signal, and Information Technology (APSIT), Jun. 2023, pp. 290–296. doi: https://doi.org/10.1109/APSIT58554.2023.10201707.

[24] B. F. Smith, B. L. Scott, and P. J. Michaels, “Metastatic seminoma presenting in an external iliac lymph node in an asymptomatic elderly patient undergoing screening in a multi-cancer early detection study: Report of a case and review of the literature,” Human Pathology Reports, vol. 40, p. 300772, Jun. 2025, doi: https://doi.org/10.1016/j.hpr.2025.300772.

[25] Y. Li et al., “Radiomics analysis of 18F-FDG PET/CT for visceral pleural invasion in non-small cell lung cancer with pleural attachment,” Clinical Radiology, p. 106867, Mar. 2025, doi: https://doi.org/10.1016/j.crad.2025.106867.

[26] A. M. Alhassan, “Driving Training-Based Optimization- Multitask Fuzzy C-Means (DTBO-MFCM) Image Segmentation and Robust Deep Learning Algorithm for Multicenter Breast Histopathological Images,” IEEE Access, vol. 11, pp. 136350–136360, 2023, doi: https://doi.org/10.1109/ACCESS.2023.3335667.

[27] B. D. Basaran, W. Zhang, M. Qiao, B. Kainz, P. M. Matthews, and W. Bai, “LesionMix: A Lesion-Level Data Augmentation Method for Medical Image Segmentation,” Aug. 17, 2023, arXiv: 2308.09026. doi: https://doi.org/10.48550/arXiv.2308.09026.

[28] M. Khalil, A. Khalil, and A. Ngom, “A Comprehensive Study of Vision Transformers in Image Classification Tasks,” Dec. 05, 2023, arXiv: arXiv:2312.01232. doi: https://doi.org/10.48550/arXiv.2312.01232.

[29] C. Chodvadiya, N. Mahla, K. G. Singh, and K. Sharad Jadhav, “FESS Loss: Feature-Enhanced Spatial Segmentation Loss for Optimizing Medical Image Analysis,” in 2024 IEEE International Symposium on Biomedical Imaging (ISBI), May 2024, pp. 1–5. doi: https://doi.org/10.1109/ISBI56570.2024.10635209.

[30] O. Leleu et al., “Lung cancer screening by low-dose CT scan in France: final results of the DEP KP80 study after three rounds,” eBioMedicine, vol. 109, p. 105396, Nov. 2024, doi: https://doi.org/10.1016/j.ebiom.2024.105396.

[31] X. Ma, H. Song, X. Jia, and Z. Wang, “An improved V-Net lung nodule segmentation model based on pixel threshold separation and attention mechanism,” Sci Rep, vol. 14, no. 1, p. 4743, Feb. 2024, doi: https://doi.org/10.1038/s41598-024-55178-3.