Benchmarking Transformer Architectures for Chest X-ray Classification

Authors

  • Joshua Pinem Telkom University
  • Widi Astuti Telkom University
  • Adiwijaya Telkom University

DOI:

https://doi.org/10.29207/resti.v10i1.7132

Keywords:

Chest X-ray, classification, deep learning, lung diseases, transformer, vision transformer, ViT

Abstract

Lung diseases remain a major global health concern, necessitating accurate and timely diagnosis. Chest X-ray (CXR) imaging is widely used but challenging to interpret due to overlapping radiographic features and subjective variability among radiologists. Deep learning approaches, particularly Convolutional Neural Networks (CNNs), have shown promise but are limited in capturing global spatial dependencies. Vision Transformers (ViTs) overcome this limitation through self-attention, making them increasingly attractive for medical image analysis. This study systematically evaluates 13 Transformer-based architectures across three CXR datasets with distinct tasks: Pneumonia (3-class: Normal, Bacterial, Viral), COVID-QU-Ex (3-class: Normal, Non-COVID Pneumonia, COVID-19), and Tuberculosis (2-class: Normal, Tuberculosis). All models were trained under a unified setup with consistent preprocessing, augmentation, and evaluation protocols. To improve robustness, a soft voting ensemble of the top five models was also implemented. Results demonstrate that Transformer-based models provide highly competitive performance. On the Pneumonia dataset, the ensemble achieved an accuracy of 0.8743 and F1-score of 0.8615, surpassing several single models such as DeiT-Base (F1 = 0.8725). On COVID-QU-Ex, the ensemble soft voting obtained 0.9593 accuracy and 0.9582 F1-score, effectively balancing precision and recall. On Tuberculosis, ViT-B/16 and MobileViT-S achieved perfect performance (F1 = 1.0), likely influenced by dataset imbalance. These findings highlight the clinical potential of Transformer-based models, particularly when combined through ensembles, for robust and accurate CXR classification.

Downloads

Download data is not yet available.

References

S. Safiri et al., “Global burden of lower respiratory infections during the last three decades,” Front Public Health, vol. 10, Jan. 2023, doi: 10.3389/fpubh.2022.1028525.

World Health Organization (WHO), “Pneumonia in Children.” Accessed: Jul. 11, 2025. [Online]. Available: https://www.who.int/news-room/fact-sheets/detail/pneumonia

A. Moussaid et al., “On the Implementation of a Post-Pandemic Deep Learning Algorithm Based on a Hybrid CT-Scan/X-ray Images Classification Applied to Pneumonia Categories,” 2023, doi: 10.3390/healthcare11050662.

T. Chen et al., “A vision transformer machine learning model for COVID-19 diagnosis using chest X-ray images,” Healthcare Analytics, vol. 5, Jun. 2024, doi: 10.1016/j.health.2024.100332.

M. Ketfi, M. Belahcene, and S. Bourennane, “Transfer Learning Fusion and Stacked Auto-encoders for Viral Lung Disease Classification,” New Gener Comput, Nov. 2024, doi: 10.1007/s00354-024-00247-4.

D. Shen, G. Wu, and H. Il Suk, “Deep Learning in Medical Image Analysis,” Annu Rev Biomed Eng, vol. 19, pp. 221–248, Jun. 2017, doi: 10.1146/annurev-bioeng-071516-044442.

S. M. Anwar, M. Majid, A. Qayyum, M. Awais, M. Alnowami, and M. K. Khan, “Medical Image Analysis using Convolutional Neural Networks: A Review,” May 2019, doi: 10.1007/s10916-018-1088-1.

P. Rajpurkar et al., “CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning,” no. Computer Vision and Pattern Recognition, Dec. 2017, doi: https://doi.org/10.48550/arXiv.1711.05225

B. Mustapha, Y. Zhou, C. Shan, and Z. Xiao, “Enhanced Pneumonia Detection in Chest x-rays using Hybrid Convolutional and Vision Transformer Networks,” Curr Med Imaging Rev, vol. 21, Jan. 2025, doi: 10.2174/0115734056326685250101113959.

A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” Oct. 2020, [Online]. Available: http://arxiv.org/abs/2010.11929

R. Fan and S. Bu, “Transfer-Learning-Based Approach for the Diagnosis of Lung Diseases from Chest X-ray Images,” Entropy, vol. 24, no. 3, Mar. 2022, doi: 10.3390/e24030313.

K. Kawadkar, “Comparative Analysis of Vision Transformers and Convolutional Neural Networks for Medical Image Classification,” Jul. 2025, [Online]. Available: http://arxiv.org/abs/2507.21156

T. Sanida and M. Dasygenis, “A novel lightweight CNN for chest X-ray-based lung disease identification on heterogeneous embedded system,” Applied Intelligence, vol. 54, no. 6, pp. 4756–4780, Mar. 2024, doi: 10.1007/s10489-024-05420-2.

M. S. Al Reshan et al., “Detection of Pneumonia from Chest X-ray Images Utilizing MobileNet Model,” Healthcare (Switzerland), vol. 11, no. 11, Jun. 2023, doi: 10.3390/healthcare11111561.

P. Guddati, S. Dash, and R. K. Tripathy, “FPGA Implementation of the Proposed DCNN Model for Detection of Tuberculosis and Pneumonia Using CXR Images,” IEEE Embed Syst Lett, 2024, doi: 10.1109/LES.2024.3370833.

A. Elrefaei, A. Hossam are, F. A. Mostafa, L. A. Elrefaei, M. M. Fouda, and A. Hossam, “Enhancing Diagnostic Accuracy With Ensemble Techniques: Detecting Covid-19 And Pneumonia On Chest X-Ray Images,” 2024, doi: https://doi.org/10.5455/jjcit.71-1716799666

M. Pal et al., “A comparative analysis of the binary and multiclass classified chest X-ray images of pneumonia and COVID-19 with ML and DL models,” Open Medicine (Poland), vol. 20, no. 1, Jan. 2025, doi: 10.1515/med-2024-1110.

Z. Liang, Z. Xue, S. Rajaraman, and S. Antani, “Automated quantification of SARS-CoV-2 pneumonia with large vision model knowledge adaptation,” New Microbes New Infect, vol. 62, Dec. 2024, doi: 10.1016/j.nmni.2024.101457.

S. Singh, M. Kumar, A. Kumar, B. K. Verma, K. Abhishek, and S. Selvarajan, “Efficient pneumonia detection using Vision Transformers on chest X-rays,” Sci Rep, vol. 14, no. 1, Dec. 2024, doi: 10.1038/s41598-024-52703-2.

T. Wang et al., “PneuNet: deep learning for COVID-19 pneumonia diagnosis on chest X-ray image analysis using Vision Transformer,” Med Biol Eng Comput, vol. 61, no. 6, pp. 1395–1408, Jun. 2023, doi: 10.1007/s11517-022-02746-2.

J. Ko, S. Park, and H. G. Woo, “Optimization of vision transformer-based detection of lung diseases from chest X-ray images,” BMC Med Inform Decis Mak, vol. 24, no. 1, Dec. 2024, doi: 10.1186/s12911-024-02591-3.

V. Padmavathi and K. Ganesan, “Metaheuristic optimizers integrated with vision transformer model for severity detection and classification via multimodal COVID-19 images,” Sci Rep, vol. 15, no. 1, Dec. 2025, doi: 10.1038/s41598-025-98593-w.

M. Pal, S. Parija, and G. Panda, “An effective ensemble approach for classification of chest X-ray images having symptoms of COVID: A precautionary measure for the COVID-19 subvariants,” e-Prime - Advances in Electrical Engineering, Electronics and Energy, vol. 8, Jun. 2024, doi: 10.1016/j.prime.2024.100547.

B. Oltu, S. Güney, S. E. Yuksel, and B. Dengiz, “Automated classification of chest X-rays: a deep learning approach with attention mechanisms,” BMC Med Imaging, vol. 25, no. 1, Dec. 2025, doi: 10.1186/s12880-025-01604-5.

E. Selvano, A. Y. Paulindino, G. N. Elwirehardja, and B. Pardamean, “Evaluating Self-Supervised Pre-Trained Vision Transformer On Imbalanced Data For Lung Disease Classification,” ICIC Express Letters, Part B: Applications, vol. 15, no. 1, pp. 83–89, Jan. 2024, doi: 10.24507/icicelb.15.01.83.

S. Kadry et al., “COVID19 detection in chest x-ray using vision-transformer with different patch dimensions,” in Procedia Computer Science, Elsevier B.V., 2024, pp. 3438–3446. doi: 10.1016/j.procs.2024.04.324.

H. Yang, L. Wang, Y. Xu, and X. Liu, “CovidViT: a novel neural network with self-attention mechanism to detect Covid-19 through X-ray images,” International Journal of Machine Learning and Cybernetics, vol. 14, no. 3, pp. 973–987, Mar. 2023, doi: 10.1007/s13042-022-01676-7.

L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, and A. Gulin, “CatBoost: unbiased boosting with categorical features,” Jan. 2019, [Online]. Available: http://arxiv.org/abs/1706.09516

A. M. Tahir et al., “COVID-19 infection localization and severity grading from chest X-ray images,” Comput Biol Med, vol. 139, Dec. 2021, doi: 10.1016/j.compbiomed.2021.105002.

T. Rahman et al., “Reliable Tuberculosis Detection Using Chest X-Ray With Deep Learning, Segmentation and Visualization,” IEEE Access, vol. 8, pp. 191586–191601, 2020, doi: 10.1109/ACCESS.2020.3031384.

R. Yulvina et al., “Hybrid Vision Transformer and Convolutional Neural Network for Multi-Class and Multi-Label Classification of Tuberculosis Anomalies on Chest X-Ray,” Computers, vol. 13, no. 12, p. 343, Dec. 2024, doi: 10.3390/computers13120343.

Y. Hadhoud et al., “From Binary to Multi-Class Classification: A Two-Step Hybrid CNN-ViT Model for Chest Disease Classification Based on X-Ray Images,” Diagnostics, vol. 14, no. 23, Dec. 2024, doi: 10.3390/diagnostics14232754.

J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” May 2016, [Online]. Available: http://arxiv.org/abs/1506.02640

Downloads

Published

2026-02-16

Issue

Section

Computer Vision and Pattern Recognition

How to Cite

Pinem, J., Astuti, W., & Adiwijaya. (2026). Benchmarking Transformer Architectures for Chest X-ray Classification. Jurnal RESTI (Rekayasa Sistem Dan Teknologi Informasi), 10(1), 62-70. https://doi.org/10.29207/resti.v10i1.7132

Most read articles by the same author(s)