Benchmarking Transformer Architectures for Chest X-ray Classification
DOI:
https://doi.org/10.29207/resti.v10i1.7132Keywords:
Chest X-ray, classification, deep learning, lung diseases, transformer, vision transformer, ViTAbstract
Lung diseases remain a major global health concern, necessitating accurate and timely diagnosis. Chest X-ray (CXR) imaging is widely used but challenging to interpret due to overlapping radiographic features and subjective variability among radiologists. Deep learning approaches, particularly Convolutional Neural Networks (CNNs), have shown promise but are limited in capturing global spatial dependencies. Vision Transformers (ViTs) overcome this limitation through self-attention, making them increasingly attractive for medical image analysis. This study systematically evaluates 13 Transformer-based architectures across three CXR datasets with distinct tasks: Pneumonia (3-class: Normal, Bacterial, Viral), COVID-QU-Ex (3-class: Normal, Non-COVID Pneumonia, COVID-19), and Tuberculosis (2-class: Normal, Tuberculosis). All models were trained under a unified setup with consistent preprocessing, augmentation, and evaluation protocols. To improve robustness, a soft voting ensemble of the top five models was also implemented. Results demonstrate that Transformer-based models provide highly competitive performance. On the Pneumonia dataset, the ensemble achieved an accuracy of 0.8743 and F1-score of 0.8615, surpassing several single models such as DeiT-Base (F1 = 0.8725). On COVID-QU-Ex, the ensemble soft voting obtained 0.9593 accuracy and 0.9582 F1-score, effectively balancing precision and recall. On Tuberculosis, ViT-B/16 and MobileViT-S achieved perfect performance (F1 = 1.0), likely influenced by dataset imbalance. These findings highlight the clinical potential of Transformer-based models, particularly when combined through ensembles, for robust and accurate CXR classification.
Downloads
References
S. Safiri et al., “Global burden of lower respiratory infections during the last three decades,” Front Public Health, vol. 10, Jan. 2023, doi: 10.3389/fpubh.2022.1028525.
World Health Organization (WHO), “Pneumonia in Children.” Accessed: Jul. 11, 2025. [Online]. Available: https://www.who.int/news-room/fact-sheets/detail/pneumonia
A. Moussaid et al., “On the Implementation of a Post-Pandemic Deep Learning Algorithm Based on a Hybrid CT-Scan/X-ray Images Classification Applied to Pneumonia Categories,” 2023, doi: 10.3390/healthcare11050662.
T. Chen et al., “A vision transformer machine learning model for COVID-19 diagnosis using chest X-ray images,” Healthcare Analytics, vol. 5, Jun. 2024, doi: 10.1016/j.health.2024.100332.
M. Ketfi, M. Belahcene, and S. Bourennane, “Transfer Learning Fusion and Stacked Auto-encoders for Viral Lung Disease Classification,” New Gener Comput, Nov. 2024, doi: 10.1007/s00354-024-00247-4.
D. Shen, G. Wu, and H. Il Suk, “Deep Learning in Medical Image Analysis,” Annu Rev Biomed Eng, vol. 19, pp. 221–248, Jun. 2017, doi: 10.1146/annurev-bioeng-071516-044442.
S. M. Anwar, M. Majid, A. Qayyum, M. Awais, M. Alnowami, and M. K. Khan, “Medical Image Analysis using Convolutional Neural Networks: A Review,” May 2019, doi: 10.1007/s10916-018-1088-1.
P. Rajpurkar et al., “CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning,” no. Computer Vision and Pattern Recognition, Dec. 2017, doi: https://doi.org/10.48550/arXiv.1711.05225
B. Mustapha, Y. Zhou, C. Shan, and Z. Xiao, “Enhanced Pneumonia Detection in Chest x-rays using Hybrid Convolutional and Vision Transformer Networks,” Curr Med Imaging Rev, vol. 21, Jan. 2025, doi: 10.2174/0115734056326685250101113959.
A. Dosovitskiy et al., “An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,” Oct. 2020, [Online]. Available: http://arxiv.org/abs/2010.11929
R. Fan and S. Bu, “Transfer-Learning-Based Approach for the Diagnosis of Lung Diseases from Chest X-ray Images,” Entropy, vol. 24, no. 3, Mar. 2022, doi: 10.3390/e24030313.
K. Kawadkar, “Comparative Analysis of Vision Transformers and Convolutional Neural Networks for Medical Image Classification,” Jul. 2025, [Online]. Available: http://arxiv.org/abs/2507.21156
T. Sanida and M. Dasygenis, “A novel lightweight CNN for chest X-ray-based lung disease identification on heterogeneous embedded system,” Applied Intelligence, vol. 54, no. 6, pp. 4756–4780, Mar. 2024, doi: 10.1007/s10489-024-05420-2.
M. S. Al Reshan et al., “Detection of Pneumonia from Chest X-ray Images Utilizing MobileNet Model,” Healthcare (Switzerland), vol. 11, no. 11, Jun. 2023, doi: 10.3390/healthcare11111561.
P. Guddati, S. Dash, and R. K. Tripathy, “FPGA Implementation of the Proposed DCNN Model for Detection of Tuberculosis and Pneumonia Using CXR Images,” IEEE Embed Syst Lett, 2024, doi: 10.1109/LES.2024.3370833.
A. Elrefaei, A. Hossam are, F. A. Mostafa, L. A. Elrefaei, M. M. Fouda, and A. Hossam, “Enhancing Diagnostic Accuracy With Ensemble Techniques: Detecting Covid-19 And Pneumonia On Chest X-Ray Images,” 2024, doi: https://doi.org/10.5455/jjcit.71-1716799666
M. Pal et al., “A comparative analysis of the binary and multiclass classified chest X-ray images of pneumonia and COVID-19 with ML and DL models,” Open Medicine (Poland), vol. 20, no. 1, Jan. 2025, doi: 10.1515/med-2024-1110.
Z. Liang, Z. Xue, S. Rajaraman, and S. Antani, “Automated quantification of SARS-CoV-2 pneumonia with large vision model knowledge adaptation,” New Microbes New Infect, vol. 62, Dec. 2024, doi: 10.1016/j.nmni.2024.101457.
S. Singh, M. Kumar, A. Kumar, B. K. Verma, K. Abhishek, and S. Selvarajan, “Efficient pneumonia detection using Vision Transformers on chest X-rays,” Sci Rep, vol. 14, no. 1, Dec. 2024, doi: 10.1038/s41598-024-52703-2.
T. Wang et al., “PneuNet: deep learning for COVID-19 pneumonia diagnosis on chest X-ray image analysis using Vision Transformer,” Med Biol Eng Comput, vol. 61, no. 6, pp. 1395–1408, Jun. 2023, doi: 10.1007/s11517-022-02746-2.
J. Ko, S. Park, and H. G. Woo, “Optimization of vision transformer-based detection of lung diseases from chest X-ray images,” BMC Med Inform Decis Mak, vol. 24, no. 1, Dec. 2024, doi: 10.1186/s12911-024-02591-3.
V. Padmavathi and K. Ganesan, “Metaheuristic optimizers integrated with vision transformer model for severity detection and classification via multimodal COVID-19 images,” Sci Rep, vol. 15, no. 1, Dec. 2025, doi: 10.1038/s41598-025-98593-w.
M. Pal, S. Parija, and G. Panda, “An effective ensemble approach for classification of chest X-ray images having symptoms of COVID: A precautionary measure for the COVID-19 subvariants,” e-Prime - Advances in Electrical Engineering, Electronics and Energy, vol. 8, Jun. 2024, doi: 10.1016/j.prime.2024.100547.
B. Oltu, S. Güney, S. E. Yuksel, and B. Dengiz, “Automated classification of chest X-rays: a deep learning approach with attention mechanisms,” BMC Med Imaging, vol. 25, no. 1, Dec. 2025, doi: 10.1186/s12880-025-01604-5.
E. Selvano, A. Y. Paulindino, G. N. Elwirehardja, and B. Pardamean, “Evaluating Self-Supervised Pre-Trained Vision Transformer On Imbalanced Data For Lung Disease Classification,” ICIC Express Letters, Part B: Applications, vol. 15, no. 1, pp. 83–89, Jan. 2024, doi: 10.24507/icicelb.15.01.83.
S. Kadry et al., “COVID19 detection in chest x-ray using vision-transformer with different patch dimensions,” in Procedia Computer Science, Elsevier B.V., 2024, pp. 3438–3446. doi: 10.1016/j.procs.2024.04.324.
H. Yang, L. Wang, Y. Xu, and X. Liu, “CovidViT: a novel neural network with self-attention mechanism to detect Covid-19 through X-ray images,” International Journal of Machine Learning and Cybernetics, vol. 14, no. 3, pp. 973–987, Mar. 2023, doi: 10.1007/s13042-022-01676-7.
L. Prokhorenkova, G. Gusev, A. Vorobev, A. V. Dorogush, and A. Gulin, “CatBoost: unbiased boosting with categorical features,” Jan. 2019, [Online]. Available: http://arxiv.org/abs/1706.09516
A. M. Tahir et al., “COVID-19 infection localization and severity grading from chest X-ray images,” Comput Biol Med, vol. 139, Dec. 2021, doi: 10.1016/j.compbiomed.2021.105002.
T. Rahman et al., “Reliable Tuberculosis Detection Using Chest X-Ray With Deep Learning, Segmentation and Visualization,” IEEE Access, vol. 8, pp. 191586–191601, 2020, doi: 10.1109/ACCESS.2020.3031384.
R. Yulvina et al., “Hybrid Vision Transformer and Convolutional Neural Network for Multi-Class and Multi-Label Classification of Tuberculosis Anomalies on Chest X-Ray,” Computers, vol. 13, no. 12, p. 343, Dec. 2024, doi: 10.3390/computers13120343.
Y. Hadhoud et al., “From Binary to Multi-Class Classification: A Two-Step Hybrid CNN-ViT Model for Chest Disease Classification Based on X-Ray Images,” Diagnostics, vol. 14, no. 23, Dec. 2024, doi: 10.3390/diagnostics14232754.
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, “You Only Look Once: Unified, Real-Time Object Detection,” May 2016, [Online]. Available: http://arxiv.org/abs/1506.02640
Downloads
Published
Issue
Section
License
Copyright in each article belongs to the author
- The author acknowledges that the RESTI Journal (System Engineering and Information Technology) is the first publisher to publish with a license Creative Commons Attribution 4.0 International License.
- Authors can enter writing separately, arrange the non-exclusive distribution of manuscripts that have been published in this journal into other versions (eg sent to the author's institutional repository, publication in a book, etc.), by acknowledging that the manuscript has been published for the first time in the RESTI (Rekayasa Sistem dan Teknologi Informasi) journal ;





