Comparative Analysis of CNN and Vision Transformer for Alzheimer’s Prediction via Cross-Dataset Validation

Jasman Pardede; Hilman  Syawaluddin; Muhammad Rama  Reyswara; Asep Nana  Hermana; Marisa  Premitasari

doi:10.29207/resti.v10i5.7532

Authors

Jasman Pardede Institut Teknologi Nasional Bandung
Hilman Syawaluddin Universitas Padjadjaran
Muhammad Rama Reyswara Institut Teknologi Nasional Bandung
Asep Nana Hermana Institut Teknologi Nasional Bandung
Marisa Premitasari Institut Teknologi Nasional Bandung

DOI:

https://doi.org/10.29207/resti.v10i5.7532

Keywords:

alzheimer's disease, cross-dataset validation, deep learning, grid search, MRI classification

Abstract

The automated detection of Alzheimer's Disease (AD) is increasingly critical for global healthcare systems, as conventional diagnostic pathways remain subjective, resource-intensive, and poorly scalable to resource-constrained clinical settings such as those prevalent in Indonesia. This study presents a rigorous comparative evaluation of three deep learning architectures - a custom Convolutional Neural Network (CNN), ResNet-50, and Vision Transformer (ViT/B-16)- for binary AD classification from Axial T1-weighted MRI. To assess genuine generalizability rather than in-distribution performance, a cross-dataset validation strategy was adopted: models were trained on the public dataset, i.e., the Falah/HuggingFace dataset, and then evaluated on the private dataset collected from Rumah Sakit Hasan Sadikin (RSHS), Bandung, Indonesia. To eliminate hyperparameter bias as a confounding variable, a systematic Grid Search across 27 configurations was conducted, combining three optimizers (Adam, SGD, and AdamW) with three learning rates (0.001, 0.0001, and 0.00001) per architecture. ResNet-50 with SGD at lr = 0.0001 emerged as the optimal configuration, achieving the highest internal accuracy of 0.989 (F1 = 0.980) and retaining a robust external accuracy of 0.823 (F1 = 0.825), surpassing all six prior benchmarked studies, none of which conducted external clinical validation. ViT exhibited severe domain sensitivity, collapsing from 0.985 internal to 0.617 external accuracy, a 37.36% degradation attributable to attention miscalibration under distribution shift. The Custom CNN achieved a moderate external accuracy of 0.653, outperforming ViT by 5.83% and confirming the structural advantage of convolutional inductive biases in limited-data regimes. These findings indicate that CNN-based architectures offer a safer, computationally more efficient, and clinically applicable solution for AD diagnosis compared to Transformer models, which rely heavily on large amounts of data.

Downloads

Download data is not yet available.

References

[1] Z. Liu, H. Mao, C. W. Christoph, F. Trevor, D. Saining, and U. C. Berkeley, “A ConvNet for the 2020s,” pp. 11976–11986, 2020, doi:10.48550/arXiv.2201.03545.

[2] L. Alzubaidi et al., Review of deep learning : concepts , CNN architectures , challenges , applications , future directions. Springer International Publishing, 2021. doi10.1186/s40537-021-00444-8.

[3] C. Peng, “Comprehensive Analysis of the Impact of Learning Rate and Dropout Rate on the Performance of Convolutional Neural Networks on the CIFAR-10 Dataset,” Appl. Comput. Eng., vol. 102, no. 1, pp. 183–192, 2024, doi: 10.54254/2755-2721/102/20241161.

[4] Eisai, “Clinical Dementia Rating – Sum of Boxes (CDR-SB): Scoring and Interpretation, Alzheimer’s Disease Education and Referral Center, National Institute on Aging,” Available, 2026, [Online]. Available: https://www.understandingalzheimersdisease.com/-/media/5BAEEF2C65894E7A89BFB0F9087688F3.ashx

[5] J. M. Amine and M. Mourad, “Toward accurate Alzheimer ’ s detection : transfer learning with ResNet50 for MRI-based diagnosis,” Front. Neurosci., vol. 2, no. 2, pp. 1–14, 2025, doi: 10.3389/fnins.2025.1664418.

[6] D. R. Hess, “How to Write an Effective Discussion,” Respir. Care, pp. 1771–1774, 2023, doi: 10.4187/respcare.11435.

[7] Y. Çetİn-kaya, “A Novel Deep Learning Architecture Optimization for Multiclass Classification of Alzheimer ’ s Disease Level,” IEEE Access, vol. 12, no. March, 2024, doi: 10.1109/ACCESS.2024.3382947

[8] J. Cohen, S. Kaur, Y. Li, J. Z. Kolter, and A. Talwalkar, “Gradient Descent On Neural Networks Typi- Cally Occurs At The Edge Of Stability,” arXiv Prepr. arXiv2103.00065, pp. 1–84, 2021, doi: 10.48550/arXiv.2103.00065.

[9] Alexey Dosovitskiy, “An Image Is Worth 16x16 Words: Transformers For Image Recognition At Scale,” arXiv Prepr. arXiv2010.11929, 2021, doi: 10.48550/arXiv.2010.11929.

[10] Y. Kong, X. Ma, and C. Wen, “A New Method of Deep Convolutional Neural Network Image Classification Based on Knowledge Transfer in Small Label,” Sensors, vol. 22, no. 3, p. 898, 2022, doi: 10.3390/s22030898.

[11] S. Jamil and J. Piran, “A Comprehensive Survey of Transformers for Computer Vision,” Drones, vol. 7, no. 5, pp. 1–27, 2023, doi: 10.3390/drones7050287.

[12] G. Hinton, “Dropout : A Simple Way to Prevent Neural Networks from Overfitting,” J. Mach. Learn. Res., vol. 15, no. 1, pp. 1929–1958, 2014, [Online]. Available: https://www.researchgate.net/publication/286794765_Dropout_A_Simple_Way_to_Prevent_Neural_Networks_from_Overfitting

[13] J. Jepkoech, D. M. Mugo, B. K. Kenduiywo, and E. C. Too, “The Effect of Adaptive Learning Rate on the Accuracy of Neural Networks,” Int. J. Adv. Comput. Sci. Appl., vol. 12, no. 8, pp. 736–751, 2021, doi:10.14569/IJACSA.2021.0120885.

[14] A. Mahavar, A. Patel, and A. Patel, “A comprehensive review on deep learning techniques for Alzheimer’s disease classification,” Multimed. Tools Appl., vol. 25, no. 4, pp. 335–349, 2023, doi: 10.2174/0115680266310776240524061252.

[15] M. H. Al-Adhaileh, “Alzheimer’s disease stage prediction using a novel transfer learning-Alzheimer’s network architecture,” Indones. J. Electr. Eng. Comput. Sci., 2025. doi:10.11591/ijeecs.v40.i1.pp518-529.

[16] G. A. Sandag, E. Djamal, G. Morris, W. Tangka, and S. W. Taju, “MRI Image Analysis for Alzheimer ’ s Disease Detection Using Transfer Learning : VGGNet vs . EfficientNet,” CogITo Smart J., vol. 10, no. 2, pp. 580–592, 2024, doi: 10.31154/cogito.v10i2.836.580-592.

[17] S. L. Smith, B. Dherin, D. G. T. Barrett, and S. De, “On The Origin Of Implicit Regularization In Stochastic Gradient Descent,” arXiv Prepr. arXiv2101.12176, pp. 1–14, 2021, doi: 10.48550/arXiv.2101.12176.

[18] R. M. Schmidt, F. Schneider, and P. Hennig, “Descending through a Crowded Valley - Benchmarking Deep Learning Optimizers,” Int. Conf. Mach. Learn., pp. 9367–9367, 2021, doi: 10.48550/arXiv.2007.01547.

[19] N. J. Tustison and J. C. Gee, “N4ITK : Nick ’ s N3 ITK Implementation For MRI Bias Field Correction,” Insight J., vol. 9, pp. 1–8, 2009, doi: 10.54294/jculxw.

[20] H. Touvron, M. Cord, M. Douze, F. Massa, and A. Sablayrolles, “Training data-efficient image transformers & distillation through attention,” Int. Conf. Mach. Learn., 2021, doi: 10.48550/arXiv.2012.12877.

[21] L. Wen, “A transfer convolutional neural network for fault diagnosis based on ResNet-50,” Neural Comput. Appl., vol. 32, pp. 6111–6124, 2020, doi: 10.1007/s00521-019-04097-w.

[22] G. Varoquaux and V. Cheplygina, “Machine learning for medical imaging : methodological failures and recommendations for the future,” NPJ Digit. Med., vol. 5, no. 48, 2022, doi: 10.1038/s41746-022-00592-y.

[23] Z. Gao and Y. Zhang, “Research on the application of convolutional neural networks in image recognition,” Int. J. Adv. Netw. Monit. Control., vol. 5, no. 3, pp. 31–36, 2020, doi: 10.21307/ijanmc-2020-014

[24] J. Maurício, I. Domingues, and J. Bernardino, “Comparing vision transformers and convolutional neural networks for image classification: A literature review,” Appl. Sci., vol. 13, no. 9, p. 5521, 2023, doi: 10.3390/app13095521.

[25] A. A. A. El-latif, S. A. Chelloug, M. Alabdulhafith, and M. Hammad, “Accurate Detection of Alzheimer ’ s Disease Using Lightweight Deep Learning Model on MRI Data,” Diagnostics, vol. 13, no. 7, pp. 1–21, 2023, doi: 10.3390/diagnostics13071216.

[26] A. C. Yu, B. Mohajer, and J. Eng, “External validation of deep learning algorithms for radiologic diagnosis: a systematic review,” Radiol. Artif. Intell., vol. 4, no. 3, p. e210064, 2022, doi: 10.1148/ryai.210064.

[27] A. Mortazi, V. Cicek, E. Keles, and U. Bagci, “Selecting the best optimizers for deep learning–based medical image segmentation,” Front. Radiol., no. September, pp. 1–17, 2023, doi: 10.3389/fradi.2023.1175473.

[28] T. Mahmud, K. Barua, S. U. Habiba, N. Sharmen, M. S. Hossain, and K. Andersson, “An explainable AI paradigm for alzheimer’s diagnosis using deep transfer learning,” Diagnostics, vol. 14, no. 3, p. 345, 2024, doi: 10.3390/diagnostics14030345.

[29] Z. Li, W. Yang, S. Peng, and F. Liu, “A Survey of Convolutional Neural Networks : Analysis , Applications , and Prospects,” 2023, doi: 10.1109/TNNLS.2021.3084827.

[30] C. Tinauer, M. Sackl, R. Stollberger, R. Schmidt, S. Ropele, and C. Langkammer, “Skull-stripping induces shortcut learning in MRI-based Alzheimer’s disease classification,” Insights Imaging, vol. 16, no. 1, p. 283, 2025, doi: 10.1186/s13244-025-02158-4.

[31] C. Matsoukas, J. F. Haslum, S. Magnus, K. Smith, and K. T. H. Royal, “Is it Time to Replace CNNs with Transformers for Medical Images ?,” arXiv Prepr. arXiv2108.09038, 2024, doi: 10.48550/arXiv.2108.09038.