Speech-Driven Optotype Recognition for Visual Acuity Assessment Using Deep Learning

Dwiny Meidelfi; Hendrick Hendrick; Fanni Sukma; Ulya Ilhami Arsyah; Jeprizal Jeprizal

doi:10.29207/resti.v10i5.7538

Authors

Dwiny Meidelfi Politeknik Negeri Padang
Hendrick Hendrick Politeknik Negeri Padang
Fanni Sukma Politeknik Negeri Padang
Ulya Ilhami Arsyah Politeknik Negeri Padang
Jeprizal Jeprizal Politeknik Negeri Padang

DOI:

https://doi.org/10.29207/resti.v10i5.7538

Keywords:

CNN, MFCC, snellen chart, speech recognition, visual acuity

Abstract

Visual acuity (VA) testing is a fundamental procedure for evaluating visual function. Conventional VA examinations require an examiner to interpret patients' responses, which may limit automation and scalability in repeated or remote testing scenarios. This study develops an embedded speech-based optotype recognition system to support digital visual acuity testing workflow. Mel-Frequency Cepstral Coefficients (MFCCs) were used to represent spoken responses, and a Convolutional Neural Network (CNN) was trained to classify nine Snellen optotype letters. Voice recordings were collected from 20 participants, with data augmentation including noise injection, pitch shifting, time stretching, and random gain adjustment to increase acoustic variability. Under the evaluated experimental protocol, the proposed MFCC-CNN model achieved 97.42% accuracy, 94.89% precision, 94.33% recall, and an F1-score of 94.61%, outperforming the evaluated kNN, Random Forest, and SVM baselines. The model was deployed on an NVIDIA Jetson Orin Nano, occupying approximately 8.3 MB and achieving an inference latency of approximately 30–40 ms per prediction. These results demonstrate the technical feasibility of embedded spoken-optotype recognition for supporting visual acuity testing workflows. However, the present study evaluates speech-recognition performance rather than clinical agreement between automatically generated VA scores and clinician-administered measurements. Formal clinical validation and stricter leakage-safe evaluation are therefore required before clinical applicability can be established.

Downloads

Download data is not yet available.

References

[1] A. Fazeenah, “A literary review on visual acuity,” Internation Journal of Creative Research Thoughts, vol. 9, no. 6, 2021. https://ijcrt.org/papers/IJCRT2106378.pdf

[2] J. H. Kim et al., “A Deep Learning Ensemble Method to Visual Acuity Measurement Using Fundus Images,” Applied Sciences (Switzerland), vol. 12, no. 6, 2022, doi: 10.3390/app12063190.

[3] J. C. Wanten et al., “Evaluation of a visual acuity eHealth tool in patients with cataract,” J. Cataract Refract. Surg., vol. 49, no. 3, 2023, doi: 10.1097/j.jcrs.00000000000011088.

[4] V. K. Patil et al., “Converging emotion recognition with AI and IoT,” in Affective Computing Applications using Artificial Intelligence in Healthcare: Methods, approaches and challenges in system design, 2024. doi: 10.1049/pbhe056e_ch9.

[5] M. E. G. Gendy and M. R. Yuce, “Emerging Technologies Used in Health Management and Efficiency Improvement During Different Contact Tracing Phases Against COVID-19 Pandemic,” IEEE Rev. Biomed. Eng., vol. 16, 2023, doi: 10.1109/RBME.2022.3219433.

[6] Evangel Chinyere Anyanwu, Femi Osasona, Opeoluwa Oluwanifemi Akomolafe, Jane Osareme Ogugua, Tolulope Olorunsogo, and Ebere Rosita Daraojimba, “Biomedical engineering advances: A review of innovations in healthcare and patient outcomes,” International Journal of Science and Research Archive, vol. 11, no. 1, 2024, doi: 10.30574/ijsra.2024.11.1.0139.

[7] Kajal, Dr. Kanchan Joshi, and Dr. Tarannum Shakeel, “Yoga for Young Eyes: Exploring the Effects on Visual Acuity in Adolescents,” International Journal of Latest Technology in Engineering Management & Applied Science, vol. 14, no. 8, 2025, doi: 10.51583/ijltemas.2025.1408000063.

[8] M. A. Ahulló-Fuster, M. L. Sánchez-Sánchez, E. Varela-Donoso, and T. Ortiz, “Measuring postural stability and health-related quality of life in people with visual impairment: a cross-sectional study,” Physiotherapy, vol. 123, 2024, doi: 10.1016/j.physio.2024.04.254.

[9] NCT05414799, “Effect of Exercises on Computer Vision Syndrome,” https://clinicaltrials.gov/show/NCT05414799, 2022.

[10] S. Das, L. Kalita, and B. Sarma, “The Impact of Digital Devices on Visual Health: Addressing Digital Eye Strain And Blue Light Exposure,” Journal of Population Therapeutics and Clinical Pharmacology, 2024, doi: 10.53555/jptcp.v31i7.7185.

[11] S. Fatima, R. W. Siddiqui, and T. W. Siddiqui, “Digital Eye Strain: A Cross-sectional Study Among Undergraduate Medical Students in UAE,” New Emirates Medical Journal, vol. 5, 2024, doi: 10.2174/0102506882361765250221072850.

[12] S. Bagaji and R. Rao, “Digital Fatigue In The Age of Screens: Eye and Postural Strain Among 18–35-Year-Old Screen Users,” International Journal of Research -GRANTHAALAYAH, vol. 13, no. 5, 2025, doi: 10.29121/granthaalayah.v13.i5.2025.6191.

[13] Y. Gao, C. Ma, L. Guo, X. Zhang, and X. Ji, “CLRD: Collaborative Learning for Retinopathy Detection Using Fundus Images,” Bioengineering, vol. 10, no. 8, 2023, doi: 10.3390/bioengineering10080978.

[14] C. Muchibwa, M. H. S. Eldaw, and M. O. Agyeman, “An Assessment of Contemporary Methods and Data-Enabled Approaches for Early Cataract Detection,” F1000Res., vol. 12, p. 998, 2024, doi: 10.12688/f1000research.138294.2.

[15] S. V. Mahadevkar et al., “A Review on Machine Learning Styles in Computer Vision - Techniques and Future Directions,” 2022. doi: 10.1109/ACCESS.2022.3209825.

[16] L. D. Mang, F. D. González Martínez, D. Martinez Muñoz, S. García Galán, and R. Cortina, “Classification of Adventitious Sounds Combining Cochleogram and Vision Transformers,” Sensors, vol. 24, no. 2, 2024, doi: 10.3390/s24020682.

[17] M. Aly and N. S. Alotaibi, “A novel deep learning model to detect COVID-19 based on wavelet features extracted from Mel-scale spectrogram of patients’ cough and breathing sounds,” Inform. Med. Unlocked, vol. 32, 2022, doi: 10.1016/j.imu.2022.101049.

[18] N. Yamsani, A. H. Abdul Hussein, S. J. Faith, K. Al-Attabi, and M. Y. Arabi, “Multimodal Sentimental Analysis Using GloVe Convolutional Neural Network with Capsule Network,” in 3rd IEEE International Conference on Mobile Networks and Wireless Communications, ICMNWC 2023, 2023. doi: 10.1109/ICMNWC60182.2023.10435957.

[19] Y. L. Chen, N. C. Wang, J. F. Ciou, S. S. Lien, C. H. Lin, and L. W. Lin, “Speaker recognition using autoencoder-based Mel-frequency cepstral coefficients with convolutional and bidirectional long short-term memory networks,” Smart Science, 2025, doi: 10.1080/23080477.2025.2565152.

[20] Anitha Mummireddygari and N Ananda Reddy, “Optimizing Speaker Recognition in Complex Environments : An Enhanced Framework with Artificial Neural Networks for Multi-Speaker Settings,” International Journal of Scientific Research in Computer Science, Engineering and Information Technology, vol. 10, no. 3, 2024, doi: 10.32628/cseit24103116.

[21] D. Subhash, J. L. G., P. B., and V. Ravi, “A robust accent classification system based on variational mode decomposition,” Eng. Appl. Artif. Intell., vol. 139, 2025, doi: 10.1016/j.engappai.2024.109512.

[22] D. J. Min and D. H. Kim, “Speech Emotion Recognition via Sparse Learning-Based Fusion Model,” IEEE Access, vol. 12, 2024, doi: 10.1109/ACCESS.2024.3506565.

[23] P. Wijonarko and A. Zahra, “Spoken language identification on 4 Indonesian local languages using deep learning,” Bulletin of Electrical Engineering and Informatics, vol. 11, no. 6, 2022, doi: 10.11591/eei.v11i6.4166.

[24] J. M. van Osta, B. Dreis, E. Meyer, L. F. Grogan, and J. G. Castley, “An active learning framework and assessment of inter-annotator agreement facilitate automated recogniser development for vocalisations of a rare species, the southern black-throated finch (Poephila cincta cincta),” Ecol. Inform., vol. 77, 2023, doi: 10.1016/j.ecoinf.2023.102233.

[25] T. Li, “Optimizing the configuration of deep learning models for music genre classification,” Heliyon, vol. 10, no. 2, 2024, doi: 10.1016/j.heliyon.2024.e24892.

[26] P. Ruan, X. Zheng, Y. Qiu, and Z. Hao, “A Binaural MFCC-CNN Sound Quality Model of High-Speed Train,” Applied Sciences (Switzerland), vol. 12, no. 23, 2022, doi: 10.3390/app122312151.

[27] R. V. Sharan, H. Xiong, and S. Berkovsky, “Benchmarking audio signal representation techniques for classification with convolutional neural networks,” Sensors, vol. 21, no. 10, 2021, doi: 10.3390/s21103434.

[28] S. J. Park, H. B. Lee, and G. W. Kim, “Eardrum-inspired soft viscoelastic diaphragms for CNN-based speech recognition with audio visualization images,” Sci. Rep., vol. 13, no. 1, 2023, doi: 10.1038/s41598-023-33755-2.

[29] H. Li, Q. Yao, and X. Li, “Voiceprint Fault Diagnosis of Converter Transformer under Load Influence Based on Multi-Strategy Improved Mel-Frequency Spectrum Coefficient and Temporal Convolutional Network,” Sensors, vol. 24, no. 3, 2024, doi: 10.3390/s24030757.

[30] T. Thomas et al., “Modelling dataset bias in machine-learned theories of economic decision-making,” Nat. Hum. Behav., vol. 8, no. 4, 2024, doi: 10.1038/s41562-023-01784-6.

[31] C. W. Baker et al., “Comparison of Snellen Visual Acuity Measurements in Retinal Clinical Practice to Electronic ETDRS Protocol Visual Acuity Assessment,” Ophthalmology, vol. 130, no. 5, 2023, doi: 10.1016/j.ophtha.2022.12.008.