MobileNetV3-based Handwritten Chinese Recognition Towards the Effectiveness of Learning Hanzi
Abstract
Writing Mandarin characters is considered the most challenging component for beginners due to the rules and character formations. This paper explores the potential of a machine learning-based digital learning tool for writing Mandarin characters. It also conducts a comparative study between MobileNetV2 and MobileNetV3, exploring different configurations. The research follows the Multimedia Development Life Cycle (MDLC) method to create both the application and machine learning models. Participants from higher education institutions that offer Mandarin courses in Batam, Indonesia, were involved in a User Acceptance Test (UAT). Data was gathered through questionnaires and analyzed using the System Usability Scale (SUS) methods. The results show positive user acceptance, with an SUS score of 77.92%, indicating a high level of acceptability. MobileNetV3Small was also preferred for recognizing the user’s handwriting, due to comparable accuracy size, rapid inference time, and smallest model size. While the application was well-received, several participants provided constructive feedback, suggesting potential improvements.
Downloads
References
I. Markov, K. Kharitonova, and E. L. Grigorenko, “Language: Its Origin and Ongoing Evolution,” Journal of Intelligence 2023, Vol. 11, Page 61, vol. 11, no. 4, p. 61, 2023, doi: 10.3390/JINTELLIGENCE11040061.
R. Canestrino, P. Magliocca, and Y. Li, “The Impact of Language Diversity on Knowledge Sharing Within International University Research Teams: Evidence From TED Project,” Front Psychol, vol. 13, 2022, doi: 10.3389/FPSYG.2022.879154.
Y. Y. Su, R. Denadai, C. T. Ho, B. R. Lai, and L. J. Lo, “Measuring Patient-Reported Outcomes in Orthognathic Surgery: Linguistic and Psychometric Validation of the Mandarin Chinese Version of Face-Q Instrument,” Biomed J, vol. 43, no. 1, pp. 62–73, 2020, doi: 10.1016/J.BJ.2019.05.011.
V. Habic, A. Semenov, and E. L. Pasiliao, “Multitask Deep Learning for Native Language Identification,” Knowl Based Syst, vol. 209, p. 106440, 2020, doi: 10.1016/J.KNOSYS.2020.106440.
B. Lyu, C. Lai, C. H. Lin, and Y. Gong, “Comparison Studies of Typing and Handwriting in Chinese Language Learning: A Synthetic Review,” Int J Educ Res, vol. 106, p. 101740, 2021, doi: 10.1016/J.IJER.2021.101740.
Y. Yang et al., “Brain Activation and Functional Connectivity during Chinese Writing: An fMRI Study,” J Neurolinguistics, vol. 51, pp. 199–211, 2019, doi: 10.1016/J.JNEUROLING.2019.03.002.
S. Huang, Y. Zhou, M. Du, R. Wang, and Z. G. Cai, “Character Amnesia in Chinese Handwriting: A Mega-Study Analysis,” Language Sciences, vol. 85, p. 101383, 2021, doi: 10.1016/J.LANGSCI.2021.101383.
S. Huang, W. Lin, M. Xu, R. Wang, and Z. G. Cai, “On the Tip of the Pen: Effects of Character-Level Lexical Variables and Handwriter-Level Individual Differences on Orthographic Retrieval Difficulties in Chinese Handwriting,” Quarterly Journal of Experimental Psychology, vol. 74, no. 9, pp. 1497–1511, Mar. 2021, doi: 10.1177/17470218211004385.
Y. Ying, D. Marchelline, and G. Wijaya, “Using Technology-Flashcard to Encourage Students Learning Mandarin,” in Journal of Physics: Conference Series, IOP Publishing, Feb. 2021, p. 012138. doi: 10.1088/1742-6596/1764/1/012138.
W. Zhou and X. Li, “Investigation of a Chinese Character Writing App: Learners’ Perspectives,” Knowledge Management and E-Learning, vol. 14, no. 1, pp. 15–29, Mar. 2022, doi: 10.34105/j.kmel.2022.14.002.
A. Bozkurt, A. Karadeniz, D. Baneres, A. E. Guerrero-Roldán, and M. E. Rodríguez, “Artificial Intelligence and Reflections from Educational Landscape: A Review of AI Studies in Half a Century,” Sustainability 2021, Vol. 13, Page 800, vol. 13, no. 2, p. 800, 2021, doi: 10.3390/SU13020800.
M. Pikhart, “Intelligent Information Processing for Language Education: The Use of Artificial Intelligence in Language Learning Apps,” Procedia Comput Sci, vol. 176, pp. 1412–1419, 2020, doi: 10.1016/J.PROCS.2020.09.151.
M. Ponticorvo, E. Dell’Aquila, and R. Di Fuccio, “Hyper-Activity Books and Serious Games: How to Promote Experiential Learning beyond Distance,” International Journal of Environmental Research and Public Health 2022, Vol. 19, Page 11132, vol. 19, no. 17, p. 11132, 2022, doi: 10.3390/IJERPH191711132.
N. Bonneton-Botté et al., “Can Tablet Apps Support the Learning of Handwriting? An Investigation of Learning Outcomes in Kindergarten Classroom,” Comput Educ, vol. 151, p. 103831, 2020, doi: 10.1016/J.COMPEDU.2020.103831.
M. Desai and M. Shah, “An Anatomization on Breast Cancer Detection and Diagnosis Employing Multi-layer Perceptron Neural Network (MLP) and Convolutional Neural Network (CNN),” Clinical eHealth, vol. 4, pp. 1–11, 2021, doi: 10.1016/J.CEH.2020.11.002.
B. Kumeda, Z. Fengli, A. Oluwasanmi, F. Owusu, M. Assefa, and T. Amenu, “Vehicle Accident and Traffic Classification Using Deep Convolutional Neural Networks,” in 2019 16th International Computer Conference on Wavelet Active Media Technology and Information Processing, ICCWAMTIP 2019, Institute of Electrical and Electronics Engineers Inc., 2019, pp. 323–328. doi: 10.1109/ICCWAMTIP47768.2019.9067530.
R. R. Chowdhury, M. S. Hossain, R. Ul Islam, K. Andersson, and S. Hossain, “Bangla Handwritten Character Recognition using Convolutional Neural Network with Data Augmentation,” in 2019 Joint 8th international conference on informatics, electronics & vision (ICIEV) and 2019 3rd international conference on imaging, vision & pattern recognition (icIVPR), Institute of Electrical and Electronics Engineers Inc., 2019, pp. 318–323. doi: 10.1109/ICIEV.2019.8858545.
S. N. Shuvo, F. Hasan, M. U. Ahmed, S. A. Hossain, and S. Abujar, “MathNET: Using CNN Bangla Handwritten Digit, Mathematical Symbols, and Trigonometric Function Recognition,” in Advances in Intelligent Systems and Computing, Springer Science and Business Media Deutschland GmbH, 2021, pp. 515–523. doi: 10.1007/978-981-15-7394-1_47.
H. Kusetogullari, A. Yavariabdi, J. Hall, and N. Lavesson, “DIGITNET: A Deep Handwritten Digit Detection and Recognition Method Using a New Historical Handwritten Digit Dataset,” Big Data Research, vol. 23, p. 100182, 2021, doi: 10.1016/J.BDR.2020.100182.
S. Ahlawat, A. Choudhary, A. Nayyar, S. Singh, and B. Yoon, “Improved Handwritten Digit Recognition Using Convolutional Neural Networks (CNN),” Sensors, vol. 20, no. 12, p. 3344, 2020, doi: 10.3390/S20123344.
M. Elkhayati, Y. Elkettani, and M. Mourchid, “Segmentation of Handwritten Arabic Graphemes Using a Directed Convolutional Neural Network and Mathematical Morphology Operations,” Pattern Recognit, vol. 122, p. 108288, 2022, doi: 10.1016/J.PATCOG.2021.108288.
W. Liu, J. Wei, and Q. Meng, “Comparisons on KNN, SVM, BP and the CNN for Handwritten Digit Recognition,” 2020 IEEE International Conference on Advances in Electrical Engineering and Computer Applications (AEECA), pp. 587–590, 2020, doi: 10.1109/AEECA49918.2020.9213482.
R. Ameri, A. Alameer, S. Ferdowsi, V. Abolghasemi, and K. Nazarpour, “Classification of Handwritten Chinese Numbers with Convolutional Neural Networks,” in Proceedings of the 5th International Conference on Pattern Recognition and Image Analysis, IPRIA 2021, Institute of Electrical and Electronics Engineers Inc., 2021. doi: 10.1109/IPRIA53572.2021.9483557.
H. M. Balaha et al., “Recognizing Arabic Handwritten Characters using Deep Learning and Genetic Algorithms,” Multimed Tools Appl, pp. 32473–32509, 2021, doi: 10.1007/S11042-021-11185-4.
M. U. Hossain, M. A. Rahman, M. M. Islam, A. Akhter, M. A. Uddin, and B. K. Paul, “Automatic Driver Distraction Detection using Deep Convolutional Neural Networks,” Intelligent Systems with Applications, vol. 14, p. 200075, 2022, doi: 10.1016/J.ISWA.2022.200075.
A. Howard et al., “Searching for MobileNetV3,” in Proceedings of the IEEE International Conference on Computer Vision, Institute of Electrical and Electronics Engineers Inc., 2019, pp. 1314–1324. doi: 10.1109/ICCV.2019.00140.
A. Nayak, S. Chakraborty, and D. K. Swain, “Application of Smartphone-Image Processing and Transfer Learning for Rice Disease and Nutrient Deficiency Detection,” Smart Agricultural Technology, vol. 4, p. 100195, 2023, doi: 10.1016/J.ATECH.2023.100195.
S. Qian, C. Ning, and Y. Hu, “MobileNetV3 for Image Classification,” in 2021 IEEE 2nd International Conference on Big Data, Artificial Intelligence and Internet of Things Engineering (ICBAIE), Institute of Electrical and Electronics Engineers Inc., 2021, pp. 490–497. doi: 10.1109/ICBAIE52039.2021.9389905.
M. Al-Sarem, M. Al-Asali, A. Y. Alqutaibi, and F. Saeed, “Enhanced Tooth Region Detection Using Pretrained Deep Learning Models,” International Journal of Environmental Research and Public Health 2022, Vol. 19, Page 15414, vol. 19, no. 22, p. 15414, 2022, doi: 10.3390/IJERPH192215414.
A. Hussain, B. Barua, A. Osman, R. Abozariba, and A. Taufiq Asyhari, “Performance of MobileNetV3 Transfer Learning on Handheld Device-based Real-Time Tree Species Identification,” 2021 26th International Conference on Automation and Computing: System Intelligence through Automation and Computing, ICAC 2021, 2021, doi: 10.23919/ICAC50006.2021.9594222.
H. Tarek et al., “Optimized Deep Learning Algorithms for Tomato Leaf Disease Detection with Hardware Deployment,” Electronics (Basel), vol. 11, no. 1, p. 140, 2022, doi: 10.3390/ELECTRONICS11010140.
Y. Li, W. C. Huang, and P. H. Song, “A Face Image Classification Method of Autistic Children Based on the Two-Phase Transfer Learning,” Front Psychol, vol. 14, 2023, doi: 10.3389/FPSYG.2023.1226470.
R. Nurcholis, A. I. Purnamasari, A. R. Dikananda, O. Nurdiawan, and S. Anwar, “Game Edukasi Pengenalan Huruf Hiragana Untuk Meningkatkan Kemampuan Berbahasa Jepang,” Building of Informatics, Technology and Science (BITS), vol. 3, no. 3, pp. 338–345, 2021, doi: 10.47065/bits.v3i3.1091.
V. D. Lestari and T. Huriah, “The Influence of Health Education Using Game-Based Learning Methods on Improving Smoking Prevention Behavior Among School-Age Children,” Jurnal Aisyah : Jurnal Ilmu Kesehatan, vol. 7, no. 4, pp. 1267–1272, 2022, doi: 10.30604/jika.v7i4.1290.
M. Ardiansyah and Riswanto, “Analisis Komparasi Ketertarikan Masyarakat Kota Batam Dalam Penggunaan Video Editor Capcut dan VN,” Jurnal Informasi dan Teknologi, vol. 5, no. 3, pp. 91–102, 2023, doi: 10.60083/JIDT.V5I3.398.
J. Pratama and R. I. Putri, “Design and Implementation of Animated Stickers as Educational Media Regarding COVID-19 Using the MDLC Method,” Science Tech: Jurnal Ilmu Pengetahuan dan Teknologi, vol. 7, no. 2, pp. 1–12, 2021, doi: 10.30738/st.vol7.no2.a9190.
M. N. I. Opu, M. R. Islam, M. A. Kabir, M. S. Hossain, and M. M. Islam, “Learn2Write: Augmented Reality and Machine Learning-Based Mobile App to Learn Writing,” Computers 2022, Vol. 11, Page 4, vol. 11, no. 1, p. 4, 2021, doi: 10.3390/COMPUTERS11010004.
R. Wibawa, A. Lokacarya, F. Kurniawan, and Y. Udjaja, “Japanese Language Learning Game ‘Miryoku’ using Android-Based Speech Recognizer API,” Procedia Comput Sci, vol. 216, pp. 547–556, 2023, doi: 10.1016/J.PROCS.2022.12.168.
P. Satti, N. Sharma, and B. Garg, “Min-Max Average Pooling Based Filter for Impulse Noise Removal,” IEEE Signal Process Lett, vol. 27, pp. 1475–1479, 2020, doi: 10.1109/LSP.2020.3016868.
N. Akhtar and U. Ragavendran, “Interpretation of Intelligence in CNN-Pooling Processes: A Methodological Survey,” Neural Comput Appl, vol. 32, no. 3, pp. 879–898, 2020, doi: 10.1007/S00521-019-04296-5.
Nurbaity Sabri, Haza Nuzly Abdull Hamed, Zaidah Ibrahim, and Kamalnizat Ibrahim, “A Comparison between Average and Max-Pooling in Convolutional Neural Network for Scoliosis Classification,” International Journal of Advanced Trends in Computer Science and Engineering, vol. 9, pp. 689–696, 2020, doi: 10.30534/ijatcse/2020/9791.42020.
Y. Gulzar, “Fruit Image Classification Model Based on MobileNetV2 with Deep Transfer Learning Technique,” Sustainability, vol. 15, no. 3, p. 1906, 2023, doi: 10.3390/SU15031906.
C. N. A. M. Gulfan and C. M. Vilela-Malabanan, “Evaluating the Usability and User Experience of Phytoplankton Cell Counter Prototype,” Procedia Comput Sci, vol. 197, pp. 309–316, 2022, doi: 10.1016/J.PROCS.2021.12.145.
S. Bordoni and G. Tang, “Development and Assessment of a Contactless 3D Joystick Approach to Industrial Manipulator Gesture Control,” Int J Ind Ergon, vol. 93, p. 103376, 2023, doi: 10.1016/J.ERGON.2022.103376.
L. D. Tamang and B. W. Kim, “FVR-Net: Finger Vein Recognition with Convolutional Neural Network Using Hybrid Pooling,” Applied Sciences, vol. 12, no. 15, p. 7538, 2022, doi: 10.3390/APP12157538.
N. W. Raharjanti et al., “Translation, Validity and Reliability of Decision Style Scale in Forensic Psychiatric Setting in Indonesia,” Heliyon, vol. 8, no. 7, 2022, doi: 10.1016/j.heliyon.2022.e09810.
S. R. Natasia, Y. T. Wiranti, and A. Parastika, “Acceptance Analysis of NUADU As E-learning Platform Using the Technology Acceptance Model (TAM) Approach,” Procedia Comput Sci, vol. 197, pp. 512–520, 2022, doi: 10.1016/J.PROCS.2021.12.168.
Copyright (c) 2023 Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi)
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright in each article belongs to the author
- The author acknowledges that the RESTI Journal (System Engineering and Information Technology) is the first publisher to publish with a license Creative Commons Attribution 4.0 International License.
- Authors can enter writing separately, arrange the non-exclusive distribution of manuscripts that have been published in this journal into other versions (eg sent to the author's institutional repository, publication in a book, etc.), by acknowledging that the manuscript has been published for the first time in the RESTI (Rekayasa Sistem dan Teknologi Informasi) journal ;
