Comparative Analysis of YOLOv8 Segmentation Variants  for Indonesian Sign Language (SIBI) Recognition

Desi Fatkhi  Azizah; Anik Nur  Handayani; Aji Prasetya  Wibawa; Osamu  Fukuda

doi:10.29207/resti.v10i2.7500

Authors

Desi Fatkhi Azizah Universitas Negeri Malang
Anik Nur Handayani Universitas Negeri Malang
Aji Prasetya Wibawa Universitas Negeri Malang
Osamu Fukuda Saga University

DOI:

https://doi.org/10.29207/resti.v10i2.7500

Keywords:

assistive communication technology, disability inclusion, public services, SIBI, sign language recognition, YOLOv8

Abstract

The Indonesian Sign Language System (SIBI) is the officially recognized communication medium for deaf communities in Indonesia, yet its limited public use continues to create barriers in education, healthcare, and public services. Automatic sign language recognition powered by artificial intelligence provides a promising pathway to reduce these inequities. This study presents a comprehensive comparative evaluation of YOLOv8 segmentation variants for SIBI recognition, aiming to identify models that stabilize accuracy and efficiency for real-time deployment. A mono-background dataset of SIBI alphabet gestures was annotated using instance segmentation, and five YOLOv8-seg models (n, s, m, l, x) were trained and tested across multiple data-split scenarios. Performance was assessed through precision, recall, F1-score, mAP50, mAP50–95, and inference time. Results show that YOLOv8m-seg consistently achieved the best trade-off (F1-score 0.972; mAP50 0.982), while YOLOv8n-seg delivered the fastest inference speed (5.163 ms), making it suitable for resource-constrained devices. Visualization further demonstrated the models’ ability to capture hand contours and distinguish gestures accurately. Beyond technical benchmarking, this research highlights the potential of YOLOv8-based SIBI recognition as an inclusive assistive technology for bridging communication gaps in schools and clinics where interpreters are often unavailable. It also identifies governance challenges, including privacy protection, misclassification risks, and equitable access, which must be addressed for actual adoption. The findings, therefore, provide not only a contribution to computer vision research but also practical guidance for policymakers and service providers, positioning SIBI recognition systems as socially embedded technologies aligned with the goals of disability inclusion and sustainable development.

Downloads

Download data is not yet available.

References

B. Joksimoski et al., “Technological Solutions for Sign Language Recognition: A Scoping Review of Research Trends, Challenges, and Opportunities,” IEEE Access, vol. 10, pp. 40979–40998, 2022, doi: 10.1109/ACCESS.2022.3161440.

B. A. Al Abdullah, G. A. Amoudi, and H. S. Alghamdi, “Advancements in Sign Language Recognition: A Comprehensive Review and Future Prospects,” IEEE Access, vol. 12, pp. 128871–128895, 2024, doi: 10.1109/ACCESS.2024.3457692.

D. Lillo-Martin and J. A. Hochgesang, “Signed languages – Unique and ordinary: A commentary on Kidd and Garcia (2022),” First Lang., vol. 42, no. 6, pp. 789–793, Dec. 2022, doi: 10.1177/01427237221098858 .

L. Ferrara et al., “A cross-linguistic comparison of reference across five signed languages,” Linguistic Typology, vol. 27, no. 3, pp. 591–627, Oct. 2023, doi: 10.1515/lingty-2021-0057.

S. Hassan et al., “ASL-Homework-RGBD Dataset: An annotated dataset of 45 fluent and non-fluent signers performing American Sign Language homeworks,” Jul. 2022, [Online]. Available: http://arxiv.org/abs/2207.04021

G. E. Shields, K. D. Rogers, A. Young, S. Dedotsi, and L. M. Davies, “Health State Values of Deaf British Sign Language (BSL) Users in the UK: An Application of the BSL Version of the EQ-5D-5L,” Appl. Health Econ. Health Policy, vol. 18, no. 4, pp. 547–556, Aug. 2020, doi: 10.1007/s40258-019-00546-8.

Y. K. Suyitno, I. W. Sudiarsa, I. N. B. Hartawan, and I. D. P. G. W. Putra, “Implementation of Sibi Sign Language Realtime Detection Program (Case Studi At Sekolah Luar Biasa Negeri 1 Tabanan),” Journal of Computer Networks, Architecture and High Performance Computing, vol. 6, no. 3, pp. 1431–1441, Jul. 2024, doi: 10.47709/cnahpc.v6i3.4405.

J. S. Izquierdo-Condoy et al., “Exploring healthcare barriers and satisfaction levels among deaf individuals in Ecuador: A video-based survey approach,” Disabil. Health J., vol. 17, no. 3, p. 101622, Jul. 2024, doi: 10.1016/j.dhjo.2024.101622.

J. A. Manganello, “Shining a light on disability and health communication,” J. Commun. Healthc., vol. 15, no. 4, pp. 309–312, Oct. 2022, doi: 10.1080/17538068.2022.2142443.

H. F. da Silva, J. F. da Silva, V. V. Cruz, N. M. A. de Figueiredo, A. C. S. Sé, and W. C. A. Machado, “Aplicabilidade da Tecnologia Assistiva para pessoas com lesão medular: Uma revisão integrativa,” Research, Society and Development, vol. 10, no. 8, p. e0210815387, Jul. 2021, doi: 10.33448/rsd-v10i8.15387.

N. G. A. P. Mayuni and I. A. O. Martini, “Bridging Health and Sustainability: Evaluating Siloam Hospital Denpasar’s Role in Advancing Medical Tourism Aligned with SDG 3,” Journal of Lifestyle and SDGs Review, vol. 5, no. 1, p. e04323, Jan. 2025, doi: 10.47172/2965-730X.SDGsReview.v5.n01.pe04323.

M. P. Bhandari, “What is next for the sustainable development goals, what are the challenges concerning SDG 10—reduced inequalities?,” Sustainable Earth Reviews, vol. 7, no. 1, p. 23, Jul. 2024, doi: 10.1186/s42055-024-00093-8.

M. Yu, J. Jia, C. Xue, G. Yan, Y. Guo, and Y. Liu, “A review of sign language recognition research,” Journal of Intelligent & Fuzzy Systems, vol. 43, no. 4, pp. 3879–3898, Aug. 2022, doi: 10.3233/JIFS-210050.

B. A. Al Abdullah, G. A. Amoudi, and H. S. Alghamdi, “Advancements in Sign Language Recognition: A Comprehensive Review and Future Prospects,” IEEE Access, vol. 12, pp. 128871–128895, 2024, doi: 10.1109/ACCESS.2024.3457692.

H. ZainEldin et al., “Silent no more: a comprehensive review of artificial intelligence, deep learning, and machine learning in facilitating deaf and mute communication,” Artif. Intell. Rev., vol. 57, no. 7, p. 188, Jun. 2024, doi: 10.1007/s10462-024-10816-0.

Y. Liu, P. Nand, M. A. Hossain, M. Nguyen, and W. Q. Yan, “Sign language recognition from digital videos using feature pyramid network with detection transformer,” Multimed. Tools Appl., vol. 82, no. 14, pp. 21673–21685, Jun. 2023, doi: 10.1007/s11042-023-14646-0.

Y. Zhou, “A YOLO-NL object detector for real-time detection,” Expert Syst. Appl., vol. 238, p. 122256, Mar. 2024, doi: 10.1016/j.eswa.2023.122256.

M. Sohaib, M. Arif, and J.-M. Kim, “Evaluating YOLO Models for Efficient Crack Detection in Concrete Structures Using Transfer Learning,” Buildings, vol. 14, no. 12, p. 3928, Dec. 2024, doi: 10.3390/buildings14123928.

J. Ma, Y. Zhao, W. Fan, and J. Liu, “An Improved YOLOv8 Model for Lotus Seedpod Instance Segmentation in the Lotus Pond Environment,” Agronomy, vol. 14, no. 6, p. 1325, Jun. 2024, doi: 10.3390/agronomy14061325.

A. Wang et al., “NVW-YOLOv8s: An improved YOLOv8s network for real-time detection and segmentation of tomato fruits at different ripeness stages,” Comput. Electron. Agric., vol. 219, p. 108833, Apr. 2024, doi: 10.1016/j.compag.2024.108833.

M. Hussain, “YOLOv1 to v8: Unveiling Each Variant–A Comprehensive Review of YOLO,” IEEE Access, vol. 12, pp. 42816–42833, 2024, doi: 10.1109/ACCESS.2024.3378568.

H.-I. Liu et al., “Lightweight Deep Learning for Resource-Constrained Environments: A Survey,” ACM Comput. Surv., vol. 56, no. 10, pp. 1–42, Oct. 2024, doi: 10.1145/3657282.

Q. Wang, G. Feng, and Z. Li, “A Lightweight Person Detector for Surveillance Footage Based on YOLOv8n,” Sensors, vol. 25, no. 2, p. 436, Jan. 2025, doi: 10.3390/s25020436.

Y. Liu, C. Zhang, W. Wu, B. Zhang, and F. Zhou, “MiniYOLO: A lightweight object detection algorithm that realizes the trade‐off between model size and detection accuracy,” International Journal of Intelligent Systems, vol. 37, no. 12, pp. 12135–12151, Dec. 2022, doi: 10.1002/int.23079.

B. Zhao, Y. Zhou, R. Song, L. Yu, X. Zhang, and J. Liu, “Modular YOLOv8 optimization for real-time UAV maritime rescue object detection,” Sci. Rep., vol. 14, no. 1, p. 24492, Oct. 2024, doi: 10.1038/s41598-024-75807-1.

A. N. Handayani, S. Amaliya, M. I. Akbar, M. Z. Wiryawan, Y. W. Liang, and W. C. Kurniawan, “Hand Keypoint-Based CNN for SIBI Sign Language Recognition,” International Journal of Robotics and Control Systems, vol. 5, no. 2, pp. 813–829, 2025, doi: 10.31763/ijrcs.v5i2.1745.

Suharjito, N. Thiracitta, and H. Gunawan, “SIBI Sign Language Recognition Using Convolutional Neural Network Combined with Transfer Learning and non-trainable Parameters,” Procedia Comput. Sci., vol. 179, pp. 72–80, 2021, doi: 10.1016/j.procs.2020.12.011.

A. Z. Nugraha, R. F. Salsabila, A. N. Handayani, A. P. Wibawa, E. Hitipeuw, and K. Arai, “Decision tree based algorithms for Indonesian Language Sign System (SIBI) recognition,” Applied Engineering and Technology, vol. 3, no. 2, pp. 86–101, Aug. 2024, doi: 10.31763/aet.v3i2.1536.

W. Hao, C. Hou, Z. Zhang, X. Zhai, L. Wang, and G. Lv, “A sensing data and deep learning-based sign language recognition approach,” Computers and Electrical Engineering, vol. 118, p. 109339, Aug. 2024, doi: 10.1016/j.compeleceng.2024.109339.

A. S. Dhanjal and W. Singh, “An optimized machine translation technique for multi-lingual speech to sign language notation,” Multimed. Tools Appl., vol. 81, no. 17, pp. 24099–24117, Jul. 2022, doi: 10.1007/s11042-022-12763-w.

C. K. M. Lee, K. K. H. Ng, C.-H. Chen, H. C. W. Lau, S. Y. Chung, and T. Tsoi, “American sign language recognition and training method with recurrent neural network,” Expert Syst. Appl., vol. 167, p. 114403, Apr. 2021, doi: 10.1016/j.eswa.2020.114403.

J. Shin, A. Matsuoka, Md. A. M. Hasan, and A. Y. Srizon, “American Sign Language Alphabet Recognition by Extracting Feature from Hand Pose Estimation,” Sensors, vol. 21, no. 17, p. 5856, Aug. 2021, doi: 10.3390/s21175856.

L. N. Hayati et al., “Improving Indonesian Sign Alphabet Recognition for Assistive Learning Robots Using Gamma-Corrected MobileNetV2 Lilis Nur Hayati (Improving Indonesian Sign Alphabet Recognition for Assistive Learning Robots Using Gamma-Corrected MobileNetV2),” Buletin Ilmiah Sarjana Teknik Elektro, vol. 7, no. 3, pp. 350–361, 2025, doi: 10.12928/biste.v7i3.13300.

S. Sharma and K. Kumar, “ASL-3DCNN: American sign language recognition technique using 3-D convolutional neural networks,” Multimed. Tools Appl., vol. 80, no. 17, pp. 26319–26331, Jul. 2021, doi: 10.1007/s11042-021-10768-5.

T. Liu, T. Tao, Y. Zhao, M. Li, and J. Zhu, “A signer-independent sign language recognition method for the single-frequency dataset,” Neurocomputing, vol. 582, p. 127479, May 2024, doi: 10.1016/j.neucom.2024.127479.

G. Varol, L. Momeni, S. Albanie, T. Afouras, and A. Zisserman, “Scaling Up Sign Spotting Through Sign Language Dictionaries,” Int. J. Comput. Vis., vol. 130, no. 6, pp. 1416–1439, Jun. 2022, doi: 10.1007/s11263-022-01589-6.

Y. Liu et al., “A wearable system for sign language recognition enabled by a convolutional neural network,” Nano Energy, vol. 116, p. 108767, Nov. 2023, doi: 10.1016/j.nanoen.2023.108767.

R. Wulanningrum, A. N. Handayani, and H. W. Herwanto, “Comparative Analysis of Yolov-8 Segmentation for Gait Performance in Individuals with Lower Limb Disabilities,” International Journal of Robotics and Control Systems, vol. 5, no. 1, pp. 516–529, Jan. 2025, doi: 10.31763/ijrcs.v5i1.1731.

C. Agnew, C. Eising, P. Denny, A. Scanlan, P. Van De Ven, and E. M. Grua, “Quantifying the Effects of Ground Truth Annotation Quality on Object Detection and Instance Segmentation Performance,” IEEE Access, vol. 11, pp. 25174–25188, 2023, doi: 10.1109/ACCESS.2023.3256723.

L. He, Y. Zhou, L. Liu, and J. Ma, “Research and Application of YOLOv11-Based Object Segmentation in Intelligent Recognition at Construction Sites,” Buildings, vol. 14, no. 12, p. 3777, Nov. 2024, doi: 10.3390/buildings14123777.

F. Tan, M. Zhai, and C. Zhai, “Foreign object detection in urban rail transit based on deep differentiation segmentation neural network,” Heliyon, vol. 10, no. 17, p. e37072, Sep. 2024, doi: 10.1016/j.heliyon.2024.e37072.

T. Li, J. Bai, and Q. Wang, “Enhancing medical text detection with vision-language pre-training and efficient segmentation,” Complex & Intelligent Systems, vol. 10, no. 3, pp. 3995–4007, Jun. 2024, doi: 10.1007/s40747-024-01378-3.

X. Yuan, X. Shen, S. Mehta, T. Li, S. Ge, and Z. Zha, “Structure injected weight normalization for training deep networks,” Multimed. Syst., vol. 28, no. 2, pp. 433–444, Apr. 2022, doi: 10.1007/s00530-021-00793-7.

H. Lai, L. Zhang, and S. Zhang, “Improving Network Training on Resource-Constrained Devices via Habituation Normalization,” Sensors, vol. 22, no. 24, p. 9940, Dec. 2022, doi: 10.3390/s22249940.

R. Jalaboi, O. Winther, and A. Galimzianova, “Explainable Image Quality Assessments in Teledermatological Photography,” Telemedicine and e-Health, vol. 29, no. 9, pp. 1342–1348, Sep. 2023, doi: 10.1089/tmj.2022.0405.

T. Rädsch et al., “Labelling instructions matter in biomedical image analysis,” Nat. Mach. Intell., vol. 5, no. 3, pp. 273–283, Mar. 2023, doi: 10.1038/s42256-023-00625-5.

E. Bellou, I. Pisica, and K. Banitsas, “Aerial Inspection of High-Voltage Power Lines Using YOLOv8 Real-Time Object Detector,” Energies (Basel)., vol. 17, no. 11, p. 2535, May 2024, doi: 10.3390/en17112535.

S. Qi, X. Song, T. Shang, X. Hu, and K. Han, “MSFE-YOLO: An Improved YOLOv8 Network for Object Detection on Drone View,” IEEE Geoscience and Remote Sensing Letters, vol. 21, pp. 1–5, 2024, doi: 10.1109/LGRS.2024.3432536.

Q. Zhou, Z. Wang, Y. Zhong, F. Zhong, and L. Wang, “Efficient Optimized YOLOv8 Model with Extended Vision,” Sensors, vol. 24, no. 20, p. 6506, Oct. 2024, doi: 10.3390/s24206506.

M. Yue, L. Zhang, Y. Zhang, and H. Zhang, “An Improved YOLOv8 Detector for Multi-Scale Target Detection in Remote Sensing Images,” IEEE Access, vol. 12, pp. 114123–114136, 2024, doi: 10.1109/ACCESS.2024.3444606.

X. Zhao, W. Zhang, H. Zhang, C. Zheng, J. Ma, and Z. Zhang, “ITD-YOLOv8: An Infrared Target Detection Model Based on YOLOv8 for Unmanned Aerial Vehicles,” Drones, vol. 8, no. 4, p. 161, Apr. 2024, doi: 10.3390/drones8040161.

X. Zuo, Z. Shao, J. Wang, X. Huang, and Y. Wang, “A cross-stage features fusion network for building extraction from remote sensing images,” Geo-spatial Information Science, vol. 28, no. 2, pp. 387–401, Mar. 2025, doi: 10.1080/10095020.2024.2307922.

Z. Zhang, X. Qiu, and Y. Li, “SEFPN: Scale-Equalizing Feature Pyramid Network for Object Detection,” Sensors, vol. 21, no. 21, p. 7136, Oct. 2021, doi: 10.3390/s21217136.

H. Wang, C. Liu, Y. Cai, L. Chen, and Y. Li, “YOLOv8-QSD: An Improved Small Object Detection Algorithm for Autonomous Vehicles Based on YOLOv8,” IEEE Trans. Instrum. Meas., vol. 73, pp. 1–16, 2024, doi: 10.1109/TIM.2024.3379090.

N. U. A. Tahir, Z. Long, Z. Zhang, M. Asim, and M. ELAffendi, “PVswin-YOLOv8s: UAV-Based Pedestrian and Vehicle Detection for Traffic Management in Smart Cities Using Improved YOLOv8,” Drones, vol. 8, no. 3, p. 84, Feb. 2024, doi: 10.3390/drones8030084.

L. Ramos, E. Casas, E. Bendek, C. Romero, and F. Rivas-Echeverría, “Hyperparameter optimization of YOLOv8 for smoke and wildfire detection: Implications for agricultural and environmental safety,” Artificial Intelligence in Agriculture, vol. 12, pp. 109–126, Jun. 2024, doi: 10.1016/j.aiia.2024.05.003.

E. Asci et al., “A Deep Learning Approach to Automatic Tooth Caries Segmentation in Panoramic Radiographs of Children in Primary Dentition, Mixed Dentition, and Permanent Dentition,” Children, vol. 11, no. 6, p. 690, Jun. 2024, doi: 10.3390/children11060690.

A. Hinterreiter et al., “ConfusionFlow: A Model-Agnostic Visualization for Temporal Analysis of Classifier Confusion,” IEEE Trans. Vis. Comput. Graph., vol. 28, no. 2, pp. 1222–1236, Feb. 2022, doi: 10.1109/TVCG.2020.3012063.

M. Owusu-Adjei, J. Ben Hayfron-Acquah, T. Frimpong, and G. Abdul-Salaam, “A systematic review of prediction accuracy as an evaluation measure for determining machine learning model performance in healthcare systems,” Jun. 04, 2023. doi: 10.1101/2023.06.01.23290837.

Z. Mawardi, D. Gautam, and T. G. Whiteside, “Utilization of Remote Sensing Dataset and a Deep Learning Object Detection Model to Map Siam Weed Infestations,” IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens., vol. 17, pp. 18939–18948, 2024, doi: 10.1109/JSTARS.2024.3465554.

R. Srilatha, Y. D. G, Y. Perumalla, D. S. Yaganti, and S. Yegoti, “Machine Learning Approaches in Financial Fraud Detection: Performance Evaluation and Statistical Insights,” in Digital Crossroads: Integrating Humanities, Science and Technology Edition 1, BP International, 2025, pp. 47–61. doi: 10.9734/bpi/mono/978-93-48859-10-5/CH5.

H. Ma, W. Zhao, B. Liu, and W. Chen, “Multi-Scale Target Detection in Autonomous Driving Scenarios Based on YOLOv5-AFAM,” Applied Sciences, vol. 14, no. 11, p. 4633, May 2024, doi: 10.3390/app14114633.

H. Peng and S. Yu, “A Systematic IoU-Related Method: Beyond Simplified Regression for Better Localization,” IEEE Transactions on Image Processing, vol. 30, pp. 5032–5044, 2021, doi: 10.1109/TIP.2021.3077144.

S. Gao, “Exploration and evaluation of faster R-CNN-based pedestrian detection techniques,” Applied and Computational Engineering, vol. 32, no. 1, pp. 185–190, Jan. 2024, doi: 10.54254/2755-2721/32/20230208.

C. Tian, Z. Liu, H. Chen, F. Dong, X. Liu, and C. Lin, “A Lightweight Model for Shine Muscat Grape Detection in Complex Environments Based on the YOLOv8 Architecture,” Agronomy, vol. 15, no. 1, p. 174, Jan. 2025, doi: 10.3390/agronomy15010174.

D. F. Azizah, A. N. Handayani, and A. P. Wibawa, “YOLOv8 Performance for SIBI Sign Language Mono Background Detection,” in 2025 International Conference on Computer Engineering, Network and Intelligent Multimedia (CENIM), IEEE, Nov. 2025, pp. 185–190. doi: 10.1109/CENIM67940.2025.11326292.