Handling Imbalance in Javanese Manuscript Character Dataset using Skeleton-based Balancing Generative Adversarial Networks
DOI:
https://doi.org/10.29207/resti.v9i4.6572Keywords:
Data imbalance, Javanese manuscript, Character classification, Generative adversarial network, Skeleton-based generationAbstract
Javanese script is an important part of Indonesia’s cultural heritage, representing cultural values from the past. However, recognizing and classifying Javanese characters within manuscripts is challenging due to the limited availability of data and uneven distribution of character classes. The decline in formal use of Javanese script has drastically reduced the pool of manuscript samples, causing certain characters to appear rarely and skewing class frequencies. Existing methods that utilize Generative Adversarial Networks (GANs) attempt to address this problem. However, they often struggle to generate characters that are both consistent and visually accurate in terms of structural details. To address these issues, this study introduces a skeleton-based balancing GAN (SkelBAGAN), which improves the structural details of the previous method for generating characters. The proposed method introduces three main enhancements: (i) a layer for extracting the character skeleton structure, (ii) an optimized pretrained network using an autoencoder for learning the skeleton distribution, and (iii) refinement of the evaluation function, preserving both the distribution and structural fidelity in the adversarial process. The performance of the proposed model is evaluated against previous methods using the Fréchet Inception Distance (FID) to assess distribution quality and the Structural Similarity Index Measure (SSIM) to evaluate structural fidelity. The results indicate that the proposed methods outperform previous methods in balancing the FID and SSIM metrics. The integration of all enhancements in SkelBAGAN achieves the lowest FID, indicating improved generative quality while maintaining competitive SSIM values. The qualitative study indicates that SkelBAGAN outperforms previous methods in character generation. These results highlight how the skeleton-based improvement of the quality of generated characters enhances the recognition performance for underrepresented Javanese characters in imbalanced datasets. Ultimately, this work contributes to the broader effort to preserve the Javanese script as a vital element of Indonesia’s cultural identity.
Downloads
References
S. Anistya Ori and H. W. Susianti, “Preserving Javanese Language and Culture in The Digital Age : Challenge and Future Prospects,” LACULTOUR: Journal of Language and Cultural Tourism, vol. 2, no. 2, pp. 79–88, Nov. 2023, doi: 10.52352/lacultour.v2i2.1212.
A. B. Perdana, “Diverse Scripts, Same Mistakes: Identifying Common Pitfalls in the Typographic Designs of Indonesian Traditional Scripts,” Nirmana, vol. 24, no. 2, pp. 167–178, Jul. 2024, doi: 10.9744/nirmana.24.2.167-178.
M. A. Rasyidi, T. Bariyah, Y. I. Riskajaya, and A. D. Septyani, “Classification of handwritten Javanese script using random forest algorithm,” Bulletin of Electrical Engineering and Informatics, vol. 10, no. 3, pp. 1308–1315, Jun. 2021, doi: 10.11591/eei.v10i3.3036.
D. Iskandar, S. Hidayat, U. Jamaludin, and S. Mukti Leksono, “Javanese script digitalization and its utilization as learning media: an etnopedagogical approach,” International Journal of Mathematics and Sciences Education, vol. 1, no. 1, pp. 21–30, Jun. 2023, doi: 10.59965/ijmsed.v1i1.24.
P. Kumar, R. Bhatnagar, K. Gaur, and A. Bhatnagar, “Classification of Imbalanced Data:Review of Methods and Applications,” IOP Conf Ser Mater Sci Eng, vol. 1099, no. 1, p. 012077, Mar. 2021, doi: 10.1088/1757-899X/1099/1/012077.
S. Cicak and U. Avci, “Handling Imbalanced Data in Predictive Maintenance: A Resampling-Based Approach,” in 2023 5th International Congress on Human-Computer Interaction, Optimization and Robotic Applications (HORA), IEEE, Jun. 2023, pp. 1–6. doi: 10.1109/HORA58378.2023.10156799.
J. Jason Jeong, B. Patel, and I. Banerjee, “GAN augmentation for multiclass image classification using hemorrhage detection as a case-study,” Journal of Medical Imaging, vol. 9, no. 03, Jun. 2022, doi: 10.1117/1.JMI.9.3.035504.
M. Tomaszewski and J. Osuchowski, “Effectiveness of Data Resampling in Mitigating Class Imbalance for Object Detection,” in Proc. ITTAP ’23: 3rd International Workshop on Information Technologies: Theoretical and Applied Problems, Nov. 2023, pp. 197–07.
S. Yang, W. Xiao, M. Zhang, S. Guo, J. Zhao, and F. Shen, “Image Data Augmentation for Deep Learning: A Survey,” arXiv preprint, Apr. 2022, doi: 10.48550/arXiv.2204.08610.
J. Rabbi, N. Ray, M. Schubert, S. Chowdhury, and D. Chao, “Small-Object Detection in Remote Sensing Images with End-to-End Edge-Enhanced GAN and Object Detector Network,” Remote Sens (Basel), vol. 12, no. 9, p. 1432, May 2020, doi: 10.3390/rs12091432.
P. Sharma, M. Kumar, H. K. Sharma, and S. M. Biju, “Generative adversarial networks (GANs): Introduction, Taxonomy, Variants, Limitations, and Applications,” Multimed Tools Appl, vol. 83, no. 41, pp. 88811–88858, Mar. 2024, doi: 10.1007/s11042-024-18767-y.
R. Nagaraju and M. Stamp, “Auxiliary-Classifier GAN for Malware Analysis,” arXiv Preprint, Jul. 2021.
A. Bourou, V. Mezger, and A. Genovesio, “GANs Conditioning Methods: A Survey,” arXiv preprint, Aug. 2024, doi: 10.48550/arXiv.2408.15640.
T. Chakraborty, U. Reddy K S, S. M. Naik, M. Panja, and B. Manvitha, “Ten years of generative adversarial nets (GANs): a survey of the state-of-the-art,” Mach Learn Sci Technol, vol. 5, no. 1, p. 011001, Mar. 2024, doi: 10.1088/2632-2153/ad1f77.
G. Zhu, K. Zhou, L. Lu, Y. Fu, Z. Liu, and X. Yang, “Partial Discharge Data Augmentation Based on Improved Wasserstein Generative Adversarial Network With Gradient Penalty,” IEEE Trans Industr Inform, vol. 19, no. 5, pp. 6565–6575, May 2023, doi: 10.1109/TII.2022.3197839.
B. Kamsu-Foguem, S. L. Msouobu Gueuwou, and C. A. K. A. Kounta, “Generative Adversarial Networks based on optimal transport: a survey,” Artif Intell Rev, vol. 56, no. 7, pp. 6723–6773, Jul. 2023, doi: 10.1007/s10462-022-10342-x.
V. Sampath, I. Maurtua, J. J. A. Martín, and A. Gutierrez, “A Survey on Generative Adversarial Networks for Imbalance Problems in Computer Vision Tasks,” J Big Data, Aug. 2020, doi: 10.21203/rs.3.rs-45616/v1.
G. Huang and A. H. Jafari, “Enhanced balancing GAN: minority-class image generation,” Neural Comput Appl, vol. 35, no. 7, pp. 5145–5154, Mar. 2023, doi: 10.1007/s00521-021-06163-8.
A. Zhang, L. Su, Y. Zhang, Y. Fu, L. Wu, and S. Liang, “EEG data augmentation for emotion recognition with a multiple generator conditional Wasserstein GAN,” Complex & Intelligent Systems, vol. 8, no. 4, pp. 3059–3071, Aug. 2022, doi: 10.1007/s40747-021-00336-7.
A. R. Widiarti and R. Pulungan, “A method for solving scriptio continua in Javanese manuscript transliteration,” Heliyon, vol. 6, no. 4, p. e03827, Apr. 2020, doi: 10.1016/j.heliyon.2020.e03827.
P. Basa Pati, G. V Chandana, C. Rithish Reddy, G. Balaji Subash, and J. SriHarsha, “Handwriting Quality Assessment using Structural Features and Support Vector Machines,” in 2022 IEEE 7th International conference for Convergence in Technology (I2CT), IEEE, Apr. 2022, pp. 1–5. doi: 10.1109/I2CT54291.2022.9824125.
J. Gan, Y. Chen, B. Hu, J. Leng, W. Wang, and X. Gao, “Characters as graphs: Interpretable handwritten Chinese character recognition via Pyramid Graph Transformer,” Pattern Recognit, vol. 137, p. 109317, May 2023, doi: 10.1016/j.patcog.2023.109317.
D. H. Ko, A. Ul Hassan, S. Majeed, and J. Choi, “SkelGAN: A Font Image Skeletonization Method,” Journal of Information Processing Systems, vol. 17, no. 1, pp. 1–13, Feb. 2021, doi: 10.3745/JIPS.02.0152.
D. H. Ko, A. U. Hassan, J. Suk, and J. Choi, “SKFont: skeleton-driven Korean font generator with conditional deep adversarial networks,” International Journal on Document Analysis and Recognition (IJDAR), vol. 24, no. 4, pp. 325–337, Dec. 2021, doi: 10.1007/s10032-021-00374-4.
S. Rana and M. Gatti, “Comparative Evaluation of Modified Wasserstein GAN-GP and State-of-the-Art GAN Models for Synthesizing Agricultural Weed Images in RGB and Infrared Domain,” MethodsX, vol. 14, p. 103309, Jun. 2025, doi: 10.1016/j.mex.2025.103309.
P. Milewski, “A Review of Image Thinning Algorithms,” in Computer Information Systems and Industrial Management: 23rd International Conference, CISIM 2024, 2024, pp. 3–16. doi: 10.1007/978-3-031-71115-2_1.
N. Mamatov, M. Jalelova, V. Fayziev, A. Samijonov, and B. Samijonov, “Hybrid quadratic diagonal algorithm for thinning contour lines,” EPJ Web Conf, vol. 321, p. 03005, Mar. 2025, doi: 10.1051/epjconf/202532103005.
H. A. Abdo, A. Abdu, R. Manza, and S. Bawiskar, “An approach to analysis of arabic text documents into text lines, words, and characters,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 26, no. 2, p. 754, May 2022, doi: 10.11591/ijeecs.v26.i2.pp754-763.
L. O. Guertler, A. Ashfahani, and A. T. Luu, “How to train your draGAN: A task oriented solution to imbalanced classification,” arXiv preprint, Nov. 2022, doi: 10.48550/arXiv.2211.10065.
Q. Wan, W. Guo, and Y. Wang, “SGBGAN: minority class image generation for class-imbalanced datasets,” Mach Vis Appl, vol. 35, no. 2, p. 22, Mar. 2024, doi: 10.1007/s00138-023-01506-y.
S. Yuan, R. Liu, M. Chen, B. Chen, Z. Qiu, and X. He, “SE-GAN: Skeleton Enhanced Gan-Based Model for Brush Handwriting Font Generation,” in IEEE International Conference on Multimedia and Expo (ICME), IEEE, Jul. 2022, pp. 1–6. doi: 10.1109/ICME52920.2022.9859964.
Y. Xu, W. Luo, A. Hu, Z. Xie, X. Xie, and L. Tao, “TE-SAGAN: An Improved Generative Adversarial Network for Remote Sensing Super-Resolution Images,” Remote Sens (Basel), vol. 14, no. 10, p. 2425, May 2022, doi: 10.3390/rs14102425.
G. Huang and A. H. Jafari, “Enhanced Balancing GAN: Minority-class Image Generation,” ArXiv Preprint, Oct. 2020, doi: 10.1007/s00521-021-06163-8.
I. Bakurov, M. Buzzelli, R. Schettini, M. Castelli, and L. Vanneschi, “Structural similarity index (SSIM) revisited: A data-driven approach,” Expert Syst Appl, vol. 189, p. 116087, Mar. 2022, doi: 10.1016/j.eswa.2021.116087.
M. ’Arif Faizin and D. Herumurti, “An Initial Study in Historical Javanese Script Recognition Data Synthetic using Neural Style Transfer,” in 2024 11th International Conference on Electrical Engineering, Computer Science and Informatics (EECSI), IEEE, Sep. 2024, pp. 787–792. doi: 10.1109/EECSI63442.2024.10776278.
M. ’Arif Faizin, N. Suciati, and C. Fatichah, “Evaluation of Synthetic Data Effectiveness using Generative Adversarial Networks (GAN) in Improving Javanese Script Recognition on Ancient Manuscript,” JUTI: Jurnal Ilmiah Teknologi Informasi, pp. 95–109, Feb. 2025, doi: 10.12962/j24068535.v23i1.a1256.
M. ’Arif Faizin, “Handwritten Javanese Characters Sewaka Detection (HJCS_DETC),” Apr. 2025, Zenodo. doi: 10.5281/zenodo.15226702.
O. Fedoruk, K. Klimaszewski, A. Ogonowski, and R. Możdżonek, “Performance of GAN-based augmentation for deep learning COVID-19 image classification,” in AIP Conference Proceedings, Apr. 2023. doi: 10.1063/5.0203379.
R. Zhu, Y. Guo, and J.-H. Xue, “Adjusting the imbalance ratio by the dimensionality of imbalanced data,” Pattern Recognit Lett, vol. 133, pp. 217–223, May 2020, doi: 10.1016/j.patrec.2020.03.004.
M. Arabboev, S. Begmatov, M. Rikhsivoev, K. Nosirov, and S. Saydiakbarov, “comprehensive review of image super-resolution metrics: classical and AI-based approaches,” Acta IMEKO, vol. 13, no. 1, pp. 1–8, Mar. 2024, doi: 10.21014/actaimeko.v13i1.1679.
Downloads
Published
Issue
Section
License
Copyright in each article belongs to the author
- The author acknowledges that the RESTI Journal (System Engineering and Information Technology) is the first publisher to publish with a license Creative Commons Attribution 4.0 International License.
- Authors can enter writing separately, arrange the non-exclusive distribution of manuscripts that have been published in this journal into other versions (eg sent to the author's institutional repository, publication in a book, etc.), by acknowledging that the manuscript has been published for the first time in the RESTI (Rekayasa Sistem dan Teknologi Informasi) journal ;





