A Hybrid VADER–IndoBERT Framework for Robust Sentiment Analysis of Long and Ambiguous Indonesian Texts:

Margareta Valencia Suci  Handayani; Ruri Suko  Basuki; Muljono; Raden Arief  Nugroho; Dhendra Marutho; Yo Ceng Giap; Deshinta Arrova Dewi

doi:10.29207/resti.v10i4.7377

Authors

Margareta Valencia Suci Handayani Universitas Dian Nuswantoro,
Ruri Suko Basuki Universitas Dian Nuswantoro
Muljono Universitas Dian Nuswantoro https://orcid.org/0000-0001-6059-3571
Raden Arief Nugroho Universitas Dian Nuswantoro
Dhendra Marutho Universitas Muhammadiyah Semarang
Yo Ceng Giap Universitas Dian Nuswantoro
Deshinta Arrova Dewi INTI International University

DOI:

https://doi.org/10.29207/resti.v10i4.7377

Keywords:

BiLSTM, indobert, natural language processing, process innovation, ruangguru, sentiment analysis

Abstract

The rapid expansion of digital learning platforms has increased the reliance on user-generated reviews for service evaluation and quality monitoring. However, sentiment analysis of Indonesian reviews remains challenging due to the prevalence of long sentences, mixed sentiments, and ambiguous linguistic expressions. This study introduces a Hybrid VADER–IndoBERT framework designed to improve sentiment classification robustness on complex Indonesian texts. A dataset of 4,904 Ruangguru application reviews was collected through web scraping and processed using a hybrid pipeline consisting of preprocessing, translation-based silver-standard sentiment labeling with VADER, and class balancing via Random Oversampling (ROS). The IndoBERT classifier was evaluated against a Bidirectional Long Short-Term Memory (BiLSTM) baseline. Experimental results show that IndoBERT achieved 90.9% accuracy, outperforming BiLSTM at 86.4%, demonstrating the superiority of Transformer-based architectures in capturing long-range dependencies and handling ambiguous sentiment cues. These findings highlight the effectiveness of integrating lexicon-based and Transformer-based approaches to achieve more robust sentiment analysis on linguistically complex Indonesian texts.

Downloads

Download data is not yet available.

References

A. Haleem, M. Javaid, M. A. Qadri, and R. Suman, “Understanding The Role of Digital Technologies in Education: A review,” Sustainable Operations and Computers, vol. 3, pp. 275–285, 2022, doi: 10.1016/j.susoc.2022.05.004.

I. Y. Alyoussef, “Acceptance of E-Learning in Higher Education: The Role of Task-Technology Fit with The Information Systems Success Model,” Heliyon, vol. 9, no. 3, p. e13751, Mar. 2023, doi: 10.1016/j.heliyon.2023.e13751.

We Are Social, “Meltwater,” Digital 2024 Indonesia, 2024. Accessed: Sep. 28, 2025. [Online]. Available: https://wearesocial.com/id/wp-content/uploads/sites/ 19/2024/02/Digital_2024_Indonesia.pdf

Mussalimun, E. H. Khasby, G. I. Dzikrillah, and Muljono, “Comparison of K- N earest Neighbor (K -NN) and Naïve Bayes Algorithm for Sentiment Analysis on Google Play Store Textual Reviews,” in 2021 8th International Conference on Information Technology, Computer and Electrical Engineering (ICITACEE), Semarang, Indonesia: IEEE, Sep. 2021, pp. 180–184. doi: 10.1109/ICITACEE53184.2021.9617217.

R. Kurniawan, H. O. L. Wijaya, and R. P. Aprisusanti, “Sentiment Analysis of Google Play Store User Reviews on Digital Population Identity App Using K-Nearest Neighbors,” SISFOKOM, vol. 13, no. 2, pp. 170–178, Jun. 2024, doi: 10.32736/sisfokom.v13i2.2071.

A. A. Mohammad et al., “Customer Sentiment Analysis for Food and Beverage Development in Restaurants using AI in Jordan,” Data and Metadata, vol. 4, p. 922, Apr. 2025, doi: 10.56294/dm2025922.

S. Al-Natour and O. Turetken, “A Comparative Assessment of Sentiment Analysis and Star Ratings for Consumer Reviews,” International Journal of Information Management, vol. 54, p. 102132, Oct. 2020, doi: 10.1016/j.ijinfomgt.2020.102132.

R. I. Perwira, V. A. Permadi, D. I. Purnamasari, and R. P. Agusdin, “Domain-Specific Fine-Tuning of IndoBERT for Aspect-Based Sentiment Analysis in Indonesian Travel User-Generated Content,” Journal of Information Systems Engineering and Business Intelligence, vol. 11, no. 1, pp. 30–40, Mar. 2025, doi: 10.20473/jisebi.11.1.30-40.

A. S. Paramita, “Fine-Grained Sentiment Analysis Approach on Customer Reviews Based on Aspect-Level Emotion Detection,” J. Appl. Data Sci., vol. 6, no. 3, pp. 2235–2247, Sep. 2025, doi: 10.47738/jads.v6i3.964.

A. Mabrouk, R. P. D. Redondo, and M. Kayed, “Deep Learning-Based Sentiment Classification: A Comparative Survey,” IEEE Access, vol. 8, pp. 85616–85638, 2020, doi: 10.1109/ACCESS.2020.2992013.

Y. Mao, Q. Liu, and Y. Zhang, “Sentiment analysis methods, applications, and challenges: A systematic literature review,” Journal of King Saud University - Computer and Information Sciences, vol. 36, no. 4, p. 102048, Apr. 2024, doi: 10.1016/j.jksuci.2024.102048.

Z. A. Diekson, M. R. B. Prakoso, M. S. Q. Putra, M. S. A. F. Syaputra, S. Achmad, and R. Sutoyo, “Sentiment analysis for customer review: Case study of Traveloka,” Procedia Computer Science, vol. 216, pp. 682–690, 2023, doi: 10.1016/j.procs.2022.12.184.

S. Fahmi, L. Purnamawati, G. F. Shidik, M. Muljono, and A. Z. Fanani, “Sentiment Analysis of Student Review in Learning Management System Based on Sastrawi Stemmer and SVM-PSO,” in 2020 International Seminar on Application for Technology of Information and Communication (iSemantic), Semarang, Indonesia: IEEE, Sep. 2020, pp. 643–648. doi: 10.1109/iSemantic50169.2020.9234291.

Y. Handayani, A. R. Hakim, and Muljono, “Sentiment Analysis of Bank BNI User Comments Using the Support Vector Machine Method,” in 2020 International Seminar on Application for Technology of Information and Communication (iSemantic), Semarang, Indonesia: IEEE, Sep. 2020, pp. 202–207. doi: 10.1109/iSemantic50169.2020.9234230.

Z. Hameed and B. Garcia-Zapirain, “Sentiment Classification Using a Single-Layered BiLSTM Model,” IEEE Access, vol. 8, pp. 73992–74001, 2020, doi: 10.1109/ACCESS.2020.2988550.

K. Kusnawi and K. Anam, “Comparison ff Sentiment Labeling Using Textblob, Vader, and Flair in Public Opinion Analysis Post-2024 Presidential Inauguration with IndoBERT,” J. Tek. Inform. (JUTIF), vol. 6, no. 2, pp. 803–818, Apr. 2025, doi: 10.52436/1.jutif.2025.6.2.4015.

W. Christian, D. Adamlu, A. Yu, and D. Suhartono, “Leveraging IndoBERT and DistilBERT for Indonesian emotion classification in e-commerce reviews,” Procedia Computer Science, vol. 269, pp. 321–330, 2025, doi: 10.1016/j.procs.2025.08.284.

B. Setiawan, “A Review of Sentiment Analysis Applications in Indonesia Between 2023-2024,” Journal Information Engineering and Educational Technology, vol. 08, 2024, doi: https://doi.org/10.26740/jieet.v8n2.p71-83.

K. Barik and S. Misra, “Analysis of Customer Reviews with an Improved VADER Lexicon Classifier,” Journal of Big Data, vol. 11, no. 1, p. 10, Jan. 2024, doi: 10.1186/s40537-023-00861-x.

D. Marutho, Muljono, S. Rustad, and Purwanto, “Sentiment Analysis Optimization Using Vader Lexicon on Machine Learning Approach,” in 2022 International Seminar on Intelligent Technology and Its Applications (ISITIA), Surabaya, Indonesia: IEEE, Jul. 2022, pp. 98–103. doi: 10.1109/ISITIA56226.2022.9855341.

F. S. Lestari, H. Harliana, M. M. Huda, and T. Prabowo, “Sentiment Analysis of iPusnas Application Reviews on Google Play Using Support Vector Machine,” Proceedings of the International Seminar on Business, Education and Science, vol. 1, pp. 178–188, Oct. 2022, doi: 10.29407/int.v1i1.2656.

F. Koto, A. Rahimi, J. H. Lau, and T. Baldwin, “IndoLEM and IndoBERT: A Benchmark Dataset and Pre-trained Language Model for Indonesian NLP,” Nov. 02, 2020, arXiv: arXiv:2011.00677. doi: 10.48550/arXiv.2011.00677.

I. Gambo, R. Massenon, R. O. Ogundokun, S. Agarwal, and W. Pak, “Identifying and Resolving Conflict in Mobile Application Features Through Contradictory Feedback Analysis,” Heliyon, vol. 10, no. 17, p. e36729, Sep. 2024, doi: 10.1016/j.heliyon.2024.e36729.

H.-T. Duong and T.-A. Nguyen-Thi, “A review: preprocessing techniques and data augmentation for sentiment analysis,” Computational Social Networks, vol. 8, no. 1, p. 1, Dec. 2021, doi: 10.1186/s40649-020-00080-x.

S. Bengesi, T. Oladunni, R. Olusegun, and H. Audu, “A Machine Learning-Sentiment Analysis on Monkeypox Outbreak: An Extensive Dataset to Show the Polarity of Public Opinion From Twitter Tweets,” IEEE Access, vol. 11, pp. 11811–11826, 2023, doi: 10.1109/ACCESS.2023.3242290.

J. R. Landis and G. G. Koch, “The Measurement of Observer Agreement for Categorical Data,” Biometrics, vol. 33, no. 1, 1977, doi: https://doi.org/10.2307/2529310.

Hairani, A. Anggrawan, Muhammad Ridho Akbar, Khasnur Hidjah, and Muhammad Innuddin, “Optimizing Sentiment Analysis for Lombok Tourism Using SMOTE and Chi-Square with Machine Learning,” J. RESTI (Rekayasa Sist. Teknol. Inf.), vol. 9, no. 4, pp. 706–713, Jul. 2025, doi: 10.29207/resti.v9i4.6623.

D. A. Sani, “A Random Oversampling and BERT-based Model Approach for Handling Imbalanced Data in Essay Answer Correction,” J.INFOTEL, vol. 16, no. 4, pp. 729–739, Dec. 2024, doi: 10.20895/infotel.v16i4.1224.

A. He and M. Abisado, “Text Sentiment Analysis of Douban Film Short Comments Based on BERT-CNN-BiLSTM-Att Model,” IEEE Access, vol. 12, pp. 45229–45237, 2024, doi: 10.1109/ACCESS.2024.3381515.

B. Rahman and Maryani, “Optimizing Customer Satisfaction Through Sentiment Analysis: A BERT-Based Machine Learning Approach to Extract Insights,” IEEE Access, vol. 12, pp. 151476–151489, 2024, doi: 10.1109/ACCESS.2024.3478835.

W. Aljedaani, F. Rustam, S. Ludi, A. Ouni, and M. W. Mkaouer, “Learning Sentiment Analysis for Accessibility User Reviews,” in 2021 36th IEEE/ACM International Conference on Automated Software Engineering Workshops (ASEW), Melbourne, Australia: IEEE, Nov. 2021, pp. 239–246. doi: 10.1109/ASEW52652.2021.00053.