HyRoBERTa: Hybrid Robustly Optimized BERT Approach Model for Sentiment and Sarcasm Detection in Post-Flood Social Media Analysis
DOI:
https://doi.org/10.29207/resti.v10i1.6963Keywords:
hybrid representation model, post floods, RoBERTa, sarcasm, transformers-basedAbstract
The detection problem is a crucial step in sentiment classification because it strengthens the validity and reliability of the model's interpretation of ambiguous text, especially in complex social contexts such as post-disaster public communication. Without this detection, the model is prone to significant classification errors. This study presents a hybrid approach for sentiment analysis with sarcasm detection after a flood disaster by combining the RoBERTa model with sequential deep learning architectures such as GRU, LSTM, and BiLSTM. We used a dataset of 17,520 tweets that were pre-processed using cleaning, normalization, and tokenization. Then, the positive class is further detected to determine whether it is sarcasm. The model was trained using a transformer-based transfer learning method with a combination of hyperparameters: the number of epochs, batch size, dropout rate, and learning rate. The experimental results show that the RoBERTa-GRU model achieved the highest accuracy for sentiment classification at 97. 26%, whereas the RoBERTa-BiLSTM model excels in detecting sarcasm with an accuracy of 98. 74%. RoBERTa-BiLSTM excels in sarcasm detection because it provides a bidirectional sequential mechanism and better long-term memory, effectively leveraging RoBERTa's rich embedding to identify contextual contradictions that are characteristic of sarcasm. Meanwhile, RoBERTa-GRU succeeds in sentiment classification because its architecture is more concise yet effective enough to infer dominant sentiments that have been filtered from the robust representation provided by RoBERTa, making the model more efficient for less complex tasks.
Downloads
References
S. Yuliyanti and S. Sholihah, “Pemodelan Analisis Sentimen Masyarakat terhadap Adaptasi Kebiasaan Baru (AKB) mengunakan Algoritma Naïve Bayes,” MIND J., vol. 6, no. 2, pp. 155–167, 2021, doi: 10.26760/mindjournal.v6i2.155-167.
S. Yuliyanti, T. Djatna, and H. Sukoco, “Sentiment mining of community development program evaluation based on social media,” Telkomnika (Telecommunication Comput. Electron. Control., vol. 15, no. 4, pp. 1858–1864, 2017, doi: 10.12928/TELKOMNIKA.v15i4.4633.
K. L. Tan, C. P. Lee, K. M. Lim, and K. S. M. Anbananthen, “Sentiment Analysis With Ensemble Hybrid Deep Learning Model,” IEEE Access, vol. 10, pp. 103694–103704, 2022, doi: 10.1109/ACCESS.2022.3210182.
T. Bey Kusuma, I. Komang, and A. Mogi, “Implementasi BERT pada Analisis Sentimen Ulasan Destinasi Wisata Bali,” J. Elektron. Ilmu Komput. Udayana, vol. 12, no. 2, pp. 409–420, 2023 https://doi.org/10.51519/journalisi.v6i3.814.
N. Singh, N. Roy, and A. Gangopadhyay, “Localized Flood DetectionWith Minimal Labeled Social Media Data Using Transfer Learning,” 2020, [Online]. Available: https://arxiv.org/abs/2003.04973.
Y. Tasya and R. A. Putri, “Klasifikasi Tingkat Kerawanan Banjir Wilayah Medan Menggunakan Metode Naive Bayes Dan Algoritma J48,” INTECOMS J. Inf. Technol. Comput. Sci., vol. 6, no. 2, pp. 657–665, 2023, doi: 10.31539/intecoms.v6i2.7392.
B. Choubin, E. Moradi, M. Golshan, J. Adamowski, F. Sajedi-Hosseini, and A. Mosavi, “An ensemble prediction of flood susceptibility using multivariate discriminant analysis, classification and regression trees, and support vector machines,” Sci. Total Environ., vol. 651, pp. 2087–2096, Feb. 2019, doi: 10.1016/j.scitotenv.2018.10.064.
A. M. Maksun, Y. A. Sari, and B. Rahayudi, “Analisis Sentimen pada Twitter Bencana Alam di Kalimantan Selatan menggunakan Metode Naïve Bayes,” J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 5, no. 12, pp. 5614–5621, 2021, [Online]. Available: https://j-ptiik.ub.ac.id/index.php/j-ptiik/article/view/10307
R. Dhawale, C. J. Schuster-Wallace, and A. Pietroniro, “Assessing the multidimensional nature of flood and drought vulnerability index: A systematic review of literature,” Int. J. Disaster Risk Reduct., vol. 112, no. August, p. 104764, 2024, doi: 10.1016/j.ijdrr.2024.104764.
T. Dadu and K. Pant, “Sarcasm detection using context separators in online discourse,” Proc. Annu. Meet. Assoc. Comput. Linguist., pp. 51–55, 2020, doi: 10.18653/v1/P17.
P. Dubey, P. Dubey, and P. N. Bokoro, “Unpacking Sarcasm: A Contextual and Transformer-Based Approach for Improved Detection,” Computers, vol. 14, no. 3, pp. 1–21, 2025, doi: 10.3390/computers14030095.
F. Ullah, A. Alsirhani, M. M. Alshahrani, A. Alomari, H. Naeem, and S. A. Shah, “Explainable Malware Detection System Using Transformers-Based Transfer Learning and Multi-Model Visual Representation,” Sensors, vol. 22, no. 18, 2022, doi: 10.3390/s22186766.
R. H. Yahya, W. Maharani, and R. Wijaya, “Disaster Management Sentiment Analysis Using the BiLSTM Method,” J. Media Inform. Budidarma, vol. 7, no. 1, p. 501, 2023, doi: 10.30865/mib.v7i1.5573.
K. L. Tan, C. P. Lee, K. S. M. Anbananthen, and K. M. Lim, “RoBERTa-LSTM: A Hybrid Model for Sentiment Analysis With Transformer and Recurrent Neural Network,” IEEE Access, vol. 10, pp. 21517–21525, 2022, doi: 10.1109/ACCESS.2022.3152828.
M. Chiny, M. Chihab, and Y. Chihab, “LSTM , VADER and TF-IDF based Hybrid Sentiment Analysis Model,” vol. 12, no. 7, pp. 265–275, 2021. Doi: 10.14569/IJACSA.2021.0120730.
M. A. Jahin, M. S. H. Shovon, M. F. Mridha, M. R. Islam, and Y. Watanobe, “A hybrid transformer and attention based recurrent neural network for robust and interpretable sentiment analysis of tweets,” Sci. Rep., vol. 14, no. 1, p. 24882, 2024, doi: 10.1038/s41598-024-76079-5.
R. Silva Barbon and A. T. Akabane, “Towards Transfer Learning Techniques—BERT, DistilBERT, BERTimbau, and DistilBERTimbau for Automatic Text Classification from Different Languages: A Case Study,” Sensors, vol. 22, no. 21, 2022, doi: 10.3390/s22218184.
S. E. Uthirapathy and D. Sandanam, “Topic Modelling and Opinion Analysis on Climate Change Twitter Data Using LDA and BERT Model.,” Procedia Comput. Sci., vol. 218, no. 2022, pp. 908–917, 2022, doi: 10.1016/j.procs.2023.01.071.
N. Yusliani, A. Nabilah, M. Raihan Habibullah, A. Darmawahyuni, and G. Athalina, “Comparing Word Representation BERT and RoBERTa in Keyphrase Extraction using TgGAT,” J. RESTI, vol. 9, no. 2, pp. 250–257, 2025, doi: 10.29207/resti.v9i2.6279.
G. Hakim, T. N. Fatyanosa, and A. W. Widodo, “Analisis Sentimen Masyarakat terhadap Kereta Cepat Whoosh pada Platform X menggunakan IndoBERT,” vol. 1, no. 1, pp. 1–10, 2023, https://doi.org/10.24076/joism.2026v7i2.2449.
D. Hazarika, S. Poria, S. Gorantla, E. Cambria, R. Zimmermann, and R. Mihalcea, “CASCADE: Contextual sarcasm detection in online discussion forums,” COLING 2018 - 27th Int. Conf. Comput. Linguist. Proc., pp. 1837–1848, 2018 https://aclanthology.org/volumes/C18-1/.
T. Wahyuningsih et al., “Text Mining an Automatic Short Answer Grading (ASAG), Comparison of Three Methods of Cosine Similarity, Jaccard Similarity and Dice’s Coefficient,” J. Teknol. Inf. dan Ilmu Komput., vol. 1, no. 2, pp. 343–348, 2021, doi: 10.17762/turcomat.v12i3.938.
M. Polignano, P. Basile, M. de Gemmis, G. Semeraro, and V. Basile, “AlBERTo: Italian BERT language understanding model for NLP challenging tasks based on tweets,” CEUR Workshop Proc., vol. 2481, 2019. https://ceur-ws.org/Vol-2481/paper57.pdf.
J. Nasional, S. Informasi, E. Teks, E. Marthen, and I. Novita, “Optimasi RoBERTa dengan Hyperparameter Tuning untuk Deteksi,” vol. 03, pp. 240–248, 2024. https://doi.org/10.52436/1.jutif.2025.6.5.4897.
S. Informasi, F. Teknik, and U. Bakrie, “Perbandingan Performa Algoritma VADER dan RoBERTa pada Twitter Comparison of VADER and RoBERTa Algorithm Performance on,” vol. 13, pp. 1547–1557, 2024.
C. A. Bahri and L. H. Suadaa, “Aspect-Based Sentiment Analysis in Bromo Tengger Semeru National Park Indonesia Based on Google Maps User Reviews,” IJCCS (Indonesian J. Comput. Cybern. Syst., vol. 17, no. 1, p. 79, 2023, doi: 10.22146/ijccs.77354.
F. Sufi, “A Sustainable Way Forward: Systematic Review of Transformer Technology in Social-Media-Based Disaster Analytics,” Sustain. , vol. 16, no. 7, 2024, doi: 10.3390/su16072742.
W. Chen et al., “Flood susceptibility modelling using novel hybrid approach of reduced-error pruning trees with bagging and random subspace ensembles,” J. Hydrol., vol. 575, pp. 864–873, Aug. 2019, doi: 10.1016/j.jhydrol.2019.05.089.
C. Sun, X. Qiu, Y. Xu, and X. Huang, “How to Fine-Tune BERT for Text Classification?,” Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 11856 LNAI, no. 2, pp. 194–206, 2019, doi: 10.1007/978-3-030-32381-3_16.
A. U. Khan, A. Mazaheri, N. da Vitoria Lobo, and M. Shah, “MMFT-BERT: Multimodal fusion transformer with BERT encodings for visual question answering,” Find. Assoc. Comput. Linguist. Find. ACL EMNLP 2020, pp. 4648–4660, 2020, doi: 10.18653/v1/2020.findings-emnlp.417.
M. G. Villar, J. B. Ballester, I. De, T. Diez, and I. Ashraf, “Analyzing Sentiments Regarding ChatGPT Using Novel BERT : A Machine Learning Approach,” pp. 1–29, 2023. https://www.mdpi.com/2078-2489/14/9/474.
Downloads
Published
Issue
Section
License
Copyright in each article belongs to the author
- The author acknowledges that the RESTI Journal (System Engineering and Information Technology) is the first publisher to publish with a license Creative Commons Attribution 4.0 International License.
- Authors can enter writing separately, arrange the non-exclusive distribution of manuscripts that have been published in this journal into other versions (eg sent to the author's institutional repository, publication in a book, etc.), by acknowledging that the manuscript has been published for the first time in the RESTI (Rekayasa Sistem dan Teknologi Informasi) journal ;





