Deep Sentiment Analysis of Halal Tourism: An Enhanced IndoBERT with Attention-Based and Coupling Latent Sampling on User-Generated Feedback

Rio Andika Malik; Marta Riri  Frimadani

doi:10.29207/resti.v10i4.7378

Authors

Rio Andika Malik Universitas Perintis Indonesia
Marta Riri Frimadani Universitas Andalas

DOI:

https://doi.org/10.29207/resti.v10i4.7378

Keywords:

contrastive learning, halal tourism, imbalanced data, IndoBERT, SMOTE

Abstract

Public sentiment analysis regarding government policies, particularly in niche sectors like Halal tourism, often faces significant computational challenges due to data scarcity and severe class imbalance. Traditional augmentation methods, such as Synthetic Minority Over-sampling Technique (SMOTE) applied to raw text or TF-IDF vectors, often degrade semantic integrity, while standard fine-tuning of pre-trained models like IndoBERT tends to bias predictions toward majority classes. To address these limitations, this study proposes IndoBERT-LSS (Latent Space Sampling), a novel decoupled two-stage deep learning framework. The first stage employs a Representation Learning approach integrating an Attention-Pooling mechanism with Supervised Contrastive Loss (SupCon) to enforce compact intra-class clustering in the embedding space. The second stage introduces a Latent Space Sampling strategy, where SMOTE is applied to the extracted high-dimensional embeddings rather than the raw text, followed by a final classification using a Multi-Layer Perceptron. Validated on a dataset of 1,051 textual responses regarding Halal tourism in Pariaman, Indonesia, the proposed model achieved an accuracy of 93.84% and a macro F1-score of 0.94. Notably, the model demonstrated exceptional robustness in identifying minority classes, achieving a 0.99 F1-score for neutral sentiments. These results conclude that decoupling representation learning from feature balancing in the latent space significantly enhances model performance on imbalanced short-text datasets compared to standard fine-tuning baselines.

Downloads

Download data is not yet available.

References

[1] J. Chen, J. Wu, D. Wang, and B. Stantic, “Beyond static rankings: A tourist experience-driven approach to measure destination competitiveness,” Tour. Manag., vol. 106, no. August 2024, p. 105022, 2025, doi: 10.1016/j.tourman.2024.105022.

[2] A. Ariyanto, M. Adam, S. Musnadi, and M. A. Djalil, “Empowering and Influencing Factors of Intellectual and Spiritual Capital in Improving Organizational Performance at Tourism Institutions,” J. Ecohumanism, vol. 3, no. 7, pp. 5074–5085, 2024, doi: 10.62754/joe.v3i7.4616.

[3] L. Alfarajat, “Halal Food and Medical Tourism: Muslim Patients’ Experiences and Satisfaction in South Korea,” J. Relig. Health, vol. 63, no. 5, pp. 3274–3290, 2022, doi: 10.1007/s10943-022-01727-x.

[4] Repelita, N. Iskandar, and Mursal, “Halal Without Label: Implementation of Maqāṣid Al-Sharī’Ah in Community-Based Tourism in Kerinci, Indonesia,” Malaysian J. Syariah Law, vol. 12, no. 1, pp. 192–205, 2024, doi: 10.33102/mjsl.vol12no1.477.

[5] A. Holder, L. Ruhanen, G. Walters, and M. Mkono, “‘ I think … I feel …’ : using projective techniques to explore socio-cultural aversions towards Indigenous tourism,” Tour. Manag., vol. 98, no. January 2022, p. 104778, 2023, doi: 10.1016/j.tourman.2023.104778.

[6] S. J. Page and M. Duignan, “Progress in Tourism Management : Is urban tourism a paradoxical research domain ? Progress since 2011 and prospects for the future,” Tour. Manag., vol. 98, no. March, p. 104737, 2023, doi: 10.1016/j.tourman.2023.104737.

[7] I. M. Permana and F. Adam, “Impact of Islamic attributes and destination image on tourist satisfaction and revisit intentions in halal tourism: A study in Banda Aceh,” Int. J. Adv. Appl. Sci., vol. 11, no. 4, pp. 118–127, 2024, doi: 10.21833/ijaas.2024.04.014.

[8] S. Pongsakornrungsilp, P. Pongsakornrungsilp, T. Chouykaew, H. Niyomdecha, S. Chansamran, and Y. He, “Integrating Sustainability and Cultural Sensitivity: Clustering Muslim Tourist Lifestyles in the Andaman Coastal Cluster, Thailand,” Sustain., vol. 16, no. 21, pp. 1–15, 2024, doi: 10.3390/su16219542.

[9] Syufa’at, E. Z. Mutaqin, A. Dharin, and Mutholaah, “Implementation of Chse Certificate for Tourism Object Management in Banyumas (Halal Tourism Perspective),” Rev. Gest. Soc. e Ambient., vol. 18, no. 9, pp. 1–18, 2024, doi: 10.24857/rgsa.v18n9-069.

[10] D. Rusyaida, “Local wisdom as Halal tourism model in West Sumatera: an `Urf perspective,” Indones. J. Islam Muslim Soc., vol. 14, no. 2, pp. 259–287, 2024, doi: 10.18326/ijims.v14i2.259-287.

[11] R. Polus, R. Nautiyal, A. Nautiyal, and P. Kari, “Caught in the maze : Exploration of the ‘ tourist trap ’ phenomenon,” Tour. Manag., vol. 106, no. June 2024, p. 105009, 2025, doi: 10.1016/j.tourman.2024.105009.

[12] L. Li, “Insights into People ’ s Perceptions Towards Urban Public Spaces Through Analysis of Social Media Reviews : A Case Study of Shanghai †,” pp. 1–19, 2025, doi: 10.3390/buildings15173033.

[13] A. Boukis, D. Caruelle, and L. Harris, “Dealing with travel influencers : Opportunity or threat ? ☆,” Tour. Manag., vol. 108, no. March 2024, p. 105112, 2025, doi: 10.1016/j.tourman.2024.105112.

[14] L. Xie-carson, T. Magor, P. Benckendorff, and K. Hughes, “All hype or the real deal ? Investigating user engagement with virtual influencers in tourism,” Tour. Manag., vol. 99, no. April, p. 104779, 2023, doi: 10.1016/j.tourman.2023.104779.

[15] Sunarmo, M. Thurisaina, A. Amri, and A. Sagita, “Effects of Halal Awareness, Service Quality and Promotion on Visitor Decisions Through Destination Imagery as a Mediator,” Indones. J. Halal Res., vol. 6, no. 1, pp. 13–25, 2024, doi: 10.15575/ijhar.v6i1.19489.

[16] A. Sutono, Sumaryadi, W. Rahtomo, F. F. Puksi, and O. Permana, “Re-modelling Muslim-friendly tourism value chain ecosystem: the case study of West Sumatera, Indonesia,” Cogent Soc. Sci., vol. 10, no. 1, p., 2024, doi: 10.1080/23311886.2024.2361536.

[17] V. Arefieva, R. Egger, and J. Yu, “A machine learning approach to cluster destination image on Instagram,” Tour. Manag., vol. 85, no. March, p. 104318, 2021, doi: 10.1016/j.tourman.2021.104318.

[18] G. Hughes, S. E. Shaw, J. Bowley, J. L. Darbyshire, and T. J. Stephens, “Choosing wisely? A frame analysis of the evolution and adoption of shared decision making in the UK,” SSM - Qual. Res. Heal., vol. 8, no. December 2024, p. 100579, 2025, doi: 10.1016/j.ssmqr.2025.100579.

[19] K. Miazek and K. Bocian, “When AI is fairer than humans: The role of egocentrism in moral and fairness judgments of AI and human decisions,” Comput. Hum. Behav. Reports, vol. 19, no. May, p. 100719, 2025, doi: 10.1016/j.chbr.2025.100719.

[20] J. Cuadrado, E. Martinez, J. C. Martinez-Santos, and E. Puertas, “The media framing dataset: Analyzing news narratives in Mexico and Colombia,” Data Br., vol. 58, p. 111284, 2025, doi: 10.1016/j.dib.2025.111284.

[21] V. A. Cordero-Vinueza, T. van Dijk, C. Lamker, and F. Niekerk, “Making child-friendly cities: A quest for local agenda-setting,” Cities, vol. 161, no. March, p. 105882, 2025, doi: 10.1016/j.cities.2025.105882.

[22] F. M. Suud, K. S. Rouzi, M. Al-Fairusy, T. Na’imah, H. Harinawati, and R. B. Yasa, “Tourism Education Development on the Coastline: Challenges Empirical Study on Islamic Law Province,” J. Ilm. Peuradeun, vol. 12, no. 3, pp. 1289–1310, 2024, doi: 10.26811/peuradeun.v12i3.1172.

[23] T. Pang, J. Liu, L. Han, H. Liu, and D. Yan, “A Deep Learning-Based Analysis of Customer Concerns and Satisfaction: Enhancing Sustainable Practices in Luxury Hotels,” Sustain., vol. 17, no. 8, 2025, doi: 10.3390/su17083603.

[24] D. Erdoğan et al., “Developing a Deep Learning-Based Sentiment Analysis System of Hotel Customer Reviews for Sustainable Tourism,” Sustain., vol. 17, no. 13, 2025, doi: 10.3390/su17135756.

[25] Y.-W. Chen and S.-C. Pei, “Always Clear Days: Degradation Type and Severity Aware All-In-One Adverse Weather Removal,” IEEE Access, vol. 13, no. January, pp. 7650–7662, 2023, doi: 10.1109/ACCESS.2025.3526168.

[26] F. Marchesani and G. Testa, “Understanding municipalities and visitors tourism perspectives : An AI-driven analysis of Instagram posts in Italian and French villages,” Tour. Manag., vol. 112, no. August 2025, p. 105279, 2026, doi: 10.1016/j.tourman.2025.105279.

[27] A. S. Ekakristi, A. F. Wicaksono, and R. Mahendra, “Intermediate-task transfer learning for Indonesian NLP tasks,” Nat. Lang. Process. J., vol. 12, no. May, p. 100161, 2025, doi: 10.1016/j.nlp.2025.100161.

[28] G. Kątek, R. Kozik, A. Pawlicka, M. Pawlicki, and M. Choraś, “In depth analysis for securing the truth: Addressing the fake news challenge with graph neural networks,” Neurocomputing, vol. 654, no. June, p. 131327, 2025, doi: 10.1016/j.neucom.2025.131327.

[29] K. Mnassri, R. Farahbakhsh, and N. Crespi, “Multilingual Hate Speech Detection: A Semi-Supervised Generative Adversarial Approach,” Entropy, vol. 26, no. 4, 2024, doi: 10.3390/e26040344.

[30] G. Z. Nabiilah, S. Y. Prasetyo, Z. N. Izdihar, and A. S. Girsang, “BERT base model for toxic comment analysis on Indonesian social media,” Procedia Comput. Sci., vol. 216, no. 2022, pp. 714–721, 2022, doi: 10.1016/j.procs.2022.12.188.

[31] A. Jazuli, Widowati, and R. Kusumaningrum, “Optimizing Aspect-Based Sentiment Analysis Using BERT for Comprehensive Analysis of Indonesian Student Feedback,” Appl. Sci., vol. 15, no. 1, pp. 1–28, 2025, doi: 10.3390/app15010172.

[32] S. Kang, “LLM-Guided Weighted Contrastive Learning with Topic-Aware Masking for Efficient Domain Adaptation: A Case Study on Pulp-Era Science Fiction,” Electron. MDPI, vol. 14, no. 17, pp. 1–24, 2025, doi: 10.3390/electronics14173351.

[33] R. Pratiwi et al., “The Collaboration of Penta Helix to Develop Halal Tourism Villages in Batang, Cental Java,” Int. J. Sustain. Dev. Plan., vol. 19, no. 7, pp. 2753–2761, 2024, doi: 10.18280/ijsdp.190732.

[34] J. Amarta, D. Tyogo, D. Utomo, and B. Ignasio, “ScienceDirect ScienceDirect Deep Learning And Machine Learning For Indonesian Online Gambling Website Promotion Detection,” Procedia Comput. Sci., vol. 269, pp. 979–992, 2025, doi: 10.1016/j.procs.2025.09.040.

[35] G. Z. Nabiilah, S. Y. Prasetyo, Z. N. Izdihar, and A. S. Girsang, “ScienceDirect ScienceDirect 7th International Conference on Computer Science and Computational Intelligence 2022 BERT base model for toxic comment analysis on BERT base model for toxic comment analysis on Indonesian social media Indonesian social media,” Procedia Comput. Sci., vol. 216, no. 2022, pp. 714–721, 2023, doi: 10.1016/j.procs.2022.12.188.

[36] F. Dornaika, Z. Ibrahim, J. Charafeddine, and A. Bosaghzadeh, “Towards dynamic self-training for scalable semi-supervised learning on graphs,” Neurocomputing, vol. 654, no. August, p. 131223, 2025, doi: 10.1016/j.neucom.2025.131223.

[37] S. K. Bharti et al., “Text-Based Emotion Recognition Using Deep Learning Approach,” Comput. Intell. Neurosci., vol. 2022, 2022, doi: 10.1155/2022/2645381.

[38] N. R. P. Syallya, A. A. Pravitasari, and A. Helen, “NLP-Based Intent Classification Model for Academic Curriculum Chatbots in Universities Study Programs,” J. RESTI, vol. 9, no. 1, pp. 111–117, 2025, doi: 10.29207/resti.v9i1.6276.

[39] S. Dehghan and M. F. Amasyali, “SupMPN: Supervised Multiple Positives and Negatives Contrastive Learning Model for Semantic Textual Similarity,” Appl. Sci., vol. 12, no. 19, 2022, doi: 10.3390/app12199659.

[40] X. Huang et al., “Optimizing sleep staging on multimodal time series: Leveraging borderline synthetic minority oversampling technique and supervised convolutional contrastive learning,” Comput. Biol. Med., vol. 166, no. August, p. 107501, 2023, doi: 10.1016/j.compbiomed.2023.107501.

[41] D. Gonzalez-Cuautle et al., “Synthetic minority oversampling technique for optimizing classification tasks in botnet and intrusion-detection-system datasets,” Appl. Sci., vol. 10, no. 3, 2020, doi: 10.3390/app10030794.

[42] A. Kummer, T. Ruppert, T. Medvegy, and J. Abonyi, “Machine learning-based software sensors for machine state monitoring - The role of SMOTE-based data augmentation,” Results Eng., vol. 16, no. November, 2022, doi: 10.1016/j.rineng.2022.100778.

[43] M. Yusoff, Y. Mahmud, P. A. R. Azmi, and M. T. M. Sallehud-din, “The improvement of SMOTE-ENN-XGBoost through Yeo Johnson strategy on Dissolved Gas Analysis dataset,” Energy Reports, vol. 13, no. April, pp. 6281–6290, 2025, doi: 10.1016/j.egyr.2025.05.013.

[44] M. Ali, “Classification of imbalanced travel mode choice dataset with SMOTE and prediction using interpretable machine learning,” Sustain. Futur., vol. 10, no. August, p. 101119, 2025, doi: 10.1016/j.sftr.2025.101119.

[45] I. Aruleba and Y. Sun, “Enhanced credit risk prediction using deep learning and SMOTE-ENN resampling,” Mach. Learn. with Appl., vol. 21, no. April, p. 100692, 2025, doi: 10.1016/j.mlwa.2025.100692.

[46] H. Jakha, S. El Houssaini, M.-A. El Houssaini, S. Ajjaj, and A. Hadir, “Optimizing Sentiment Analysis in Multilingual Balanced Datasets: A New Comparative Approach to Enhancing Feature Extraction Performance with ML and DL Classifiers,” Appl. Syst. Innov., vol. 8, no. 4, p. 104, 2025, doi: 10.3390/asi8040104.

[47] Z. He and H. Wang, “Fake News Detection Based on Contrastive Learning and Cross-Modal Interaction,” Electron. MDPI, vol. 14, no. 7, pp. 1–24, 2025, doi: 10.3390/electronics14173351.

[48] H. Honda and M. Hagiwara, “Context-dependent neuro-symbolic AI through self-supervised learning with large language models,” Neurocomputing, vol. 654, no. August, p. 131269, 2025, doi: 10.1016/j.neucom.2025.131269.

[49] E. Hashmi, S. Y. Yayilgan, M. M. Yamin, M. Abomhara, and M. Ullah, “Self-supervised hate speech detection in Norwegian texts with lexical and semantic augmentations,” Expert Syst. Appl., vol. 264, no. April 2024, p. 125843, 2025, doi: 10.1016/j.eswa.2024.125843.

[50] J. W. Bi, Y. Liu, Z. P. Fan, and J. Zhang, “Wisdom of crowds: Conducting importance-performance analysis (IPA) through online reviews,” Tour. Manag., vol. 70, no. March 2018, pp. 460–478, 2019, doi: 10.1016/j.tourman.2018.09.010.

[51] L. C. Cheng, K. Chen, M. C. Lee, and K. M. Li, “User-Defined SWOT analysis – A change mining perspective on user-generated content,” Inf. Process. Manag., vol. 58, no. 5, p. 102613, 2021, doi: 10.1016/j.ipm.2021.102613.

[52] L. Anthony, “Concordancing with AI : Applications of word and sentence embeddings,” Appl. Corpus Linguist., vol. 5, no. 3, p. 100164, 2025, doi: 10.1016/j.acorp.2025.100164.

[53] L. Kong, W. Yang, and F. Wei, “Implicit Emotion Analysis Based on Improved Supervised Contrastive Learning,” Electron., vol. 12, no. 10, pp. 1–12, 2023, doi: 10.3390/electronics12102172.

[54] Q. Wang, W. Zhang, T. Lei, and D. Peng, “Grouped Contrastive Learning of Self-Supervised Sentence Representation,” Appl. Sci., vol. 13, no. 17, 2023, doi: 10.3390/app13179873.

[55] H. Xiao, F. Dornaika, J. Charafeddine, and J. Bi, “Metric learning-enhanced semi-supervised Graph Convolutional Network for multi-view learning,” Inf. Fusion, vol. 124, no. October 2024, p. 103420, 2025, doi: 10.1016/j.inffus.2025.103420.

[56] C. Ohams, S. Nair, S. Bhattasali, and P. Resnik, “A predictive coding model for online sentence processing $,” J. Mem. Lang., vol. 146, no. August 2025, p. 104705, 2026, doi: 10.1016/j.jml.2025.104705.

[57] B. Zhu and W. Pan, “Heliyon Chinese text classification method based on sentence information enhancement and feature fusion,” Heliyon, vol. 10, no. 17, p. e36861, 2024, doi: 10.1016/j.heliyon.2024.e36861.

[58] O. Wallaart, F. Frasincar, and F. Van Der Knaap, “Knowledge-Based Systems Weakly-supervised sentence-based aspect category and sentiment classification,” vol. 317, no. March, 2025, doi: 10.1016/j.knosys.2025.113403.

[59] S. Begum, M. U. Ahmed, and S. Barua, “Intelligent Systems with Applications Attention-based fuzzy neural networks for self-supervised data annotation Md Rakibul Islam,” Intell. Syst. with Appl., vol. 28, no. November, p. 200610, 2025, doi: 10.1016/j.iswa.2025.200610.

[60] T. Wu, M. Wang, Y. Xi, and Z. Zhao, “Neurocomputing Intent recognition model based on sequential information and sentence features,” Neurocomputing, vol. 566, no. June 2023, p. 127054, 2024, doi: 10.1016/j.neucom.2023.127054.

[61] K. Bannigan, “Results in Engineering EnCo SupCon : Entropy-driven supervised contrastive learning for discriminative feature representations from remote sensing imagery,” Results Eng., vol. 29, no. November 2025, p. 108311, 2026, doi: 10.1016/j.rineng.2025.108311.

[62] L. Alexander, R. Gian, A. Ananta, J. Harefa, L. Alexander, and K. Jingga, “ScienceDirect ScienceDirect Evaluating IndoBERT for Indonesian Hoax News Detection : A Evaluating IndoBERT for Ensemble Indonesian Hoax News Detection : Comparative Study with and Models Comparative Study with Ensemble and CNN-LSTM Models,” Procedia Comput. Sci., vol. 269, pp. 1625–1633, 2025, doi: 10.1016/j.procs.2025.09.105.

[63] W. Christian et al., “Leveraging Leveraging IndoBERT IndoBERT and and DistilBERT DistilBERT for for Indonesian Indonesian emotion emotion Leveraging IndoBERT and DistilBERT for Indonesian classification in reviews classification in e-commerce reviews emotion classification in e-commerce reviews ScienceDirect,” Procedia Comput. Sci., vol. 269, pp. 321–330, 2025, doi: 10.1016/j.procs.2025.08.284.

[64] M. Alfhi, “ScienceDirect Online Gambling Promotion Detection in Indonesian YouTube Comments Using Semi-Supervised IndoBERT Classification,” Procedia Comput. Sci., vol. 269, pp. 1269–1278, 2025, doi: 10.1016/j.procs.2025.09.068.

[65] E. Supriyadi and P. Nurhuda, “ScienceDirect ScienceDirect Sentiment Analysis of TikTok User Comments on QRIS Sentiment Analysis TikTok User Comments on QRIS Adoption in of Indonesia Using IndoBERT Adoption in Indonesia Using IndoBERT,” Procedia Comput. Sci., vol. 269, pp. 121–130, 2025, doi: 10.1016/j.procs.2025.08.265.

[66] R. A. Malik, F. Riza, and S. Defit, “Quickly Assess the Acceptability Sentiment of White Paracetamol Intake Using KNN-SMOTE Based On Receptive Deciding,” Lontar Komput. J. Ilm. Teknol. Inf., vol. 15, no. 1, p. 51, 2024, doi: 10.24843/lkjiti.2024.v15.i01.p05.

[67] S. M. Octafia, R. A. Malik, A. Weriframayeni, and D. Delpa, “IndoBERT-SupCon : A Supervised Contrastive Learning Model for Analyzing Public Perception on Halal Tourism,” vol. 7, no. 1, pp. 218–231, 2026, doi: 10.47738/jads.v7i1.1045.

[68] M. K. Anam et al., “Improved Performance of Hybrid GRU-BiLSTM for Detection Emotion on Twitter Dataset,” J. Appl. Data Sci., vol. 6, no. 1, pp. 354–365, 2025, doi: 10.47738/jads.v6i1.459.

[69] P. P. Putra, M. K. Anam, A. S. Chan, A. Hadi, and N. Hendri, “Optimizing Sentiment Analysis on Imbalanced Hotel Review Data Using SMOTE and Ensemble Machine Learning Techniques,” J. Appl. Data Scie, vol. 6, no. 2, pp. 936–951, 2025, doi: 10.47738/jads.v6i2.618.

[70] C. Owen and D. Suhartono, “Intent Classification in Artificial Intelligence-Based Customer Service Chatbot for E-Wallet Service Providers,” Int. J. Comput. Digit. Syst., vol. 17, no. 1, pp. 1–9, 2025, doi: 10.12785/ijcds/1571046212.