Improving Government Helpdesk Service  With an AI-Powered Chatbot Built on the Rasa Framework

Wirat Moko Hadi Sasmita; Surya Sumpeno; Reza Fuad Rachmadi

doi:10.29207/resti.v9i2.6293

Wirat Moko Hadi Sasmita Institut Teknologi Sepuluh Nopember
Surya Sumpeno Institut Teknologi Sepuluh Nopember
Reza Fuad Rachmadi Institut Teknologi Sepuluh Nopember

DOI: https://doi.org/10.29207/resti.v9i2.6293

Keywords: Chatbot, Rasa, Helpdesk, Natural Language Understanding

Abstract

Helpdesk services are an important component in supporting Information Technology (IT) services. The helpdesk operates as the initial interface for managing and resolving concerns. Helpdesk helps user to get solutions when facing problems while using an IT service. This research focuses on the impact of artificial intelligence (AI)-powered chatbots on the performance of the initial response of government helpdesk services. The chatbot is designed to improve service performance by quickly identifying and classifying reported issues and automatically responding to messages, enabling faster responses. The research proposed a new System Design of a helpdesk system with an AI-based chatbot. The data used comes from Telegram group chat logs, exported in JSON format. We find that the Rasa NLU model with DIET Classifier successfully achieved an accuracy rate of 0.825 in classifying intents, with the precision value of 0.838, recall of 0.829, and F1 score of 0.821 using a Rasa model with cross-validation, where folds is 5 in evaluation. And initial response time was highly improved after using chatbot artificial intelligence from more than 3 hours on the telegram group helpdesk based to an average of 2.15 seconds. These research results suggest AI-Chatbot-based ability to assist the helpdesk team in handling user queries and reports, and improving initial time response.

Downloads

Download data is not yet available.

References

C. Cassandra, S. Hartono, and M. Karsen, “Online Helpdesk Support System for Handling Complaints and Service,” in 2019 International Conference on Information Management and Technology (ICIMTech), 2019, pp. 314–319. doi: 10.1109/ICIMTech.2019.8843726.

Presiden Republik Indonesia, Peraturan Presiden Nomor 82 Tahun 2023 Percepatan Transformasi Digital dan Keterpaduan Layanan Digital Nasional. Peraturan Presiden, 2023.

Presiden Republik Indonesia, Nomor 95 Tahun 2018 Tentang Sistem Pemerintahan Berbasis Elektronik. Peraturan Presiden, 2018.

A. Følstad et al., “Future directions for chatbot research: an interdisciplinary research agenda,” Computing, vol. 103, no. 12, pp. 2915–2942, Dec. 2021, doi: 10.1007/s00607-021-01016-7.

A. Abdellatif, K. Badran, D. E. Costa, and E. Shihab, “A Comparison of Natural Language Understanding Platforms for Chatbots in Software Engineering,” IEEE Transactions on Software Engineering, vol. 48, no. 8, pp. 3087–3102, Aug. 2022, doi: 10.1109/TSE.2021.3078384.

Rasa Technologies GmbH, “Introduction to Rasa Open Source & Rasa Pro.” Accessed: Aug. 06, 2024. [Online]. Available: https://rasa.com/docs/rasa/

K. N. Lam, N. N. Le, and J. Kalita, “Building a Chatbot on a Closed Domain using RASA,” in ACM International Conference Proceeding Series, Association for Computing Machinery, Dec. 2020, pp. 144–148. doi: 10.1145/3443279.3443308.

W. Hadikurniawati, S. Wijono, D. Manongga, I. Sembiring, and K. D. Hartomo, “Toddler Stunting Consulting Chatbot using Rasa Framework,” Jurnal Rekayasa Elektrika, vol. 19, no. 4, Dec. 2023, doi: 10.17529/jre.v19i4.33014.

F. A. Anam and S. N. Anwar, “Telegram Chatbot Implementation Using Rasa Framework to Recommend Tourism in Semarang City,” Mar. 2024.

L. Fauzia, R. B. Hadiprakoso, and Girinoto, “Implementation of Chatbot on University Website Using RASA Framework,” in 2021 4th International Seminar on Research of Information Technology and Intelligent Systems (ISRITI), 2021, pp. 373–378. doi: 10.1109/ISRITI54043.2021.9702821.

A. Paranjape, Y. Patwardhan, V. Deshpande, A. Darp, and J. Jagdale, “Voice-Based Smart Assistant System for Vehicles Using RASA,” in 2023 International Conference on Computational Intelligence, Networks and Security, ICCINS 2023, Institute of Electrical and Electronics Engineers Inc., 2023. doi: 10.1109/ICCINS58907.2023.10450143.

S. Meshram, N. Naik, V. R. Megha, T. More, and S. Kharche, “College Enquiry Chatbot using Rasa Framework,” in 2021 Asian Conference on Innovation in Technology, ASIANCON 2021, Institute of Electrical and Electronics Engineers Inc., Aug. 2021. doi: 10.1109/ASIANCON51346.2021.9544650.

Zein Hanni Pradana, Hanin Nafi’ah, and Raditya Artha Rochmanto, “in Chatbot-based Information Service using RASA Open-SourceFrameworkin Prambanan Temple Tourism Object,” Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi), vol. 6, no. 4, pp. 656–662, Aug. 2022, doi: 10.29207/resti.v6i4.3913.

N. Widiari, I. M. A. D. Suarjaya, and D. Githa, “Teknik Data Cleaning Menggunakan Snowflake untuk Studi Kasus Objek Pariwisata di Bali,” Jurnal Ilmiah Merpati (Menara Penelitian Akademika Teknologi Informasi), p. 137, Jul. 2020, doi: 10.24843/JIM.2020.v08.i02.p07.

M. S. Santos, J. P. Soares, P. H. Abreu, H. Araujo, and J. Santos, “Cross-Validation for Imbalanced Datasets: Avoiding Overoptimistic and Overfitting Approaches [Research Frontier],” IEEE Comput Intell Mag, vol. 13, no. 4, pp. 59–76, 2018, doi: 10.1109/MCI.2018.2866730.

M. Grandini, E. Bagli, and G. Visani, “Metrics for Multi-Class Classification: an Overview,” Aug. 2020, [Online]. Available: http://arxiv.org/abs/2008.05756

Telegram, “Telegram FAQ.” Accessed: Oct. 01, 2024. [Online]. Available: https://telegram.org/faq?

R. Vineeth, S. Maskey, U. S. Vishakan, and Y. Singh, “A Proposed Chatbot Psykh Your Personal Therapist and Stress Buster Using RASA Open-Source Framework,” in 2022 OPJU International Technology Conference on Emerging Technologies for Sustainable Development, OTCON 2022, Institute of Electrical and Electronics Engineers Inc., 2023. doi: 10.1109/OTCON56053.2023.10114025.