Abstractive Dialogue Summarization using Fine-Tuning Pre-Trained Language Model BART
DOI:
https://doi.org/10.29207/resti.v10i3.7488Keywords:
abstractive summarization, BART, dialogue, natural language processing, sequence-to-sequenceAbstract
The increasing volume of conversational data on digital platforms necessitates effective automatic summarization methods. This study investigates the use of a pre-trained BART (Bidirectional and Auto-Regressive Transformers) model for abstractive dialogue summarization on the DialogSum dataset, which consists of 14,460 English dialogues. The model is fine-tuned using a sequence-to-sequence framework with systematic hyperparameter tuning, including variations in learning rate, beam size, and training epochs. The optimal configuration is achieved using a learning rate of 2e-5, a beam size of 6, and 5 training epochs. Model performance is evaluated using ROUGE and BERTScore metrics. The experimental results show that the proposed model attains an average ROUGE-L F1-score of 0.40, indicating moderate structural similarity between generated and reference summaries, and an average BERTScore F1-score of 0.69, reflecting strong semantic alignment. These findings suggest that fine-tuned BART is effective in preserving semantic relevance in abstractive dialogue summarization, although further improvements are required to enhance lexical precision and discourse-level coherence.
Downloads
References
M. Zhong, Y. Liu, Y. Xu, C. Zhu, and M. Zeng, “DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization,” Proc. AAAI Conf. Artif. Intell., vol. 36, no. 10, pp. 11765–11773, Jun. 2022, doi: 10.1609/aaai.v36i10.21432.
X. Feng, X. Feng, and B. Qin, “A Survey on Dialogue Summarization: Recent Advances and New Frontiers,” in Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence, California: International Joint Conferences on Artificial Intelligence Organization, Jul. 2022, pp. 5453–5460. doi: 10.24963/ijcai.2022/764.
S. Gupta and S. K. Gupta, “Abstractive summarization: An overview of the state of the art,” Expert Syst. Appl., vol. 121, pp. 49–65, May 2019, doi: 10.1016/j.eswa.2018.12.011.
A. A. Syed, F. L. Gaol, A. Boediman, T. Matsuo, and W. Budiharto, “A Survey of Abstractive Text Summarization Utilising Pretrained Language Models,” N. T. Nguyen, T. K. Tran, U. Tukayev, T.-P. Hong, B. Trawiński, and E. Szczerbicki, Eds., Cham: Springer International Publishing, 2022, pp. 532–544. doi: 10.1007/978-3-031-21743-2_42.
L. Xiao, L. Wang, H. He, and Y. Jin, “Modeling Content Importance for Summarization with Pre-trained Language Models,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), B. Webber, T. Cohn, Y. He, and Y. Liu, Eds., Stroudsburg, PA, USA: Association for Computational Linguistics, Nov. 2020, pp. 3606–3611. doi: 10.18653/v1/2020.emnlp-main.293.
T. Shi, Y. Keneshloo, N. Ramakrishnan, and C. K. Reddy, “Neural Abstractive Text Summarization with Sequence-to-Sequence Models,” ACM/IMS Trans. Data Sci., vol. 2, no. 1, pp. 1–37, Feb. 2021, doi: 10.1145/3419106.
C. Prasad, J. S. Kallimani, D. Harekal, and N. Sharma, “Automatic Text Summarization Model using Seq2Seq Technique,” in 2020 Fourth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC), IEEE, Oct. 2020, pp. 599–604. doi: 10.1109/I-SMAC49090.2020.9243572.
C. Zheng, K. Zhang, H. J. Wang, L. Fan, and Z. Wang, “Enhanced Seq2Seq Autoencoder via Contrastive Learning for Abstractive Text Summarization,” in 2021 IEEE International Conference on Big Data (Big Data), IEEE, Dec. 2021, pp. 1764–1771. doi: 10.1109/BigData52589.2021.9671819.
S. Macharla, A. Madamanchi, and N. Kancharla, “nav-nlp at RadSum23: Abstractive Summarization of Radiology Reports using BART Finetuning,” in The 22nd Workshop on Biomedical Natural Language Processing and BioNLP Shared Tasks, Stroudsburg, PA, USA: Association for Computational Linguistics, 2023, pp. 541–544. doi: 10.18653/v1/2023.bionlp-1.55.
Y. Chen, Y. Liu, L. Chen, and Y. Zhang, “DialogSum: A Real-Life Scenario Dialogue Summarization Dataset,” in Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, Stroudsburg, PA, USA: Association for Computational Linguistics, 2021, pp. 5062–5074. doi: 10.18653/v1/2021.findings-acl.449.
C. Lundberg, L. Sánchez Viñuela, and S. Biales, “Dialogue Summarization using BART,” in Proceedings of the 15th International Conference on Natural Language Generation: Generation Challenges, Association for Computational Linguistics, Jul. 2022, pp. 121–125. [Online]. Available: https://aclanthology.org/2022.inlg-genchal.17/
V. Chauhan, P. Roy, L. Dey, and T. Goel, “TCS_WITM_2022 @ DialogSum : Topic oriented Summarization using Transformer based Encoder Decoder Model,” in Proceedings of the 15th International Conference on Natural Language Generation: Generation Challenges, Association for Computational Linguistics, Jul. 2022, pp. 104–109. [Online]. Available: https://aclanthology.org/2022.inlg-genchal.15/
T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi, “BERTScore: Evaluating Text Generation with BERT,” CoRR, vol. abs/1904.0, Feb. 2020, doi: 10.48550/arXiv.1904.09675.
P. Verma and A. Verma, “A Review on Text Summarization Techniques,” J. Sci. Res., vol. 64, no. 01, pp. 251–257, 2020, doi: 10.37398/JSR.2020.640148.
A. P. Widyassari et al., “Review of automatic text summarization techniques & methods,” J. King Saud Univ. - Comput. Inf. Sci., vol. 34, no. 4, pp. 1029–1046, Apr. 2022, doi: 10.1016/j.jksuci.2020.05.006.
N. Almohaimeed and A. M. Azmi, “Abstractive text summarization: A comprehensive survey of techniques, systems, and challenges,” Comput. Sci. Rev., vol. 57, p. 100762, Aug. 2025, doi: 10.1016/j.cosrev.2025.100762.
H. Shakil, A. Farooq, and J. Kalita, “Abstractive text summarization: State of the art, challenges, and improvements,” Neurocomputing, vol. 603, p. 128255, Oct. 2024, doi: 10.1016/j.neucom.2024.128255.
T. Bao, H. Zhang, and C. Zhang, “Enhancing abstractive summarization of scientific papers using structure information,” Expert Syst. Appl., vol. 261, p. 125529, Feb. 2025, doi: 10.1016/j.eswa.2024.125529.
O. Büyük, “A Comprehensive Evaluation of Large Language Models for Turkish Abstractive Dialogue Summarization,” IEEE Access, vol. 12, pp. 124391–124401, Sep. 2024, doi: 10.1109/ACCESS.2024.3454342.
Y. Altameemi, M. Altamimi, A. Alkhalil, D. Uliyan, and R. F. Mansour, “Text summarization method of argumentative discourse by combining the BERT-transformer model,” Front. Artif. Intell., vol. 8, Nov. 2025, doi: 10.3389/frai.2025.1654496.
Z. Nie et al., “Domain-Specific Multi-Document Political News Summarization Using BART and ACT-GAN,” IEEE Access, vol. 13, pp. 143737–143750, 2025, doi: 10.1109/ACCESS.2025.3598058.
C. Adhik, S. Sri Lakshmi, and C. Muralidharan, “Text summarization using BART,” 2024, p. 020038. doi: 10.1063/5.0217004.
K. Jin, Y. Wang, L. Santos, H. G. Oliveira, X. Yang, and S. K. Im, “Role-aware adapters for dialogue summarization in Seq2Seq models,” Appl. Soft Comput., vol. 187, p. 114293, Feb. 2026, doi: 10.1016/j.asoc.2025.114293.
A. R. Fabbri et al., “Improving Zero and Few-Shot Abstractive Summarization with Intermediate Fine-tuning and Data Augmentation,” Apr. 2021, doi: 10.48550/arXiv.2010.12836.
W. S. El-Kassas, C. R. Salama, A. A. Rafea, and H. K. Mohamed, “Automatic text summarization: A comprehensive survey,” Expert Syst. Appl., vol. 165, p. 113679, Mar. 2021, doi: 10.1016/j.eswa.2020.113679.
H. Lin and V. Ng, “Abstractive Summarization: A Survey of the State of the Art,” Proc. AAAI Conf. Artif. Intell., vol. 33, no. 01, pp. 9815–9822, Jul. 2019, doi: 10.1609/aaai.v33i01.33019815.
Y. Li, H. Su, X. Shen, W. Li, Z. Cao, and S. Niu, “DailyDialog: A Manually Labelled Multi-turn Dialogue Dataset,” Oct. 2017, doi: 10.48550/arXiv.1710.03957.
K. Sun, D. Yu, J. Chen, D. Yu, Y. Choi, and C. Cardie, “DREAM: A Challenge Data Set and Models for Dialogue-Based Reading Comprehension,” Trans. Assoc. Comput. Linguist., vol. 7, pp. 217–231, Nov. 2019, doi: 10.1162/tacl_a_00264.
L. Cui, Y. Wu, S. Liu, Y. Zhang, and M. Zhou, “MuTual: A Dataset for Multi-Turn Dialogue Reasoning,” Apr. 2020, doi: 10.48550/arXiv.2004.04494.
L. Hickman, S. Thapa, L. Tay, M. Cao, and P. Srinivasan, “Text Preprocessing for Text Mining in Organizational Research: Review and Recommendations,” Organ. Res. Methods, vol. 25, no. 1, pp. 114–146, Jan. 2022, doi: 10.1177/1094428120971683.
N. Garg and K. Sharma, “Text pre-processing of multilingual for sentiment analysis based on social network data,” Int. J. Electr. Comput. Eng., vol. 12, no. 1, p. 776, Feb. 2022, doi: 10.11591/ijece.v12i1.pp776-784.
R. Sennrich, B. Haddow, and A. Birch, “Neural Machine Translation of Rare Words with Subword Units,” in Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), K. Erk and N. A. Smith, Eds., Stroudsburg, PA, USA: Association for Computational Linguistics, Aug. 2016, pp. 1715–1725. doi: 10.18653/v1/P16-1162.
M. Lewis et al., “BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension,” Oct. 2019, doi: https://doi.org/10.48550/arXiv.1910.13461.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “{BERT}: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of the 2019 Conference of the North, J. Burstein, C. Doran, and T. Solorio, Eds., Stroudsburg, PA, USA: Association for Computational Linguistics, Jun. 2019, pp. 4171–4186. doi: 10.18653/v1/N19-1423.
I. Sutskever, O. Vinyals, and Q. V Le, “Sequence to Sequence Learning with Neural Networks,” Adv. Neural Inf. Process. Syst., vol. 27, Dec. 2014, doi: https://doi.org/10.48550/arXiv.1409.3215.
A. Auriemma Citarella, M. Barbella, M. G. Ciobanu, F. De Marco, L. Di Biasi, and G. Tortora, “Assessing the effectiveness of ROUGE as unbiased metric in Extractive vs. Abstractive summarization techniques,” J. Comput. Sci., vol. 87, no. November 2024, p. 102571, May 2025, doi: 10.1016/j.jocs.2025.102571.
M. Barbella and G. Tortora, “Rouge Metric Evaluation for Text Summarization Techniques,” SSRN Electron. J., 2022, doi: 10.2139/ssrn.4120317.
C.-Y. Lin, “ROUGE: A Package for Automatic Evaluation of Summaries,” in Association for Computational Linguistics, Association for Computational Linguistics, Jul. 2004, pp. 74–81. [Online]. Available: https://aclanthology.org/W04-1013/
Halimah, Surya Agustian, and Siti Ramadhani, “Peringkasan teks otomatis (automated text summarization) pada artikel berbahasa indonesia menggunakan algoritma lexrank,” J. CoSciTech (Computer Sci. Inf. Technol., vol. 3, no. 3, pp. 371–381, Dec. 2022, doi: 10.37859/coscitech.v3i3.4300.
E. Davoodijam and M. Alambardar Meybodi, “Evaluation metrics on text summarization: comprehensive survey,” Knowl. Inf. Syst., vol. 66, no. 12, pp. 7717–7738, Dec. 2024, doi: 10.1007/s10115-024-02217-0.
Downloads
Published
Issue
Section
License
Copyright in each article belongs to the author
- The author acknowledges that the RESTI Journal (System Engineering and Information Technology) is the first publisher to publish with a license Creative Commons Attribution 4.0 International License.
- Authors can enter writing separately, arrange the non-exclusive distribution of manuscripts that have been published in this journal into other versions (eg sent to the author's institutional repository, publication in a book, etc.), by acknowledging that the manuscript has been published for the first time in the RESTI (Rekayasa Sistem dan Teknologi Informasi) journal ;
