Hybrid Video Transcription Summarization with a BERT-Based Clustering and BART
DOI:
https://doi.org/10.29207/resti.v9i6.7066Keywords:
BART, BERT, clustering, hybrid approach, video summarizationAbstract
The use of video as a medium for information and education is rapidly increasing across online platforms. However, long durations and unstructured delivery often hinder audiences from grasping the core message, presenting challenges for the development of automatic summarization methods for monologues, interviews, and podcasts. Extractive methods often yield less coherent summaries, while abstractive methods may overlook important details. To address this issue, this study proposes a hybrid approach combining extractive and abstractive techniques. In the extractive stage, sentences are represented using BERT embeddings and clustered using two methods, namely K-Means Clustering and Hierarchical Clustering (agglomerative). The abstractive stage then employs the BART model to generate summaries that are more coherent and informative. Experimental evaluations on 20 Human Metapneumovirus (HMPV) videos indicate the strongest performance on monologues, with ROUGE-1 of 57%, ROUGE-2 of 30%, and ROUGE-L of 32%. Although lower performance was observed for interviews and podcasts due to dynamic interactions and frequent speaker shifts, the hybrid approach consistently surpassed extractive-only and abstractive-only baselines. These results highlight the effectiveness of the hybrid approach and its potential for developing more adaptive video summarization in the future.
Downloads
References
J. P. Suharsono and D. Nurahman, “Pemanfaatan Youtube Sebagai Media Peningkatan Pelayanan Dan Informasi,” Ganaya J. Ilmu Sos. Dan Hum., vol. 7, no. 1, pp. 298–304, Mar. 2024, doi: 10.37329/ganaya.v7i1.3157.
L. Parabhoi, R. R. Sahu, R. S. Dewey, M. K. Verma, A. Kumar Seth, and D. Parabhoi, “YouTube as a source of information during the Covid-19 pandemic: a content analysis of YouTube videos published during January to March 2020,” BMC Med. Inform. Decis. Mak., vol. 21, no. 1, p. 255, Dec. 2021, doi: 10.1186/s12911-021-01613-8.
J. K. Höhne, T. Lenzner, and J. Claassen, “Automatic speech-to-text transcription: evidence from a smartphone survey with voice answers,” Int. J. Soc. Res. Methodol., pp. 1–8, Jan. 2025, doi: 10.1080/13645579.2024.2443633.
A. Koenecke, A. S. G. Choi, K. X. Mei, H. Schellmann, and M. Sloane, “Careless Whisper: Speech-to-Text Hallucination Harms,” 2024, doi: 10.48550/ARXIV.2402.08021.
Le Ngoc Thang, Nguyen Minh Tien, Do Nhat Minh, Nguyen Chi Thanh, and Le Quang Minh, “A method to utilize prior knowledge for extractive summarization based on pre-trained language models,” Vietnam J. Sci. Technol., Dec. 2024, doi: 10.15625/2525-2518/20241.
Wicaksono, G. W., Azhar, Y., Hidayah, N. P., & Andreawana, A, "Automatic summarization of court decision documents over narcotic cases using bert", JOIV: International Journal on Informatics Visualization, 7(2), 416-422. 2023. doi: https://dx.doi.org/10.30630/joiv.7.2.1811
G. Hartawan, D. S. Maylawati, and W. Uriawan, “Bidirectional and Auto-Regressive Transformer (BART) for Indonesian Abstractive Text Summarization,” J. Inform. Polinema, vol. 10, no. 4, pp. 535–542, Aug. 2024, doi: 10.33795/jip.v10i4.5242.
M. Ülker and A. B. Özer, “The Bart-based Model for Scientific Articles Summarization,” JUCS - J. Univers. Comput. Sci., vol. 30, no. 13, pp. 1807–1828, Dec. 2024, doi: 10.3897/jucs.115121.
N. B. Raut, A. S. Pranesh, B. Nagulan, S. Pranesh, and R. Vasantharajan, “An Extensive Survey on Audio-to-Text and Text Summarization for Video Content,” in 2023 3rd International Conference on Innovative Mechanisms for Industry Applications (ICIMIA), Bengaluru, India: IEEE, Dec. 2023, pp. 1251–1257. doi: 10.1109/ICIMIA60377.2023.10426376.
M. Y. Jaradeh, M. Stocker, and S. Auer, “MORTY: Structured Summarization for Targeted Information Extraction from Scholarly Articles,” in From Born-Physical to Born-Virtual: Augmenting Intelligence in Digital Libraries, vol. 13636, Y.-H. Tseng, M. Katsurai, and H. N. Nguyen, Eds., in Lecture Notes in Computer Science, vol. 13636. , Cham: Springer International Publishing, 2022, pp. 290–300. doi: 10.1007/978-3-031-21756-2_23.
A. I. Jony, A. T. Rithin, and S. I. Edrish, “A Comparative Study and Analysis of Text Summarization Methods,” Malays. J. Sci. Adv. Technol., pp. 118–129, Mar. 2024, doi: 10.56532/mjsat.v4i2.231.
Z. Edress and Y. Ortakci, “Optimizing Text Summarization with Sentence Clustering and Natural Language Processing,” Int. J. Adv. Comput. Sci. Appl., vol. 15, no. 10, 2024, doi: 10.14569/IJACSA.2024.01510115.
J. Yang, F. Wei, N. Huber-Fliflet, A. Dabrowski, Q. Mao, and H. Qin, “An Empirical Analysis of Text Segmentation for BERT Classification in Extended Documents,” in 2023 IEEE International Conference on Big Data (BigData), Sorrento, Italy: IEEE, Dec. 2023, pp. 2793–2797. doi: 10.1109/BigData59044.2023.10386783.
A. Firdaus, N. Yusliani, and D. Rodiah, “Text Summarization with K-Means Method,” Sriwij. J. Inform. Appl., vol. 2, no. 2, July 2021, doi: 10.36706/sjia.v2i2.25.
Lviv Polytechnic National University, Knyaz Roman Str. 5 Lviv, 79013, Ukraine, N. I. Boyko, O. A. Tkachyk, and Lviv Polytechnic National University, Knyaz Roman Str. 5 Lviv, 79013; Ukraine, “Hierarchical clustering algorithm for dendrogram construction and cluster counting,” Inform. Math. Methods Simul., vol. 13, no. 1–2, pp. 5–15, Apr. 2023, doi: 10.15276/imms.v13.no1-2.5.
A. L. Haz, E. D. Fajrianti, N. Funabiki, and S. Sukaridhoto, “A Study of Audio-to-Text Conversion Software Using Whispers Model,” in 2023 Sixth International Conference on Vocational Education and Electrical Engineering (ICVEE), Surabaya, Indonesia: IEEE, Oct. 2023, pp. 268–273. doi: 10.1109/ICVEE59738.2023.10348186.
M. Salıcı and Ü. E. Ölçer, “Impact of Transformer-Based Models in NLP: An In-Depth Study on BERT and GPT,” in 2024 8th International Artificial Intelligence and Data Processing Symposium (IDAP), Malatya, Turkiye: IEEE, Sept. 2024, pp. 1–6. doi: 10.1109/IDAP64064.2024.10710796.
M. S. Islam and L. Zhang, “A Review on BERT: Language Understanding for Different Types of NLP Task,” Jan. 26, 2024, Computer Science and Mathematics. doi: 10.20944/preprints202401.1857.v1.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proceedings of the 2019 Conference of the North, Minneapolis, Minnesota: Association for Computational Linguistics, 2019, pp. 4171–4186. doi: 10.18653/v1/N19-1423.
J. Wang et al., “Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges,” ACM Comput. Surv., vol. 56, no. 7, pp. 1–33, July 2024, doi: 10.1145/3648471.
Suharso, W., Arifianto, S., Wibowo, H., Chandranegara, D.R. and Syaifuddin, S., "Music Features Pada Bidang Ilmu Komputer Menggunakan Modularity Clustering", Just IT: Jurnal Sistem Informasi, Teknologi Informasi, dan Komputer, vol 12, no 1, pp. 39-46, December 2022, doi: https://doi.org/10.24853/justit.13.1.%25p
Mrs. J. Sirisha, V. Lakshmi Prathyusha, P. Naga Anupriya, M. Suma Sri, and P. Naga Hema, “Customer Segmentation using K-Means Clustering,” Int. J. Adv. Res. Sci. Commun. Technol., pp. 170–175, Nov. 2022, doi: 10.48175/IJARSCT-7618.
Devaghi. P and S. Sudha, “Exploratory Data Analysis and Data Segmentation using K means Clustering,” in 2023 3rd International Conference on Advance Computing and Innovative Technologies in Engineering (ICACITE), Greater Noida, India: IEEE, May 2023, pp. 145–147. doi: 10.1109/ICACITE57410.2023.10183143.
R. R. Yager, “Intelligent control of the hierarchical agglomerative clustering process,” IEEE Trans. Syst. Man Cybern. Part B Cybern., vol. 30, no. 6, pp. 835–845, Dec. 2000, doi: 10.1109/3477.891145.
J.-T. Chien, “Hierarchical Theme and Topic Modeling,” IEEE Trans. Neural Netw. Learn. Syst., vol. 27, no. 3, pp. 565–578, Mar. 2016, doi: 10.1109/TNNLS.2015.2414658.
D. Jiang, S. Cao, and S. Yang, “Abstractive summarization of long texts based on BERT and sequence-to-sequence model,” in 2021 2nd International Conference on Information Science and Education (ICISE-IE), Chongqing, China: IEEE, Nov. 2021, pp. 460–466. doi: 10.1109/ICISE-IE53922.2021.00112.
C.-Y. Lin, “ROUGE: A Package for Automatic Evaluation of Summaries,” Assoc. Comput. Linguist., vol. Text Summarization Branches Out, pp. 74–81, July 2004.
T. A. Mir, D. Banerjee, D. Upadhyay, and R. S. Rawat, “Binary Classification of Lung Diseases Using Convolutional Neural Networks and Support Vector Machines,” in 2024 4th Asian Conference on Innovation in Technology (ASIANCON), Pimari Chinchwad, India: IEEE, Aug. 2024, pp. 1–6. doi: 10.1109/ASIANCON62057.2024.10837890.
J. Chen, S. Wang, Z. Qi, Z. Zhang, C. Wang, and H. Zheng, “A Combined Encoder and Transformer Approach for Coherent and High-Quality Text Generation,” 2024, arXiv. doi: 10.48550/ARXIV.2411.12157.
B. M. Gurusamy, P. K. Rengarajan, and P. Srinivasan, “A hybrid approach for text summarization using semantic latent Dirichlet allocation and sentence concept mapping with transformer,” Int. J. Electr. Comput. Eng. IJECE, vol. 13, no. 6, p. 6663, Dec. 2023, doi: 10.11591/ijece.v13i6.pp6663-6672.
Downloads
Published
Issue
Section
License
Copyright in each article belongs to the author
- The author acknowledges that the RESTI Journal (System Engineering and Information Technology) is the first publisher to publish with a license Creative Commons Attribution 4.0 International License.
- Authors can enter writing separately, arrange the non-exclusive distribution of manuscripts that have been published in this journal into other versions (eg sent to the author's institutional repository, publication in a book, etc.), by acknowledging that the manuscript has been published for the first time in the RESTI (Rekayasa Sistem dan Teknologi Informasi) journal ;
