GraphiBERT-ML: A Knowledge-Enhanced NER Approach for Cross-Domain Comparative Analysis of Machine Learning Literature
DOI:
https://doi.org/10.29207/resti.v10i2.7160Keywords:
academic research, language models, machine learning, natural language processing, named entity recognitionAbstract
The exponential growth of scientific literature on platforms such as arXiv presents a major challenge in identifying and comparing key contributions to machine learning across diverse academic domains. To address this, we propose GraphiBERT-ML, a knowledge-enhanced extension of BERT that integrates semantic embeddings extracted from DBpedia to improve named entity recognition (NER) in scientific articles. To the best of our knowledge, this study presents the first knowledge-enhanced NER model that explicitly integrates DBpedia-based embeddings for large-scale cross-domain scientific analyses. The model was evaluated on a cross-domain dataset spanning eight fields, including computer science, physics, biology, finance, and economics. Experimental results show that GraphiBERT-ML achieves its highest performance in computer science, with an accuracy of 0.9372, an F1-score of 0.9368, and a precision of 0.9376. Physics and mathematics also demonstrate strong performance (F1-scores of 0.9115 and 0.8970), while more heterogeneous domains such as biology and finance show lower scores (F1-scores of 0.7946 and 0.7872), reflecting the complexity and variability of their terminology. Across all domains, GraphiBERT-ML consistently outperformed the baseline BERT model, confirming the benefit of external knowledge integration for scientific NER. These findings highlight domain-specific challenges in entity extraction and demonstrate the potential of knowledge-augmented models to advance cross-disciplinary analysis of machine learning research.
Downloads
References
B. S. Lubis et A. Yus, « Mapping knowledge and research trend on technology adoption in higher education: A bibliometric analysis », Educ Inf Technol, juin 2024, doi: 10.1007/s10639-024-12801-0.
A. Sharma, Amrita, S. Chakraborty, et S. Kumar, « Named Entity Recognition in Natural Language Processing: A Systematic Review », in Proceedings of Second Doctoral Symposium on Computational Intelligence, D. Gupta, A. Khanna, V. Kansal, G. Fortino, et A. E. Hassanien, Éd., Singapore: Springer, 2022, p. 817 828. doi: 10.1007/978-981-16-3346-1_66.
ArXiv:arXiv.org e-Print archive
A. Goyal, V. Gupta, et M. Kumar, « Recent Named Entity Recognition and Classification techniques: A systematic review », Computer Science Review, vol. 29, p. 21 43, août 2018, doi: 10.1016/j.cosrev.2018.06.001.
S. Naseer, M. M. Ghafoor, S. bin K. Alvi, A. Kiran, G. M. Shafique Ur Rahmand, et G. Murtaza, « Named Entity Recognition (NER) in NLP Techniques, Tools Accuracy and Performance. », Pakistan Journal of Multidisciplinary Research, vol. 2, no 2, Art. no 2, 2021.
W. Otto, M. Zloch, L. Gan, S. Karmakar, et S. Dietze, « GSAP-NER: A Novel Task, Corpus, and Baseline for Scholarly Entity Extraction Focused on Machine Learning Models and Datasets », in Findings of the Association for Computational Linguistics: EMNLP 2023, H. Bouamor, J. Pino, et K. Bali, Éd., Singapore: Association for Computational Linguistics, déc. 2023, p. 8166 8176. doi: 10.18653/v1/2023.findings-emnlp.548.
O. Ovtšarenko, « Opportunities of machine learning algorithms for education », Discov Educ, vol. 3, no 1, p. 209, nov. 2024, doi: 10.1007/s44217-024-00313-5.
J. Li, A. Sun, J. Han, et C. Li, « A Survey on Deep Learning for Named Entity Recognition », IEEE Transactions on Knowledge and Data Engineering, vol. 34, no 1, p. 50 70, janv. 2022, doi:10.1109/TKDE.2020.2981314.
S. F. Yilmaz, F. B. Mutlu, I. Balaban, et S. S. Kozat, « TMD-NER: Turkish multi-domain named entity recognition for informal texts », SIViP, vol. 18, no 3, p. 2255 2263, avr. 2024, doi: 10.1007/s11760-023-02898-0.
H. Sarker, « Machine Learning: Algorithms, Real-World Applications and Research Directions », SN COMPUT. SCI., vol. 2, no 3, p. 160, mars 2021, doi: 10.1007/s42979-021-00592-x.
E. Harsányi et al., « Assessment of Advanced Machine and Deep Learning Approaches for Predicting CO2 Emissions from Agricultural Lands: Insights Across Diverse Agroclimatic Zones », Earth Syst Environ, vol. 8, no 4, p. 1109 1125, déc. 2024, doi: 10.1007/s41748-024-00424-x.
J. Dunstan et al., « A pseudonymized corpus of occupational health narratives for clinical entity recognition in Spanish », BMC Med Inform Decis Mak, vol. 24, no 1, p. 204, juill. 2024, doi: 10.1186/s12911-024-02609-w.
Z. Liu et al., « CrossNER: Evaluating Cross-Domain Named Entity Recognition », Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no 15, Art. no 15, mai 2021, doi: 10.1609/aaai.v35i15.17587.
S. D. Baum, « Artificial Interdisciplinarity: Artificial Intelligence for Research on Complex Societal Problems », Philos. Technol., vol. 34, no 1, p. 45 63, nov. 2021, doi: 10.1007/s13347-020-00416-5.
P. Sun, X. Yang, X. Zhao, et Z. Wang, « An Overview of Named Entity Recognition », in 2018 International Conference on Asian Language Processing (IALP), nov. 2018, p. 273 278. doi: 10.1109/IALP.2018.8629225.
W. Shao, Y. Hu, P. Ji, X. Yan, D. Fan, et R. Zhang, « Prompt-NER: Zero-shot Named Entity Recognition in Astronomy Literature via Large Language Models », 27 octobre 2023, arXiv: arXiv:2310.17892. doi: 10.48550/arXiv.2310.17892.
S. Wu et al., « Benchmarking Open-Source Large Language Models, GPT-4 and Claude 2 on Multiple-Choice Questions in Nephrology », NEJM AI, vol. 1, no 2, janv. 2024, doi: 10.1056/AIdbp2300092.
J. Yeom et al., « Tc-llama 2: fine-tuning LLM for technology and commercialization applications », J Big Data, vol. 11, no 1, p. 100, août 2024, doi: 10.1186/s40537-024-00963-0.
J. Dietrich et A. Hollstein, « Performance and Reproducibility of Large Language Models in Named Entity Recognition: Considerations for the Use in Controlled Environments », Drug Saf, déc. 2024, doi: 10.1007/s40264-024-01499-1.
A. Thukral, S. Dhiman, R. Meher, et P. Bedi, « Knowledge graph enrichment from clinical narratives using NLP, NER, and biomedical ontologies for healthcare applications », Int. j. inf. tecnol., vol. 15, no 1, p. 53 65, janv. 2023, doi: 10.1007/s41870-022-01145-y.
B. Pang, E. Nijkamp, et Y. N. Wu, « Deep Learning With TensorFlow: A Review », Journal of Educational and Behavioral Statistics, vol. 45, no 2, p. 227 248, avr. 2020, doi: 10.3102/1076998619872761.
H. Dai, X. Peng, X. Shi, L. He, Q. Xiong, et H. Jin, « Reveal training performance mystery between TensorFlow and PyTorch in the single GPU environment », Sci. China Inf. Sci., vol. 65, no 1, p. 112103, déc. 2021, doi: 10.1007/s11432-020-3182-1.
Honnibal, M., Montani, I.: SpaCy 2: natural language understanding with bloom embeddings, convolutional neural networks and incremental parsing. neural machine translation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, pp. 688–697 (2017)
D. Khurana, A. Koli, K. Khatter, et S. Singh, « Natural language processing: state of the art, current trends and challenges », Multimed Tools Appl, vol. 82, no 3, p. 3713 3744, janv. 2023, doi: 10.1007/s11042-022-13428-4.
S. Gorla, S. S. Tangeda, L. B. M. Neti, et A. Malapati, « Telugu named entity recognition using bert », Int J Data Sci Anal, vol. 14, no 2, p. 127 140, août 2022, doi: 10.1007/s41060-021-00305-w.
R. Balamurugan, S. Mohite, et S. P. Raja, « Protein Sequence Classification Using Bidirectional Encoder Representations from Transformers (BERT) Approach », SN COMPUT. SCI., vol. 4, no 5, p. 481, juin 2023, doi: 10.1007/s42979-023-01980-1.
S. Nahali, L. Safari, A. Khanteymoori, et J. Huang, « StructmRNA a BERT based model with dual level and conditional masking for mRNA representation », Sci Rep, vol. 14, no 1, p. 26043, oct. 2024, doi: 10.1038/s41598-024-77172-5.
N. Khouya, A. Retbi, et S. Bennani, « Enriching Ontology with Named Entity Recognition (NER) Integration », in Proceedings of the Second International Conference on Advances in Computing Research (ACR’24), K. Daimi et A. Al Sadoon, Éd., Cham: Springer Nature Switzerland, 2024, p. 151 159. doi: 10.1007/978-3-031-56950-0_13.
S. Chopra, P. Agarwal, J. Ahmed, S. S. Biswas, et A. J. Obaid, « Roberta and BERT: Revolutionizing Mental Healthcare Through Natural Language », SN COMPUT. SCI., vol. 5, no 7, p. 889, sept. 2024, doi: 10.1007/s42979-024-03202-8.
M. Jojoa, P. Eftekhar, B. Nowrouzi-Kia, et B. Garcia-Zapirain, « Natural language processing analysis applied to COVID-19 open-text opinions using a distilBERT model for sentiment categorization », AI & Soc, vol. 39, no 3, p. 883 890, juin 2024, doi: 10.1007/s00146-022-01594-w.
Downloads
Published
Issue
Section
License
Copyright in each article belongs to the author
- The author acknowledges that the RESTI Journal (System Engineering and Information Technology) is the first publisher to publish with a license Creative Commons Attribution 4.0 International License.
- Authors can enter writing separately, arrange the non-exclusive distribution of manuscripts that have been published in this journal into other versions (eg sent to the author's institutional repository, publication in a book, etc.), by acknowledging that the manuscript has been published for the first time in the RESTI (Rekayasa Sistem dan Teknologi Informasi) journal ;





