Development of Reviewer Assignment Method with Latent Dirichlet Allocation and Link Prediction to Avoid Conflict of Interest

Adi Setyo Nugroho; Ahmad Saikhu; Ratih Nur Esti Anggraini

doi:10.29207/resti.v7i4.4900

Adi Setyo Nugroho Institut Teknologi Sepuluh Nopember
Ahmad Saikhu Institut Teknologi Sepuluh Nopember
Ratih Nur Esti Anggraini Institut Teknologi Sepuluh Nopember

DOI: https://doi.org/10.29207/resti.v7i4.4900

Keywords: reviewer, dblp dataset, latent dirichlet allocation (LDA), random forest, link prediction, conflict of interest (CoI)

Abstract

The number of published academic papers has been increasing rapidly from year to year. However, this increase in publications must be linear with an emphasis on quality. To ensure that academic papers meet the required quality standard, the peer review process is necessary. The main objective of the assignment of reviewers is to find the appropriate reviewer who can conduct a review based on their field of research. However, there are potential obstacles when there is a conflict of interest in the process. This study aims to develop a method for assigning reviewers that overcomes such obstacles. Our approach involves combining the Latent Dirichlet Allocation (LDA), Classification, and Link Prediction methods. LDA is used to find topics from the research data of prospective reviewers to ensure that the assigned reviewers are well suited to the submitted article. These data were used as training data for classification using Random Forest. Finally, link prediction implemented to make reviewer recommendations. We evaluated and compared our proposed method with previous research that used cosine similarity as the last step in recommendation, using Mean Average Precision (MAP). Our proposed method achieved a MAP value of 0.87, which was an improvement compared to the previous approach. These results suggest that our approach has the potential to improve the effectiveness of academic peer review.

Downloads

Download data is not yet available.

References

“Scopus Fact Sheet,” 2019. https://www.elsevier.com/__data/assets/pdf_file/0010/891397/Scopus_GlobalResearch_Factsheet2019_FINAL_WEB.pdf (accessed Jan. 02, 2022).

“DBLP Statistic,” 2022. https://dblp.uni-trier.de/statistics/newrecordsperyear.html (accessed Feb. 02, 2022).

L. Bornmann, R. Haunschild, and R. Mutz, “Growth rates of modern science: a latent piecewise growth curve approach to model publication numbers from established and new literature databases,” Humanit. Soc. Sci. Commun., vol. 8, no. 1, 2021, doi: 10.1057/s41599-021-00903-w.

H. Jelodar et al., “Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey,” Multimed. Tools Appl., vol. 78, no. 11, pp. 15169–15211, 2019, doi: 10.1007/s11042-018-6894-4.

D. Annane, N. Lerolle, S. Meuris, J. Sibilla, and K. M. Olsen, “Academic conflict of interest,” Intensive Care Med., vol. 45, no. 1, pp. 13–20, 2019, doi: 10.1007/s00134-018-5458-4.

L. Schenk and G. Johanson, “Management of bias and conflict of interest among occupational exposure limit expert groups,” Regul. Toxicol. Pharmacol., vol. 123, no. March, p. 104929, 2021, doi: 10.1016/j.yrtph.2021.104929.

S. A. Al-Zboon, S. K. Tawalbeh, H. Ai-Jarrah, M. Al-Asa’D, M. Hammad, and M. Al-Smadi, “Resolving Conflict of Interests in Recommending Reviewers for Academic Publications Using Link Prediction Techniques,” 2019 2nd Int. Conf. New Trends Comput. Sci. ICTCS 2019 - Proc., pp. 1–6, 2019, doi: 10.1109/ICTCS.2019.8923033.

C. R. Kirman, T. W. Simon, and S. M. Hays, “Science peer review for the 21st century: Assessing scientific consensus for decision-making while managing conflict of interests, reviewer and process bias,” Regul. Toxicol. Pharmacol., vol. 103, no. November 2018, pp. 73–85, 2019, doi: 10.1016/j.yrtph.2019.01.003.

H. Al-Jarrah, M. Al-Asa’D, S. A. Al-Zboon, S. K. Tawalbeh, M. M. Hammad, and M. Al-Smadi, “Resolving Conflict of Interests and Recommending Expert Reviewers for Academic Publications Using Linked Open Data,” 2019 6th Int. Conf. Soc. Networks Anal. Manag. Secur. SNAMS 2019, pp. 91–98, 2019, doi: 10.1109/SNAMS.2019.8931826.

A. S. Nugroho, A. F. Iswafaza, R. N. E. Anggraini, and R. Sarno, “A Novel Approach on Conducting Reviewer Recommendations Based on Conflict of Interest,” in 2021 13th International Conference on Information & Communication Technology and System (ICTS), Oct. 2021, pp. 195–200. doi: 10.1109/ICTS52701.2021.9609054.

“DBLP Dataset,” 2022. https://dblp.org/ (accessed Sep. 09, 2022).

A. S. Szalay, “DBLP — Some Lessons Learned,” Commun. ACM, vol. 51, no. 11, pp. 59–65, 2008, doi: 10.1145/1400214.1400231.

R. A. Priyantina and R. Sarno, “Sentiment analysis of hotel reviews using Latent Dirichlet Allocation, semantic similarity and LSTM,” Int. J. Intell. Eng. Syst., vol. 12, no. 4, pp. 142–155, 2019, doi: 10.22266/ijies2019.0831.14.

T. Cvitanic, B. Lee, H. I. Song, K. Fu, and D. Rosen, “LDA v. LSA: A comparison of two computational text analysis tools for the functional categorization of patents,” CEUR Workshop Proc., vol. 1815, pp. 41–50, 2016.

Wiranto and R. M. Uswatunnisa, “Topic Modeling for Support Ticket using Latent Dirichlet Allocation,” J. RESTI (Rekayasa Sist. dan Teknol. Informasi), vol. 6, pp. 998–1005, 2022, doi: 10.29207/resti.v6i6.4542.

T. Mikolov, K. Chen, G. Corrado, and J. Dean, “Efficient estimation of word representations in vector space,” 1st Int. Conf. Learn. Represent. ICLR 2013 - Work. Track Proc., pp. 1–12, 2013.

B. Jang, I. Kim, and J. W. Kim, “Word2vec convolutional neural networks for classification of news articles and tweets,” PLoS One, vol. 14, no. 8, pp. 1–20, 2019, doi: 10.1371/journal.pone.0220976.

R. P. Nawangsari, R. Kusumaningrum, and A. Wibowo, “Word2vec for Indonesian sentiment analysis towards hotel reviews: An evaluation study,” Procedia Comput. Sci., vol. 157, pp. 360–366, 2019, doi: 10.1016/j.procs.2019.08.178.

C. G. Siji George and B. Sumathi, “Grid search tuning of hyperparameters in random forest classifier for customer feedback sentiment prediction,” Int. J. Adv. Comput. Sci. Appl., vol. 11, no. 9, pp. 173–178, 2020, doi: 10.14569/IJACSA.2020.0110920.

F. Tan, Y. Xia, and B. Zhu, “Link prediction in complex networks: A mutual information perspective,” PLoS One, vol. 9, no. 9, 2014, doi: 10.1371/journal.pone.0107056.

P. Raut, H. Khandelwal, and G. Vyas, “A Comparative Study of Classification Algorithms for Link Prediction,” 2nd Int. Conf. Innov. Mech. Ind. Appl. ICIMIA 2020 - Conf. Proc., no. Icimia, pp. 479–483, 2020, doi: 10.1109/ICIMIA48430.2020.9074840.

F. Gao, K. Musial, C. Cooper, and S. Tsoka, “Link prediction methods and their accuracy for different social networks and network metrics,” Sci. Program., vol. 2015, no. i, 2015, doi: 10.1155/2015/172879.

C. D. Manning, P. Raghavan, and H. Schutze, “An Introduction to Information Retrieval,” 1st ed., Cambridge: Cambridge University Press, 2009, p. 160. [Online]. Available: http://www.informationretrieval.org/