Comparison of Sentiment Analysis Methods Based on Accuracy Value Case Study: Twitter Mentions of Academic Article

  • Muhamad Fahmi Fakhrezi Universitas Diponegoro
  • Adian Fatchur Rochim Universitas Diponegoro
  • Dinar Mutiara Kusomo Nugraheni Universitas Diponegoro
DOI: https://doi.org/10.29207/resti.v7i1.4767
Keywords: Sentiment Analysis, Decision Tree, K-Nearest Neighbors, Naïve Bayes, Random Forest

Abstract

The assessment of academic articles is based on the number of citations, but the number only is not enough. So now there is Altmetric which can measure the impact of academic articles from the number of citations and using social media, usually Twitter. Still, the number of mentions on Twitter is not enough because the expressions of the sentences vary. Mentions must be classified according to neutral, positive, and negative criteria. Sentiment analysis is performed on tweets to measure social media volume and attention related to research findings from academic articles. There are many sentiment analysis methods, so this study aims to compare sentiment analysis methods using Decision Tree, K-NN, Naïve Bayes, and Random Forest to get the most suitable methods. The evaluation method in this study uses the Confusion Matrix by searching for Accuracy, Precision, and Recall values. The results show that the most suitable sentiment analysis method is Naïve Bayes by obtaining the highest classification suitability value of the other methods, which has an actual positive sentiment value of neutral 2056, positive 1200, and negative 1292. In addition, Naïve Bayes gets the highest accuracy score of 95, 45%.

 

Downloads

Download data is not yet available.

References

D. Hantono and D. Pramitasari, “Aspek Perilaku Manusia Sebagai Makhluk Individu dan Sosial Pada Ruang Terbuka Publik,” Nature: National Academic Journal of Architecture, vol. 5, no. 2, p. 85, Dec. 2018.

https://doi.org/10.24252/nature.v5i2a1.

A. Hadiy, D. Fatra, N. Hayatin, C. Sri, and K. Aditya, “Analisa Sentimen Tweet Berbahasa Indonesia Dengan Menggunakan Metode Lexicon Pada Topik Perpindahan Ibu Kota Indonesia,” REPOSITOR, vol. 2, no. 7, pp. 977–984, 2020.

https://doi.org/10.22219/repositor.v2i7.937.

M. Thelwall, “Measuring Societal Impacts of Research with Altmetrics? Common Problems and Mistakes,” in Journal of Economic Surveys, Dec. 2021, vol. 35, no. 5, pp. 1302–1314.

https://doi.org/10.1111/joes.12381.

J. Kolahi, S. Khazaei, P. Iranmanesh, J. Kim, H. Bang, and A. Khademi, “Meta-Analysis of Correlations between Altmetric Attention Score and Citations in Health Sciences,” BioMed Research International, vol. 2021. Hindawi Limited, 2021.

https://doi.org/10.1155/2021/6680764.

A. F. Rochim, F. R. Rahadian, and D. Eridani, “An Alternative Analysis Method to Measure Impact of Academic Papers Shared to Social Media and Number of Citations Obtained Based on Support Vector Machine Algorithm,” in 18th International Conference on Scientometrics and Informetrics Conference 2021, 2021, pp. 393–400.

A. Rahmaniar, D. Pratiwi, and E. B. Setiawan, “Implementation of Rumor Detection on Twitter Using the SVM Classification Method,” System Engineering and Information Technology (RESTI), vol. 4, no. 2, pp. 782–789, 2020.

https://doi.org/10.29207/resti.v4i5.2031.

A. Bayhaqy, S. Sfenrianto, K. Nainggolan, and E. R. Kaburuan, “Sentiment Analysis about E-Commerce from Tweets Using Decision Tree, K-Nearest Neighbor, and Naïve Bayes,” in 2018 International Conference on Orange Technologies, ICOT 2018, Jul. 2018, pp. 1–6.

http://doi.org/10.1109/ICOT.2018.8705796.

A. R. Isnain, J. Supriyanto, and M. P. Kharisma, “Implementation of K-Nearest Neighbor (K-NN) Algorithm For Public Sentiment Analysis of Online Learning,” IJCCS (Indonesian Journal of Computing and Cybernetics Systems), vol. 15, no. 2, pp. 121–130, Apr. 2021.

https://doi.org/10.22146/ijccs.65176.

M. Abbas, K. Ali Memon, and A. Aleem Jamali, “Multinomial Naive Bayes Classification Model for Sentiment Analysis,” IJCSNS International Journal of Computer Science and Network Security, vol. 19, no. 3, p. 62, 2019.

https://doi.org/10.13140/RG.2.2.30021.40169.

M. A. Fauzi, “Random forest approach fo sentiment analysis in Indonesian language,” Indonesian Journal of Electrical Engineering and Computer Science, vol. 12, no. 1, pp. 46–50, Oct. 2018.

http://doi.org/10.11591/ijeecs.v12.i1.pp46-50.

M. S. Hossain, N. Nayla, and A. A. Rasel, “Product Market Demand Analysis Using NLP in Banglish Text with Sentiment Analysis and Named Entity Recognition,” In 2022 56th Annual Conference on Information Sciences and Systems (CISS) , pp. 166–171, Apr. 2022.

https://doi.org/10.1109/CISS53076.2022.975118.

K. G. Andersen, A. Rambaut, W. I. Lipkin, E. C. Holmes, and R. F. Garry, “The proximal origin of SARS-CoV-2,” Nature Medicine, vol. 26, no. 4. Nature Research, pp. 450–452, Apr. 01, 2020.

https://doi.org/10.1038/s41591-020-0820-9.

S. Zhang, H. Xu, G. Zhu, X. Chen, and K. Li, “A Data Processing Method Based on Sequence Labelling and Syntactic Analysis for Extracting New Sentiment Words from Product Reviews,” Soft comput, vol. 26, no. 22, pp. 853–866, 2022.

https://doi.org/10.3390/math9121449.

M. Anandarajan, C. Hill, and T. Nolan, “Text Preprocessing,” in Practical Text Analytics, vol. 2, 2019, pp. 45–59.

https://doi.org/10.1007/978-3-319-95663-3_4.

A. Gaydhani, V. Doma, S. Kendre, and L. Bhagwat, “Detecting Hate Speech and Offensive Language on Twitter using Machine Learning: An N-gram and TFIDF based Approach,” in IEEE International Advance Computing Conference 2018, Sep. 2018.

https://doi.org/10.48550/arXiv.1809.08651.

R. Ahuja, A. Chug, S. Kohli, S. Gupta, and P. Ahuja, “The impact of features extraction on the sentiment analysis,” in Procedia Computer Science, 2019, vol. 152, pp. 341–348.

https://doi.org/10.1016/j.procs.2019.05.008.

Pristiyono, M. Ritonga, M. A. al Ihsan, A. Anjar, and F. H. Rambe, “Sentiment analysis of COVID-19 vaccine in Indonesia using Naïve Bayes Algorithm,” IOP Conf Ser Mater Sci Eng, vol. 1088, no. 1, p. 012045, Feb. 2021.

https://doi.org/10.1088/1757-899x/1088/1/012045.

Y. al Amrani, M. Lazaar, and K. E. el Kadirp, “Random forest and support vector machine based hybrid approach to sentiment analysis,” in Procedia Computer Science, 2018, vol. 127, pp. 511–520.

https://doi.org/10.1016/j.procs.2018.01.150.

A. Tharwat, “Classification assessment methods,” Applied Computing and Informatics, vol. 17, no. 1, pp. 168–192, 2020.

https://doi.org/10.1016/j.aci.2018.08.003.

Published
2023-02-05
How to Cite
Fakhrezi, M. F., Adian Fatchur Rochim, & Dinar Mutiara Kusomo Nugraheni. (2023). Comparison of Sentiment Analysis Methods Based on Accuracy Value Case Study: Twitter Mentions of Academic Article. Jurnal RESTI (Rekayasa Sistem Dan Teknologi Informasi), 7(1), 161 - 167. https://doi.org/10.29207/resti.v7i1.4767
Issue
Vol 7 No 1 (2023): February 2023
Section
Information Systems Engineering Articles

Copyright (c) 2023 Jurnal RESTI (Rekayasa Sistem dan Teknologi Informasi)

Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Copyright in each article belongs to the author

  1. The author acknowledges that the RESTI Journal (System Engineering and Information Technology) is the first publisher to publish with a license Creative Commons Attribution 4.0 International License.
  2. Authors can enter writing separately, arrange the non-exclusive distribution of manuscripts that have been published in this journal into other versions (eg sent to the author's institutional repository, publication in a book, etc.), by acknowledging that the manuscript has been published for the first time in the RESTI (Rekayasa Sistem dan Teknologi Informasi) journal ;