Leveraging Sarcasm Text Detection in Meme Polarity Classification

Authors

  • Zeva Patu Assyadid Telkom University
  • Ema Rachmawati Telkom University

DOI:

https://doi.org/10.29207/resti.v10i2.7516

Keywords:

deep learning, meme, memotion, sarcasm detection, sentiment classification

Abstract

Memes are a popular internet format and spread quickly across many social media platforms. People can express their ideas, criticisms, interests, or aversions through memes. But in some cases, other people may interpret memes differently and feel bad about it. This variation in meme interpretation is a challenge in sentiment analysis, as a meme can be judged negative or positive by different individuals. Therefore, there is a need for an automated system that can consistently predict the sentiment polarity of memes. A meme is multimodal content that could consist of visual and textual components, which is suitable for a sentiment polarity analysis study. To model a system that effectively leverages multimodal features, the model needs to understand the meme features. This study proposes a joint deep learning model—BERT and Densenet121—that concatenates text, image, and cluster features based on the extracted face encodings. To assess the context of the texts better, BERT was trained with a sarcasm dataset. The widely used ‘SemEval 2020 Task 8: Memotion Analysis dataset’ was used in this study due to its comprehensive annotation of meme-based sentiment and sarcasm, which aligns with this study’s approach. The result demonstrates that the model achieved 0.3738 Accuracy (+2.52%) and 0.3735 Weighted F1 (+1.04%), while maintaining competitive Macro F1 (0.3047). This result shows effective sarcasm adaptation on imbalanced datasets, with improved ability to detect positive and neutral sentiments and reduce sarcastic false negatives compared to the base model. This highlights the effectiveness of integrating sarcasm detection into the model framework for robust sentiment classification in memes.

Downloads

Download data is not yet available.

References

S. Tabatabaei and Elena Anatolievna Ivanova, “The Role of Memes on Emotional Contagion,” Elementary Education Online, vol. 20, no. 5, pp. 6028–6036, 2021, doi: 10.17051/ilkonline.2021.05.676.

M. Velmala, S. Rajiakodi, K. Pannerselvam, and B. Sivagnanam, “Multimodal Sentiment Analysis of Online Memes: Integrating Text and Image Features for Enhanced Classification,” in Procedia Computer Science, Elsevier, Jan. 2025, pp. 355–364. doi: 10.1016/J.PROCS.2025.04.272.

M. Hazman, S. McKeever, and J. Griffith, “What Makes a Meme a Meme? Identifying Memes for Memetics-Aware Dataset Creation,” Proceedings of the International AAAI Conference on Web and Social Media, vol. 19, pp. 745–759, Jul. 2024, doi: 10.1609/icwsm.v19i1.35843.

R. K. Routhu and U. Baruah, “Sentiment Analysis on Memes: A Review,” Expert Syst., vol. 42, no. 11, p. e70133, Nov. 2025, doi: https://doi.org/10.1111/exsy.70133.

C. Ling, I. Abuhilal, J. Blackburn, E. De Cristofaro, S. Zannettou, and G. Stringhini, “Dissecting the Meme Magic: Understanding Indicators of Virality in Image Memes,” in Proceedings of the ACM on Human-Computer Interaction, Association for Computing Machinery, Apr. 2021. doi: https://doi.org/10.1145/3449155.

M. S. Hee, R. Cao, T. Chakraborty, and R. K. W. Lee, “Understanding (Dark) Humour with Internet Meme Analysis,” in WWW 2024 Companion - Companion Proceedings of the ACM Web Conference, Association for Computing Machinery, Inc, May 2024, pp. 1276–1279. doi: https://doi.org/10.1145/3589335.3641249.

L. Bonheme and M. Grzes, “SESAM at SemEval-2020 Task 8: Investigating the Relationship between Image and Text in Sentiment Analysis of Memes,” 14th International Workshops on Semantic Evaluation, SemEval 2020 - co-located 28th International Conference on Computational Linguistics, COLING 2020, Proceedings, pp. 804–816, 2020, doi: 10.18653/V1/2020.SEMEVAL-1.102.

Z. Zhao et al., “MemeReaCon: Probing Contextual Meme Understanding in Large Vision-Language Models,” in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Suzhou, China: Association for Computational Linguistics (ACL), Nov. 2025, pp. 3559–3582. doi: 10.18653/V1/2025.EMNLP-MAIN.176.

S. Sharma et al., “Detecting and Understanding Harmful Memes: A Survey,” IJCAI International Joint Conference on Artificial Intelligence, vol. 6, pp. 5597–5606, Jul. 2022, doi: 10.24963/IJCAI.2022/781.

K. P. N. Nguyen and V. Ng, “Computational Meme Understanding: A Survey,” EMNLP 2024 - 2024 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, pp. 21251–21267, 2024, doi: 10.18653/V1/2024.EMNLP-MAIN.1184.

L. Bojić et al., “Comparing large Language models and human annotators in latent content analysis of sentiment, political leaning, emotional intensity and sarcasm,” Scientific Reports 2025 15:1, vol. 15, no. 1, pp. 11477-, Apr. 2025, doi: 10.1038/s41598-025-96508-3.

A. Konyspay, P. Shamoi, M. Ziyada, and Z. Smambayev, “Meme Similarity and Emotion Detection using Multimodal Analysis,” in 2025 International Conference on Activity and Behavior Computing (ABC), Al Ain, United Arab Emirates: IEEE, Mar. 2025. doi: https://doi.org/10.1109/ABC64332.2025.11118558.

M. Hazman, S. McKeever, and J. Griffith, “Meme Sentiment Analysis Enhanced with Multimodal Spatial Encoding and Face Embedding,” Communications in Computer and Information Science, vol. 1662 CCIS, pp. 318–331, 2023, doi: 10.1007/978-3-031-26438-2_25/TABLES/6.

S. Gundapu and R. Mamidi, “Gundapusunil at SemEval-2020 Task 8: Multimodal Memotion Analysis,” 14th International Workshops on Semantic Evaluation, SemEval 2020 - co-located 28th International Conference on Computational Linguistics, COLING 2020, Proceedings, pp. 1112–1119, 2020, doi: 10.18653/V1/2020.SEMEVAL-1.147.

S. Chi, G. Sang, and X. Shi, “Research on Multimodal Sentiment Classification of Internet Memes Based on Transformer,” in ACM International Conference Proceeding Series, Association for Computing Machinery, Jan. 2024, pp. 445–450. doi: https://doi.org/10.1145/3673277.3673354.

E. Hossain, O. Sharif, and M. Moshiul Hoque, “MemoSen: A Multimodal Dataset for Sentiment Analysis of Memes,” in Proceedings of the Thirteenth Language Resources and Evaluation Conference, Marseille, France: European Language Resources Association, Jun. 2022, pp. 1542–1554. Accessed: Jan. 21, 2026. [Online]. Available: https://aclanthology.org/2022.lrec-1.165/

Y. Duan, X. Zhao, and H. Guo, “Sentiment-aware cross-modal semantic interaction model for harmful meme detection,” Decis. Support Syst., vol. 196, p. 114509, Sep. 2025, doi: 10.1016/J.DSS.2025.114509.

T. H. Afridi, A. Alam, M. N. Khan, J. Khan, and Y. K. Lee, “A Multimodal Memes Classification: A Survey and Open Research Issues,” Lecture Notes in Networks and Systems, vol. 183, pp. 1451–1466, 2021, doi: 10.1007/978-3-030-66840-2_109.

C. Sharma et al., “SemEval-2020 Task 8: Memotion Analysis- the Visuo-Lingual Metaphor!,” 14th International Workshops on Semantic Evaluation, SemEval 2020 - co-located 28th International Conference on Computational Linguistics, COLING 2020, Proceedings, pp. 759–773, 2020, doi: 10.18653/V1/2020.SEMEVAL-1.99.

Y. Guo, J. Huang, Y. Dong, and M. Xu, “Guoym at SemEval-2020 Task 8: Ensemble-based Classification of Visuo-Lingual Metaphor in Memes,” 14th International Workshops on Semantic Evaluation, SemEval 2020 - co-located 28th International Conference on Computational Linguistics, COLING 2020, Proceedings, pp. 1120–1125, 2020, doi: 10.18653/V1/2020.SEMEVAL-1.148.

A. Maladry, E. Lefever, C. Van Hee, and V. Hoste, “The limitations of irony detection in Dutch social media,” Language Resources and Evaluation 2023 58:4, vol. 58, no. 4, pp. 1355–1386, Jul. 2023, doi: 10.1007/S10579-023-09656-1.

R. A. Potamias, G. Siolas, and A. G. Stafylopatis, “A transformer-based approach to irony and sarcasm detection,” Neural Computing and Applications 2020 32:23, vol. 32, no. 23, pp. 17309–17320, Jun. 2020, doi: 10.1007/S00521-020-05102-3.

C. Mello, · Gullal, S. Cheema, · Gaurish Thakkar, G. S. Cheema, and G. Thakkar, “Combining sentiment analysis classifiers to explore multilingual news articles covering London 2012 and Rio 2016 Olympics,” International Journal of Digital Humanities 2022 5:2, vol. 5, no. 2, pp. 131–157, Nov. 2022, doi: 10.1007/S42803-022-00052-9.

N. Novielli et al., “Assessment of off-the-shelf SE-specific sentiment analysis tools: An extended replication study,” Empirical Software Engineering 2021 26:4, vol. 26, no. 4, pp. 77-, Jun. 2021, doi: 10.1007/S10664-021-09960-W.

U. Naseem, I. Razzak, P. Eklund, and K. Musial, “Towards Improved Deep Contextual Embedding for the identification of Irony and Sarcasm,” Proceedings of the International Joint Conference on Neural Networks, Jul. 2020, doi: 10.1109/IJCNN48605.2020.9207237.

X. Guo, J. Ma, and A. Zubiaga, “Cluster-based deep ensemble learning for emotion classification in Internet memes,” J. Inf. Sci., vol. 51, no. 1, pp. 265–283, Feb. 2025, doi: https://doi.org/10.1177/01655515221136241.

W. Chen, F. Lin, G. Li, and B. Liu, “A survey of automatic sarcasm detection: Fundamental theories, formulation, datasets, detection methods, and opportunities,” Neurocomputing, vol. 578, p. 127428, Apr. 2024, doi: 10.1016/J.NEUCOM.2024.127428.

S. Farabi, T. Ranasinghe, D. Kanojia, Y. Kong, and M. Zampieri, “A Survey of Multimodal Sarcasm Detection,” IJCAI International Joint Conference on Artificial Intelligence, pp. 8020–8028, 2024, doi: 10.24963/IJCAI.2024/887.

P. Parameswaran, A. Trotman, V. Liesaputra, and D. Eyers, “BERT’s The Word : Sarcasm Target Detection using BERT,” in Proceedings of the 19th Annual Workshop of the Australasian Language Technology Association, Australasian Language Technology Association, Dec. 2021, pp. 185–191. Accessed: Jan. 21, 2026. [Online]. Available: https://aclanthology.org/2021.alta-1.21/

I. N. Y. Sumadewa, A. Anggrawan, H. Hairani, I. P. Hariyadi, C. Satria, and D. S. C. Saputri, “Emotion Classification on Customer Reviews of Drinking Water Services Using IndoBERT and Machine Learning Algorithms,” in 2025 7th International Conference on Cybernetics and Intelligent System (ICORIS), Mataram, Indonesia: Institute of Electrical and Electronics Engineers (IEEE), Dec. 2025, pp. 1–5. doi: 10.1109/icoris67789.2025.11296077.

A. Baruah, K. A. Das, F. A. Barbhuiya, and K. Dey, “IIITG-ADBU at SemEval-2020 Task 8: A Multimodal Approach to Detect Offensive, Sarcastic and Humorous Memes,” 14th International Workshops on Semantic Evaluation, SemEval 2020 - co-located 28th International Conference on Computational Linguistics, COLING 2020, Proceedings, pp. 885–890, 2020, doi: 10.18653/v1/2020.semeval-1.112.

A.-M. Bucur, A. Cosma, and I.-B. Iordache, “BLUE at Memotion 2.0 2022: You have my Image, my Text and my Transformer,” CEUR Workshop Proc., vol. 3168, Apr. 2022, Accessed: Feb. 26, 2026. [Online]. Available: http://arxiv.org/abs/2202.07543

Downloads

Published

2026-04-22

Issue

Section

Computer Science Applications

How to Cite

Assyadid, Z. P., & Rachmawati, E. (2026). Leveraging Sarcasm Text Detection in Meme Polarity Classification. Jurnal RESTI (Rekayasa Sistem Dan Teknologi Informasi), 10(2), 355-365. https://doi.org/10.29207/resti.v10i2.7516