Mu2STS: A Multitask Multimodal Sarcasm-Humor-Differential Teacher-Student Model for Sarcastic Meme Detection

Kumari, G; Adak, C; Ekbal, A

Mu2STS: A Multitask Multimodal Sarcasm-Humor-Differential Teacher-Student Model for Sarcastic Meme Detection

Kumari, G Adak, C

Ekbal, A

Permalink

Publisher:: SPRINGER INTERNATIONAL PUBLISHING AG
Publication Type:: Chapter
Citation:: Advances in Information Retrieval, 2024, 14610 LNCS, pp. 19-37
Issue Date:: 2024-01-01

Recently Added

	Filename	Description	Size
	978-3-031-56063-7_2.pdf	Published version	1.65 MB	Adobe PDF	View/Open

Copyright Clearance Process

Recently Added
In Progress
Open Access

This item is new to OPUS and is not currently available.

Full metadata record

Field	Value	Language
dc.contributor.author	Kumari, G
dc.contributor.author	Adak, C https://orcid.org/0000-0002-9085-2770
dc.contributor.author	Ekbal, A
dc.contributor.editor	Goharian, N
dc.contributor.editor	Tonellotto, N
dc.contributor.editor	He, Y
dc.contributor.editor	Lipani, A
dc.contributor.editor	McDonald, G
dc.contributor.editor	Macdonald, C
dc.contributor.editor	Ounis, I
dc.date.accessioned	2024-09-09T05:26:13Z
dc.date.available	2024-09-09T05:26:13Z
dc.date.issued	2024-01-01
dc.identifier.citation	Advances in Information Retrieval, 2024, 14610 LNCS, pp. 19-37
dc.identifier.isbn	978-3-031-56062-0
dc.identifier.uri	http://hdl.handle.net/10453/180762
dc.description.abstract	Memes, a prevalent form of online communication, often express opinions, emotions, and creativity concisely and entertainingly. Amidst the diverse landscape of memes, the realm of sarcastic memes holds a unique position with its foundation in irony, mockery, satire, and messages that diverge from literal meanings. Detecting sarcasm in memes is challenging due to the intricate interplay between sarcasm and humor. While prior research has primarily concentrated on leveraging the relationship between sarcasm and humor for identifying sarcastic memes, our goal in this paper extends beyond establishing a fundamental connection between the two; instead, we aspire to unravel their distinct characteristics and nuances that differentiate sarcasm from humor. To accomplish this, we introduce a novel deep learning model, i.e., Mu2STS (MultitaskMultimodalSarcasm-Humor-DifferentialTeacher-Student), for sarcasm detection in memes, with a special focus on humor. To bolster Mu2STS, we have developed the SHMH (WARNING: This paper contains meme samples that are offensive in nature.) (Sarcasm-with-Humorous-Meme-in-Hindi) dataset, designed for detecting sarcasm and humor in memes written in the Hindi language, which is the first of its kind to the best of our knowledge. Our empirical evaluation, which includes both qualitative and quantitative analyses conducted on the SHMH dataset and some benchmark meme datasets, clearly illustrates the effectiveness of Mu2STS, which outperformed major state-of-the-art models. (The dataset and codes are available at https://www.iitp.ac.in/~ai-nlp-ml/resources.html.)
dc.language	en
dc.publisher	SPRINGER INTERNATIONAL PUBLISHING AG
dc.relation.ispartof	Advances in Information Retrieval
dc.relation.ispartofseries	Lecture Notes in Computer Science
dc.relation.isbasedon	10.1007/978-3-031-56063-7_2
dc.rights	info:eu-repo/semantics/restrictedAccess
dc.subject.classification	Artificial Intelligence & Image Processing
dc.subject.classification	46 Information and computing sciences
dc.title	Mu2STS: A Multitask Multimodal Sarcasm-Humor-Differential Teacher-Student Model for Sarcastic Meme Detection
dc.type	Chapter
utslib.citation.volume	14610 LNCS
pubs.organisational-group	University of Technology Sydney
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology
pubs.organisational-group	University of Technology Sydney/Faculty of Engineering and Information Technology/School of Computer Science
utslib.copyright.status	recently_added	*
dc.date.updated	2024-09-09T05:26:11Z
pubs.publication-status	Published
pubs.volume	14610 LNCS

Abstract:

Memes, a prevalent form of online communication, often express opinions, emotions, and creativity concisely and entertainingly. Amidst the diverse landscape of memes, the realm of sarcastic memes holds a unique position with its foundation in irony, mockery, satire, and messages that diverge from literal meanings. Detecting sarcasm in memes is challenging due to the intricate interplay between sarcasm and humor. While prior research has primarily concentrated on leveraging the relationship between sarcasm and humor for identifying sarcastic memes, our goal in this paper extends beyond establishing a fundamental connection between the two; instead, we aspire to unravel their distinct characteristics and nuances that differentiate sarcasm from humor. To accomplish this, we introduce a novel deep learning model, i.e., Mu2STS (MultitaskMultimodalSarcasm-Humor-DifferentialTeacher-Student), for sarcasm detection in memes, with a special focus on humor. To bolster Mu2STS, we have developed the SHMH (WARNING: This paper contains meme samples that are offensive in nature.) (Sarcasm-with-Humorous-Meme-in-Hindi) dataset, designed for detecting sarcasm and humor in memes written in the Hindi language, which is the first of its kind to the best of our knowledge. Our empirical evaluation, which includes both qualitative and quantitative analyses conducted on the SHMH dataset and some benchmark meme datasets, clearly illustrates the effectiveness of Mu2STS, which outperformed major state-of-the-art models. (The dataset and codes are available at https://www.iitp.ac.in/~ai-nlp-ml/resources.html.)

Please use this identifier to cite or link to this item:

http://hdl.handle.net/10453/180762