# TIB-VA at SemEval-2022 Task 5: A Multimodal Architecture for the Detection and Classification of Misogynous Memes

arXiv: https://arxiv.org/abs/2204.06299 · HTML: https://mlsanity.com/paper/2204.06299/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 16 references that are in the map.
- First posted on arXiv 13 Apr 2022.

## Builds on (most cited first, 7 of 7)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://mlsanity.com/paper/2005.04790/) (2020)
- [Exploring Hate Speech Detection in Multimodal Publications](https://mlsanity.com/paper/1910.03814/) (2019)
- [SemEval-2020 Task 8: Memotion Analysis -- The Visuo-Lingual Metaphor!](https://mlsanity.com/paper/2008.03781/) (2020)
- [Detecting Harmful Memes and Their Targets](https://mlsanity.com/paper/2110.00413/) (2021)
- [Creating a Multimodal Dataset of Images and Text to Study Abusive Language](https://mlsanity.com/paper/2005.02235/) (2020)

## Built on by (most cited first, 4 of 4)

- [Toxic Memes: A Survey of Computational Perspectives on the Detection and Explanation of Meme Toxicities](https://mlsanity.com/paper/2406.07353/) (2024)
- [SemiMemes: A Semi-supervised Learning Approach for Multimodal Memes Analysis](https://mlsanity.com/paper/2304.00020/) (2023)
- [Leveraging LLMs for Context-Aware Implicit Textual and Multimodal Hate Speech Detection](https://mlsanity.com/paper/2510.15685/) (2025)
- [Multimodal Sexism Identification and Characterization using Large Language Models and Gradient Boosting](https://mlsanity.com/paper/2606.05997/) (2026)

## Abstract (arXiv)

The detection of offensive, hateful content on social media is a challenging problem that affects many online users on a daily basis. Hateful content is often used to target a group of people based on ethnicity, gender, religion and other factors. The hate or contempt toward women has been increasing on social platforms. Misogynous content detection is especially challenging when textual and visual modalities are combined to form a single context, e.g., an overlay text embedded on top of an image, also known as meme. In this paper, we present a multimodal architecture that combines textual and visual features in order to detect misogynous meme content. The proposed architecture is evaluated in the SemEval-2022 Task 5: MAMI - Multimedia Automatic Misogyny Identification challenge under the team name TIB-VA. Our solution obtained the best result in the Task-B where the challenge is to classify whether a given document is misogynous and further identify the main sub-classes of shaming, stereotype, objectification, and violence.