# Multimodal Classification for Analysing Social Media

arXiv: https://arxiv.org/abs/1708.02099 · HTML: https://mlsanity.com/paper/1708.02099/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Aug 2017.

## Built on by (most cited first, 8 of 8)

- [Detecting Harmful Memes and Their Targets](https://mlsanity.com/paper/2110.00413/) (2021)
- [Overcoming Language Disparity in Online Content Classification with Multimodal Learning](https://mlsanity.com/paper/2205.09744/) (2022)
- [MM-HSD: Multi-Modal Hate Speech Detection in Videos](https://mlsanity.com/paper/2508.20546/) (2025)
- [Robustness of Fusion-based Multimodal Classifiers to Cross-Modal Content Dilutions](https://mlsanity.com/paper/2211.02646/) (2022)
- [NLP-CUET@DravidianLangTech-EACL2021: Investigating Visual and Textual Features to Identify Trolls from Multimodal Social Media Memes](https://mlsanity.com/paper/2103.00466/) (2021)
- [Self-Augmented Multi-Modal Feature Embedding](https://mlsanity.com/paper/2103.04731/) (2021)
- [A Comprehensive Review of Visual-Textual Sentiment Analysis from Social Media Networks](https://mlsanity.com/paper/2207.02160/) (2022)
- [MultiSiam: A Multiple Input Siamese Network For Social Media Text Classification And Duplicate Text Detection](https://mlsanity.com/paper/2401.06783/) (2024)

## Abstract (arXiv)

Classification of social media data is an important approach in understanding user behavior on the Web. Although information on social media can be of different modalities such as texts, images, audio or videos, traditional approaches in classification usually leverage only one prominent modality. Techniques that are able to leverage multiple modalities are often complex and susceptible to the absence of some modalities. In this paper, we present simple models that combine information from different modalities to classify social media content and are able to handle the above problems with existing techniques. Our models combine information from different modalities using a pooling layer and an auxiliary learning task is used to learn a common feature space. We demonstrate the performance of our models and their robustness to the missing of some modalities in the emotion classification domain. Our approaches, although being simple, can not only achieve significantly higher accuracies than traditional fusion approaches but also have comparable results when only one modality is available.