# Vision-Language Pre-Training for Multimodal Aspect-Based Sentiment Analysis

arXiv: https://arxiv.org/abs/2204.07955 · HTML: https://mlsanity.com/paper/2204.07955/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 13 of its 40 references that are in the map.
- First posted on arXiv 17 Apr 2022.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-training](https://mlsanity.com/paper/2002.10638/) (2020)
- [ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graph](https://mlsanity.com/paper/2006.16934/) (2020)
- [A Unified Generative Framework for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2106.04300/) (2021)
- [A Unified Model for Opinion Target Extraction and Target Sentiment Prediction](https://mlsanity.com/paper/1811.05082/) (2018)
- [An Interactive Multi-Task Learning Network for End-to-End Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/1906.06906/) (2019)
- [Open-Domain Targeted Sentiment Analysis via Span-Based Extraction and Classification](https://mlsanity.com/paper/1906.03820/) (2019)
- [Exploiting BERT For Multimodal Target Sentiment Classification Through Input Space Translation](https://mlsanity.com/paper/2108.01682/) (2021)
- [RpBERT: A Text-image Relation Propagation-based BERT Model for Multimodal NER](https://mlsanity.com/paper/2102.02967/) (2021)
- [DeepSentiBank: Visual Sentiment Concept Classification with Deep Convolutional Neural Networks](https://mlsanity.com/paper/1410.8586/) (2014)
- [KM-BART: Knowledge Enhanced Multimodal BART for Visual Commonsense Generation](https://mlsanity.com/paper/2101.00419/) (2021)

## Built on by (most cited first, 21 of 21)

- [Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey](https://mlsanity.com/paper/2302.10035/) (2023)
- [All in One: Multi-task Prompting for Graph Neural Networks](https://mlsanity.com/paper/2307.01504/) (2023)
- [MM-BigBench: Evaluating Multimodal Models on Multimodal Content Comprehension Tasks](https://mlsanity.com/paper/2310.09036/) (2023)
- [WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledge](https://mlsanity.com/paper/2401.06659/) (2024)
- [Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A Survey](https://mlsanity.com/paper/2406.08068/) (2024)
- [PanoSent: A Panoptic Sextuple Extraction Benchmark for Multimodal Conversational Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2408.09481/) (2024)
- [Recent Advances in Multimodal Affective Computing: An NLP Perspective](https://mlsanity.com/paper/2409.07388/) (2024)
- [Few-shot Multimodal Sentiment Analysis based on Multimodal Probabilistic Fusion Prompts](https://mlsanity.com/paper/2211.06607/) (2022)
- [Provoking Multi-modal Few-Shot LVLM via Exploration-Exploitation In-Context Learning](https://mlsanity.com/paper/2506.09473/) (2025)
- [Exploring Cognitive and Aesthetic Causality for Multimodal Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2504.15848/) (2025)
- [An Empirical Study on Configuring In-Context Learning Demonstrations for Unleashing MLLMs' Sentimental Perception Capability](https://mlsanity.com/paper/2505.16193/) (2025)
- [RNG: Reducing Multi-level Noise and Multi-grained Semantic Gap for Joint Multimodal Aspect-Sentiment Analysis](https://mlsanity.com/paper/2405.13059/) (2024)
- [Target-oriented Sentiment Classification with Sequential Cross-modal Semantic Graph](https://mlsanity.com/paper/2208.09417/) (2022)
- [TCAB: A Large-Scale Text Classification Attack Benchmark](https://mlsanity.com/paper/2210.12233/) (2022)
- [RethinkingTMSC: An Empirical Study for Target-Oriented Multimodal Sentiment Classification](https://mlsanity.com/paper/2310.09596/) (2023)
- [Enhanced Multimodal Aspect-Based Sentiment Analysis by LLM-Generated Rationales](https://mlsanity.com/paper/2505.14499/) (2025)
- [AdaptiSent: Context-Aware Adaptive Attention for Multimodal Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2507.12695/) (2025)
- [DualKanbaFormer: An Efficient Selective Sparse Framework for Multimodal Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2408.15379/) (2024)
- [Resource-Limited Joint Multimodal Sentiment Reasoning and Classification via Chain-of-Thought Enhancement and Distillation](https://mlsanity.com/paper/2508.05234/) (2025)
- [GateMABSA: Aspect-Image Gated Fusion for Multimodal Aspect-based Sentiment Analysis](https://mlsanity.com/paper/2509.25037/) (2025)
- [Semantic-Preserving Cross-Style Visual Reasoning for Robust Multi-Modal Understanding in Large Vision-Language Models](https://mlsanity.com/paper/2510.22838/) (2025)

## Abstract (arXiv)

As an important task in sentiment analysis, Multimodal Aspect-Based Sentiment Analysis (MABSA) has attracted increasing attention in recent years. However, previous approaches either (i) use separately pre-trained visual and textual models, which ignore the crossmodal alignment or (ii) use vision-language models pre-trained with general pre-training tasks, which are inadequate to identify finegrained aspects, opinions, and their alignments across modalities. To tackle these limitations, we propose a task-specific Vision-Language Pre-training framework for MABSA (VLPMABSA), which is a unified multimodal encoder-decoder architecture for all the pretraining and downstream tasks. We further design three types of task-specific pre-training tasks from the language, vision, and multimodal modalities, respectively. Experimental results show that our approach generally outperforms the state-of-the-art approaches on three MABSA subtasks. Further analysis demonstrates the effectiveness of each pretraining task. The source code is publicly released at https://github.com/NUSTM/VLP-MABSA.