# New Benchmark Dataset and Fine-Grained Cross-Modal Fusion Framework for Vietnamese Multimodal Aspect-Category Sentiment Analysis

arXiv: https://arxiv.org/abs/2405.00543 · HTML: https://mlsanity.com/paper/2405.00543/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 82 references that are in the map.
- First posted on arXiv 1 May 2024.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Multimodal Transformer for Unaligned Multimodal Language Sequences](https://mlsanity.com/paper/1906.00295/) (2019)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [Relation Networks for Object Detection](https://mlsanity.com/paper/1711.11575/) (2017)
- [MemNet: A Persistent Memory Network for Image Restoration](https://mlsanity.com/paper/1708.02209/) (2017)
- [MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos](https://mlsanity.com/paper/1606.06259/) (2016)
- [Image Captioning: Transforming Objects into Words](https://mlsanity.com/paper/1906.05963/) (2019)
- [PhoBERT: Pre-trained language models for Vietnamese](https://mlsanity.com/paper/2003.00744/) (2020)
- [Utilizing BERT for Aspect-Based Sentiment Analysis via Constructing Auxiliary Sentence](https://mlsanity.com/paper/1903.09588/) (2019)
- [Interactive Attention Networks for Aspect-Level Sentiment Classification](https://mlsanity.com/paper/1709.00893/) (2017)
- [Aspect Based Sentiment Analysis with Gated Convolutional Networks](https://mlsanity.com/paper/1805.07043/) (2018)
- [A Survey on Aspect-Based Sentiment Analysis: Tasks, Methods, and Challenges](https://mlsanity.com/paper/2203.01054/) (2022)
- [Exploiting BERT For Multimodal Target Sentiment Classification Through Input Space Translation](https://mlsanity.com/paper/2108.01682/) (2021)
- [Emotion Recognition for Vietnamese Social Media Text](https://mlsanity.com/paper/1911.09339/) (2019)
- [Solving Aspect Category Sentiment Analysis as a Text Generation Task](https://mlsanity.com/paper/2110.07310/) (2021)
- [OpenViVQA: Task, Dataset, and Multimodal Fusion Models for Visual Question Answering in Vietnamese](https://mlsanity.com/paper/2305.04183/) (2023)
- [ASAP: A Chinese Review Dataset Towards Aspect Category Sentiment Analysis and Rating Prediction](https://mlsanity.com/paper/2103.06605/) (2021)
- [SA2SL: From Aspect-Based Sentiment Analysis to Social Listening System for Business Intelligence](https://mlsanity.com/paper/2105.15079/) (2021)
- [ViSoBERT: A Pre-Trained Language Model for Vietnamese Social Media Text Processing](https://mlsanity.com/paper/2310.11166/) (2023)
- [EVJVQA Challenge: Multilingual Visual Question Answering](https://mlsanity.com/paper/2302.11752/) (2023)
- [ViCLEVR: A Visual Reasoning Dataset and Hybrid Multimodal Fusion Model for Visual Question Answering in Vietnamese](https://mlsanity.com/paper/2310.18046/) (2023)
- [MACSA: A Multimodal Aspect-Category Sentiment Analysis Dataset with Multimodal Fine-grained Aligned Annotations](https://mlsanity.com/paper/2206.13969/) (2022)

## Abstract (arXiv)

The emergence of multimodal data on social media platforms presents new opportunities to better understand user sentiments toward a given aspect. However, existing multimodal datasets for Aspect-Category Sentiment Analysis (ACSA) often focus on textual annotations, neglecting fine-grained information in images. Consequently, these datasets fail to fully exploit the richness inherent in multimodal. To address this, we introduce a new Vietnamese multimodal dataset, named ViMACSA, which consists of 4,876 text-image pairs with 14,618 fine-grained annotations for both text and image in the hotel domain. Additionally, we propose a Fine-Grained Cross-Modal Fusion Framework (FCMF) that effectively learns both intra- and inter-modality interactions and then fuses these information to produce a unified multimodal representation. Experimental results show that our framework outperforms SOTA models on the ViMACSA dataset, achieving the highest F1 score of 79.73%. We also explore characteristics and challenges in Vietnamese multimodal sentiment analysis, including misspellings, abbreviations, and the complexities of the Vietnamese language. This work contributes both a benchmark dataset and a new framework that leverages fine-grained multimodal information to improve multimodal aspect-category sentiment analysis. Our dataset is available for research purposes: https://github.com/hoangquy18/Multimodal-Aspect-Category-Sentiment-Analysis.