# Improving Multimodal Sentiment Analysis: Supervised Angular Margin-based Contrastive Learning for Enhanced Fusion Representation

arXiv: https://arxiv.org/abs/2312.02227 · HTML: https://mlsanity.com/paper/2312.02227/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 39 references that are in the map.
- First posted on arXiv 4 Dec 2023.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [SphereFace: Deep Hypersphere Embedding for Face Recognition](https://mlsanity.com/paper/1704.08063/) (2017)
- [Multimodal Transformer for Unaligned Multimodal Language Sequences](https://mlsanity.com/paper/1906.00295/) (2019)
- [Deep Metric Learning via Lifted Structured Feature Embedding](https://mlsanity.com/paper/1511.06452/) (2015)
- [MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2005.03545/) (2020)
- [SMIL: Multimodal Learning with Severely Missing Modality](https://mlsanity.com/paper/2103.05677/) (2021)
- [MOSI: Multimodal Corpus of Sentiment Intensity and Subjectivity Analysis in Online Opinion Videos](https://mlsanity.com/paper/1606.06259/) (2016)
- [Learning Modality-Specific Representations with Self-Supervised Multi-Task Learning for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2102.04830/) (2021)
- [Improving Multimodal Fusion with Hierarchical Mutual Information Maximization for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2109.00412/) (2021)
- [Learning Relationships between Text, Audio, and Video via Deep Canonical Correlation for Multimodal Language Analysis](https://mlsanity.com/paper/1911.05544/) (2019)
- [Contrastive Learning for Neural Topic Model](https://mlsanity.com/paper/2110.12764/) (2021)
- [Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning](https://mlsanity.com/paper/2211.12878/) (2022)
- [Tag-assisted Multimodal Sentiment Analysis under Uncertain Missing Modalities](https://mlsanity.com/paper/2204.13707/) (2022)
- [Learning to Attend via Word-Aspect Associative Fusion for Aspect-based Sentiment Analysis](https://mlsanity.com/paper/1712.05403/) (2017)
- [Contrastive Fine-tuning Improves Robustness for Neural Rankers](https://mlsanity.com/paper/2105.12932/) (2021)
- [Holistic Multi-modal Memory Network for Movie Question Answering](https://mlsanity.com/paper/1811.04595/) (2018)
- [Gradient-Boosted Decision Tree for Listwise Context Model in Multimodal Review Helpfulness Prediction](https://mlsanity.com/paper/2305.12678/) (2023)

## Built on by (most cited first, 12 of 12)

- [Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectives](https://mlsanity.com/paper/2406.05615/) (2024)
- [KDMCSE: Knowledge Distillation Multimodal Sentence Embeddings with Adaptive Angular margin Contrastive Learning](https://mlsanity.com/paper/2403.17486/) (2024)
- [Topic Modeling as Multi-Objective Contrastive Optimization](https://mlsanity.com/paper/2402.07577/) (2024)
- [Motion-aware Contrastive Learning for Temporal Panoptic Scene Graph Generation](https://mlsanity.com/paper/2412.07160/) (2024)
- [Encoding and Controlling Global Semantics for Long-form Video Question Answering](https://mlsanity.com/paper/2405.19723/) (2024)
- [MAMA: Meta-optimized Angular Margin Contrastive Framework for Video-Language Representation Learning](https://mlsanity.com/paper/2407.03788/) (2024)
- [TI-JEPA: An Innovative Energy-based Joint Embedding Strategy for Text-Image Multimodal Systems](https://mlsanity.com/paper/2503.06380/) (2025)
- [Multi-Scale Contrastive Learning for Video Temporal Grounding](https://mlsanity.com/paper/2412.07157/) (2024)
- [Enhancing Multimodal Entity Linking with Jaccard Distance-based Conditional Contrastive Learning and Contextual Visual Augmentation](https://mlsanity.com/paper/2501.14166/) (2025)
- [DALR: Dual-level Alignment Learning for Multimodal Sentence Representation Learning](https://mlsanity.com/paper/2506.21096/) (2025)
- [Topic-aware Causal Intervention for Counterfactual Detection](https://mlsanity.com/paper/2409.16668/) (2024)
- [More Bias, Less Bias: BiasPrompting for Enhanced Multiple-Choice Question Answering](https://mlsanity.com/paper/2511.20086/) (2025)

## Abstract (arXiv)

The effectiveness of a model is heavily reliant on the quality of the fusion representation of multiple modalities in multimodal sentiment analysis. Moreover, each modality is extracted from raw input and integrated with the rest to construct a multimodal representation. Although previous methods have proposed multimodal representations and achieved promising results, most of them focus on forming positive and negative pairs, neglecting the variation in sentiment scores within the same class. Additionally, they fail to capture the significance of unimodal representations in the fusion vector. To address these limitations, we introduce a framework called Supervised Angular-based Contrastive Learning for Multimodal Sentiment Analysis. This framework aims to enhance discrimination and generalizability of the multimodal representation and overcome biases in the fusion vector's modality. Our experimental results, along with visualizations on two widely used datasets, demonstrate the effectiveness of our approach.