# An Enhanced Dual Transformer Contrastive Network for Multimodal Sentiment Analysis

arXiv: https://arxiv.org/abs/2510.23617 · HTML: https://mlsanity.com/paper/2510.23617/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 20 Oct 2025.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [CLMLF:A Contrastive Learning and Multi-Layer Fusion Method for Multimodal Sentiment Detection](https://mlsanity.com/paper/2204.05515/) (2022)
- [General Debiasing for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2307.10511/) (2023)
- [Data Warehouse and Decision Support on Integrated Crop Big Data](https://mlsanity.com/paper/2003.04470/) (2020)
- [Designing and Implementing Data Warehouse for Agricultural Big Data](https://mlsanity.com/paper/1905.12411/) (2019)
- [Structural Textile Pattern Recognition and Processing Based on Hypergraphs](https://mlsanity.com/paper/2103.11271/) (2021)
- [A Fair and Comprehensive Comparison of Multimodal Tweet Sentiment Analysis Methods](https://mlsanity.com/paper/2106.08829/) (2021)
- [A Similarity Measure for Weaving Patterns in Textiles](https://mlsanity.com/paper/1810.04604/) (2018)
- [Holistic Visual-Textual Sentiment Analysis with Prior Models](https://mlsanity.com/paper/2211.12981/) (2022)
- [Semantic Search by Latent Ontological Features](https://mlsanity.com/paper/1807.05576/) (2018)
- [A Graph Based Raman Spectral Processing Technique for Exosome Classification](https://mlsanity.com/paper/2504.15324/) (2025)

## Abstract (arXiv)

Multimodal Sentiment Analysis (MSA) seeks to understand human emotions by jointly analyzing data from multiple modalities typically text and images offering a richer and more accurate interpretation than unimodal approaches. In this paper, we first propose BERT-ViT-EF, a novel model that combines powerful Transformer-based encoders BERT for textual input and ViT for visual input through an early fusion strategy. This approach facilitates deeper cross-modal interactions and more effective joint representation learning. To further enhance the model's capability, we propose an extension called the Dual Transformer Contrastive Network (DTCN), which builds upon BERT-ViT-EF. DTCN incorporates an additional Transformer encoder layer after BERT to refine textual context (before fusion) and employs contrastive learning to align text and image representations, fostering robust multimodal feature learning. Empirical results on two widely used MSA benchmarks MVSA-Single and TumEmo demonstrate the effectiveness of our approach. DTCN achieves best accuracy (78.4%) and F1-score (78.3%) on TumEmo, and delivers competitive performance on MVSA-Single, with 76.6% accuracy and 75.9% F1-score. These improvements highlight the benefits of early fusion and deeper contextual modeling in Transformer-based multimodal sentiment analysis.