# A Novel Energy based Model Mechanism for Multi-modal Aspect-Based Sentiment Analysis

arXiv: https://arxiv.org/abs/2312.08084 · HTML: https://mlsanity.com/paper/2312.08084/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 15 references that are in the map.
- First posted on arXiv 13 Dec 2023.

## Builds on (most cited first, 5 of 5)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Unified Generative Framework for Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2106.04300/) (2021)
- [Position-Aware Tagging for Aspect Sentiment Triplet Extraction](https://mlsanity.com/paper/2010.02609/) (2020)
- [Exploiting BERT For Multimodal Target Sentiment Classification Through Input Space Translation](https://mlsanity.com/paper/2108.01682/) (2021)
- [FSUIE: A Novel Fuzzy Span Mechanism for Universal Information Extraction](https://mlsanity.com/paper/2306.14913/) (2023)

## Built on by (most cited first, 3 of 3)

- [Large Language Models Meet Text-Centric Multimodal Sentiment Analysis: A Survey](https://mlsanity.com/paper/2406.08068/) (2024)
- [An Empirical Study on Configuring In-Context Learning Demonstrations for Unleashing MLLMs' Sentimental Perception Capability](https://mlsanity.com/paper/2505.16193/) (2025)
- [AdaptiSent: Context-Aware Adaptive Attention for Multimodal Aspect-Based Sentiment Analysis](https://mlsanity.com/paper/2507.12695/) (2025)

## Abstract (arXiv)

Multi-modal aspect-based sentiment analysis (MABSA) has recently attracted increasing attention. The span-based extraction methods, such as FSUIE, demonstrate strong performance in sentiment analysis due to their joint modeling of input sequences and target labels. However, previous methods still have certain limitations: (i) They ignore the difference in the focus of visual information between different analysis targets (aspect or sentiment). (ii) Combining features from uni-modal encoders directly may not be sufficient to eliminate the modal gap and can cause difficulties in capturing the image-text pairwise relevance. (iii) Existing span-based methods for MABSA ignore the pairwise relevance of target span boundaries. To tackle these limitations, we propose a novel framework called DQPSA for multi-modal sentiment analysis. Specifically, our model contains a Prompt as Dual Query (PDQ) module that uses the prompt as both a visual query and a language query to extract prompt-aware visual information and strengthen the pairwise relevance between visual information and the analysis target. Additionally, we introduce an Energy-based Pairwise Expert (EPE) module that models the boundaries pairing of the analysis target from the perspective of an Energy-based Model. This expert predicts aspect or sentiment span based on pairwise stability. Experiments on three widely used benchmarks demonstrate that DQPSA outperforms previous approaches and achieves a new state-of-the-art performance.