# LLM-Driven Multimodal Opinion Expression Identification

arXiv: https://arxiv.org/abs/2406.18088 · HTML: https://mlsanity.com/paper/2406.18088/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 25 references that are in the map.
- First posted on arXiv 26 Jun 2024.

## Builds on (most cited first, 13 of 13)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Sentiment Analysis in the Era of Large Language Models: A Reality Check](https://mlsanity.com/paper/2305.15005/) (2023)
- [BLSP: Bootstrapping Language-Speech Pre-training via Behavior Alignment of Continuation Writing](https://mlsanity.com/paper/2309.00916/) (2023)
- [Emotional Intelligence of Large Language Models](https://mlsanity.com/paper/2307.09042/) (2023)
- [Whispering LLaMA: A Cross-Modal Generative Error Correction Framework for Speech Recognition](https://mlsanity.com/paper/2310.06434/) (2023)
- [Sequence-to-Sequence Speech Recognition with Time-Depth Separable Convolutions](https://mlsanity.com/paper/1904.02619/) (2019)
- [Sentiment Analysis through LLM Negotiations](https://mlsanity.com/paper/2311.01876/) (2023)
- [Identifying Chinese Opinion Expressions with Extremely-Noisy Crowdsourcing Annotations](https://mlsanity.com/paper/2204.10714/) (2022)
- [Leveraging Label Information for Multimodal Emotion Recognition](https://mlsanity.com/paper/2309.02106/) (2023)
- [Exploring Sentiment Analysis Techniques in Natural Language Processing: A Comprehensive Review](https://mlsanity.com/paper/2305.14842/) (2023)

## Built on by (most cited first, 3 of 3)

- [Affective Computing in the Era of Large Language Models: A Survey from the NLP Perspective](https://mlsanity.com/paper/2408.04638/) (2024)
- [LLM-Mixer: Multiscale Mixing in LLMs for Time Series Forecasting](https://mlsanity.com/paper/2410.11674/) (2024)
- [Grammar Induction from Visual, Speech and Text](https://mlsanity.com/paper/2410.03739/) (2024)

## Abstract (arXiv)

Opinion Expression Identification (OEI) is essential in NLP for applications ranging from voice assistants to depression diagnosis. This study extends OEI to encompass multimodal inputs, underlining the significance of auditory cues in delivering emotional subtleties beyond the capabilities of text. We introduce a novel multimodal OEI (MOEI) task, integrating text and speech to mirror real-world scenarios. Utilizing CMU MOSEI and IEMOCAP datasets, we construct the CI-MOEI dataset. Additionally, Text-to-Speech (TTS) technology is applied to the MPQA dataset to obtain the CIM-OEI dataset. We design a template for the OEI task to take full advantage of the generative power of large language models (LLMs). Advancing further, we propose an LLM-driven method STOEI, which combines speech and text modal to identify opinion expressions. Our experiments demonstrate that MOEI significantly improves the performance while our method outperforms existing methods by 9.20% and obtains SOTA results.