# Multilogue-Net: A Context Aware RNN for Multi-modal Emotion Detection and Sentiment Analysis in Conversation

arXiv: https://arxiv.org/abs/2002.08267 · HTML: https://mlsanity.com/paper/2002.08267/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2020.

## Built on by (most cited first, 27 of 27)

- [Multimodal Information Bottleneck: Learning Minimal Sufficient Unimodal and Multimodal Representations](https://mlsanity.com/paper/2210.17444/) (2022)
- [Learning in Audio-visual Context: A Review, Analysis, and New Perspective](https://mlsanity.com/paper/2208.09579/) (2022)
- [COGMEN: COntextualized GNN based Multimodal Emotion recognitioN](https://mlsanity.com/paper/2205.02455/) (2022)
- [Multimodal Multi-loss Fusion Network for Sentiment Analysis](https://mlsanity.com/paper/2308.00264/) (2023)
- [TVLT: Textless Vision-Language Transformer](https://mlsanity.com/paper/2209.14156/) (2022)
- [A Review of Text Style Transfer using Deep Learning](https://mlsanity.com/paper/2109.15144/) (2021)
- [Conversation Understanding using Relational Temporal Graph Neural Networks with Auxiliary Cross-Modality Interaction](https://mlsanity.com/paper/2311.04507/) (2023)
- [Joyful: Joint Modality Fusion and Graph Contrastive Learning for Multimodal Emotion Recognition](https://mlsanity.com/paper/2311.11009/) (2023)
- [Shapes of Emotions: Multimodal Emotion Recognition in Conversations via Emotion Shifts](https://mlsanity.com/paper/2112.01938/) (2021)
- [Trustworthy Multimodal Fusion for Sentiment Analysis in Ordinal Sentiment Space](https://mlsanity.com/paper/2404.08923/) (2024)
- [Multimodal Emotion Recognition with Modality-Pairwise Unsupervised Contrastive Loss](https://mlsanity.com/paper/2207.11482/) (2022)
- [MMLatch: Bottom-up Top-down Fusion for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2201.09828/) (2022)
- [Which is Making the Contribution: Modulating Unimodal and Cross-modal Dynamics for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2111.08451/) (2021)
- [Multimodal Emotion Recognition using Audio-Video Transformer Fusion with Cross Attention](https://mlsanity.com/paper/2407.18552/) (2024)
- [Multi-Task Multi-Modal Self-Supervised Learning for Facial Expression Recognition](https://mlsanity.com/paper/2404.10904/) (2024)
- [Target conversation extraction: Source separation using turn-taking dynamics](https://mlsanity.com/paper/2407.11277/) (2024)
- [End-to-end Semantic-centric Video-based Multimodal Affective Computing](https://mlsanity.com/paper/2408.07694/) (2024)
- [Multimodal Vision Transformers with Forced Attention for Behavior Analysis](https://mlsanity.com/paper/2212.03968/) (2022)
- [Adapted Multimodal BERT with Layer-wise Fusion for Sentiment Analysis](https://mlsanity.com/paper/2212.00678/) (2022)
- [Curriculum Learning Meets Weakly Supervised Modality Correlation Learning](https://mlsanity.com/paper/2212.07619/) (2022)
- [Multi-Modality Collaborative Learning for Sentiment Analysis](https://mlsanity.com/paper/2501.12424/) (2025)
- [Proactive Hearing Assistants that Isolate Egocentric Conversations](https://mlsanity.com/paper/2511.11473/) (2025)
- [Multi-scale Cooperative Multimodal Transformers for Multimodal Sentiment Analysis in Videos](https://mlsanity.com/paper/2206.07981/) (2022)
- [Sentiment Analysis in Digital Spaces: An Overview of Reviews](https://mlsanity.com/paper/2310.19687/) (2023)
- [CM3T: Framework for Efficient Multimodal Learning for Inhomogeneous Interaction Datasets](https://mlsanity.com/paper/2501.03332/) (2025)
- [Multimodal Mixture of Low-Rank Experts for Sentiment Analysis and Emotion Recognition](https://mlsanity.com/paper/2505.14143/) (2025)
- [Centering Emotion Hotspots: Multimodal Local-Global Fusion and Cross-Modal Alignment for Emotion Recognition in Conversations](https://mlsanity.com/paper/2510.08606/) (2025)

## Abstract (arXiv)

Sentiment Analysis and Emotion Detection in conversation is key in several real-world applications, with an increase in modalities available aiding a better understanding of the underlying emotions. Multi-modal Emotion Detection and Sentiment Analysis can be particularly useful, as applications will be able to use specific subsets of available modalities, as per the available data. Current systems dealing with Multi-modal functionality fail to leverage and capture - the context of the conversation through all modalities, the dependency between the listener(s) and speaker emotional states, and the relevance and relationship between the available modalities. In this paper, we propose an end to end RNN architecture that attempts to take into account all the mentioned drawbacks. Our proposed model, at the time of writing, out-performs the state of the art on a benchmark dataset on a variety of accuracy and regression metrics.