# Expanding the Role of Affective Phenomena in Multimodal Interaction Research

arXiv: https://arxiv.org/abs/2305.10827 · HTML: https://mlsanity.com/paper/2305.10827/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 86 references that are in the map.
- First posted on arXiv 18 May 2023.

## Builds on (most cited first, 12 of 12)

- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [A Systematic Review on Affective Computing: Emotion Models, Databases, and Recent Advances](https://mlsanity.com/paper/2203.06935/) (2022)
- [FakeFlow: Fake News Detection by Modeling the Flow of Affective Information](https://mlsanity.com/paper/2101.09810/) (2021)
- [Classifying Idiomatic and Literal Expressions Using Topic Models and Intensity of Emotions](https://mlsanity.com/paper/1802.09961/) (2018)
- [Joint Modelling of Emotion and Abusive Language Detection](https://mlsanity.com/paper/2005.14028/) (2020)
- [Introducing Representations of Facial Affect in Automated Multimodal Deception Detection](https://mlsanity.com/paper/2008.13369/) (2020)
- [Towards Emotion-Aware Agents For Negotiation Dialogues](https://mlsanity.com/paper/2107.13165/) (2021)
- [Us vs. Them: A Dataset of Populist Attitudes, News Bias and Emotions](https://mlsanity.com/paper/2101.11956/) (2021)
- [Mitigating Toxic Degeneration with Empathetic Data: Exploring the Relationship Between Toxicity and Empathy](https://mlsanity.com/paper/2205.07233/) (2022)
- [Supervised Contrastive Learning for Affect Modelling](https://mlsanity.com/paper/2208.12238/) (2022)
- [Play with Emotion: Affect-Driven Reinforcement Learning](https://mlsanity.com/paper/2208.12622/) (2022)
- [MUSER: MUltimodal Stress Detection using Emotion Recognition as an Auxiliary Task](https://mlsanity.com/paper/2105.08146/) (2021)

## Built on by (most cited first, 1 of 1)

- [Advancing Social Intelligence in AI Agents: Technical Challenges and Open Questions](https://mlsanity.com/paper/2404.11023/) (2024)

## Abstract (arXiv)

In recent decades, the field of affective computing has made substantial progress in advancing the ability of AI systems to recognize and express affective phenomena, such as affect and emotions, during human-human and human-machine interactions. This paper describes our examination of research at the intersection of multimodal interaction and affective computing, with the objective of observing trends and identifying understudied areas. We examined over 16,000 papers from selected conferences in multimodal interaction, affective computing, and natural language processing: ACM International Conference on Multimodal Interaction, AAAC International Conference on Affective Computing and Intelligent Interaction, Annual Meeting of the Association for Computational Linguistics, and Conference on Empirical Methods in Natural Language Processing. We identified 910 affect-related papers and present our analysis of the role of affective phenomena in these papers. We find that this body of research has primarily focused on enabling machines to recognize and express affect and emotion. However, we find limited research on how affect and emotion predictions might be used by AI systems to enhance machine understanding of human social behaviors and cognitive states. Based on our analysis, we discuss directions to expand the role of affective phenomena in multimodal interaction research.