mlsanityData to 14 Oct 2026

Multi-Party Conversational Agents: A Survey

arXiv 2505.18845 ↗First posted 24 May 2025cs.CLcited by 7

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Multi-party Conversational Agents (MPCAs) are systems designed to engage in dialogue with more than two participants simultaneously. Unlike traditional two-party agents, designing MPCAs faces additional challenges due to the need to interpret both utterance semantics and social dynamics. This survey explores recent progress in MPCAs by addressing three key questions: 1) Can agents model each participants' mental states? (State of Mind Modeling); 2) Can they properly understand the dialogue content? (Semantic Understanding); and 3) Can they reason about and predict future conversation flow? (Agent Action Modeling). We review methods ranging from classical machine learning to Large Language Models (LLMs) and multi-modal systems. Our analysis underscores Theory of Mind (ToM) as essential for building intelligent MPCAs and highlights multi-modal understanding as a promising yet underexplored direction. Finally, this survey offers guidance to future researchers on developing more capable MPCAs.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 91 of its 190 references that are in the map. First posted on arXiv 24 May 2025.

Builds on

91 of 190 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Beyond Turn-Based Interfaces: Synchronous LLMs as Full-Duplex Dialogue Agents23 Sep
  2. 2024MuMA-ToM: Multi-modal Multi-Agent Theory of Mind22 Aug
  3. 2023Multi-Party Chat: Conversational Agents in Group Settings with Humans and Models26 Apr

Other references · 88

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  3. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
  4. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 652
  5. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 643
  6. 2019SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarizationcited by 535
  7. 2018Wizard of Wikipedia: Knowledge-Powered Conversational agentscited by 530
  8. 2021QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarizationcited by 365
  9. 2023Large Language Models Fail on Trivial Alterations to Theory-of-Mind Taskscited by 292
  10. 2023PersonaLLM: Investigating the Ability of Large Language Models to Express Personality Traitscited by 234
  11. 2023Evaluating Large Language Models in Theory of Mind Taskscited by 222
  12. 2018DialogueRNN: An Attentive RNN for Emotion Detection in Conversationscited by 218
38 more
  1. 2015The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systemscited by 172
  2. 2017Emotion Detection on TV Show Transcripts with Sequence-based Convolutional Neural Networkscited by 102
  3. 2023Minding Language Models' (Lack of) Theory of Mind: A Plug-and-Play Multi-Character Belief Trackercited by 95
  4. 2018EmotionLines: An Emotion Corpus of Multi-Party Conversationscited by 80
  5. 2020DialogXL: All-in-One XLNet for Multi-Party Conversation Emotion Recognitioncited by 79
  6. 2020Speaker-Aware BERT for Multi-Turn Response Selection in Retrieval-Based Chatbotscited by 73
  7. 2020Multi-View Sequence-to-Sequence Models with Conversational Structure for Abstractive Dialogue Summarizationcited by 72
  8. 2024TofuEval: Evaluating Hallucinations of LLMs on Topic-Focused Dialogue Summarizationcited by 66
  9. 2021Summ^N: A Multi-Stage Summarization Framework for Long Input Dialogues and Documentscited by 65
  10. 2020Molweni: A Challenge Multiparty Dialogues-based Machine Reading Comprehension Dataset with Discourse Structurecited by 60
  11. 2022MMDialog: A Large-scale Multi-turn Dialogue Dataset Towards Multi-modal Open-domain Conversationcited by 56
  12. 2023Is ChatGPT Equipped with Emotional Dialogue Capabilities?cited by 54
  13. 2024BIG5-CHAT: Shaping LLM Personalities Through Training on Human-Grounded Datacited by 50
  14. 2022M2FNet: Multi-modal Fusion Network for Emotion Recognition in Conversationcited by 47
  15. 2021Language Model as an Annotator: Exploring DialoGPT for Dialogue Summarizationcited by 46
  16. 2018A Deep Sequential Model for Discourse Parsing on Multi-Party Dialoguescited by 38
  17. 2021PLATO-XL: Exploring the Large-scale Pre-training of Dialogue Generationcited by 37
  18. 2021MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understandingcited by 36
  19. 2021Contrast and Generation Make BART a Good Dialogue Emotion Recognizercited by 35
  20. 2019Automatic Text-based Personality Recognition on Monologues and Multiparty Dialogues Using Attentive Networks and Contextual Embeddingscited by 31
  21. 2020Co-GAT: A Co-Interactive Graph Attention Network for Joint Dialog Act Recognition and Sentiment Classificationcited by 29
  22. 2020DCR-Net: A Deep Co-Interactive Relation Network for Joint Dialog Act Recognition and Sentiment Classificationcited by 28
  23. 2021An Exploratory Study on Long Dialogue Summarization: What Works and What's Nextcited by 28
  24. 2018Augmenting Neural Response Generation with Context-Aware Topical Attentioncited by 27
  25. 2024Beyond Silent Letters: Amplifying LLMs in Emotion Recognition with Vocal Nuancescited by 27
  26. 2019Dialogue Act Classification with Context-Aware Self-Attentioncited by 26
  27. 2020Response Selection for Multi-Party Conversations with Dynamic Topic Trackingcited by 26
  28. 2024MIntRec2.0: A Large-scale Benchmark Dataset for Multimodal Intent Recognition and Out-of-scope Detection in Conversationscited by 26
  29. 2020Multi-turn Response Selection using Dialogue Dependency Relationscited by 23
  30. 2023Target-oriented Proactive Dialogue Systems with Personalization: Problem Formulation and Dataset Curationcited by 23
  31. 2017Addressee and Response Selection in Multi-Party Conversations with Speaker Interaction RNNscited by 21
  32. 2024TimeToM: Temporal Space is the Key to Unlocking the Door of Large Language Models' Theory-of-Mindcited by 21
  33. 2022UX Research on Conversational Human-AI Interaction: A Literature Review of the ACM Digital Librarycited by 20
  34. 2024Turn-taking and Backchannel Prediction with Acoustic and Large Language Model Fusioncited by 20
  35. 2021Speaker Turn Modeling for Dialogue Act Classificationcited by 18
  36. 2020Open Domain Dialogue Generation with Latent Imagescited by 17
  37. 2019ALOHA: Artificial Learning of Human Attributes for Dialogue Agentscited by 16
  38. 2022HeterMPC: A Heterogeneous Graph Neural Network for Response Generation in Multi-Party Conversationscited by 16

50 most cited of 88 All 88 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026Evaluating Long-Horizon Memory for Multi-Party Collaborative Dialoguescited by 22
  2. 2026Adaptive Turn-Taking for Real-time Multi-Party Voice Agentscited by 2
  3. 2026ProEvent: An Event-centric Benchmark for Proactive Agentscited by 2
  4. 2026Designing Explainable Conversational Agentic Systems for Guaran'i Speakerscited by 0
  5. 2026Leveraging Turn-taking Dynamics for Intent Recognition in Multi-party Conversationscited by 0
  6. 2026Speak for Me: Giving LLMs the Situational Awareness to Participate in a Meetingcited by 0
  7. 2026EGCForest: Structured Agent Memory Organization for Enterprise Group Chatscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions
  2. 2025Infusing Theory of Mind into Socially Intelligent LLM Agents
  3. 2024Proactive Conversational Agents with Inner Thoughts
  4. 2025MetaMind: Modeling Human Social Thoughts with Metacognitive Multi-Agent Systems
  5. 2025Multi-Agent Language Models: Advancing Cooperation, Coordination, and Adaptation
  6. 2025Large Language Models as Theory of Mind Aware Generative Agents with Counterfactual Reflection
  7. 2023Dialogue Agents 101: A Beginner's Guide to Critical Ingredients for Designing Effective Conversational Systems
  8. 2026Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action