# An LLM Benchmark for Addressee Recognition in Multi-modal Multi-party Dialogue

arXiv: https://arxiv.org/abs/2501.16643 · HTML: https://mlsanity.com/paper/2501.16643/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 2 of its 26 references that are in the map.
- First posted on arXiv 28 Jan 2025.

## Builds on (most cited first, 2 of 2)

- [Rendering of Eyes for Eye-Shape Registration and Gaze Estimation](https://mlsanity.com/paper/1505.05916/) (2015)
- [Is ChatGPT a Good Multi-Party Conversation Solver?](https://mlsanity.com/paper/2310.16301/) (2023)

## Built on by (most cited first, 15 of 15)

- [Towards Online Multi-Modal Social Interaction Understanding](https://mlsanity.com/paper/2503.19851/) (2025)
- [DICE-BENCH: Evaluating the Tool-Use Capabilities of Large Language Models in Multi-Round, Multi-Party Dialogues](https://mlsanity.com/paper/2506.22853/) (2025)
- [Speak or Stay Silent: Context-Aware Turn-Taking in Multi-Party Dialogue](https://mlsanity.com/paper/2603.11409/) (2026)
- [Multimodal Conversation Structure Understanding](https://mlsanity.com/paper/2505.17536/) (2025)
- [Omni-MMSI: Toward Identity-attributed Social Interaction Understanding](https://mlsanity.com/paper/2604.00267/) (2026)
- [MP-Bench: Evaluating Voice Agents as a Multiparty Conversation Participant](https://mlsanity.com/paper/2609.13076/) (2026)
- [MultiTalk: Scaling Full-Duplex Speech Models to Long, Multi-Party, Bilingual Conversation](https://mlsanity.com/paper/2609.36903/) (2026)
- [Evaluating Large Language Models Abilities for Addressee, Turn-change, and Next Speaker Prediction in Meetings](https://mlsanity.com/paper/2606.17542/) (2026)
- [On the Structure of Address in Multi-Party Dialogue: From Discrete Labels to Continuous Levels](https://mlsanity.com/paper/2607.15648/) (2026)
- [Unified Agent: Managing Interactions across Devices](https://mlsanity.com/paper/2608.05729/) (2026)
- [From Speech to Interaction: Analyzing Multimodal Systems in Cocktail-Party Scenarios](https://mlsanity.com/paper/2608.08510/) (2026)
- [Leveraging Turn-taking Dynamics for Intent Recognition in Multi-party Conversations](https://mlsanity.com/paper/2608.28926/) (2026)
- [TEIDAN: A Multilingual Multiparty Dialogue Corpus](https://mlsanity.com/paper/2609.00802/) (2026)
- [Live Assistant: Learning Whether, When, and Whom to Assist in Real-World Live Social Streams](https://mlsanity.com/paper/2609.27303/) (2026)
- [Beyond Dyadic Memory: Interaction-Aware Multimodal Memory with Adaptive Agentic Retrieval for Multi-Party Spoken Conversations](https://mlsanity.com/paper/2609.32522/) (2026)

## Abstract (arXiv)

Handling multi-party dialogues represents a significant step for advancing spoken dialogue systems, necessitating the development of tasks specific to multi-party interactions. To address this challenge, we are constructing a multi-modal multi-party dialogue corpus of triadic (three-participant) discussions. This paper focuses on the task of addressee recognition, identifying who is being addressed to take the next turn, a critical component unique to multi-party dialogue systems. A subset of the corpus was annotated with addressee information, revealing that explicit addressees are indicated in approximately 20% of conversational turns. To evaluate the task's complexity, we benchmarked the performance of a large language model (GPT-4o) on addressee recognition. The results showed that GPT-4o achieved an accuracy only marginally above chance, underscoring the challenges of addressee recognition in multi-party dialogue. These findings highlight the need for further research to enhance the capabilities of large language models in understanding and navigating the intricacies of multi-party conversational dynamics.