You need to MIMIC to get FAME: Solving Meeting Transcript Scarcity with a Multi-Agent Conversations
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
Meeting summarization suffers from limited high-quality data, mainly due to privacy restrictions and expensive collection processes. We address this gap with FAME, a dataset of 500 meetings in English and 300 in German produced by MIMIC, our new multi-agent meeting synthesis framework that generates meeting transcripts on a given knowledge source by defining psychologically grounded participant profiles, outlining the conversation, and orchestrating a large language model (LLM) debate. A modular post-processing step refines these outputs, mitigating potential repetitiveness and overly formal tones, ensuring coherent, credible dialogues at scale. We also propose a psychologically grounded evaluation framework assessing naturalness, social behavior authenticity, and transcript difficulties. Human assessments show that FAME approximates real-meeting spontaneity (4.5/5 in naturalness), preserves speaker-centric challenges (3/5 in spoken language), and introduces richer information-oriented difficulty (4/5 in difficulty). These findings highlight that FAME is a good and scalable proxy for real-world meeting conditions. It enables new test scenarios for meeting summarization research and other conversation-centric applications in tasks requiring conversation data or simulating social scenarios under behavioral constraints.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 28 of its 49 references that are in the map. First posted on arXiv 18 Feb 2025.
Builds on
28 of 49 references.
Closest ancestors · 1
Same direction, newest first
Other references · 27
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
- 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
- 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,784
- 2018Hierarchical Neural Story Generationcited by 1,193
- 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
- 2018A Discourse-Aware Attention Model for Abstractive Summarization of Long Documentscited by 478
- 2021QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarizationcited by 365
- 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 331
- 2023MeetingBank: A Benchmark Dataset for Meeting Summarizationcited by 84
- 2023PLACES: Prompting Language Models for Social Conversation Synthesiscited by 72
15 more
- 2024Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMscited by 59
- 2023Building Real-World Meeting Summarization Systems using Large Language Models: A Practical Perspectivecited by 34
- 2020Ten Social Dimensions of Conversations and Relationshipscited by 18
- 2023Can Large Language Models emulate an inductive Thematic Analysis of semi-structured interviews? An exploration and provocation on the limits of the approach and the modelcited by 15
- 2024Improving Model Factuality with Fine-grained Critique-based Evaluatorcited by 15
- 2023Large Language Models on Wikipedia-Style Survey Generation: an Evaluation in NLP Conceptscited by 11
- 2024What's under the hood: Investigating Automatic Metrics on Meeting Summarizationcited by 9
- 2024CADS: A Systematic Literature Review on the Challenges of Abstractive Dialogue Summarizationcited by 9
- 2024Tell me what I need to know: Exploring LLM-based (Personalized) Abstractive Multi-Source Meeting Summarizationcited by 9
- 2024What's Wrong? Refining Meeting Summaries with LLM Feedbackcited by 8
- 2024Who Speaks Next? Multi-party AI Discussion Leveraging the Systematics of Turn-taking in Murder Mystery Gamescited by 7
- 2024Is my Meeting Summary Good? Estimating Quality with a Multi-LLM Evaluatorcited by 4
- 2022Leveraging Non-dialogue Summaries for Dialogue Summarizationcited by 3
- 2023FREDSum: A Dialogue Summarization Corpus for French Political Debatescited by 3
- 2024Prompting and Fine-Tuning of Small LLMs for Length-Controllable Telephone Call Summarizationcited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2023Improving Interpersonal Communication by Simulating Audiences with Language Models
- 2024CREAM: Comparison-Based Reference-Free ELO-Ranked Automatic Evaluation for Meeting Summarization
- 2026Speak for Me: Giving LLMs the Situational Awareness to Participate in a Meeting
- 2026Wiki-Talkie: Multilingual Benchmarking of Persona-Based Agents on Real-World Discussions
- 2025Re-FRAME the Meeting Summarization SCOPE: Fact-Based Summarization and Personalization via Questions
- 2026ToolRACER: A Robust Agentic Conversation Emulation Resource for Agent Training and Evaluation
- 2026GroupMemBench: Benchmarking LLM Agent Memory in Multi-Party Conversations
- 2024Cohesive Conversations: Enhancing Authenticity in Multi-Agent Simulated Dialogues