# Multi-party Goal Tracking with LLMs: Comparing Pre-training, Fine-tuning, and Prompt Engineering arXiv: https://arxiv.org/abs/2308.15231 ยท HTML: https://mlsanity.com/paper/2308.15231/ ## Key facts - Cited by 14 papers in the map (data through 14 Oct 2026). - Builds on 24 of its 52 references that are in the map. - First posted on arXiv 29 Aug 2023. ## Builds on (most cited first, 24 of 24) - [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018) - [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020) - [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022) - [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019) - [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022) - [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021) - [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020) - [Calibrate Before Use: Improving Few-Shot Performance of Language Models](https://mlsanity.com/paper/2102.09690/) (2021) - [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://mlsanity.com/paper/2306.08302/) (2023) - [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023) - [Complexity-Based Prompting for Multi-Step Reasoning](https://mlsanity.com/paper/2210.00720/) (2022) - [UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training](https://mlsanity.com/paper/2002.12804/) (2020) - [Dialogue State Tracking with a Language Model using Schema-Driven Prompting](https://mlsanity.com/paper/2109.07506/) (2021) - [DialogLM: Pre-trained Model for Long Dialogue Understanding and Summarization](https://mlsanity.com/paper/2109.02492/) (2021) - [Opening up ChatGPT: Tracking openness, transparency, and accountability in instruction-tuned text generators](https://mlsanity.com/paper/2307.05532/) (2023) - [MPC-BERT: A Pre-Trained Language Model for Multi-Party Conversation Understanding](https://mlsanity.com/paper/2106.01541/) (2021) - [Investigating the Effectiveness of Task-Agnostic Prefix Prompt for Instruction Following](https://mlsanity.com/paper/2302.14691/) (2023) - [GSN: A Graph-Structured Network for Multi-Party Dialogues](https://mlsanity.com/paper/1905.13637/) (2019) - [Dynamic Schema Graph Fusion Network for Multi-Domain Dialogue State Tracking](https://mlsanity.com/paper/2204.06677/) (2022) - [Label Semantic Aware Pre-training for Few-shot Text Classification](https://mlsanity.com/paper/2204.07128/) (2022) - [HeterMPC: A Heterogeneous Graph Neural Network for Response Generation in Multi-Party Conversations](https://mlsanity.com/paper/2203.08500/) (2022) - [Crowd-sourcing NLG Data: Pictures Elicit Better Data](https://mlsanity.com/paper/1608.00339/) (2016) - [A Comparative Study on Language Models for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2201.08687/) (2022) - [MuMMER: Socially Intelligent Human-Robot Interaction in Public Spaces](https://mlsanity.com/paper/1909.06749/) (2019) ## Built on by (most cited first, 14 of 14) - [Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!](https://mlsanity.com/paper/2310.03693/) (2023) - [Large Language Models Meet NLP: A Survey](https://mlsanity.com/paper/2405.12819/) (2024) - [Generative Students: Using LLM-Simulated Student Profiles to Support Question Item Evaluation](https://mlsanity.com/paper/2405.11591/) (2024) - [Multi-Party Conversational Agents: A Survey](https://mlsanity.com/paper/2505.18845/) (2025) - [Automatically Generating Numerous Context-Driven SFT Data for LLMs across Diverse Granularity](https://mlsanity.com/paper/2405.16579/) (2024) - [The Future of Intelligent Healthcare: A Systematic Analysis and Discussion on the Integration and Impact of Robots Using Large Language Models for Healthcare](https://mlsanity.com/paper/2411.03287/) (2024) - [Intent Recognition and Out-of-Scope Detection using LLMs in Multi-party Conversations](https://mlsanity.com/paper/2507.22289/) (2025) - [Detecting Agreement in Multi-party Conversational AI](https://mlsanity.com/paper/2311.03026/) (2023) - [Socially Pertinent Robots in Gerontological Healthcare](https://mlsanity.com/paper/2404.07560/) (2024) - [LLMs and their Limited Theory of Mind: Evaluating Mental State Annotations in Situated Dialogue](https://mlsanity.com/paper/2509.02292/) (2025) - [How DDAIR you? Disambiguated Data Augmentation for Intent Recognition](https://mlsanity.com/paper/2601.11234/) (2026) - [Alignment-Aware Model Adaptation via Feedback-Guided Optimization](https://mlsanity.com/paper/2602.02258/) (2026) - [Leveraging Turn-taking Dynamics for Intent Recognition in Multi-party Conversations](https://mlsanity.com/paper/2608.28926/) (2026) - [EGCForest: Structured Agent Memory Organization for Enterprise Group Chats](https://mlsanity.com/paper/2610.19836/) (2026) ## Abstract (arXiv) This paper evaluates the extent to which current Large Language Models (LLMs) can capture task-oriented multi-party conversations (MPCs). We have recorded and transcribed 29 MPCs between patients, their companions, and a social robot in a hospital. We then annotated this corpus for multi-party goal-tracking and intent-slot recognition. People share goals, answer each other's goals, and provide other people's goals in MPCs - none of which occur in dyadic interactions. To understand user goals in MPCs, we compared three methods in zero-shot and few-shot settings: we fine-tuned T5, created pre-training tasks to train DialogLM using LED, and employed prompt engineering techniques with GPT-3.5-turbo, to determine which approach can complete this novel task with limited data. GPT-3.5-turbo significantly outperformed the others in a few-shot setting. The `reasoning' style prompt, when given 7% of the corpus as example annotated conversations, was the best performing method. It correctly annotated 62.32% of the goal tracking MPCs, and 69.57% of the intent-slot recognition MPCs. A `story' style prompt increased model hallucination, which could be detrimental if deployed in safety-critical settings. We conclude that multi-party conversations still challenge state-of-the-art LLMs.