# Large Language Models as Zero-shot Dialogue State Tracker through Function Calling

arXiv: https://arxiv.org/abs/2402.10466 · HTML: https://mlsanity.com/paper/2402.10466/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 36 of its 46 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 30 of 36)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://mlsanity.com/paper/2302.04023/) (2023)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Baichuan 2: Open Large-scale Language Models](https://mlsanity.com/paper/2309.10305/) (2023)
- [API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs](https://mlsanity.com/paper/2304.08244/) (2023)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [Transferable Multi-Domain State Generator for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1905.08743/) (2019)
- [MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines](https://mlsanity.com/paper/2007.12720/) (2020)
- [Neural Belief Tracker: Data-Driven Dialogue State Tracking](https://mlsanity.com/paper/1606.03777/) (2016)
- [Taskmaster-1: Toward a Realistic and Diverse Dialog Dataset](https://mlsanity.com/paper/1909.05358/) (2019)
- [Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System](https://mlsanity.com/paper/2109.14739/) (2021)
- [TripPy: A Triple Copy Strategy for Value Independent Neural Dialog State Tracking](https://mlsanity.com/paper/2005.02877/) (2020)
- [MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2009.12005/) (2020)
- [Guiding Large Language Models via Directional Stimulus Prompting](https://mlsanity.com/paper/2302.11520/) (2023)
- [SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching](https://mlsanity.com/paper/2005.05298/) (2020)
- [Zero-Shot Transfer Learning with Synthesized Data for Multi-Domain Dialogue State Tracking](https://mlsanity.com/paper/2005.00891/) (2020)
- [SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Tracking](https://mlsanity.com/paper/1907.07421/) (2019)
- [Leveraging Slot Descriptions for Zero-Shot Cross-Domain Dialogue State Tracking](https://mlsanity.com/paper/2105.04222/) (2021)
- [Zero-Shot Dialogue State Tracking via Cross-Task Transfer](https://mlsanity.com/paper/2109.04655/) (2021)
- [Description-Driven Task-Oriented Dialog Modeling](https://mlsanity.com/paper/2201.08904/) (2022)
- [Microsoft Dialogue Challenge: Building End-to-End Task-Completion Dialogue Systems](https://mlsanity.com/paper/1807.11125/) (2018)
- [CoCo: Controllable Counterfactuals for Evaluating Dialogue State Trackers](https://mlsanity.com/paper/2010.12850/) (2020)

## Built on by (most cited first, 30 of 34)

- [Can a Single Model Master Both Multi-turn Conversations and Tool Use? CoALM: A Unified Conversational Agentic Language Model](https://mlsanity.com/paper/2502.08820/) (2025)
- [Autonomous Agents on Blockchains: Standards, Execution Models, and Trust Boundaries](https://mlsanity.com/paper/2601.04583/) (2026)
- [Pre3: Enabling Deterministic Pushdown Automata for Faster Structured LLM Generation](https://mlsanity.com/paper/2506.03887/) (2025)
- [TD-EVAL: Revisiting Task-Oriented Dialogue Evaluation by Combining Turn-Level Precision with Dialogue-Level Comparisons](https://mlsanity.com/paper/2504.19982/) (2025)
- [A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions](https://mlsanity.com/paper/2504.16939/) (2025)
- [Improving Small-Scale Large Language Models Function Calling for Reasoning Tasks](https://mlsanity.com/paper/2410.18890/) (2024)
- [Dialogue Benchmark Generation from Knowledge Graphs with Cost-Effective Retrieval-Augmented LLMs](https://mlsanity.com/paper/2501.09928/) (2025)
- [DiMA: An LLM-Powered Ride-Hailing Assistant at DiDi](https://mlsanity.com/paper/2503.04768/) (2025)
- [Unsupervised End-to-End Task-Oriented Dialogue with LLMs: The Power of the Noisy Channel](https://mlsanity.com/paper/2404.15219/) (2024)
- [You Have Thirteen Hours in Which to Solve the Labyrinth: Enhancing AI Game Masters with Function Calling](https://mlsanity.com/paper/2409.06949/) (2024)
- [Interpretable and Robust Dialogue State Tracking via Natural Language Summarization with LLMs](https://mlsanity.com/paper/2503.08857/) (2025)
- [Controllable and Reliable Knowledge-Intensive Task-Oriented Conversational Agents with Declarative Genie Worksheets](https://mlsanity.com/paper/2407.05674/) (2024)
- [All Learning Has an Emotional Basis, So Does Task-Oriented Dialogue](https://mlsanity.com/paper/2507.01594/) (2025)
- [Beyond Task-Oriented and Chitchat Dialogues: Proactive and Transition-Aware Conversational Agents](https://mlsanity.com/paper/2511.08835/) (2025)
- [Reasoning Gets Harder for LLMs Inside A Dialogue](https://mlsanity.com/paper/2603.20133/) (2026)
- [LLM should think and action as a human](https://mlsanity.com/paper/2502.13475/) (2025)
- [CoDial: Interpretable Task-Oriented Dialogue Systems Through Dialogue Flow Alignment](https://mlsanity.com/paper/2506.02264/) (2025)
- [PyTOD: Programmable Task-Oriented Dialogue with Execution Feedback](https://mlsanity.com/paper/2508.15456/) (2025)
- [NOEM3A: a Neuro-symbolic Ontology-Enhanced Method for Multi-intent understanding in Mobile Agents](https://mlsanity.com/paper/2511.19780/) (2025)
- [ReacTOD: Bounded Neuro-Symbolic Agentic NLU for Zero-Shot Dialogue State Tracking](https://mlsanity.com/paper/2605.19077/) (2026)
- [Multi-turn Conversational AI from Text to Multimodal Interaction: Data, Models, Evaluation, and Open Challenges](https://mlsanity.com/paper/2608.17605/) (2026)
- [Toward Robust Personalized Alignment for LLMs: Mitigating Persona Drift in Multi-Turn Dialogue](https://mlsanity.com/paper/2609.12373/) (2026)
- [Beyond Ontology in Dialogue State Tracking for Goal-Oriented Chatbot](https://mlsanity.com/paper/2410.22767/) (2024)
- [Know Your Mistakes: Towards Preventing Overreliance on Task-Oriented Conversational AI Through Accountability Modeling](https://mlsanity.com/paper/2501.10316/) (2025)
- [Auto Review: Second Stage Error Detection for Highly Accurate Information Extraction from Phone Conversations](https://mlsanity.com/paper/2506.05400/) (2025)
- [Spec-TOD: A Specialized Instruction-Tuned LLM Framework for Efficient Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2507.04841/) (2025)
- [HiCoLoRA: Addressing Context-Prompt Misalignment via Hierarchical Collaborative LoRA for Zero-Shot DST](https://mlsanity.com/paper/2509.19742/) (2025)
- [Prompt reinforcing for long-term planning of large language models](https://mlsanity.com/paper/2510.05921/) (2025)
- [Aligning Large Language Models with Procedural Rules: An Autoregressive State-Tracking Prompting for In-Game Trading](https://mlsanity.com/paper/2510.25014/) (2025)
- [MAC: A Multi-Agent Framework for Interactive User Clarification in Multi-turn Conversations](https://mlsanity.com/paper/2512.13154/) (2025)

## Abstract (arXiv)

Large language models (LLMs) are increasingly prevalent in conversational systems due to their advanced understanding and generative capabilities in general contexts. However, their effectiveness in task-oriented dialogues (TOD), which requires not only response generation but also effective dialogue state tracking (DST) within specific tasks and domains, remains less satisfying. In this work, we propose a novel approach FnCTOD for solving DST with LLMs through function calling. This method improves zero-shot DST, allowing adaptation to diverse domains without extensive data collection or model tuning. Our experimental results demonstrate that our approach achieves exceptional performance with both modestly sized open-source and also proprietary LLMs: with in-context prompting it enables various 7B or 13B parameter models to surpass the previous state-of-the-art (SOTA) achieved by ChatGPT, and improves ChatGPT's performance beating the SOTA by 5.6% average joint goal accuracy (JGA). Individual model results for GPT-3.5 and GPT-4 are boosted by 4.8% and 14%, respectively. We also show that by fine-tuning on a small collection of diverse task-oriented dialogues, we can equip modestly sized models, specifically a 13B parameter LLaMA2-Chat model, with function-calling capabilities and DST performance comparable to ChatGPT while maintaining their chat capabilities. We have made the code publicly available at https://github.com/facebookresearch/FnCTOD