# Unsupervised End-to-End Task-Oriented Dialogue with LLMs: The Power of the Noisy Channel

arXiv: https://arxiv.org/abs/2404.15219 · HTML: https://mlsanity.com/paper/2404.15219/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 31 references that are in the map.
- First posted on arXiv 23 Apr 2024.

## Builds on (most cited first, 22 of 22)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset](https://mlsanity.com/paper/1909.05855/) (2019)
- [MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines](https://mlsanity.com/paper/2007.12720/) (2020)
- [Noisy Channel Language Model Prompting for Few-Shot Text Classification](https://mlsanity.com/paper/2108.04106/) (2021)
- [Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System](https://mlsanity.com/paper/2109.14739/) (2021)
- [Task Contamination: Language Models May Not Be Few-Shot Anymore](https://mlsanity.com/paper/2312.16337/) (2023)
- [Are LLMs All You Need for Task-Oriented Dialogue?](https://mlsanity.com/paper/2304.06556/) (2023)
- [Shades of BLEU, Flavours of Success: The Case of MultiWOZ](https://mlsanity.com/paper/2106.05555/) (2021)
- [A Probabilistic End-To-End Task-Oriented Dialog Model with Latent Belief States towards Semi-Supervised Learning](https://mlsanity.com/paper/2009.08115/) (2020)
- [SGP-TOD: Building Task Bots Effortlessly via Schema-Guided LLM Prompting](https://mlsanity.com/paper/2305.09067/) (2023)
- [ChatGPT for Zero-shot Dialogue State Tracking: A Solution or an Opportunity?](https://mlsanity.com/paper/2306.01386/) (2023)
- [Large Language Models as Zero-shot Dialogue State Tracker through Function Calling](https://mlsanity.com/paper/2402.10466/) (2024)
- [Pretraining the Noisy Channel Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2103.10518/) (2021)
- [Explicit State Tracking with Semi-Supervision for Neural Dialogue Generation](https://mlsanity.com/paper/1808.10596/) (2018)
- [Variational Latent-State GPT for Semi-Supervised Task-Oriented Dialog Systems](https://mlsanity.com/paper/2109.04314/) (2021)
- [Towards Universal Dialogue Act Tagging for Task-Oriented Dialogues](https://mlsanity.com/paper/1907.03020/) (2019)
- [Learning Interpretable Latent Dialogue Actions With Less Supervision](https://mlsanity.com/paper/2209.11128/) (2022)

## Built on by (most cited first, 5 of 5)

- [Can a Single Model Master Both Multi-turn Conversations and Tool Use? CoALM: A Unified Conversational Agentic Language Model](https://mlsanity.com/paper/2502.08820/) (2025)
- [A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions](https://mlsanity.com/paper/2504.16939/) (2025)
- [RECAP: REwriting Conversations for Intent Understanding in Agentic Planning](https://mlsanity.com/paper/2509.04472/) (2025)
- [MATA: Multi-Agent Framework for Reliable and Flexible Table Question Answering](https://mlsanity.com/paper/2602.09642/) (2026)
- [Multi-Intent Spoken Language Understanding: Methods, Trends, and Challenges](https://mlsanity.com/paper/2512.11258/) (2025)

## Abstract (arXiv)

Training task-oriented dialogue systems typically requires turn-level annotations for interacting with their APIs: e.g. a dialogue state and the system actions taken at each step. These annotations can be costly to produce, error-prone, and require both domain and annotation expertise. With advances in LLMs, we hypothesize that unlabeled data and a schema definition are sufficient for building a working task-oriented dialogue system, completely unsupervised. We consider a novel unsupervised setting of only (1) a well-defined API schema (2) a set of unlabeled dialogues between a user and agent. We propose an innovative approach using expectation-maximization (EM) that infers turn-level annotations as latent variables using a noisy channel model to build an end-to-end dialogue agent. Evaluating our approach on the MultiWOZ benchmark, our method more than doubles the dialogue success rate of a strong GPT-3.5 baseline.