# Autoregressive Entity Generation for End-to-End Task-Oriented Dialog

arXiv: https://arxiv.org/abs/2209.08708 · HTML: https://mlsanity.com/paper/2209.08708/

## Key facts

- Cited by 8 papers in the map (data through 7 Oct 2026).
- Builds on 24 of its 28 references that are in the map.
- First posted on arXiv 19 Sep 2022.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [Autoregressive Entity Retrieval](https://mlsanity.com/paper/2010.00904/) (2020)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [Key-Value Retrieval Networks for Task-Oriented Dialogue](https://mlsanity.com/paper/1705.05414/) (2017)
- [UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2](https://mlsanity.com/paper/2012.03539/) (2020)
- [SOLOIST: Building Task Bots at Scale with Transfer Learning and Machine Teaching](https://mlsanity.com/paper/2005.05298/) (2020)
- [Mem2Seq: Effectively Incorporating Knowledge Bases into End-to-End Task-Oriented Dialog Systems](https://mlsanity.com/paper/1804.08217/) (2018)
- [Global-to-local Memory Pointer Networks for Task-Oriented Dialogue](https://mlsanity.com/paper/1901.04713/) (2019)
- [Structured Fusion Networks for Dialog](https://mlsanity.com/paper/1907.10016/) (2019)
- [Dynamic Fusion Network for Multi-Domain End-to-end Task-Oriented Dialog](https://mlsanity.com/paper/2004.11019/) (2020)
- [Learning Knowledge Bases with Parameters for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2009.13656/) (2020)
- [Entity-Consistent End-to-end Task-Oriented Dialogue System with KB Retriever](https://mlsanity.com/paper/1909.06762/) (2019)
- [Don't be Contradicted with Anything! CI-ToD: Towards Benchmarking Consistency for Task-oriented Dialogue System](https://mlsanity.com/paper/2109.11292/) (2021)
- [Disentangling Language and Knowledge in Task-Oriented Dialogs](https://mlsanity.com/paper/1805.01216/) (2018)
- [Sequence-to-Sequence Learning for Task-oriented Dialogue with Dialogue State Representation](https://mlsanity.com/paper/1806.04441/) (2018)
- [Multi-level Memory for Task Oriented Dialogs](https://mlsanity.com/paper/1810.10647/) (2018)
- [DualTKB: A Dual Learning Bridge between Text and Knowledge Base](https://mlsanity.com/paper/2010.14660/) (2020)
- [Retrieve & Memorize: Dialog Policy Learning with Multi-Action Memory](https://mlsanity.com/paper/2106.02317/) (2021)
- [HyKnow: End-to-End Task-Oriented Dialog Modeling with Hybrid Knowledge Management](https://mlsanity.com/paper/2105.06041/) (2021)

## Built on by (most cited first, 8 of 8)

- [End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions](https://mlsanity.com/paper/2311.09008/) (2023)
- [Multi-Grained Knowledge Retrieval for End-to-End Task-Oriented Dialog](https://mlsanity.com/paper/2305.10149/) (2023)
- [Optimal Transport Guided Correlation Assignment for Multimodal Entity Linking](https://mlsanity.com/paper/2406.01934/) (2024)
- [Synergizing In-context Learning with Hints for End-to-end Task-oriented Dialog Systems](https://mlsanity.com/paper/2405.15585/) (2024)
- [EAVE: Efficient Product Attribute Value Extraction via Lightweight Sparse-layer Interaction](https://mlsanity.com/paper/2406.06839/) (2024)
- [Retrieval-Generation Alignment for End-to-End Task-Oriented Dialogue System](https://mlsanity.com/paper/2310.08877/) (2023)
- [Dual-Feedback Knowledge Retrieval for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2310.14528/) (2023)
- [Exploring Large Language Models for Product Attribute Value Identification](https://mlsanity.com/paper/2409.12695/) (2024)

## Abstract (arXiv)

Task-oriented dialog (TOD) systems often require interaction with an external knowledge base to retrieve necessary entity (e.g., restaurant) information to support the response generation. Most current end-to-end TOD systems either retrieve the KB information explicitly or embed it into model parameters for implicit access.~While the former approach demands scanning the KB at each turn of response generation, which is inefficient when the KB scales up, the latter approach shows higher flexibility and efficiency. In either approach, the systems may generate a response with conflicting entity information. To address this issue, we propose to generate the entity autoregressively first and leverage it to guide the response generation in an end-to-end system. To ensure entity consistency, we impose a trie constraint on entity generation. We also introduce a logit concatenation strategy to facilitate gradient backpropagation for end-to-end training. Experiments on MultiWOZ 2.1 single and CAMREST show that our system can generate more high-quality and entity-consistent responses.