# BORT: Back and Denoising Reconstruction for End-to-End Task-Oriented Dialog

arXiv: https://arxiv.org/abs/2205.02471 · HTML: https://mlsanity.com/paper/2205.02471/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 41 references that are in the map.
- First posted on arXiv 5 May 2022.

## Builds on (most cited first, 26 of 26)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [MultiWOZ -- A Large-Scale Multi-Domain Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling](https://mlsanity.com/paper/1810.00278/) (2018)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [A Network-based End-to-End Trainable Task-oriented Dialogue System](https://mlsanity.com/paper/1604.04562/) (2016)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [Unsupervised Neural Machine Translation](https://mlsanity.com/paper/1710.11041/) (2017)
- [Phrase-Based & Neural Unsupervised Machine Translation](https://mlsanity.com/paper/1804.07755/) (2018)
- [UBAR: Towards Fully End-to-End Task-Oriented Dialog Systems with GPT-2](https://mlsanity.com/paper/2012.03539/) (2020)
- [Hello, It's GPT-2 -- How Can I Help You? Towards the Use of Pretrained Language Models for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/1907.05774/) (2019)
- [MinTL: Minimalist Transfer Learning for Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2009.12005/) (2020)
- [Task-Oriented Dialog Systems that Consider Multiple Appropriate Responses under the Same Context](https://mlsanity.com/paper/1911.10484/) (2019)
- [SUMBT: Slot-Utterance Matching for Universal and Scalable Belief Tracking](https://mlsanity.com/paper/1907.07421/) (2019)
- [Rethinking Action Spaces for Reinforcement Learning in End-to-end Dialog Agents with Latent Variable Models](https://mlsanity.com/paper/1902.08858/) (2019)
- [Shades of BLEU, Flavours of Success: The Case of MultiWOZ](https://mlsanity.com/paper/2106.05555/) (2021)
- [A Probabilistic End-To-End Task-Oriented Dialog Model with Latent Belief States towards Semi-Supervised Learning](https://mlsanity.com/paper/2009.08115/) (2020)
- [Semantically Conditioned Dialog Response Generation via Hierarchical Disentangled Self-Attention](https://mlsanity.com/paper/1905.12866/) (2019)
- [LAVA: Latent Action Spaces via Variational Auto-encoding for Dialogue Policy Optimization](https://mlsanity.com/paper/2011.09378/) (2020)
- [Neural Machine Translation with Reconstruction](https://mlsanity.com/paper/1611.01874/) (2016)
- [Modelling Hierarchical Structure between Dialogue Policy and Natural Language Generator with Option Framework for Task-oriented Dialogue System](https://mlsanity.com/paper/2006.06814/) (2020)
- [Alternating Recurrent Dialog Model with Large-scale Pre-trained Language Models](https://mlsanity.com/paper/1910.03756/) (2019)
- [Domain State Tracking for a Simplified Dialogue System](https://mlsanity.com/paper/2103.06648/) (2021)
- [Translating Pro-Drop Languages with Reconstruction Models](https://mlsanity.com/paper/1801.03257/) (2018)

## Built on by (most cited first, 20 of 20)

- [Task-Optimized Adapters for an End-to-End Task-Oriented Dialogue System](https://mlsanity.com/paper/2305.02468/) (2023)
- [ByGPT5: End-to-End Style-conditioned Poetry Generation with Token-free Language Models](https://mlsanity.com/paper/2212.10474/) (2022)
- [End-to-end Task-oriented Dialogue: A Survey of Tasks, Methods, and Future Directions](https://mlsanity.com/paper/2311.09008/) (2023)
- [Mars: Modeling Context & State Representations with Contrastive Learning for End-to-End Task-Oriented Dialog](https://mlsanity.com/paper/2210.08917/) (2022)
- [Q-TOD: A Query-driven Task-oriented Dialogue System](https://mlsanity.com/paper/2210.07564/) (2022)
- [KRLS: Improving End-to-End Response Generation in Task Oriented Dialog with Reinforced Keywords Learning](https://mlsanity.com/paper/2211.16773/) (2022)
- [DiactTOD: Learning Generalizable Latent Dialogue Acts for Controllable Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2308.00878/) (2023)
- [InstructTODS: Large Language Models for End-to-End Task-Oriented Dialogue Systems](https://mlsanity.com/paper/2310.08885/) (2023)
- [DFM: Dialogue Foundation Model for Universal Large-Scale Dialogue-Oriented Task Learning](https://mlsanity.com/paper/2205.12662/) (2022)
- [MoNET: Tackle State Momentum via Noise-Enhanced Training for Dialogue State Tracking](https://mlsanity.com/paper/2211.05503/) (2022)
- [Zero-Shot Generalizable End-to-End Task-Oriented Dialog System using Context Summarization and Domain Schema](https://mlsanity.com/paper/2303.16252/) (2023)
- [UBARv2: Towards Mitigating Exposure Bias in Task-Oriented Dialogs](https://mlsanity.com/paper/2209.07239/) (2022)
- [CSS: Combining Self-training and Self-supervised Learning for Few-shot Dialogue State Tracking](https://mlsanity.com/paper/2210.05146/) (2022)
- [Multijugate Dual Learning for Low-Resource Task-Oriented Dialogue System](https://mlsanity.com/paper/2305.16106/) (2023)
- [TA&AT: Enhancing Task-Oriented Dialog with Turn-Level Auxiliary Tasks and Action-Tree Based Scheduled Sampling](https://mlsanity.com/paper/2401.15626/) (2024)
- [Curricular Transfer Learning for Sentence Encoded Tasks](https://mlsanity.com/paper/2308.01849/) (2023)
- [Comparing Data Augmentation Methods for End-to-End Task-Oriented Dialog Systems](https://mlsanity.com/paper/2406.06127/) (2024)
- [Training Zero-Shot Generalizable End-to-End Task-Oriented Dialog System Without Turn-level Dialog Annotations](https://mlsanity.com/paper/2407.15055/) (2024)
- [Collaborative and Proactive Management of Task-Oriented Conversations](https://mlsanity.com/paper/2510.05110/) (2025)
- [Automated evaluation of LLMs for effective machine translation of Mandarin Chinese to English](https://mlsanity.com/paper/2603.09998/) (2026)

## Abstract (arXiv)

A typical end-to-end task-oriented dialog system transfers context into dialog state, and upon which generates a response, which usually faces the problem of error propagation from both previously generated inaccurate dialog states and responses, especially in low-resource scenarios. To alleviate these issues, we propose BORT, a back and denoising reconstruction approach for end-to-end task-oriented dialog system. Squarely, to improve the accuracy of dialog states, back reconstruction is used to reconstruct the original input context from the generated dialog states since inaccurate dialog states cannot recover the corresponding input context. To enhance the denoising capability of the model to reduce the impact of error propagation, denoising reconstruction is used to reconstruct the corrupted dialog state and response. Extensive experiments conducted on MultiWOZ 2.0 and CamRest676 show the effectiveness of BORT. Furthermore, BORT demonstrates its advanced capabilities in the zero-shot domain and low-resource scenarios.