# Adapting Task-Oriented Dialogue Models for Email Conversations

arXiv: https://arxiv.org/abs/2208.09439 · HTML: https://mlsanity.com/paper/2208.09439/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 32 references that are in the map.
- First posted on arXiv 19 Aug 2022.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [A Simple Language Model for Task-Oriented Dialogue](https://mlsanity.com/paper/2005.00796/) (2020)
- [Extractive Summarization as Text Matching](https://mlsanity.com/paper/2004.08795/) (2020)
- [TOD-BERT: Pre-trained Natural Language Understanding for Task-Oriented Dialogue](https://mlsanity.com/paper/2004.06871/) (2020)
- [Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1609.01454/) (2016)
- [Fine-tune BERT for Extractive Summarization](https://mlsanity.com/paper/1903.10318/) (2019)
- [Hierarchical Transformers for Long Document Classification](https://mlsanity.com/paper/1910.10781/) (2019)
- [A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1907.00390/) (2019)
- [Intent Mining from past conversations for conversational agent](https://mlsanity.com/paper/2005.11014/) (2020)
- [Learning with Weak Supervision for Email Intent Detection](https://mlsanity.com/paper/2005.13084/) (2020)
- [Adapting to the Long Tail: A Meta-Analysis of Transfer Learning Research for Language Understanding Tasks](https://mlsanity.com/paper/2111.01340/) (2021)
- [Smart To-Do : Automatic Generation of To-Do Items from Emails](https://mlsanity.com/paper/2005.06282/) (2020)
- [NaRLE: Natural Language Models using Reinforcement Learning with Emotion Feedback](https://mlsanity.com/paper/2110.02148/) (2021)
- [Multi-Task Learning for Interpretable Weakly Labelled Sound Event Detection](https://mlsanity.com/paper/2008.07085/) (2020)
- [Improving weakly supervised sound event detection with self-supervised auxiliary tasks](https://mlsanity.com/paper/2106.06858/) (2021)
- [ScopeIt: Scoping Task Relevant Sentences in Documents](https://mlsanity.com/paper/2003.04988/) (2020)
- [Actionable Email Intent Modeling with Reparametrized RNNs](https://mlsanity.com/paper/1712.09185/) (2017)

## Built on by (most cited first, 1 of 1)

- [MAILEX: Email Event and Argument Extraction](https://mlsanity.com/paper/2305.13469/) (2023)

## Abstract (arXiv)

Intent detection is a key part of any Natural Language Understanding (NLU) system of a conversational assistant. Detecting the correct intent is essential yet difficult for email conversations where multiple directives and intents are present. In such settings, conversation context can become a key disambiguating factor for detecting the user's request from the assistant. One prominent way of incorporating context is modeling past conversation history like task-oriented dialogue models. However, the nature of email conversations (long form) restricts direct usage of the latest advances in task-oriented dialogue models. So in this paper, we provide an effective transfer learning framework (EMToD) that allows the latest development in dialogue models to be adapted for long-form conversations. We show that the proposed EMToD framework improves intent detection performance over pre-trained language models by 45% and over pre-trained dialogue models by 30% for task-oriented email conversations. Additionally, the modular nature of the proposed framework allows plug-and-play for any future developments in both pre-trained language and task-oriented dialogue models.