# EM Pre-training for Multi-party Dialogue Response Generation

arXiv: https://arxiv.org/abs/2305.12412 · HTML: https://mlsanity.com/paper/2305.12412/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 29 references that are in the map.
- First posted on arXiv 21 May 2023.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation](https://mlsanity.com/paper/1911.00536/) (2019)
- [The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems](https://mlsanity.com/paper/1506.08909/) (2015)
- [PLATO: Pre-trained Dialogue Generation Model with Discrete Latent Variable](https://mlsanity.com/paper/1910.07931/) (2019)
- [A Discrete Hard EM Approach for Weakly Supervised Question Answering](https://mlsanity.com/paper/1909.04849/) (2019)
- [Molweni: A Challenge Multiparty Dialogues-based Machine Reading Comprehension Dataset with Discourse Structure](https://mlsanity.com/paper/2004.05080/) (2020)
- [A Deep Sequential Model for Discourse Parsing on Multi-Party Dialogues](https://mlsanity.com/paper/1812.00176/) (2018)
- [DialogVED: A Pre-trained Latent Variable Encoder-Decoder Model for Dialog Response Generation](https://mlsanity.com/paper/2204.13031/) (2022)
- [GSN: A Graph-Structured Network for Multi-Party Dialogues](https://mlsanity.com/paper/1905.13637/) (2019)
- [Response Selection for Multi-Party Conversations with Dynamic Topic Tracking](https://mlsanity.com/paper/2010.07785/) (2020)
- [Multi-turn Response Selection using Dialogue Dependency Relations](https://mlsanity.com/paper/2010.01502/) (2020)
- [Addressee and Response Selection in Multi-Party Conversations with Speaker Interaction RNNs](https://mlsanity.com/paper/1709.04005/) (2017)
- [HeterMPC: A Heterogeneous Graph Neural Network for Response Generation in Multi-Party Conversations](https://mlsanity.com/paper/2203.08500/) (2022)
- [Self- and Pseudo-self-supervised Prediction of Speaker and Key-utterance for Multi-party Dialogue Reading Comprehension](https://mlsanity.com/paper/2109.03772/) (2021)
- [Structural Characterization for Dialogue Disentanglement](https://mlsanity.com/paper/2110.08018/) (2021)
- [Back to the Future: Bidirectional Information Decoupling Network for Multi-turn Dialogue Modeling](https://mlsanity.com/paper/2204.08152/) (2022)
- [Semantic-Preserving Adversarial Code Comprehension](https://mlsanity.com/paper/2209.05130/) (2022)

## Built on by (most cited first, 7 of 7)

- [Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges](https://mlsanity.com/paper/2405.15604/) (2024)
- [Is ChatGPT a Good Multi-Party Conversation Solver?](https://mlsanity.com/paper/2310.16301/) (2023)
- [UAlign: Pushing the Limit of Template-free Retrosynthesis Prediction with Unsupervised SMILES Alignment](https://mlsanity.com/paper/2404.00044/) (2024)
- [Speak or Stay Silent: Context-Aware Turn-Taking in Multi-Party Dialogue](https://mlsanity.com/paper/2603.11409/) (2026)
- [Pre-training Multi-party Dialogue Models with Latent Discourse Inference](https://mlsanity.com/paper/2305.15175/) (2023)
- [Improving the Robustness of Distantly-Supervised Named Entity Recognition via Uncertainty-Aware Teacher Learning and Student-Student Collaborative Learning](https://mlsanity.com/paper/2311.08010/) (2023)
- [SS-MPC: A Sequence-Structured Multi-Party Conversation System](https://mlsanity.com/paper/2502.16920/) (2025)

## Abstract (arXiv)

Dialogue response generation requires an agent to generate a response according to the current dialogue history, in terms of which two-party dialogues have been well studied, but leaving a great gap for multi-party dialogues at the same time. Different from two-party dialogues where each response is a direct reply to its previous utterance, the addressee of a response utterance should be specified before it is generated in the multi-party scenario. Thanks to the huge amount of two-party conversational data, various pre-trained language models for two-party dialogue response generation have been proposed. However, due to the lack of annotated addressee labels in multi-party dialogue datasets, it is hard to use them to pre-train a response generation model for multi-party dialogues. To tackle this obstacle, we propose an Expectation-Maximization (EM) approach that iteratively performs the expectation steps to generate addressee labels, and the maximization steps to optimize a response generation model. Theoretical analyses and extensive experiments have justified the feasibility and effectiveness of our proposed method.