mlsanityData to 7 Oct 2026

Diversifying Dialog Generation via Adaptive Label Smoothing

arXiv 2105.14556 ↗First posted 30 May 2021cs.CL cs.AIcited by 25

Not on the map. Why?

Abstract (arXiv)

Neural dialogue generation models trained with the one-hot target distribution suffer from the over-confidence issue, which leads to poor generation diversity as widely reported in the literature. Although existing approaches such as label smoothing can alleviate this issue, they fail to adapt to diverse dialog contexts. In this paper, we propose an Adaptive Label Smoothing (AdaLabel) approach that can adaptively estimate a target label distribution at each time step for different contexts. The maximum probability in the predicted distribution is used to modify the soft target distribution produced by a novel light-weight bi-directional decoder module. The resulting target distribution is aware of both previous and future contexts and is adjusted to avoid over-training the dialogue model. Our model can be trained in an end-to-end manner. Extensive experiments on two benchmark datasets show that our approach outperforms various competitive baselines in producing diverse responses.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 36 of its 49 references that are in the map. First posted on arXiv 30 May 2021.

Builds on

36 of 49 references.

References · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  6. 2017On Calibration of Modern Neural Networkscited by 4,860
  7. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  8. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  9. 2019When Does Label Smoothing Help?cited by 985
  10. 2016Sequence-Level Knowledge Distillationcited by 761
  11. 2017DailyDialog: A Manually Labelled Multi-turn Dialogue Datasetcited by 649
  12. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
24 more
  1. 2017Regularizing Neural Networks by Penalizing Confident Output Distributionscited by 461
  2. 2020Calibrating Deep Neural Networks using Focal Losscited by 353
  3. 2017OpenNMT: Open-source Toolkit for Neural Machine Translationcited by 251
  4. 2017Learning Discourse-level Diversity for Neural Dialog Models using Conditional Variational Autoencoderscited by 222
  5. 2019Challenges in Building Intelligent Open-domain Dialog Systemscited by 147
  6. 2018Generating Informative and Diverse Conversational Responses via Adversarial Information Maximizationcited by 114
  7. 2019Don't Say That! Making Inconsistent Dialogue Unlikely with Unlikelihood Trainingcited by 110
  8. 2020Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translationcited by 104
  9. 2019Multilingual Neural Machine Translation with Knowledge Distillationcited by 95
  10. 2020Understanding and Improving Knowledge Distillationcited by 83
  11. 2019Comparison of Diverse Decoding Methods from Conditional Language Modelscited by 78
  12. 2020A Large-Scale Chinese Short-Text Conversation Datasetcited by 77
  13. 2019A Pre-training Based Personalized Dialogue Generation Model with Persona-sparse Datacited by 74
  14. 2019Calibration of Encoder Decoder Models for Neural Machine Translationcited by 67
  15. 2018Importance of Search and Evaluation Strategies in Neural Dialogue Modelingcited by 44
  16. 2020Data Manipulation: Towards Effective Instance Learning for Neural Dialogue Generation via Learning to Augment and Reweightcited by 34
  17. 2019Improving Neural Response Diversity with Frequency-Aware Cross-Entropy Losscited by 29
  18. 2019How Decoding Strategies Affect the Verifiability of Generated Textcited by 27
  19. 2019Negative Training for Neural Dialogue Response Generationcited by 24
  20. 2019Jointly Optimizing Diversity and Relevance in Neural Response Generationcited by 22
  21. 2020Dialogue Distillation: Open-Domain Dialogue Augmentation Using Unpaired Datacited by 20
  22. 2020Stylized Dialogue Response Generation Using Stylized Unpaired Textscited by 16
  23. 2018Why are Sequence-to-Sequence Models So Dull? Understanding the Low-Diversity Problem of Chatbotscited by 12
  24. 2020F^2-Softmax: Diversifying Neural Text Generation via Frequency Factorized Softmaxcited by 8

Built on by

Cited by year

Citing papers per year: 2021: 2; 2022: 12; 2023: 6; 2024: 3; 2025: 2.

25 papers.

Sort:

  1. 2022Duplex Conversation: Towards Human-like Interaction in Spoken Dialogue Systemscited by 28
  2. 2022Rethinking and Refining the Distinct Metriccited by 27
  3. 2022Measuring and Improving Semantic Diversity of Dialogue Generationcited by 16
  4. 2022An Equal-Size Hard EM Algorithm for Diverse Dialogue Generationcited by 11
  5. 2022Teacher Forcing Recovers Reward Functions for Text Generationcited by 11
  6. 2022Estimating Soft Labels for Out-of-Domain Intent Detectioncited by 11
  7. 2023SimOAP: Improve Coherence and Consistency in Persona-based Dialogue Generation via Over-sampling and Post-evaluationcited by 11
  8. 2022An Empirical Study on the Overlapping Problem of Open-Domain Dialogue Datasetscited by 8
  9. 2022Prompt Conditioned VAE: Enhancing Generative Replay for Lifelong Learning in Task-Oriented Dialoguecited by 8
  10. 2022Diversifying Neural Dialogue Generation via Negative Distillationcited by 6
  11. 2023EZInterviewer: To Improve Job Interview Performance with Mock Interview Generatorcited by 6
  12. 2023Diversifying Question Generation over Knowledge Base via External Natural Questionscited by 6
13 more
  1. 2021Taming Repetition in Dialogue Generationcited by 5
  2. 2024Targeted Augmentation for Low-Resource Event Extractioncited by 4
  3. 2024For Better or For Worse? Learning Minimum Variance Features With Label Augmentationcited by 3
  4. 2021Similarity Based Label Smoothing For Dialogue Generationcited by 2
  5. 2022Adversarial Ensemble Training by Jointly Learning Label Dependencies and Member Modelscited by 2
  6. 2022CORAL: Contextual Response Retrievability Loss Function for Training Dialog Generation Modelscited by 1
  7. 2023Partially Randomizing Transformer Weights for Dialogue Response Diversitycited by 1
  8. 2022Accuracy on In-Domain Samples Matters When Building Out-of-Domain detectors: A Reply to Marek et al. (2021)cited by 0
  9. 2023Hi Model, generating 'nice' instead of 'good' is not as bad as generating 'rice'! Towards Context and Semantic Infused Dialogue Generation Loss Function and Evaluation Metriccited by 0
  10. 2023Learning to Diversify Neural Text Generation via Degenerative Modelcited by 0
  11. 2024Multi-party Response Generation with Relation Disentanglementcited by 0
  12. 2025Bridge the Gap between Past and Future: Siamese Model Optimization for Context-Aware Document Rankingcited by 0
  13. 2025Label Smoothing is a Pragmatic Information Bottleneckcited by 0