mlsanityData to 7 Oct 2026

Tailoring Language Generation Models under Total Variation Distance

arXiv 2302.13344 ↗First posted 26 Feb 2023cs.CLcited by 27

Not on the map. Why?

Abstract (arXiv)

The standard paradigm of neural language generation adopts maximum likelihood estimation (MLE) as the optimizing method. From a distributional view, MLE in fact minimizes the Kullback-Leibler divergence (KLD) between the distribution of the real data and that of the model. However, this approach forces the model to distribute non-zero (sometimes large) probability mass to all training samples regardless of their quality. Moreover, in the attempt to cover the low-probability regions in the data distribution, the model systematically overestimates the probability of corrupted text sequences, which we conjecture is one of the main reasons for text degeneration during autoregressive decoding. To remedy this problem, we leverage the total variation distance (TVD) with its robustness to outliers, and develop practical bounds to apply it to language generation. Then, we introduce the TaiLr objective that balances the tradeoff of estimating TVD. Intuitively, TaiLr downweights real data samples that have low model probabilities with tunable penalization intensity. Experimental results show that our method alleviates the overestimation of degenerated sequences without sacrificing diversity and improves generation quality on a wide range of text generation tasks.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 39 of its 57 references that are in the map. First posted on arXiv 26 Feb 2023.

Builds on

39 of 57 references.

References · 39

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  5. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  6. 2018Hierarchical Neural Story Generationcited by 1,182
  7. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  8. 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 490
  9. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  10. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 484
  11. 2021MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontierscited by 466
  12. 2015A Neural Attention Model for Abstractive Sentence Summarizationcited by 394
27 more
  1. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 375
  2. 2019SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimizationcited by 253
  3. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  4. 2019Unifying Human and Statistical Evaluation for Natural Language Generationcited by 127
  5. 2022Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generationcited by 106
  6. 2019Reverse KL-Divergence Training of Prior Networks: Improved Uncertainty and Adversarial Robustnesscited by 104
  7. 2017Long Text Generation via Adversarial Training with Leaked Informationcited by 100
  8. 2018Language GANs Falling Shortcited by 97
  9. 2021Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Textcited by 89
  10. 2019On the Weaknesses of Reinforcement Learning for Neural Machine Translationcited by 82
  11. 2016Reward Augmented Maximum Likelihood for Neural Structured Predictioncited by 80
  12. 2015How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?cited by 76
  13. 2020Improved Natural Language Generation via Loss Truncationcited by 75
  14. 2018A Study of Reinforcement Learning for Neural Machine Translationcited by 74
  15. 2018Multi-Reward Reinforced Summarization with Saliency and Entailmentcited by 73
  16. 2019Bias Correction of Learned Generative Models using Likelihood-Free Importance Weightingcited by 70
  17. 2020Text Generation by Learning from Demonstrationscited by 64
  18. 2021Long Text Generation by Modeling Sentence-Level and Discourse-Level Coherencecited by 45
  19. 2021BERT, mBERT, or BiBERT? A Study on Contextualized Embeddings for Neural Machine Translationcited by 35
  20. 2019Exposure Bias versus Self-Recovery: Are Distortions Really Incremental for Autoregressive Text Generation?cited by 33
  21. 2020Reducing Quantity Hallucinations in Abstractive Summarizationcited by 22
  22. 2014On distinguishability criteria for estimating generative modelscited by 20
  23. 2022Evaluating Distributional Distortion in Neural Language Modelingcited by 18
  24. 2021DiscoDVT: Generating Long Text with Discourse-Aware Discrete Variational Transformercited by 14
  25. 2021Characterizing and addressing the issue of oversmoothing in neural autoregressive sequence modelingcited by 10
  26. 2020Robust Bayesian Inference for Discrete Outcomes with the Total Variation Distancecited by 6
  27. 2020Improving Maximum Likelihood Training for Text Generation with Density Ratio Estimationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 8; 2025: 6; 2026: 8.

27 papers.

Sort:

By area:

  1. 2024DistiLLM: Towards Streamlined Distillation for Large Language Modelscited by 145
  2. 2023MiniLLM: On-Policy Distillation of Large Language Modelscited by 129
  3. 2024Latent Space Chain-of-Embedding Enables Output-free LLM Self-Evaluationcited by 56
  4. 2024Towards Efficient Exact Optimization of Language Model Alignmentcited by 37
  5. 2025SimPER: A Minimalist Approach to Preference Alignment without Hyperparameterscited by 25
  6. 2026LK Losses: Direct Acceptance Rate Optimization for Speculative Decodingcited by 17
  7. 2023MixCE: Training Autoregressive Language Models by Mixing Forward and Reverse Cross-Entropiescited by 16
  8. 2024Sinkhorn Distance Minimization for Knowledge Distillationcited by 15
  9. 2025Sparse Logit Sampling: Accelerating Knowledge Distillation in LLMscited by 14
  10. 2023Language Model Decoding as Direct Metrics Optimizationcited by 8
  11. 2024Language Generation with Strictly Proper Scoring Rulescited by 8
  12. 2023EMO: Earth Mover Distance Optimization for Auto-Regressive Language Modelingcited by 7
15 more
  1. 2025Improving Diversity in Language Models: When Temperature Fails, Change the Losscited by 7
  2. 2023Error Norm Truncation: Robust Training in the Presence of Data Noise for Text Generation Modelscited by 6
  3. 2024Video DataFlywheel: Resolving the Impossible Data Trinity in Video-Language Understandingcited by 6
  4. 2024Predict the Next Word: Humans exhibit uncertainty in this task and language models _____cited by 4
  5. 2026Diversity in Large Language Models under Supervised Fine-Tuningcited by 4
  6. 2024Exploring and Enhancing the Transfer of Distribution in Knowledge Distillation for Autoregressive Language Modelscited by 3
  7. 2025Mitigation of Gender and Ethnicity Bias in AI-Generated Stories through Model Explanationscited by 1
  8. 2025ReviewScore: Misinformed Peer Review Detection with Large Language Modelscited by 1
  9. 2025Short-Context Dominance: How Much Local Context Natural Language Actually Needs?cited by 1
  10. 2026Equalized Generative Treatment: Matching f-divergences for Fairness in Generative Modelscited by 1
  11. 2026Perturbation is All You Need for Extrapolating Language Modelscited by 1
  12. 2026Learning Perturbations to Extrapolate Your LLMcited by 0
  13. 2026DEL: Digit Entropy Loss for Numerical Learning of Large Language Modelscited by 0
  14. 2026Guardrail-Agnostic Societal Bias Evaluation in Large Vision-Language Modelscited by 0
  15. 2026How Divergence Becomes Decision Flips in Compressed Language Modelscited by 0