mlsanityData to 7 Oct 2026

Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation

arXiv 2204.01171 ↗First posted 3 Apr 2022cs.CL cs.AI cs.LGcited by 106

Not on the map. Why?

Abstract (arXiv)

Current language generation models suffer from issues such as repetition, incoherence, and hallucinations. An often-repeated hypothesis is that this brittleness of generation models is caused by the training and the generation procedure mismatch, also referred to as exposure bias. In this paper, we verify this hypothesis by analyzing exposure bias from an imitation learning perspective. We show that exposure bias leads to an accumulation of errors, analyze why perplexity fails to capture this accumulation, and empirically show that this accumulation results in poor generation quality. Source code to reproduce these experiments is available at https://github.com/kushalarora/quantifying_exposure_bias

Key facts. Cited by 106 papers in the map (data through 7 Oct 2026). Builds on 32 of its 40 references that are in the map. First posted on arXiv 3 Apr 2022.

Builds on

32 of 40 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  5. 2016Pointer Sentinel Mixture Modelscited by 2,992
  6. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  7. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  8. 2018Hierarchical Neural Story Generationcited by 1,182
  9. 2019PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarizationcited by 1,100
  10. 2015A Diversity-Promoting Objective Function for Neural Conversation Modelscited by 1,068
  11. 2019DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generationcited by 842
  12. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
20 more
  1. 2020Recipes for building an open-domain chatbotcited by 593
  2. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  3. 2019Neural Text Generation with Unlikelihood Trainingcited by 489
  4. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 375
  5. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 299
  6. 2015Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Modelscited by 207
  7. 2016An Actor-Critic Algorithm for Sequence Predictioncited by 158
  8. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  9. 2019Bridging the Gap between Training and Inference for Neural Machine Translationcited by 123
  10. 2019Generalization in Generation: A closer look at Exposure Biascited by 123
  11. 2015Minimum Risk Training for Neural Machine Translationcited by 114
  12. 2017Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responsescited by 98
  13. 2020Residual Energy-Based Models for Text Generationcited by 95
  14. 2019Reinforcement Learning Based Graph-to-Sequence Model for Natural Question Generationcited by 57
  15. 2018Composable Planning with Attributescited by 21
  16. 2017SEARNN: Training RNNs with Global-Local Lossescited by 17
  17. 2019An Empirical Comparison on Imitation Learning and Reinforcement Learning for Paraphrase Generationcited by 11
  18. 2019Teacher-Student Training for Robust Tacotron-based TTScited by 11
  19. 2019Rethinking Exposure Bias In Language Modelingcited by 9
  20. 2020F^2-Softmax: Diversifying Neural Text Generation via Frequency Factorized Softmaxcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 6; 2023: 20; 2024: 21; 2025: 26; 2026: 33.

106 papers.

Sort:

By area:

  1. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  2. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  3. 2023How Language Model Hallucinations Can Snowballcited by 310
  4. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 244
  5. 2024BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languagescited by 171
  6. 2024DistiLLM: Towards Streamlined Distillation for Large Language Modelscited by 145
  7. 2024Stream of Search (SoS): Learning to Search in Languagecited by 141
  8. 2023MiniLLM: On-Policy Distillation of Large Language Modelscited by 129
  9. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 122
  10. 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 82
  11. 2025Beyond Next-Token: Next-X Prediction for Autoregressive Visual Generationcited by 71
  12. 2024Massively Multi-Cultural Knowledge Acquisition & LM Benchmarkingcited by 63
38 more
  1. 2026Stable On-Policy Distillation through Adaptive Target Reformulationcited by 47
  2. 2023Diffusion Language Models Can Perform Many Tasks with Scaling and Instruction-Finetuningcited by 39
  3. 2022Reward Gaming in Conditional Text Generationcited by 29
  4. 2023Tailoring Language Generation Models under Total Variation Distancecited by 27
  5. 2024CulturalBench: A Robust, Diverse, and Challenging Cultural Benchmark by Human-AI CulturalTeamingcited by 27
  6. 2025Frequency Autoregressive Image Generation with Continuous Tokenscited by 27
  7. 2024AdvPrefix: An Objective for Nuanced LLM Jailbreakscited by 25
  8. 2024PromptKD: Distilling Student-Friendly Knowledge for Generative Language Models via Prompt Tuningcited by 24
  9. 2026On the Position Bias of On-Policy Distillationcited by 23
  10. 2024PRISE: LLM-Style Sequence Compression for Learning Temporal Action Abstractions in Controlcited by 21
  11. 2024Imitating Language via Scalable Inverse Reinforcement Learningcited by 21
  12. 2024Self-Improving Robust Preference Optimizationcited by 19
  13. 2025Macro-from-Micro Planning for High-Quality and Parallelized Autoregressive Long Video Generationcited by 19
  14. 2025SelecTKD: Selective Token-Weighted Knowledge Distillation for LLMscited by 17
  15. 2023A Systematic Study of Knowledge Distillation for Natural Language Generation with Pseudo-Target Trainingcited by 16
  16. 2024HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Accelerationcited by 16
  17. 2023Understanding and Bridging the Modality Gap for Speech Translationcited by 15
  18. 2023Butterfly Effects of SGD Noise: Error Amplification in Behavior Cloning and Autoregressioncited by 14
  19. 2025Beyond Multi-Token Prediction: Pretraining LLMs with Future Summariescited by 14
  20. 2023SequenceMatch: Imitation Learning for Autoregressive Sequence Modelling with Backtrackingcited by 13
  21. 2023Towards Understanding and Improving Knowledge Distillation for Neural Machine Translationcited by 10
  22. 2025Few-shot LLM Synthetic Data with Distribution Matchingcited by 9
  23. 2023Directed Acyclic Transformer Pre-training for High-quality Non-autoregressive Text Generationcited by 8
  24. 2023Language Model Decoding as Direct Metrics Optimizationcited by 8
  25. 2024Seeing is Believing: Mitigating Hallucination in Large Vision-Language Models via CLIP-Guided Decodingcited by 8
  26. 2024The Mystery of the Pathological Path-star Task for Language Modelscited by 8
  27. 2024SWITCH: Studying with Teacher for Knowledge Distillation of Large Language Modelscited by 8
  28. 2025Jakiro: Boosting Speculative Decoding with Decoupled Multi-Head via MoEcited by 6
  29. 2025Skill-Targeted Adaptive Trainingcited by 6
  30. 2024SeRA: Self-Reviewing and Alignment of Large Language Models using Implicit Reward Marginscited by 5
  31. 2025Every Expert Matters: Towards Effective Knowledge Distillation for Mixture-of-Experts Language Modelscited by 5
  32. 2025Dimathtt[M]O: Distilling Masked Diffusion Models into One-step Generatorcited by 5
  33. 2025Supervised Optimism Correction: Be Confident When LLMs Are Surecited by 5
  34. 2025Looking beyond the next tokencited by 5
  35. 2025Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddingscited by 5
  36. 2025Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMscited by 5
  37. 2026Beyond Next-Token Alignment: Distilling Multimodal Large Language Models via Token Interactionscited by 5
  38. 2026Cross-Tokenizer On-Policy Distillation via Byte-Prefix Marginalizationcited by 5

50 most cited of 106 All 106 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Overthinking the Truth: Understanding how Language Models Process False Demonstrations
  2. 2026Time-Annealed Perturbation Sampling: Diverse Generation for Diffusion Language Models
  3. 2026Simple Diffusion Language Models Are More Effective Few-Step Generators Than Reported
  4. 2024Does Liking Yellow Imply Driving a School Bus? Semantic Leakage in Language Models
  5. 2026Why Instruction-Based Unlearning Fails in Diffusion Models?
  6. 2024Unveiling Imitation Learning: Exploring the Impact of Data Falsity to Large Language Model
  7. 2024Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models
  8. 2025DoubleGen: Debiased Generative Modeling of Counterfactuals