mlsanityData to 14 Oct 2026

Generalization in Generation: A closer look at Exposure Bias

arXiv 1910.00292 ↗First posted 1 Oct 2019cs.LG cs.CL stat.MLcited by 123

Not on the map. Why?

Cited by 42 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Exposure bias refers to the train-test discrepancy that seemingly arises when an autoregressive generative model uses only ground-truth contexts at training time but generated ones at test time. We separate the contributions of the model and the learning framework to clarify the debate on consequences and review proposed counter-measures. In this light, we argue that generalization is the underlying property to address and propose unconditional generation as its fundamental benchmark. Finally, we combine latent variable modeling with a recent formulation of exploration in reinforcement learning to obtain a rigorous handling of true and generated contexts. Results on language modeling and variational sentence auto-encoding confirm the model's generalization capability.

Key facts. Cited by 123 papers in the map (data through 14 Oct 2026). First posted on arXiv 1 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 2; 2021: 8; 2022: 13; 2023: 20; 2024: 12; 2025: 30; 2026: 38.

123 papers.

Sort:

By area:

  1. 2020Learning to summarize from human feedbackcited by 2,588
  2. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 774
  3. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
  4. 2022Diffsound: Discrete Diffusion Model for Text-to-sound Generationcited by 257
  5. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Timecited by 234
  6. 2023Factuality Challenges in the Era of Large Language Modelscited by 126
  7. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  8. 2022Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generationcited by 108
  9. 2023Elucidating the Exposure Bias in Diffusion Modelscited by 101
  10. 2022State-of-the-art generalisation research in NLP: A taxonomy and reviewcited by 98
  11. 2022Fast Unsupervised Brain Anomaly Detection and Segmentation with Diffusion Modelscited by 97
  12. 2025On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weightingcited by 91
38 more
  1. 2023Input Perturbation Reduces Exposure Bias in Diffusion Modelscited by 90
  2. 2023Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Stepscited by 73
  3. 2024Magic-Me: Identity-Specific Video Customized Diffusioncited by 59
  4. 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resamplingcited by 52
  5. 2022G2P-DDM: Generating Sign Pose Sequence from Gloss Sequence with Discrete Diffusion Modelcited by 38
  6. 2023Tuning computer vision models with task rewardscited by 37
  7. 2023Exploiting the Signal-Leak Bias in Diffusion Modelscited by 36
  8. 2023MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methodscited by 33
  9. 2022Reward Gaming in Conditional Text Generationcited by 30
  10. 2025Taming Teacher Forcing for Masked Autoregressive Video Generationcited by 30
  11. 2026ShotStream: Streaming Multi-Shot Video Generation for Interactive Storytellingcited by 28
  12. 2026Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Modelscited by 25
  13. 2023Coarse-to-Fine: a Hierarchical Diffusion Model for Molecule Generation in 3Dcited by 24
  14. 2021Relating Neural Text Degeneration to Exposure Biascited by 19
  15. 2024HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Accelerationcited by 17
  16. 2021Implicit Unlikelihood Training: Improving Neural Text Generation with Reinforcement Learningcited by 16
  17. 2023A Systematic Study of Knowledge Distillation for Natural Language Generation with Pseudo-Target Trainingcited by 16
  18. 2025GRIFFIN: Effective Token Alignment for Faster Speculative Decodingcited by 16
  19. 2022Robust Probabilistic Time Series Forecastingcited by 15
  20. 2024Timestep-Aware Correction for Quantized Diffusion Modelscited by 13
  21. 2022Flipped Classroom: Effective Teaching for Time Series Forecastingcited by 12
  22. 2023Learning from Predictions: Fusing Training and Autoregressive Inference for Long-Term Spatiotemporal Forecastscited by 11
  23. 2023Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!cited by 11
  24. 2026Sparse Forcing: Native Trainable Sparse Attention for Real-time Autoregressive Diffusion Video Generationcited by 11
  25. 2025EdiVal-Agent: An Object-Centric Framework for Automated, Fine-Grained Evaluation of Multi-Turn Editingcited by 10
  26. 2026Causal Motion Diffusion Models for Autoregressive Motion Generationcited by 10
  27. 2021Modulating Language Models with Emotionscited by 9
  28. 2022Auto-regressive Image Synthesis with Integrated Quantizationcited by 9
  29. 2024DoughNet: A Visual Predictive Model for Topological Manipulation of Deformable Objectscited by 9
  30. 2024PICASSO: A Feed-Forward Framework for Parametric Inference of CAD Sketches via Rendering Self-Supervisioncited by 9
  31. 2025PersonaLive! Expressive Portrait Image Animation for Live Streamingcited by 9
  32. 2025LiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillationcited by 9
  33. 2021Mixed Cross Entropy Loss for Neural Machine Translationcited by 8
  34. 2024A Survey on Responsible Generative AI: What to Generate and What Notcited by 8
  35. 2025Language Model Based Text-to-Audio Generation: Anti-Causally Aligned Collaborative Residual Transformerscited by 8
  36. 2026ThinkRouter: Efficient Reasoning via Routing Thinking between Latent and Discrete Spacescited by 8
  37. 2022Keywords and Instances: A Hierarchical Contrastive Learning Framework Unifying Hybrid Granularities for Text Generationcited by 7
  38. 2024Bridging the Training-Inference Gap in LLMs by Leveraging Self-Generated Tokenscited by 7

50 most cited of 123 All 123 (JSON)