# Rethinking Exposure Bias In Language Modeling

arXiv: https://arxiv.org/abs/1910.11235 · HTML: https://mlsanity.com/paper/1910.11235/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2019.

## Built on by (most cited first, 9 of 9)

- [Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation](https://mlsanity.com/paper/2204.01171/) (2022)
- [Self-Improving Robust Preference Optimization](https://mlsanity.com/paper/2406.01660/) (2024)
- [Motion Generation Using Bilateral Control-Based Imitation Learning with Autoregressive Learning](https://mlsanity.com/paper/2011.06192/) (2020)
- [Collaborate, Deliberate, Evaluate: How LLM Alignment Affects Coordinated Multi-Agent Outcomes](https://mlsanity.com/paper/2509.05882/) (2025)
- [DiffusionRollout: Uncertainty-Aware Rollout Planning in Long-Horizon PDE Solving](https://mlsanity.com/paper/2602.13616/) (2026)
- [Knowledge Distillation for Visual Autoregressive Models](https://mlsanity.com/paper/2606.06078/) (2026)
- [Interactive Symbolic Regression through Offline Reinforcement Learning: A Co-Design Framework](https://mlsanity.com/paper/2402.05306/) (2024)
- [Probing to Refine: Reinforcement Distillation of LLMs via Explanatory Inversion](https://mlsanity.com/paper/2603.19266/) (2026)
- [Supervised Fine-Tuning Needs to Unlock the Potential of Token Priority](https://mlsanity.com/paper/2602.01227/) (2026)

## Abstract (arXiv)

Exposure bias describes the phenomenon that a language model trained under the teacher forcing schema may perform poorly at the inference stage when its predictions are conditioned on its previous predictions unseen from the training corpus. Recently, several generative adversarial networks (GANs) and reinforcement learning (RL) methods have been introduced to alleviate this problem. Nonetheless, a common issue in RL and GANs training is the sparsity of reward signals. In this paper, we adopt two simple strategies, multi-range reinforcing, and multi-entropy sampling, to amplify and denoise the reward signal. Our model produces an improvement over competing models with regards to BLEU scores and road exam, a new metric we designed to measure the robustness against exposure bias in language models.