# Differentiable Scheduled Sampling for Credit Assignment

arXiv: https://arxiv.org/abs/1704.06970 · HTML: https://mlsanity.com/paper/1704.06970/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2017.

## Built on by (most cited first, 13 of 13)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization](https://mlsanity.com/paper/2210.01241/) (2022)
- [Autoregressive Knowledge Distillation through Imitation Learning](https://mlsanity.com/paper/2009.07253/) (2020)
- [Learning to Generate Better Than Your LLM](https://mlsanity.com/paper/2306.11816/) (2023)
- [Scheduled Sampling Based on Decoding Steps for Neural Machine Translation](https://mlsanity.com/paper/2108.12963/) (2021)
- [Confidence-Aware Scheduled Sampling for Neural Machine Translation](https://mlsanity.com/paper/2107.10427/) (2021)
- [Mixed Cross Entropy Loss for Neural Machine Translation](https://mlsanity.com/paper/2106.15880/) (2021)
- [Bridging the Training-Inference Gap in LLMs by Leveraging Self-Generated Tokens](https://mlsanity.com/paper/2410.14655/) (2024)
- [Language Model Evaluation in Open-ended Text Generation](https://mlsanity.com/paper/2108.03578/) (2021)
- [Improving Scheduled Sampling with Elastic Weight Consolidation for Neural Machine Translation](https://mlsanity.com/paper/2109.06308/) (2021)
- [Reinforcement Replaces Supervision: Query focused Summarization using Deep Reinforcement Learning](https://mlsanity.com/paper/2311.17514/) (2023)
- [Alleviate Exposure Bias in Sequence Prediction  with Recurrent Neural Networks](https://mlsanity.com/paper/2103.11603/) (2021)
- [Improving Autoregressive Training with Dynamic Oracles](https://mlsanity.com/paper/2406.09393/) (2024)

## Abstract (arXiv)

We demonstrate that a continuous relaxation of the argmax operation can be used to create a differentiable approximation to greedy decoding for sequence-to-sequence (seq2seq) models. By incorporating this approximation into the scheduled sampling training procedure (Bengio et al., 2015)--a well-known technique for correcting exposure bias--we introduce a new training objective that is continuous and differentiable everywhere and that can provide informative gradients near points where previous decoding decisions change their value. In addition, by using a related approximation, we demonstrate a similar approach to sampled-based training. Finally, we show that our approach outperforms cross-entropy training and scheduled sampling procedures in two sequence prediction tasks: named entity recognition and machine translation.