mlsanityData to 7 Oct 2026

Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks

arXiv 1506.03099 ↗First posted 9 Jun 2015cs.LG cs.CL cs.CVcited by 779

Not on the map. Why?

Cited by 54 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recurrent Neural Networks can be trained to produce sequences of tokens given some input, as exemplified by recent results in machine translation and image captioning. The current approach to training them consists of maximizing the likelihood of each token in the sequence given the current (recurrent) state and the previous token. At inference, the unknown previous token is then replaced by a token generated by the model itself. This discrepancy between training and inference can yield errors that can accumulate quickly along the generated sequence. We propose a curriculum learning strategy to gently change the training process from a fully guided scheme using the true previous token, towards a less guided scheme which mostly uses the generated token instead. Experiments on several sequence prediction tasks show that this approach yields significant improvements. Moreover, it was used successfully in our winning entry to the MSCOCO image captioning challenge, 2015.

Key facts. Cited by 779 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Jun 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 2; 2019: 1; 2020: 9; 2021: 143; 2022: 100; 2023: 114; 2024: 82; 2025: 96; 2026: 232.

779 papers.

Sort:

By area:

  1. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  2. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  3. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  4. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  5. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  6. 2024Simplified and Generalized Masked Diffusion for Discrete Datacited by 606
  7. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  8. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 479
  9. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 344
  10. 2022CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learningcited by 342
  11. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 329
  12. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
38 more
  1. 2023Photorealistic Video Generation with Diffusion Modelscited by 304
  2. 2021History Aware Multimodal Transformer for Vision-and-Language Navigationcited by 268
  3. 2026Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipecited by 262
  4. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 244
  5. 2022VectorMapNet: End-to-end Vectorized HD Map Learningcited by 243
  6. 2022Quark: Controllable Text Generation with Reinforced Unlearningcited by 236
  7. 2025Why Language Models Hallucinatecited by 205
  8. 2021Stochastic Scene-Aware Motion Predictioncited by 203
  9. 2022Online Decision Transformercited by 194
  10. 2022Think Global, Act Local: Dual-scale Graph Transformer for Vision-and-Language Navigationcited by 187
  11. 2021Character Controllers Using Motion VAEscited by 182
  12. 2021A Survey on Neural Speech Synthesiscited by 182
  13. 2021PredRNN: A Recurrent Neural Network for Spatiotemporal Predictive Learningcited by 181
  14. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
  15. 2023ETPNav: Evolving Topological Planning for Vision-Language Navigation in Continuous Environmentscited by 165
  16. 2022BRIO: Bringing Order to Abstractive Summarizationcited by 161
  17. 2024AmpleGCG: Learning a Universal and Transferable Generative Model of Adversarial Suffixes for Jailbreaking Both Open and Closed LLMscited by 160
  18. 2021Dynamic Graph Convolutional Recurrent Network for Traffic Prediction: Benchmark and Solutioncited by 156
  19. 2024The pitfalls of next-token predictioncited by 156
  20. 2021Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIPcited by 151
  21. 2021SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarizationcited by 131
  22. 2023MiniLLM: On-Policy Distillation of Large Language Modelscited by 129
  23. 2022Bridging the Gap Between Learning in Discrete and Continuous Environments for Vision-and-Language Navigationcited by 127
  24. 2023End-to-End Speech Recognition: A Surveycited by 127
  25. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  26. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimationcited by 116
  27. 2022Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generationcited by 106
  28. 2023Synthesizing Physical Character-Scene Interactionscited by 102
  29. 2023Enhancing Uncertainty-Based Hallucination Detection with Stronger Focuscited by 98
  30. 2021Survey of Low-Resource Machine Translationcited by 92
  31. 2020Retrieval-Augmented Generation for Code Summarization via Hybrid GNNcited by 89
  32. 2023Input Perturbation Reduces Exposure Bias in Diffusion Modelscited by 88
  33. 2025On-Policy RL Meets Off-Policy Experts: Harmonizing Supervised Fine-Tuning and Reinforcement Learning via Dynamic Weightingcited by 88
  34. 2023Monocular Depth Estimation using Diffusion Modelscited by 87
  35. 2021FitVid: Overfitting in Pixel-Level Video Predictioncited by 86
  36. 2024Energy-Based Diffusion Language Models for Text Generationcited by 86
  37. 2025LaViDa: A Large Diffusion Language Model for Multimodal Understandingcited by 81
  38. 2023On Learning to Summarize with Large Language Models as Referencescited by 80

50 most cited of 779 All 779 (JSON)