mlsanityData to 14 Oct 2026

How (not) to Train your Generative Model: Scheduled Sampling, Likelihood, Adversary?

arXiv 1511.05101 ↗First posted 16 Nov 2015stat.ML cs.AI cs.ITcited by 78

Not on the map. Why?

Abstract (arXiv)

Modern applications and progress in deep learning research have created renewed interest for generative models of text and of images. However, even today it is unclear what objective functions one should use to train and evaluate these models. In this paper we present two contributions. Firstly, we present a critique of scheduled sampling, a state-of-the-art training method that contributed to the winning entry to the MSCOCO image captioning benchmark in 2015. Here we show that despite this impressive empirical performance, the objective function underlying scheduled sampling is improper and leads to an inconsistent learning algorithm. Secondly, we revisit the problems that scheduled sampling was meant to address, and present an alternative interpretation. We argue that maximum likelihood is an inappropriate training objective when the end-goal is to generate natural-looking samples. We go on to derive an ideal objective function to use in this situation instead. We introduce a generalisation of adversarial training, and show how such method can interpolate between maximum likelihood training and our ideal training objective. To our knowledge this is the first theoretical analysis that explains why adversarial training tends to produce samples with higher perceived quality.

Key facts. Cited by 78 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Nov 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 3; 2021: 18; 2022: 8; 2023: 20; 2024: 8; 2025: 5; 2026: 15.

78 papers.

Sort:

By area:

  1. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  2. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 252
  3. 2022Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimizationcited by 246
  4. 2023DistillSpec: Improving Speculative Decoding via Knowledge Distillationcited by 165
  5. 2023MiniLLM: On-Policy Distillation of Large Language Modelscited by 139
  6. 2023Aligning Language Models with Preferences through f-divergence Minimizationcited by 120
  7. 2021Survey of Low-Resource Machine Translationcited by 92
  8. 2020Autoregressive Knowledge Distillation through Imitation Learningcited by 74
  9. 2020Text Generation by Learning from Demonstrationscited by 64
  10. 2024FakeInversion: Learning to Detect Images from Unseen Text-to-Image Models by Inverting Stable Diffusioncited by 61
  11. 2024Survey on Knowledge Distillation for Large Language Models: Methods, Evaluation, and Applicationcited by 56
  12. 2021f-Domain-Adversarial Learning: Theory and Algorithmscited by 47
38 more
  1. 2021Introduction to Normalizing Flows for Lattice Field Theorycited by 35
  2. 2026Domino: Decoupling Causal Modeling from Autoregressive Drafting in Speculative Decodingcited by 34
  3. 2022Reward Gaming in Conditional Text Generationcited by 30
  4. 2023A Review of Deep Learning for Video Captioningcited by 28
  5. 2023Tailoring Language Generation Models under Total Variation Distancecited by 27
  6. 2021Learning to Generate 3D Shapes with Generative Cellular Automatacited by 22
  7. 2023Diffusion in the Dark: A Diffusion Model for Low-Light Text Recognitioncited by 20
  8. 2021Relating Neural Text Degeneration to Exposure Biascited by 19
  9. 2023On the Efficacy of Sampling Adapterscited by 17
  10. 2023MixCE: Training Autoregressive Language Models by Mixing Forward and Reverse Cross-Entropiescited by 16
  11. 2026Explain in Your Own Words: Improving Reasoning via Token-Selective Dual Knowledge Distillationcited by 16
  12. 2021TextBox: A Unified, Modularized, and Extensible Framework for Text Generationcited by 15
  13. 2023Free-Editor: Zero-shot Text-driven 3D Scene Editingcited by 14
  14. 2023Fundamentals of Generative Large Language Models and Perspectives in Cyber-Defensecited by 13
  15. 2023Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!cited by 11
  16. 2024Efficient Inference for Large Language Model-based Generative Recommendationcited by 11
  17. 2021Realizing GANs via a Tunable Loss Functioncited by 10
  18. 2023Time-series Generation by Contrastive Imitationcited by 9
  19. 2023Stochastic Parrots Looking for Stochastic Parrots: LLMs are Easy to Fine-Tune and Hard to Detect with other LLMscited by 8
  20. 2023Language Model Decoding as Direct Metrics Optimizationcited by 8
  21. 2022DeepCreativity: Measuring Creativity with Deep Learning Techniquescited by 7
  22. 2024Bridging the Training-Inference Gap in LLMs by Leveraging Self-Generated Tokenscited by 7
  23. 2026Semantic Tube Prediction: Beating LLM Data Efficiency with JEPAcited by 7
  24. 2023Addressing GAN Training Instabilities via Tunable Classification Lossescited by 5
  25. 2019On Value Discrepancy of Imitation Learningcited by 4
  26. 2021Generative Adversarial Networks and Adversarial Autoencoders: Tutorial and Surveycited by 4
  27. 2021Denoised Labels for Financial Time-Series Data via Self-Supervised Learningcited by 4
  28. 2024Entropy Controllable Direct Preference Optimizationcited by 4
  29. 2025Knowledge Distillation with Training Wheelscited by 4
  30. 2020Nested-Wasserstein Self-Imitation Learning for Sequence Generationcited by 3
  31. 2021A Brief Introduction to Generative Modelscited by 3
  32. 2022Unsupervised Clustering of Roman Potsherds via Variational Autoencoderscited by 3
  33. 2024A Statistical Framework for Data-dependent Retrieval-Augmented Modelscited by 3
  34. 2025Whatever Remains Must Be True: Filtering Drives Reasoning in LLMs, Shaping Diversitycited by 3
  35. 2026Spec-AUF: Accept-Until-Fail Training under Train-Inference Misalignment for Masked Block Drafterscited by 3
  36. 2021Language Model Evaluation in Open-ended Text Generationcited by 2
  37. 2021Can the Transformer Be Used as a Drop-in Replacement for RNNs in Text-Generating GANs?cited by 2
  38. 2021Photon detection probability prediction using one-dimensional generative neural networkcited by 2

50 most cited of 78 All 78 (JSON)