mlsanityData to 14 Oct 2026

Professor Forcing: A New Algorithm for Training Recurrent Networks

arXiv 1610.09038 ↗First posted 27 Oct 2016stat.ML cs.LGcited by 216

Not on the map. Why?

Cited by 20 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The Teacher Forcing algorithm trains recurrent networks by supplying observed sequence values as inputs during training and using the network's own one-step-ahead predictions to do multi-step sampling. We introduce the Professor Forcing algorithm, which uses adversarial domain adaptation to encourage the dynamics of the recurrent network to be the same when training the network and when sampling from the network over multiple time steps. We apply Professor Forcing to language modeling, vocal synthesis on raw waveforms, handwriting generation, and image generation. Empirically we find that Professor Forcing acts as a regularizer, improving test likelihood on character level Penn Treebank and sequential MNIST. We also find that the model qualitatively improves samples, especially when sampling for a large number of time steps. This is supported by human evaluation of sample quality. Trade-offs between Professor Forcing and Scheduled Sampling are discussed. We produce T-SNEs showing that Professor Forcing successfully makes the dynamics of the network during training and sampling more similar.

Key facts. Cited by 216 papers in the map (data through 14 Oct 2026). First posted on arXiv 27 Oct 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2020: 1; 2021: 34; 2022: 33; 2023: 36; 2024: 26; 2025: 35; 2026: 51.

216 papers.

Sort:

By area:

  1. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
  2. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  3. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
  4. 2023AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretrainingcited by 344
  5. 2021A Survey on Neural Speech Synthesiscited by 184
  6. 2024The pitfalls of next-token predictioncited by 160
  7. 2021Zero-Shot Out-of-Distribution Detection Based on the Pre-trained Model CLIPcited by 152
  8. 2024PyramidInfer: Pyramid KV Cache Compression for High-throughput LLM Inferencecited by 147
  9. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  10. 2023Scaling Data Generation in Vision-and-Language Navigationcited by 117
  11. 2023The Surprising Effectiveness of Diffusion Models for Optical Flow and Monocular Depth Estimationcited by 116
  12. 2025KVzip: Query-Agnostic KV Cache Compression with Context Reconstructioncited by 95
38 more
  1. 2023Monocular Depth Estimation using Diffusion Modelscited by 87
  2. 2024Neural Exec: Learning (and Learning from) Execution Triggers for Prompt Injection Attackscited by 85
  3. 2023Bird's-Eye-View Scene Graph for Vision-Language Navigationcited by 72
  4. 2023PLANNER: Generating Diversified Paragraph via Latent Language Diffusion Modelcited by 58
  5. 2023On the Constrained Time-Series Generation Problemcited by 54
  6. 2022Generalizing to New Physical Systems via Context-Informed Dynamics Modelcited by 50
  7. 2023Generative Modeling of Regular and Irregular Time Series Data via Koopman VAEscited by 49
  8. 2021SASRA: Semantically-aware Spatio-temporal Reasoning Agent for Vision-and-Language Navigation in Continuous Environmentscited by 48
  9. 2022GT-GAN: General Purpose Time Series Synthesis with Generative Adversarial Networkscited by 44
  10. 2024Utilizing Image Transforms and Diffusion Models for Generative Modeling of Short and Long Time Seriescited by 43
  11. 2022MAPL: Parameter-Efficient Adaptation of Unimodal Pre-Trained Models for Vision-Language Few-Shot Promptingcited by 41
  12. 2021LEADS: Learning Dynamical Systems that Generalize Across Environmentscited by 40
  13. 2023Multilingual Mathematical Autoformalizationcited by 39
  14. 2026Rolling Sink: Bridging Limited-Horizon Training and Open-Ended Testing in Autoregressive Video Diffusioncited by 32
  15. 2021Core Challenges in Embodied Vision-Language Planningcited by 30
  16. 2024DySLIM: Dynamics Stable Learning by Invariant Measure for Chaotic Systemscited by 30
  17. 2025Taming Teacher Forcing for Masked Autoregressive Video Generationcited by 30
  18. 2025DiffAD: A Unified Diffusion Modeling Approach for Autonomous Drivingcited by 26
  19. 2021Joint Visual Semantic Reasoning: Multi-Stage Decoder for Text Recognitioncited by 24
  20. 2021Unsupervised Neural Machine Translation with Generative Language Models Onlycited by 24
  21. 2023Reliable Generation of Privacy-preserving Synthetic Electronic Health Record Time Series via Diffusion Modelscited by 24
  22. 2022Multiple-Choice Question Generation: Towards an Automated Assessment Frameworkcited by 23
  23. 2023Regular Time-series Generation using SGMcited by 23
  24. 2022Ham2Pose: Animating Sign Language Notation into Pose Sequencescited by 21
  25. 2025From Few to Many: Self-Improving Many-Shot Reasoners Through Iterative Optimization and Generationcited by 21
  26. 2021Deep Structured Reactive Planningcited by 19
  27. 2021Relating Neural Text Degeneration to Exposure Biascited by 19
  28. 2025Unlocking Efficient, Scalable, and Continual Knowledge Editing with Basis-Level Representation Fine-Tuningcited by 19
  29. 2021Improved Autoregressive Modeling with Distribution Smoothingcited by 18
  30. 2021Sequence-Level Training for Non-Autoregressive Neural Machine Translationcited by 18
  31. 2021Sparsely Changing Latent States for Prediction and Planning in Partially Observable Domainscited by 18
  32. 2026Streaming Autoregressive Video Generation via Diagonal Distillationcited by 18
  33. 2021Structured Co-reference Graph Attention for Video-grounded Dialoguecited by 17
  34. 2021PETGEN: Personalized Text Generation Attack on Deep Sequence Embedding-based Classification Modelscited by 17
  35. 2023FABind: Fast and Accurate Protein-Ligand Bindingcited by 17
  36. 2021Teaching Autoregressive Language Models Complex Tasks By Demonstrationcited by 15
  37. 2026Multi-Turn On-Policy Distillation with Prefix Replaycited by 15
  38. 2021To Beam Or Not To Beam: That is a Question of Cooperation for Language GANscited by 14

50 most cited of 216 All 216 (JSON)