mlsanityData to 14 Oct 2026

Adaptive teachers for amortized samplers

arXiv 2410.01432 ↗First posted 2 Oct 2024cs.LG stat.MLcited by 23

Not on the map. Why?

Abstract (arXiv)

Amortized inference is the task of training a parametric model, such as a neural network, to approximate a distribution with a given unnormalized density where exact sampling is intractable. When sampling is implemented as a sequential decision-making process, reinforcement learning (RL) methods, such as generative flow networks, can be used to train the sampling policy. Off-policy RL training facilitates the discovery of diverse, high-reward candidates, but existing methods still face challenges in efficient exploration. We propose to use an adaptive training distribution (the teacher) to guide the training of the primary amortized sampler (the student). The teacher, an auxiliary behavior model, is trained to sample high-loss regions of the student and can generalize across unexplored modes, thereby enhancing mode coverage by providing an efficient training curriculum. We validate the effectiveness of this approach in a synthetic environment designed to present an exploration challenge, two diffusion-based sampling tasks, and four biochemical discovery tasks demonstrating its ability to improve sample efficiency and mode coverage. Source code is available at https://github.com/alstn12088/adaptive-teacher.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 56 of its 70 references that are in the map. First posted on arXiv 2 Oct 2024.

Builds on

56 of 70 references.

References · 56

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2015Prioritized Experience Replaycited by 1,099
  4. 2017Deep Bayesian Active Learning with Image Datacited by 819
  5. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 672
  6. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 610
  7. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 432
  8. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 392
  9. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  10. 2019Learning Loss for Active Learningcited by 336
  11. 2021GFlowNet Foundationscited by 290
  12. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 288
38 more
  1. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 251
  2. 2021Path Integral Sampler: a stochastic control approach for samplingcited by 181
  3. 2019Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitcited by 176
  4. 2022Biological Sequence Design with GFlowNetscited by 174
  5. 2022Bayesian Structure Learning with Generative Flow Networkscited by 153
  6. 2022Learning GFlowNets from partial episodes for improved convergence and stabilitycited by 132
  7. 2020Sample-Efficient Optimization in the Latent Space of Deep Generative Models via Weighted Retrainingcited by 123
  8. 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 114
  9. 2022GFlowNets and variational inferencecited by 106
  10. 2023A theory of continuous generative flow networkscited by 101
  11. 2022Design-Bench: Benchmarks for Data-Driven Offline Model-Based Optimizationcited by 97
  12. 2023Amortizing intractable inference in large language modelscited by 97
  13. 2024Curiosity-driven Red-teaming for Large Language Modelscited by 88
  14. 2023Better Training of GFlowNets with Local Credit and Incomplete Trajectoriescited by 85
  15. 2023Towards Understanding and Improving GFlowNet Trainingcited by 75
  16. 2024Amortizing intractable inference in diffusion models for vision, language, and controlcited by 72
  17. 2020AdaLead: A simple and robust adaptive greedy search algorithm for sequence designcited by 65
  18. 2023Generative Flow Networks as Entropy-Regularized RLcited by 64
  19. 2023GFlowNets for AI-Driven Scientific Discoverycited by 63
  20. 2023Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimizationcited by 57
  21. 2024Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Samplingcited by 56
  22. 2024Learning diverse attacks on large language models for robust red-teaming and safety tuningcited by 54
  23. 2024Ant Colony Sampling with GFlowNets for Combinatorial Optimizationcited by 53
  24. 2022Generative Augmented Flow Networkscited by 50
  25. 2023Joint Bayesian Inference of Graphical Structure and Parameters with a Single Generative Flow Networkcited by 50
  26. 2023Local Search GFlowNetscited by 48
  27. 2024Improved off-policy training of diffusion samplerscited by 48
  28. 2020Molecular Mechanics-Driven Graph Neural Network with Multiplex Graph for Molecular Structurescited by 45
  29. 2024Discrete Probabilistic Inference as Control in Multi-path Environmentscited by 44
  30. 2023GFlowNet-EM for learning compositional latent variable modelscited by 40
  31. 2024Genetic-guided GFlowNets for Sample Efficient Molecular Optimizationcited by 39
  32. 2022Bayesian learning of Causal Structure and Mechanisms with GFlowNets and Variational Bayescited by 30
  33. 2023Amortized Variational Inference: When and Why?cited by 29
  34. 2023Learning to Scale Logits for Temperature-Conditional GFlowNetscited by 27
  35. 2023Learning Energy Decompositions for Partial Inference of GFlowNetscited by 24
  36. 2023An Empirical Study of the Effectiveness of Using a Replay Buffer on Mode Discovery in GFlowNetscited by 18
  37. 2024QGFN: Controllable Greediness with Action Valuescited by 18
  38. 2023Improving Gradient-guided Nested Sampling for Posterior Inferencecited by 14

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 13; 2026: 10.

23 papers.

Sort:

By area:

  1. 2025Self-Evolving Curriculum for LLM Reasoningcited by 112
  2. 2025Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inferencecited by 23
  3. 2025Outsourced diffusion sampling: Efficient posterior inference in latent spaces of generative modelscited by 17
  4. 2025Learning to Sample Effective and Diverse Prompts for Text-to-Image Generationcited by 17
  5. 2025On scalable and efficient training of diffusion samplerscited by 10
  6. 2026Bridge Matching Sampler: Scalable Sampling via Generalized Fixed-Point Diffusion Matchingcited by 10
  7. 2025Active Attacks: Red-teaming LLMs via Adaptive Environmentscited by 7
  8. 2025Adaptive Destruction Processes for Diffusion Samplerscited by 6
  9. 2025Reinforced sequential Monte Carlo for amortised samplingcited by 6
  10. 2025Loss-Guided Auxiliary Agents for Overcoming Mode Collapse in GFlowNetscited by 5
  11. 2025Posterior Inference in Latent Space for Scalable Constrained Black-box Optimizationcited by 5
  12. 2026Avoid What You Know: Divergent Trajectory Balance for GFlowNetscited by 4
11 more
  1. 2025gfnx: Fast and Scalable Library for Generative Flow Networks in JAXcited by 3
  2. 2026Discrete diffusion samplers and bridges: Off-policy algorithms and applications in latent spacescited by 2
  3. 2026Synthesizable Molecular Generation via Soft-constrained GFlowNets with Rich Chemical Priorscited by 1
  4. 2026Exploring Multiple High-Scoring Subspaces in Generative Flow Networkscited by 1
  5. 2026Stable GFlowNets with TV Monitoring and Probabilistic Guaranteescited by 1
  6. 2026Information-Geometric Forward Policy Training in GFlowNetscited by 1
  7. 2025Beyond the Proxy: Trajectory-Distilled Guidance for Offline GFlowNet Trainingcited by 0
  8. 2025Planning-Augmented Sampling with Early Guidance for High-Reward Discoverycited by 0
  9. 2026Controlling Exploration-Exploitation in GFlowNets via Markov Chain Perspectivescited by 0
  10. 2026Path-dependent Discrete Amortized Inferencecited by 0
  11. 2026Particle GFlowNets: Rethinking Generative Marginalization Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025PriorGuide: Test-Time Prior Adaptation for Simulation-Based Inference
  2. 2025ALINE: Joint Amortization for Bayesian Inference and Active Data Acquisition
  3. 2025Single-Step Consistent Diffusion Samplers
  4. 2024All-in-one simulation-based inference
  5. 2026Learning To Sample From Diffusion Models Via Inverse Reinforcement Learning
  6. 2024Amortized Active Causal Induction with Deep Reinforcement Learning
  7. 2026Hierarchical Variational Policies for Reward-Guided Diffusion
  8. 2025ReDiF: Reinforced Distillation for Few Step Diffusion