mlsanityData to 7 Oct 2026

Reinforcement Learning with Deep Energy-Based Policies

arXiv 1702.08165 ↗First posted 27 Feb 2017cs.LG cs.AIcited by 659

Not on the map. Why?

Cited by 97 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We propose a method for learning expressive energy-based policies for continuous states and actions, which has been feasible only in tabular domains before. We apply our method to learning maximum entropy policies, resulting into a new algorithm, called soft Q-learning, that expresses the optimal policy via a Boltzmann distribution. We use the recently proposed amortized Stein variational gradient descent to learn a stochastic sampling network that approximates samples from this distribution. The benefits of the proposed algorithm include improved exploration and compositionality that allows transferring skills between tasks, which we confirm in simulated experiments with swimming and walking robots. We also draw a connection to actor-critic methods, which can be viewed performing approximate inference on the corresponding energy-based model.

Key facts. Cited by 659 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Feb 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 4; 2019: 5; 2020: 9; 2021: 115; 2022: 100; 2023: 107; 2024: 103; 2025: 116; 2026: 100.

659 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  3. 2025Process Reinforcement through Implicit Rewardscited by 478
  4. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Modelscited by 463
  5. 2021Implicit Behavioral Cloningcited by 442
  6. 2021Flow Network based Generative Models for Non-Iterative Diverse Candidate Generationcited by 384
  7. 2019Causal Confusion in Imitation Learningcited by 286
  8. 2021GFlowNet Foundationscited by 286
  9. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 283
  10. 2021How to Train Your Energy-Based Modelscited by 271
  11. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 244
  12. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 237
38 more
  1. 2021How to Train Your Robot with Deep Reinforcement Learning; Lessons We've Learnedcited by 201
  2. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 191
  3. 2022Biological Sequence Design with GFlowNetscited by 172
  4. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  5. 2018Self-Imitation Learningcited by 135
  6. 2021Goal-Conditioned Reinforcement Learning with Imagined Subgoalscited by 129
  7. 2022Learning GFlowNets from partial episodes for improved convergence and stabilitycited by 129
  8. 2023MiniLLM: On-Policy Distillation of Large Language Modelscited by 129
  9. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 127
  10. 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 119
  11. 2024Web Agents with World Models: Learning and Leveraging Environment Dynamics in Web Navigationcited by 116
  12. 2023Offline RL with No OOD Actions: In-Sample Learning via Implicit Value Regularizationcited by 114
  13. 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 113
  14. 2023Extreme Q-Learning: MaxEnt RL without Entropycited by 113
  15. 2021Provable Benefits of Actor-Critic Methods for Offline Reinforcement Learningcited by 109
  16. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learningcited by 105
  17. 2022Discovered Policy Optimisationcited by 103
  18. 2021Recent Advances in Reinforcement Learning in Financecited by 102
  19. 2022Multi-Objective GFlowNetscited by 100
  20. 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learningcited by 100
  21. 2023Amortizing intractable inference in large language modelscited by 94
  22. 2020Human-centric Dialog Training via Offline Reinforcement Learningcited by 92
  23. 2024Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Reviewcited by 92
  24. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learningcited by 88
  25. 2021S4RL: Surprisingly Simple Self-Supervision for Offline Reinforcement Learningcited by 85
  26. 2023Better Training of GFlowNets with Local Credit and Incomplete Trajectoriescited by 83
  27. 2024Diffusion Actor-Critic with Entropy Regulatorcited by 81
  28. 2022On Reinforcement Learning and Distribution Matching for Fine-Tuning Language Models with no Catastrophic Forgettingcited by 80
  29. 2023Contrastive Preference Learning: Learning from Human Feedback without RLcited by 77
  30. 2024Simplifying Deep Temporal Difference Learningcited by 71
  31. 2021Maximum Entropy Population-Based Training for Zero-Shot Human-AI Coordinationcited by 69
  32. 2022Constrained Variational Policy Optimization for Safe Reinforcement Learningcited by 69
  33. 2021A Survey of Exploration Methods in Reinforcement Learningcited by 68
  34. 2021A Workflow for Offline Model-Free Robotic Reinforcement Learningcited by 68
  35. 2023Recent Advances in Optimal Transport for Machine Learningcited by 66
  36. 2021Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learningcited by 65
  37. 2021The Option Keyboard: Combining Skills in Reinforcement Learningcited by 65
  38. 2023Language Instructed Reinforcement Learning for Human-AI Coordinationcited by 61

50 most cited of 659 All 659 (JSON)