mlsanityData to 14 Oct 2026

Energy-Based Transfer for Reinforcement Learning

arXiv 2506.16590 ↗First posted 19 Jun 2025cs.LG cs.AIcited by 1

On the map: Deep reinforcement learning methods → Deep RL algorithms and representations

Abstract (arXiv)

Reinforcement learning algorithms often suffer from poor sample efficiency, making them challenging to apply in multi-task or continual learning settings. Efficiency can be improved by transferring knowledge from a previously trained teacher policy to guide exploration in new but related tasks. However, if the new task sufficiently differs from the teacher's training task, the transferred guidance may be sub-optimal and bias exploration toward low-reward behaviors. We propose an energy-based transfer learning method that uses out-of-distribution detection to selectively issue guidance, enabling the teacher to intervene only in states within its training distribution. We theoretically show that energy scores reflect the teacher's state-visitation density and empirically demonstrate improved sample efficiency and performance across both single-task and multi-task settings.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Deep RL algorithms and representations, one of 15 directions in Deep reinforcement learning methods. Builds on 35 of its 53 references that are in the map. First posted on arXiv 19 Jun 2025.

Builds on

35 of 53 references.

References · 35

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  3. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  4. 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,328
  5. 2020Energy-based Out-of-distribution Detectioncited by 1,299
  6. 2014How transferable are features in deep neural networks?cited by 1,225
  7. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  8. 2017Hindsight Experience Replaycited by 1,136
  9. 2021Generalized Out-of-Distribution Detection: A Surveycited by 802
  10. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 776
  11. 2019Solving Rubik's Cube with a Robot Handcited by 734
  12. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
23 more
  1. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
  2. 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 453
  3. 2015Policy Distillationcited by 440
  4. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  5. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
  6. 2017Deep Reinforcement Learning: An Overviewcited by 333
  7. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  8. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  9. 2020Transfer Learning in Deep Reinforcement Learning: A Surveycited by 265
  10. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 260
  11. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 150
  12. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
  13. 2014Reinforcement and Imitation Learning via Interactive No-Regret Learningcited by 118
  14. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 116
  15. 2022Jump-Start Reinforcement Learningcited by 112
  16. 2018Kickstarting Deep Reinforcement Learningcited by 81
  17. 2021The Option Keyboard: Combining Skills in Reinforcement Learningcited by 67
  18. 2024Achieving Human Level Competitive Robot Table Tenniscited by 27
  19. 2023Characterizing Out-of-Distribution Error via Optimal Transportcited by 20
  20. 2024Offline Multi-task Transfer RL with Representational Penalizationcited by 14
  21. 2020Policy Entropy for Out-of-Distribution Classificationcited by 11
  22. 2022Explainable Action Advising for Multi-Agent Reinforcement Learningcited by 10
  23. 2023Introspective Action Advising for Interpretable Transfer Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Learning Implicit Causal World Models from Multi-Agent Demonstrationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learning
  2. 2025Energy-Weighted Flow Matching for Offline Reinforcement Learning
  3. 2025Analytic Energy-Guided Policy Optimization for Offline Reinforcement Learning
  4. 2025Non-conflicting Energy Minimization in Reinforcement Learning based Robot Control
  5. 2025FlowQ: Energy-Guided Flow Policies for Offline Reinforcement Learning
  6. 2024Sampling from Energy-based Policies using Diffusion
  7. 2023TGRL: An Algorithm for Teacher Guided Reinforcement Learning
  8. 2020Energy-Based Imitation Learning