mlsanityData to 7 Oct 2026

Maximum Entropy Behavior Exploration for Sim2Real Zero-Shot Reinforcement Learning

arXiv 2603.25464 ↗First posted 26 Mar 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Zero-shot reinforcement learning (RL) algorithms aim to learn a family of policies from a reward-free dataset, and recover optimal policies for any reward function directly at test time. Naturally, the quality of the pretraining dataset determines the performance of the recovered policies across tasks. However, pre-collecting a relevant, diverse dataset without prior knowledge of the downstream tasks of interest remains a challenge. In this work, we study textitonline zero-shot RL for quadrupedal control on real robotic systems, building upon the Forward-Backward (FB) algorithm. We observe that undirected exploration yields low-diversity data, leading to poor downstream performance and rendering policies impractical for direct hardware deployment. Therefore, we introduce FB-MEBE, an online zero-shot RL algorithm that combines an unsupervised behavior exploration strategy with a regularization critic. FB-MEBE promotes exploration by maximizing the entropy of the achieved behavior distribution. Additionally, a regularization critic shapes the recovered policies toward more natural and physically plausible behaviors. We empirically demonstrate that FB-MEBE achieves and improved performance compared to other exploration strategies in a range of simulated downstream tasks, and that it renders natural policies that can be seamlessly deployed to hardware without further finetuning. Videos and code available on our website.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 44 references that are in the map. First posted on arXiv 26 Mar 2026.

Builds on

36 of 44 references.

References · 36

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  3. 2015Variational Inference with Normalizing Flowscited by 2,161
  4. 2018Exploration by Random Network Distillationcited by 966
  5. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  6. 2016Successor Features for Transfer in Reinforcement Learningcited by 331
  7. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  8. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 252
  9. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 171
  10. 2021URLB: Unsupervised Reinforcement Learning Benchmarkcited by 138
  11. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 131
  12. 2021Discovering and Achieving Goals via World Modelscited by 131
24 more
  1. 2021Learning One Representation to Optimize All Rewardscited by 110
  2. 2021APS: Active Pretraining with Successor Featurescited by 109
  3. 2018Universal Successor Features Approximatorscited by 102
  4. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 100
  5. 2020C-Learning: Learning to Achieve Goals via Recursive Classificationcited by 99
  6. 2024Foundation Policies with Hilbert Representationscited by 89
  7. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
  8. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 79
  9. 2021Learning Successor States and Goal-Dependent Values: A Mathematical Viewpointcited by 67
  10. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 51
  11. 2025BFM-Zero: A Promptable Behavioral Foundation Model for Humanoid Control Using Unsupervised Reinforcement Learningcited by 33
  12. 2025FastTD3: Simple, Fast, and Capable Reinforcement Learning for Humanoid Controlcited by 32
  13. 2020Emergent Real-World Robotic Skills via Unsupervised Off-Policy Reinforcement Learningcited by 30
  14. 2023Zero-Shot Reinforcement Learning from Low Quality Datacited by 21
  15. 2025Fast Adaptation with Behavioral Foundation Modelscited by 20
  16. 2024Proto Successor Measure: Representing the Behavior Space of an RL Agentcited by 17
  17. 2025TD-JEPA: Latent-predictive Representations for Zero-Shot Reinforcement Learningcited by 17
  18. 2025Normalizing Flows are Capable Models for Continuous Controlcited by 13
  19. 2024Finer Behavioral Foundation Models via Auto-Regressive Features and Advantage Weightingcited by 12
  20. 2025DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learningcited by 11
  21. 2025Zero-Shot Adaptation of Behavioral Foundation Models to Unseen Dynamicscited by 8
  22. 2025Epistemically-guided forward-backward explorationcited by 6
  23. 2025Optimistic Task Inference for Behavior Foundation Modelscited by 5
  24. 2026Soft Forward-Backward Representations for Zero-shot Reinforcement Learning with General Utilitiescited by 2

Built on by

Not cited yet.