mlsanityData to 14 Oct 2026

Improving Zero-Shot Offline RL via Behavioral Task Sampling

arXiv 2604.25496 ↗First posted 28 Apr 2026cs.AIcited by 1

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Offline zero-shot reinforcement learning (RL) aims to learn agents that optimize unseen reward functions without additional environment interaction. The standard approach to this problem trains task-conditioned policies by sampling task vectors that define linear reward functions over learned state representations. In most existing algorithms, these task vectors are randomly sampled, implicitly assuming this adequately captures the structure of the task space. We argue that doing so leads to suboptimal zero-shot generalization. To address this limitation, we propose extracting task vectors directly from the offline dataset and using them to define the task distribution used for policy training. We introduce a simple and general reward function extraction procedure that integrates into existing offline zero-shot RL algorithms. Across multiple benchmark environments and baselines, our approach improves zero-shot performance by an average of 20%, highlighting the importance of principled task sampling in offline zero-shot RL.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 44 of its 56 references that are in the map. First posted on arXiv 28 Apr 2026.

Builds on

44 of 56 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Don't Change the Algorithm, Change the Data: Exploratory Data for Offline Reinforcement Learning31 Jan

Other references · 43

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2023Segment Anythingcited by 8,234
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  8. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  9. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  10. 2016Generative Adversarial Imitation Learningcited by 1,597
  11. 2017Hindsight Experience Replaycited by 1,136
  12. 2018Exploration by Random Network Distillationcited by 993
31 more
  1. 2018DeepMind Control Suitecited by 956
  2. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  3. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
  4. 2016Successor Features for Transfer in Reinforcement Learningcited by 340
  5. 2023Language to Rewards for Robotic Skill Synthesiscited by 312
  6. 2017Automatic Goal Generation for Reinforcement Learning Agentscited by 259
  7. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 193
  8. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 174
  9. 2020Automatic Curriculum Learning For Deep RL: A Short Surveycited by 150
  10. 2021Learning One Representation to Optimize All Rewardscited by 115
  11. 2018Universal Successor Features Approximatorscited by 103
  12. 2020Maximum Entropy Gain Exploration for Long Horizon Multi-goal Reinforcement Learningcited by 102
  13. 2020Learning with AMIGo: Adversarially Motivated Intrinsic Goalscited by 88
  14. 2020Autotelic Agents with Intrinsically Motivated Goal-Conditioned Reinforcement Learning: a Short Surveycited by 85
  15. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 83
  16. 2024Open-Endedness is Essential for Artificial Superhuman Intelligencecited by 80
  17. 2025Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Modelscited by 56
  18. 2019Exploration via Hindsight Goal Generationcited by 54
  19. 2024Diffusion World Model: Future Modeling Beyond Step-by-Step Rollout for Offline Reinforcement Learningcited by 31
  20. 2024Unsupervised Zero-Shot Reinforcement Learning via Functional Reward Encodingscited by 31
  21. 2023Zero-Shot Reinforcement Learning from Low Quality Datacited by 21
  22. 2023A Definition of Open-Ended Learning Problems for Goal-Conditioned Agentscited by 18
  23. 2024Zero-Shot Reinforcement Learning via Function Encoderscited by 18
  24. 2025Self-Predictive Representations for Combinatorial Generalization in Behavioral Cloningcited by 12
  25. 2023Self-Supervised Reinforcement Learning that Transfers using Random Featurescited by 11
  26. 2023H-GAP: Humanoid Control with a Generalist Plannercited by 11
  27. 2024Inverse Reinforcement Learning by Estimating Expertise of Demonstratorscited by 11
  28. 2025Zero-Shot Adaptation of Behavioral Foundation Models to Unseen Dynamicscited by 8
  29. 2022Stein Variational Goal Generation for adaptive Exploration in Multi-Goal Reinforcement Learningcited by 6
  30. 2025Inverse Reinforcement Learning with Switching Rewards and History Dependency for Characterizing Animal Behaviorscited by 6
  31. 2024On Feasible Rewards in Multi-Agent Inverse Reinforcement Learningcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Behavioral Foundation Models for Quality Diversitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Unified Framework for Zero-Shot Reinforcement Learning
  2. 2025Tackling the Zero-Shot Reinforcement Learning Loss Directly
  3. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
  4. 2021Improving Zero-shot Generalization in Offline Reinforcement Learning using Generalized Similarity Functions
  5. 2025Provable Zero-Shot Generalization in Offline Reinforcement Learning
  6. 2025Zero-Shot Reinforcement Learning Under Partial Observability
  7. 2026Out-of-Distribution Generalisation with Sequence Models in Offline Multi-Agent Reinforcement Learning
  8. 2021A Minimalist Approach to Offline Reinforcement Learning