mlsanityData to 7 Oct 2026

Reward-Conditioned Policies

arXiv 1912.13465 ↗First posted 31 Dec 2019cs.LG stat.MLcited by 101

Not on the map. Why?

Cited by 56 in Offline RL and imitation learning from data, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Reinforcement learning offers the promise of automating the acquisition of complex behavioral skills. However, compared to commonly used and well-understood supervised learning methods, reinforcement learning algorithms can be brittle, difficult to use and tune, and sensitive to seemingly innocuous implementation decisions. In contrast, imitation learning utilizes standard and well-understood supervised learning methods, but requires near-optimal expert data. Can we learn effective policies via supervised learning without demonstrations? The main idea that we explore in this work is that non-expert trajectories collected from sub-optimal policies can be viewed as optimal supervision, not for maximizing the reward, but for matching the reward of the given trajectory. By then conditioning the policy on the numerical value of the reward, we can obtain a policy that generalizes to larger returns. We show how such an approach can be derived as a principled method for policy search, discuss several variants, and compare the method experimentally to a variety of current reinforcement learning methods on standard benchmarks.

Key facts. Cited by 101 papers in the map (data through 7 Oct 2026). First posted on arXiv 31 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 20; 2023: 30; 2024: 20; 2025: 10; 2026: 15.

101 papers.

Sort:

By area:

  1. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  2. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  3. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
  4. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  5. 2023Pretraining Language Models with Human Preferencescited by 245
  6. 2022Multi-Game Decision Transformerscited by 212
  7. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 184
  8. 2024Rewards-in-Context: Multi-objective Alignment of Foundation Models with Dynamic Preference Adjustmentcited by 171
  9. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  10. 2022Goal-Conditioned Reinforcement Learning: Problems and Solutionscited by 145
  11. 2023Q-Transformer: Scalable Offline Reinforcement Learning via Autoregressive Q-Functionscited by 129
  12. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 116
38 more
  1. 2021Generalized Decision Transformer for Offline Hindsight Information Matchingcited by 105
  2. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RLcited by 103
  3. 2023Guided Flows for Generative Modeling and Decision Makingcited by 99
  4. 2022Can Wikipedia Help Offline Reinforcement Learning?cited by 76
  5. 2022When does return-conditioned supervised learning work for offline reinforcement learning?cited by 73
  6. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbonecited by 71
  7. 2022You Can't Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environmentscited by 62
  8. 2022A Policy-Guided Imitation Approach for Offline Reinforcement Learningcited by 58
  9. 2025Diffusion Guidance Is a Controllable Policy Improvement Operatorcited by 51
  10. 2022Dichotomy of Control: Separating What You Can Control from What You Cannotcited by 47
  11. 2023AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agentscited by 36
  12. 2022On Transforming Reinforcement Learning by Transformer: The Development Trajectorycited by 35
  13. 2023Beyond Reward: Offline Preference-guided Policy Optimizationcited by 35
  14. 2023Reasoning with Latent Diffusion in Offline Reinforcement Learningcited by 33
  15. 2024Q-value Regularized Transformer for Offline Reinforcement Learningcited by 28
  16. 2021StARformer: Transformer with State-Action-Reward Representations for Visual Reinforcement Learningcited by 27
  17. 2023Future-conditioned Unsupervised Pretraining for Decision Transformercited by 26
  18. 2023Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Makingcited by 26
  19. 2024Closing the Gap between TD Learning and Supervised Learning -- A Generalisation Point of Viewcited by 26
  20. 2024CtRL-Sim: Reactive and Controllable Driving Agents with Offline Reinforcement Learningcited by 26
  21. 2025Latent Diffusion Planning for Imitation Learningcited by 26
  22. 2023Critic-Guided Decision Transformer for Offline Reinforcement Learningcited by 25
  23. 2023ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioningcited by 24
  24. 2026GigaBrain-0.5M*: a VLA That Learns From World Model-Based Reinforcement Learningcited by 23
  25. 2022Pareto Conditioned Networkscited by 22
  26. 2023Harnessing Mixed Offline Reinforcement Learning Datasets via Trajectory Weightingcited by 22
  27. 2023Waypoint Transformer: Reinforcement Learning via Supervised Learning with Intermediate Targetscited by 22
  28. 2022Know Your Boundaries: The Necessity of Explicit Behavioral Cloning in Offline RLcited by 21
  29. 2022Imitating Past Successes can be Very Suboptimalcited by 20
  30. 2023Graph Decision Transformercited by 20
  31. 2023When should we prefer Decision Transformers for Offline Reinforcement Learning?cited by 20
  32. 2023Constraint-Conditioned Policy Optimization for Versatile Safe Reinforcement Learningcited by 17
  33. 2022Phasic Self-Imitative Reduction for Sparse-Reward Goal-Conditioned Reinforcement Learningcited by 16
  34. 2025Curating Demonstrations using Online Experiencecited by 16
  35. 2023Learning Video-Conditioned Policies for Unseen Manipulation Taskscited by 15
  36. 2023Hundreds Guide Millions: Adaptive Offline Reinforcement Learning with Expert Guidancecited by 13
  37. 2023CEIL: Generalized Contextual Imitation Learningcited by 12
  38. 2023Free from Bellman Completeness: Trajectory Stitching via Model-based Return-conditioned Supervised Learningcited by 11

50 most cited of 101 All 101 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021Imitation Learning by Reinforcement Learning
  2. 2019Learning to Reach Goals via Iterated Supervised Learning
  3. 2022A Ranking Game for Imitation Learning
  4. 2018Fast Policy Learning through Imitation and Reinforcement
  5. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation
  6. 2023Backward Learning for Goal-Conditioned Policies
  7. 2025Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroids
  8. 2022Imitation Learning by Estimating Expertise of Demonstrators