mlsanityData to 14 Oct 2026

RAD: Retrieval High-quality Demonstrations to Enhance Decision-making

arXiv 2507.15356 ↗First posted 21 Jul 2025cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Offline RL algorithms

also in 1 more
Abstract (arXiv)

Offline reinforcement learning (RL) learns policies from fixed datasets, thereby avoiding costly or unsafe environment interactions. However, its reliance on finite static datasets inherently restricts the ability to generalize beyond the training distribution. Prior solutions based on synthetic data augmentation often fail to generalize to unseen scenarios in the (augmented) dataset. To address these challenges, we propose Retrieval High-quAlity Demonstrations (RAD) for decision-making, which innovatively introduces a retrieval mechanism into offline RL. Specifically, RAD retrieves high-return and reachable states from the offline dataset as target states, and leverages a generative model to generate sub-trajectories conditioned on these targets for planning. Since the targets are high-return states, once the agent reaches such a target, it can continue to obtain high returns by following the associated high-return actions, thereby improving policy generalization. Extensive experiments confirm that RAD achieves competitive or superior performance compared to baselines across diverse benchmarks, validating its effectiveness. Our code is available at https://github.com/LeahGL/RAD.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 37 of its 41 references that are in the map. First posted on arXiv 21 Jul 2025.

Builds on

37 of 41 references.

Closest ancestors · 14

Same direction, newest first

  1. 2024Is Value Learning Really the Main Bottleneck in Offline RL?13 Jun
  2. 2024DiffStitch: Boosting Offline Reinforcement Learning with Diffusion-based Trajectory Stitching4 Feb
  3. 2023Uncertainty-driven Trajectory Truncation for Data Augmentation in Offline Reinforcement Learning10 Apr
  4. 2022Double Check Your State Before Trusting It: Confidence-Aware Bidirectional Offline Model-Based Imagination16 Jun
  5. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems2 Mar
  6. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  7. 2021Offline Reinforcement Learning with Reverse Model-based Imagination1 Oct
  8. 2021COMBO: Conservative Offline Model-Based Policy Optimization16 Feb
  9. 2020Critic Regularized Regression26 Jun
  10. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  11. 2020MOPO: Model-based Offline Policy Optimization27 May
  12. 2020MOReL : Model-Based Offline Reinforcement Learning12 May
2 more
  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems4 May
  2. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr

Other references · 23

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2020Denoising Diffusion Implicit Modelscited by 9,781
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  4. 2022Scalable Diffusion Models with Transformerscited by 6,054
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  6. 2016OpenAI Gymcited by 1,736
  7. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  8. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  9. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 671
  10. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  11. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 516
  12. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 384
11 more
  1. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 229
  2. 2023Synthetic Experience Replaycited by 101
  3. 2019BAIL: Best-Action Imitation Learning for Batch Deep Reinforcement Learningcited by 91
  4. 2024DiffuserLite: Towards Real-time Diffusion Planningcited by 42
  5. 2021Offline Reinforcement Learning for Autonomous Driving with Safety and Exploration Enhancementcited by 29
  6. 2023Efficient Exploration in Continuous-time Model-based Reinforcement Learningcited by 22
  7. 2022Discrete-Constrained Regression for Local Counting Modelscited by 19
  8. 2022Semi-Markov Offline Reinforcement Learning for Healthcarecited by 18
  9. 2023CEIL: Generalized Contextual Imitation Learningcited by 13
  10. 2023Decision Stacks: Flexible Reinforcement Learning via Modular Generative Modelscited by 8
  11. 2024Learning Versatile Skills with Curriculum Maskingcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Large-Scale Retrieval for Reinforcement Learning
  2. 2024Retrieval-Augmented Decision Transformer: External Memory for In-context RL
  3. 2021Representation Matters: Offline Pretraining for Sequential Decision Making
  4. 2022Retrieval-Augmented Reinforcement Learning
  5. 2025Enhancing Online Reinforcement Learning with Meta-Learned Objective from Offline Data
  6. 2025Learning from Reward-Free Offline Data: A Case for Planning with Latent Dynamics Models
  7. 2025Unsupervised Data Generation for Offline Reinforcement Learning: A Perspective from Model
  8. 2023Using Offline Data to Speed Up Reinforcement Learning in Procedurally Generated Environments