mlsanityData to 7 Oct 2026

Discovering Multiple Solutions from a Single Task in Offline Reinforcement Learning

arXiv 2406.05993 ↗First posted 10 Jun 2024cs.LG stat.MLcited by 4

On the map: Offline RL and imitation learning from data → Offline RL algorithms

Abstract (arXiv)

Recent studies on online reinforcement learning (RL) have demonstrated the advantages of learning multiple behaviors from a single task, as in the case of few-shot adaptation to a new environment. Although this approach is expected to yield similar benefits in offline RL, appropriate methods for learning multiple solutions have not been fully investigated in previous studies. In this study, we therefore addressed the problem of finding multiple solutions from a single task in offline RL. We propose algorithms that can learn multiple solutions in offline RL, and empirically investigate their performance. Our experimental results show that the proposed algorithm learns multiple qualitatively and quantitatively distinctive solutions in offline RL.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Offline RL algorithms, one of 13 directions in Offline RL and imitation learning from data. Builds on 21 of its 33 references that are in the map. First posted on arXiv 10 Jun 2024.

Builds on

21 of 33 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023CLUE: Calibrated Latent Guidance for Offline Reinforcement Learning23 Jun
  2. 2021Offline Reinforcement Learning with Implicit Q-Learning12 Oct
  3. 2021A Minimalist Approach to Offline Reinforcement Learning12 Jun
  4. 2020PLAS: Latent Action Space for Offline Reinforcement Learning14 Nov
  5. 2020OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning26 Oct
  6. 2020Conservative Q-Learning for Offline Reinforcement Learning8 Jun
  7. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learning15 Apr

Other references · 14

  1. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  2. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  3. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  4. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  5. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  6. 2018Maximum a Posteriori Policy Optimisationcited by 318
  7. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  8. 2020Effective Diversity in Population Based Reinforcement Learningcited by 116
  9. 2017Quality and Diversity Optimization: A Unifying Modular Frameworkcited by 111
  10. 2020One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RLcited by 79
  11. 2017Interpretable Policies for Reinforcement Learning by Genetic Programmingcited by 47
  12. 2021Discovering Diverse Solutions in Deep Reinforcement Learning by Maximizing State-Action-Based Mutual Informationcited by 18
2 more
  1. 2021Motion Planning by Learning the Solution Manifold in Trajectory Optimizationcited by 13
  2. 2018Generating Interpretable Fuzzy Controllers using Particle Swarm Optimization and Genetic Programmingcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025The impact of intrinsic rewards on exploration in Reinforcement Learningcited by 8
  2. 2025Video-Enhanced Offline Reinforcement Learning: A Model-Based Approachcited by 1
  3. 2025Diverse Transformer Decoding for Offline Reinforcement Learning Using Financial Algorithmic Approachescited by 0
  4. 2026Learning Hierarchical Skill Policies with Offline Quality-Diversity Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Tutorial: An Intuitive Explanation of Offline Reinforcement Learning Theory
  2. 2024A Non-Monolithic Policy Approach of Offline-to-Online Reinforcement Learning
  3. 2022Bridging the Gap Between Offline and Online Reinforcement Learning Evaluation Methodologies
  4. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problems
  5. 2023Finetuning from Offline Reinforcement Learning: Challenges, Trade-offs and Practical Solutions
  6. 2025Behavior-Adaptive Q-Learning: A Unifying Framework for Offline-to-Online RL
  7. 2025On the Statistical Complexity for Offline and Low-Adaptive Reinforcement Learning with Structures
  8. 2020MOReL : Model-Based Offline Reinforcement Learning