mlsanityData to 7 Oct 2026

Task-agnostic Exploration in Reinforcement Learning

arXiv 2006.09497 ↗First posted 16 Jun 2020cs.LG cs.AI stat.MLcited by 37

Not on the map. Why?

Abstract (arXiv)

Efficient exploration is one of the main challenges in reinforcement learning (RL). Most existing sample-efficient algorithms assume the existence of a single reward function during exploration. In many practical scenarios, however, there is not a single underlying reward function to guide the exploration, for instance, when an agent needs to learn many skills simultaneously, or multiple conflicting objectives need to be balanced. To address these challenges, we propose the textittask-agnostic RL framework: In the exploration phase, the agent first collects trajectories by exploring the MDP without the guidance of a reward function. After exploration, it aims at finding near-optimal policies for N tasks, given the collected trajectories augmented with textitsampled rewards for each task. We present an efficient task-agnostic RL algorithm, textscUCBZero, that finds ε-optimal policies for N arbitrary tasks after at most tilde O(log(N)H5SA/ε2) exploration episodes. We also provide an Omega(log (N)H2SA/ε2) lower bound, showing that the log dependency on N is unavoidable. Furthermore, we provide an N-independent sample complexity bound of textscUCBZero in the statistically easier setting when the ground truth reward functions are known.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). First posted on arXiv 16 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 9; 2023: 8; 2024: 7; 2025: 2; 2026: 3.

37 papers.

Sort:

By area:

  1. 2022The Role of Coverage in Online Reinforcement Learningcited by 82
  2. 2022Provable Benefits of Representational Transfer in Reinforcement Learningcited by 38
  3. 2022Sample-Efficient Reinforcement Learning with loglog(T) Switching Costcited by 32
  4. 2023On the Importance of Exploration for Generalization in Reinforcement Learningcited by 30
  5. 2021A Simple Reward-free Approach to Constrained Reinforcement Learningcited by 28
  6. 2022On the Statistical Efficiency of Reward-Free Exploration in Non-Linear RLcited by 27
  7. 2021Reward Poisoning in Reinforcement Learning: Attacks Against Unknown Learners in Unknown Environmentscited by 26
  8. 2021Unsupervised Reinforcement Learning in Multiple Environmentscited by 17
  9. 2022Near Instance-Optimal PAC Reinforcement Learning for Deterministic MDPscited by 17
  10. 2023Provably Feedback-Efficient Reinforcement Learning via Active Reward Learningcited by 15
  11. 2023Reward-agnostic Fine-tuning: Provable Statistical Benefits of Hybrid Reinforcement Learningcited by 15
  12. 2021Optimal Uniform OPE and Model-based Offline Reinforcement Learning in Time-Homogeneous, Reward-Free and Task-Agnostic Settingscited by 14
25 more
  1. 2023Minimax-Optimal Reward-Agnostic Exploration in Reinforcement Learningcited by 14
  2. 2021Near-Optimal Reward-Free Exploration for Linear Mixture MDPs with Plug-in Solvercited by 13
  3. 2022Near-Optimal Deployment Efficiency in Reward-Free Reinforcement Learning with Linear Function Approximationcited by 13
  4. 2023Learning to Make Adherence-Aware Advicecited by 13
  5. 2024Offline Multi-task Transfer RL with Representational Penalizationcited by 13
  6. 2023Policy Finetuning in Reinforcement Learning via Design of Experiments using Offline Datacited by 11
  7. 2021Gap-Dependent Unsupervised Exploration for Reinforcement Learningcited by 10
  8. 2023Offline Retraining for Online RL: Decoupled Policy Learning to Mitigate Exploration Biascited by 7
  9. 2021Adaptive Multi-Goal Explorationcited by 4
  10. 2024Reinforcement Learning from Human Feedback without Reward Inference: Model-Free Algorithm and Instance-Dependent Analysiscited by 4
  11. 2024Efficient Reinforcement Learning in Probabilistic Reward Machinescited by 4
  12. 2024Near-Optimal Reinforcement Learning with Self-Play under Adaptivity Constraintscited by 3
  13. 2024Informativeness of Reward Functions in Reinforcement Learningcited by 3
  14. 2021Deep Learning for Embodied Vision Navigation: A Surveycited by 2
  15. 2025Statistical and Algorithmic Foundations of Reinforcement Learningcited by 2
  16. 2022Operator Deep Q-Learning: Zero-Shot Reward Transferring in Reinforcement Learningcited by 1
  17. 2023Between accurate prediction and poor decision making: the AI/ML gapcited by 1
  18. 2024The Limits of Transfer Reinforcement Learning with Latent Low-rank Structurecited by 1
  19. 2026Improved Bounds for Reward-Agnostic and Reward-Free Explorationcited by 1
  20. 2026Multi-agent imitation learning with function approximation: Linear Markov games and beyondcited by 1
  21. 2022SEREN: Knowing When to Explore and When to Exploitcited by 0
  22. 2022Task-Agnostic Learning to Accomplish New Taskscited by 0
  23. 2024Reward Augmentation in Reinforcement Learning for Testing Distributed Systemscited by 0
  24. 2025DIAL: Distribution-Informed Adaptive Learning of Multi-Task Constraints for Safety-Critical Systemscited by 0
  25. 2026Optimal Multi-Reward Reinforcement Learningcited by 0