mlsanityData to 7 Oct 2026

Efficient Model-free Reinforcement Learning in Metric Spaces

arXiv 1905.00475 ↗First posted 1 May 2019cs.LG stat.MLcited by 18

Not on the map. Why?

Abstract (arXiv)

Model-free Reinforcement Learning (RL) algorithms such as Q-learning [Watkins, Dayan 92] have been widely used in practice and can achieve human level performance in applications such as video games [Mnih et al. 15]. Recently, equipped with the idea of optimism in the face of uncertainty, Q-learning algorithms [Jin, Allen-Zhu, Bubeck, Jordan 18] can be proven to be sample efficient for discrete tabular Markov Decision Processes (MDPs) which have finite number of states and actions. In this work, we present an efficient model-free Q-learning based algorithm in MDPs with a natural metric on the state-action space--hence extending efficient model-free Q-learning algorithms to continuous state-action space. Compared to previous model-based RL algorithms for metric spaces [Kakade, Kearns, Langford 03], our algorithm does not require access to a black-box planning oracle.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 0; 2023: 2; 2024: 7; 2025: 3; 2026: 1.

18 papers.

Sort:

  1. 2021Metrics and continuity in reinforcement learningcited by 26
  2. 2021Sublinear Least-Squares Value Iteration via Locality Sensitive Hashingcited by 15
  3. 2021Adaptive Discretization in Online Reinforcement Learningcited by 12
  4. 2023Minimax Optimal Q Learning with Nearest Neighborscited by 8
  5. 2021Q-Learning for MDPs with General Spaces: Convergence and Near Optimality via Quantization under Weak Continuitycited by 7
  6. 2024No-Regret Reinforcement Learning in Smooth MDPscited by 6
  7. 2024Projection by Convolution: Optimal Sample Complexity for Reinforcement Learning in Continuous-Space MDPscited by 5
  8. 2023Low-Rank MDPs with Continuous Action Spacescited by 4
  9. 2024Local Linearity: the Key for No-regret Reinforcement Learning in Continuous MDPscited by 4
  10. 2021Coarse-Grained Smoothness for RL in Metric Spacescited by 2
  11. 2025Provably Efficient and Agile Randomized Q-Learningcited by 2
  12. 2024Policy Zooming: Adaptive Discretization-based Infinite-Horizon Average-Reward Reinforcement Learningcited by 1
6 more
  1. 2024Topological Foundations of Reinforcement Learningcited by 1
  2. 2025Bellman operator convergence enhancements in reinforcement learning algorithmscited by 1
  3. 2025Improving Reinforcement Learning Sample-Efficiency using Local Approximationcited by 1
  4. 2024Scrutinize What We Ignore: Reining In Task Representation Shift Of Context-Based Offline Meta Reinforcement Learningcited by 0
  5. 2024Provably Adaptive Average Reward Reinforcement Learning for Metric Spacescited by 0
  6. 2026Improved Model-based Reinforcement Learning with Smooth Kernelscited by 0