mlsanityData to 7 Oct 2026

A Kernel-Based Approach to Non-Stationary Reinforcement Learning in Metric Spaces

arXiv 2007.05078 ↗First posted 9 Jul 2020cs.LG stat.MLcited by 32

Not on the map. Why?

Abstract (arXiv)

In this work, we propose KeRNS: an algorithm for episodic reinforcement learning in non-stationary Markov Decision Processes (MDPs) whose state-action set is endowed with a metric. Using a non-parametric model of the MDP built with time-dependent kernels, we prove a regret bound that scales with the covering dimension of the state-action space and the total variation of the MDP with time, which quantifies its level of non-stationarity. Our method generalizes previous approaches based on sliding windows and exponential discounting used to handle changing environments. We further propose a practical implementation of KeRNS, we analyze its regret and validate it experimentally.

Key facts. Cited by 32 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 8; 2023: 7; 2024: 6; 2025: 4; 2026: 3.

32 papers.

Sort:

By area:

  1. 2021Non-stationary Reinforcement Learning without Prior Knowledge: An Optimal Black-box Approachcited by 98
  2. 2021Robust Policy Gradient against Strong Data Corruptioncited by 30
  3. 2022Performative Reinforcement Learningcited by 25
  4. 2023Provably Efficient Model-Free Algorithms for Non-stationary CMDPscited by 21
  5. 2021Optimistic Policy Optimization is Provably Efficient in Non-stationary MDPscited by 16
  6. 2022Dynamic Regret of Online Markov Decision Processescited by 13
  7. 2022Dynamics-Adaptive Continual Reinforcement Learning via Progressive Contextualizationcited by 13
  8. 2022Testing Stationarity and Change Point Detection in Reinforcement Learningcited by 12
  9. 2022Provably Efficient Primal-Dual Reinforcement Learning for CMDPs with Non-stationary Objectives and Constraintscited by 10
  10. 2023Tracking Most Significant Shifts in Nonparametric Contextual Banditscited by 10
  11. 2024Jointly Training and Pruning CNNs via Learnable Agent Guidance and Alignmentcited by 9
  12. 2023Minimax Optimal Q Learning with Nearest Neighborscited by 8
20 more
  1. 2023The complexity of non-stationary reinforcement learningcited by 7
  2. 2022Near-Optimal Goal-Oriented Reinforcement Learning in Non-Stationary Environmentscited by 6
  3. 2024Sample-efficient Learning of Infinite-horizon Average-reward MDPs with General Function Approximationcited by 6
  4. 2024Kernel-Based Function Approximation for Average Reward Reinforcement Learning: An Optimist No-Regret Algorithmcited by 6
  5. 2021Dealing with Non-Stationarity in MARL via Trust-Region Decompositioncited by 5
  6. 2023Non-stationary Reinforcement Learning under General Function Approximationcited by 5
  7. 2024Near-Optimal Dynamic Regret for Adversarial Linear Mixture MDPscited by 5
  8. 2022Non-stationary Risk-sensitive Reinforcement Learning: Near-optimal Dynamic Regret, Adaptive Detection, and Separation Designcited by 4
  9. 2024OMPO: A Unified Framework for RL under Policy and Dynamics Shiftscited by 4
  10. 2024MetaCURL: Non-stationary Concave Utility Reinforcement Learningcited by 4
  11. 2023Distributionally Time-Varying Online Stochastic Optimization under Polyak-Lojasiewicz Condition with Application in Conditional Value-at-Risk Statistical Learningcited by 2
  12. 2025Reinforcement Learning in Switching Non-Stationary Markov Decision Processes: Algorithms and Convergence Analysiscited by 2
  13. 2025Achieve Performatively Optimal Policy for Performative Reinforcement Learningcited by 2
  14. 2022Online Reinforcement Learning for Periodic MDPcited by 0
  15. 2023Online Reinforcement Learning in Periodic MDPcited by 0
  16. 2025Natural Policy Gradient for Average Reward Non-Stationary RLcited by 0
  17. 2025Optimal Dynamic Regret by Transformers for Non-Stationary Reinforcement Learningcited by 0
  18. 2026DARLING: Detection Augmented Reinforcement Learning with Non-Stationary Guaranteescited by 0
  19. 2026Improved Model-based Reinforcement Learning with Smooth Kernelscited by 0
  20. 2026Information-Based Exploration via Random Features for Reinforcement Learningcited by 0