mlsanityData to 14 Oct 2026

Safe Reinforcement Learning in Tensor Reproducing Kernel Hilbert Space

arXiv 2312.00727 ↗First posted 1 Dec 2023cs.LG cs.AI cs.SYcited by 0

Not on the map. Why?

Abstract (arXiv)

This paper delves into the problem of safe reinforcement learning (RL) in a partially observable environment with the aim of achieving safe-reachability objectives. In traditional partially observable Markov decision processes (POMDP), ensuring safety typically involves estimating the belief in latent states. However, accurately estimating an optimal Bayesian filter in POMDP to infer latent states from observations in a continuous state space poses a significant challenge, largely due to the intractable likelihood. To tackle this issue, we propose a stochastic model-based approach that guarantees RL safety almost surely in the face of unknown system dynamics and partial observation environments. We leveraged the Predictive State Representation (PSR) and Reproducing Kernel Hilbert Space (RKHS) to represent future multi-step observations analytically, and the results in this context are provable. Furthermore, we derived essential operators from the kernel Bayes' rule, enabling the recursive estimation of future observations using various operators. Under the assumption of textitundercompleness, a polynomial sample complexity is established for the RL algorithm for the infinite size of observation and action spaces, ensuring an ε-suboptimal safe policy guarantee.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 52 references that are in the map. First posted on arXiv 1 Dec 2023.

Builds on

26 of 52 references.

References · 26

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  4. 2017Constrained Policy Optimizationcited by 983
  5. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  6. 2021Neural Operator: Learning Maps Between Function Spacescited by 414
  7. 2015An Introduction to Matrix Concentration Inequalitiescited by 397
  8. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  9. 2016Kernel Mean Embedding of Distributions: A Review and Beyondcited by 364
  10. 2017Safe Policy Improvement with Baseline Bootstrappingcited by 144
  11. 2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 115
  12. 2022When Is Partially Observable Reinforcement Learning Not Scary?cited by 100
14 more
  1. 2019Learning for Safety-Critical Control with Control Barrier Functionscited by 69
  2. 2020Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functionscited by 68
  3. 2020Sample-Efficient Reinforcement Learning of Undercomplete POMDPscited by 66
  4. 2020Learning Stability Certificates from Datacited by 45
  5. 2022PAC Reinforcement Learning for Predictive State Representationscited by 43
  6. 2022Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learningcited by 43
  7. 2022Provable Reinforcement Learning with a Short-Term Memorycited by 40
  8. 2022Provably Efficient Reinforcement Learning in Partially Observable Dynamical Systemscited by 35
  9. 2021Density Constrained Reinforcement Learningcited by 25
  10. 2013Hilbert Space Embeddings of Predictive State Representationscited by 16
  11. 2021High-order Barrier Functions: Robustness, Safety and Performance-Critical Controlcited by 8
  12. 2021Learning Good State and Action Representations via Tensor Decompositioncited by 7
  13. 2016Kernel Bayesian Inference with Posterior Regularizationcited by 2
  14. 2022Adaptive Kernel Kalman Filtercited by 1

Built on by

Not cited yet.