mlsanityData to 14 Oct 2026

Safe Policy Optimization with Local Generalized Linear Function Approximations

arXiv 2111.04894 ↗First posted 9 Nov 2021cs.LG cs.AI cs.ROcited by 10

Not on the map. Why?

Abstract (arXiv)

Safe exploration is a key to applying reinforcement learning (RL) in safety-critical systems. Existing safe exploration methods guaranteed safety under the assumption of regularity, and it has been difficult to apply them to large-scale real problems. We propose a novel algorithm, SPO-LF, that optimizes an agent's policy while learning the relation between a locally available feature obtained by sensors and environmental reward/safety using generalized linear function approximations. We provide theoretical guarantees on its safety and optimality. We experimentally show that our algorithm is 1) more efficient in terms of sample complexity and computational cost and 2) more applicable to large-scale problems than previous safe RL methods with theoretical guarantees, and 3) comparably sample-efficient and safer compared with existing advanced deep RL methods with safety constraints.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 21 of its 42 references that are in the map. First posted on arXiv 9 Nov 2021.

Builds on

21 of 42 references.

References · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2017Constrained Policy Optimizationcited by 983
  4. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 520
  5. 2017Safe Reinforcement Learning via Shieldingcited by 449
  6. 2018Reward Constrained Policy Optimizationcited by 417
  7. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  8. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 337
  9. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  10. 2019Reinforcement Learning in Feature Space: Matrix Bandit, Kernels, and Regret Boundcited by 201
  11. 2020Provably Efficient Safe Exploration via Primal-Dual Policy Optimizationcited by 134
  12. 2019IPO: Interior-point Policy Optimization under Constraintscited by 132
9 more
  1. 2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 115
  2. 2014Generalization and Exploration via Randomized Value Functionscited by 112
  3. 2019Optimism in Reinforcement Learning with Generalized Linear Function Approximationcited by 111
  4. 2019Linear Stochastic Bandits Under Safety Constraintscited by 90
  5. 2019Safe Policies for Reinforcement Learning via Primal-Dual Methodscited by 79
  6. 2016Safe Exploration in Finite Markov Decision Processes with Gaussian Processescited by 75
  7. 2019Safe Exploration for Interactive Machine Learningcited by 51
  8. 2020Constrained Markov Decision Processes via Backward Value Functionscited by 33
  9. 2019Efficient and Safe Exploration in Deterministic Markov Decision Processes with Unknown Transition Modelscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 2; 2025: 2; 2026: 1.

10 papers.

  1. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  2. 2023Safe Exploration in Reinforcement Learning: A Generalized Formulation and Algorithmscited by 28
  3. 2025Generalized Linear Bandits: Almost Optimal Regret with One-Pass Updatecited by 16
  4. 2024Distributionally Robust Constrained Reinforcement Learning under Strong Dualitycited by 15
  5. 2023Gradient Shaping for Multi-Constraint Safe Reinforcement Learningcited by 10
  6. 2025HASARD: A Benchmark for Vision-Based Safe Reinforcement Learning in Embodied Agentscited by 6
  7. 2024Long-term Safe Reinforcement Learning with Binary Feedbackcited by 4
  8. 2023Modeling Risk in Reinforcement Learning: A Literature Mappingcited by 1
  9. 2023Adaptive Aggregation for Safety-Critical Controlcited by 0
  10. 2026Learning Safe Agent Behaviour from Human Preferences and Justifications via World Modelscited by 0