mlsanityData to 14 Oct 2026

Feasibility Consistent Representation Learning for Safe Reinforcement Learning

arXiv 2405.11718 ↗First posted 20 May 2024cs.LGcited by 3

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

In the field of safe reinforcement learning (RL), finding a balance between satisfying safety constraints and optimizing reward performance presents a significant challenge. A key obstacle in this endeavor is the estimation of safety constraints, which is typically more difficult than estimating a reward metric due to the sparse nature of the constraint signals. To address this issue, we introduce a novel framework named Feasibility Consistent Safe Reinforcement Learning (FCSRL). This framework combines representation learning with feasibility-oriented objectives to identify and extract safety-related information from the raw state for safe RL. Leveraging self-supervised learning techniques and a more learnable safety metric, our approach enhances the policy learning and constraint estimation. Empirical evaluations across a range of vector-state and image-based tasks demonstrate that our method is capable of learning a better safety-aware embedding and achieving superior performance than previous representation learning baselines.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 58 of its 74 references that are in the map. First posted on arXiv 20 May 2024.

Builds on

58 of 74 references.

References · 58

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  6. 2015Continuous control with deep reinforcement learningcited by 3,407
  7. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  8. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  9. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  10. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  11. 2017Hindsight Experience Replaycited by 1,136
  12. 2020Mastering Atari with Discrete World Modelscited by 1,007
38 more
  1. 2017Constrained Policy Optimizationcited by 983
  2. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 786
  3. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  4. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  5. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 721
  6. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  7. 2019Model-Based Reinforcement Learning for Ataricited by 502
  8. 2018Reward Constrained Policy Optimizationcited by 417
  9. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  10. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  11. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 337
  12. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  13. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  14. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 290
  15. 2022Contrastive Learning as Goal-Conditioned Reinforcement Learningcited by 253
  16. 2018Safe Exploration in Continuous Action Spacescited by 247
  17. 2021Mastering Atari Games with Limited Datacited by 237
  18. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
  19. 2020Projection-Based Constrained Policy Optimizationcited by 203
  20. 2017Hamilton-Jacobi Reachability: A Brief Overview and Recent Advancescited by 201
  21. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  22. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 167
  23. 2015Deep Spatial Autoencoders for Visuomotor Learningcited by 153
  24. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 143
  25. 2018State Representation Learning for Control: An Overviewcited by 127
  26. 2017Value Prediction Networkcited by 117
  27. 2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 115
  28. 2022A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspectivecited by 105
  29. 2020The Value Equivalence Principle for Model-Based Reinforcement Learningcited by 94
  30. 2023For SALE: State-Action Representation Learning for Deep Reinforcement Learningcited by 75
  31. 2022Constrained Variational Policy Optimization for Safe Reinforcement Learningcited by 71
  32. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 69
  33. 2022Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environmentscited by 60
  34. 2022Reachability Constrained Reinforcement Learningcited by 59
  35. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learningcited by 59
  36. 2022Constrained Policy Optimization via Bayesian World Modelscited by 57
  37. 2023SafeDreamer: Safe Reinforcement Learning with World Modelscited by 54
  38. 2019Dealing with Sparse Rewards in Reinforcement Learningcited by 42

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024OASIS: Conditional Distribution Shaping for Offline Safe Reinforcement Learningcited by 10
  2. 2025Enhance Exploration in Safe Reinforcement Learning with Contrastive Representation Learningcited by 0
  3. 2026VERA: Verifiable Feasibility Representations with Counterfactual Credit for Constrained Multi-Agent Controlcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Survey of Constraint Formulations in Safe Reinforcement Learning
  2. 2025Safety Representations for Safer Policy Learning
  3. 2025Feasibility-Aware Pessimistic Estimation: Toward Long-Horizon Safety in Offline RL
  4. 2024Safe Reinforcement Learning with Learned Non-Markovian Safety Constraints
  5. 2026Evaluation Metrics for Safe Reinforcement Learning
  6. 2024Safe Offline Reinforcement Learning with Feasibility-Guided Diffusion Model
  7. 2024Safety through Permissibility: Shield Construction for Fast and Safe Reinforcement Learning
  8. 2026Sampling-Based Safe Reinforcement Learning