mlsanityData to 7 Oct 2026

Context-Aware Safe Reinforcement Learning for Non-Stationary Environments

arXiv 2101.00531 ↗First posted 2 Jan 2021cs.LGcited by 33

Not on the map. Why?

Abstract (arXiv)

Safety is a critical concern when deploying reinforcement learning agents for realistic tasks. Recently, safe reinforcement learning algorithms have been developed to optimize the agent's performance while avoiding violations of safety constraints. However, few studies have addressed the non-stationary disturbances in the environments, which may cause catastrophic outcomes. In this paper, we propose the context-aware safe reinforcement learning (CASRL) method, a meta-learning framework to realize safe adaptation in non-stationary environments. We use a probabilistic latent variable model to achieve fast inference of the posterior environment transition distribution given the context data. Safety constraints are then evaluated with uncertainty-aware trajectory sampling. The high cost of safety violations leads to the rareness of unsafe records in the dataset. We address this issue by enabling prioritized sampling during model training and formulating prior safety constraints with domain knowledge during constrained planning. The algorithm is evaluated in realistic safety-critical environments with non-stationary disturbances. Results show that the proposed algorithm significantly outperforms existing baselines in terms of safety and robustness.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Builds on 34 of its 48 references that are in the map. First posted on arXiv 2 Jan 2021.

Builds on

34 of 48 references.

References · 34

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2015Continuous control with deep reinforcement learningcited by 3,279
  5. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  6. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  7. 2015Prioritized Experience Replaycited by 1,070
  8. 2017Constrained Policy Optimizationcited by 928
  9. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  10. 2020Overfitting in adversarially robust deep learningcited by 504
  11. 2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 490
  12. 2017Robust Adversarial Reinforcement Learningcited by 439
22 more
  1. 2019Learning agile and dynamic motor skills for legged robotscited by 402
  2. 2018Neural Processescited by 307
  3. 2019Attentive Neural Processescited by 280
  4. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 278
  5. 2018Learning to Adapt in Dynamic, Real-World Environments Through Meta-Reinforcement Learningcited by 270
  6. 2018Safe Exploration in Continuous Action Spacescited by 237
  7. 2020Projection-Based Constrained Policy Optimizationcited by 195
  8. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 140
  9. 2019Convolutional Conditional Neural Processescited by 139
  10. 2018Learning-based Model Predictive Control for Safe Explorationcited by 116
  11. 2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 109
  12. 2014Optimizing the CVaR via Samplingcited by 98
  13. 2015Gaussian Processes for Data-Efficient Learning in Robotics and Controlcited by 97
  14. 2017Cautious Model Predictive Control using Gaussian Process Regressioncited by 91
  15. 2018Deep Online Learning via Meta-Learning: Continual Adaptation for Model-Based RLcited by 80
  16. 2020Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Methodcited by 74
  17. 2017OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real Worldcited by 64
  18. 2018Meta Reinforcement Learning with Latent Variable Gaussian Processescited by 44
  19. 2019Assistive Gym: A Physics Simulation Framework for Assistive Roboticscited by 44
  20. 2020Delay-Aware Model-Based Reinforcement Learning for Continuous Controlcited by 33
  21. 2020Task-Agnostic Online Reinforcement Learning with an Infinite Mixture of Gaussian Processescited by 23
  22. 2019Recurrent Attentive Neural Process for Sequential Datacited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 5; 2023: 10; 2024: 1; 2025: 7; 2026: 7.

33 papers.

Sort:

By area:

  1. 2022Constrained Variational Policy Optimization for Safe Reinforcement Learningcited by 69
  2. 2022A Survey on Model-based Reinforcement Learningcited by 65
  3. 2023Constrained Decision Transformer for Offline Safe Reinforcement Learningcited by 65
  4. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learningcited by 58
  5. 2023TAIL: Task-specific Adapters for Imitation Learning with Large Pretrained Modelscited by 48
  6. 2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
  7. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  8. 2023Constraint-Conditioned Policy Optimization for Versatile Safe Reinforcement Learningcited by 17
  9. 2023Safety-aware Causal Representation for Trustworthy Offline Reinforcement Learning in Autonomous Drivingcited by 16
  10. 2023RiskBench: A Scenario-based Benchmark for Risk Identificationcited by 12
  11. 2025A Survey of Safe Reinforcement Learning and Constrained MDPs: A Technical Survey on Single-Agent and Multi-Agent Safetycited by 12
  12. 2025Out-of-Distribution Generalization with a SPARC: Racing 100 Unseen Vehicles with a Single Policycited by 4
21 more
  1. 2021Model-Based Safe Reinforcement Learning with Time-Varying State and Control Constraints: An Application to Intelligent Vehiclescited by 3
  2. 2021Scalable Safety-Critical Policy Evaluation with Accelerated Rare Event Samplingcited by 2
  3. 2025On the Design of Safe Continual RL Methods for Control of Nonlinear Systemscited by 2
  4. 2025Adaptive Shielding for Safe Reinforcement Learning under Hidden-Parameter Dynamics Shiftscited by 2
  5. 2025PNAct: Crafting Backdoor Attacks in Safe Reinforcement Learningcited by 2
  6. 2022Variational Meta Reinforcement Learning for Social Roboticscited by 1
  7. 2023Life-inspired Interoceptive Artificial Intelligence for Autonomous and Adaptive Agentscited by 1
  8. 2023Learning to Recover for Safe Reinforcement Learningcited by 1
  9. 2023C-Procgen: Empowering Procgen with Controllable Contextscited by 1
  10. 2024Robust Model Based Reinforcement Learning Using L1 Adaptive Controlcited by 1
  11. 2025Safe Continual Domain Adaptation after Sim2Real Transfer of Reinforcement Learning Policies in Roboticscited by 1
  12. 2026Safe Continual Reinforcement Learning Methods for Nonstationary Environments. Towards a Survey of the State of the Artcited by 1
  13. 2021Robustifying Reinforcement Learning Policies with L1 Adaptive Controlcited by 0
  14. 2023Adaptive Aggregation for Safety-Critical Controlcited by 0
  15. 2025UACER: An Uncertainty-Adaptive Critic Ensemble Framework for Robust Adversarial Reinforcement Learningcited by 0
  16. 2026Safe Continual Reinforcement Learning in Non-stationary Environmentscited by 0
  17. 2026BAPR: Bayesian amnesic piecewise-robust reinforcement learning for non-stationary continuous controlcited by 0
  18. 2026Safe Continual Reinforcement Learning under Nonstationarity via Adaptive Safety Constraintscited by 0
  19. 2026Latent Q-Barrier Shielding for Safe In-Context Reinforcement Learningcited by 0
  20. 2026Adjustment Speed as a Safety Constraint for Nonstationary Reinforcement Learningcited by 0
  21. 2026Proactive Context-Forecasted Safety Constraints for Nonstationary Reinforcement Learningcited by 0