Latent Q-Barrier Shielding for Safe In-Context Reinforcement Learning

arXiv 2605.25267 ↗First posted 24 May 2026cs.LG cs.AIcited by 1

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Safe in-context reinforcement learning (ICRL) adapts online from interaction history without test-time parameter updates while controlling episode cost under a safety budget. Under out-of-distribution (OOD) deployment shifts, pretraining-only safe ICRL can give poor reward-safety tradeoffs because the remaining budget affects behavior only through frozen policy conditioning, not an explicit action-level check against predicted future cost. We propose a latent Q-Barrier shield that learns a context representation, latent dynamics, and an ensemble cost critic before deployment. Without parameter updates, the shield infers context from history and filters or softly reweights candidate actions using the remaining budget and predicted future cost. We prove a conditional, error-decomposed barrier-margin result: a Q-Barrier-satisfying action leaves the next latent-budget state with an approximately budget-safe continuation under the learned critic, up to Bellman and latent-prediction errors. Across five safe ICRL benchmarks, the shield improves deployment-time reward-safety tradeoffs over a strong safe-ICRL baseline: after a short context window, it achieves higher return in four of five benchmarks while matching or lowering average episode cost in all five.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 55 of its 70 references that are in the map. First posted on arXiv 24 May 2026.

Builds on

55 of 70 references.

Closest ancestors · 3

Same direction, newest first

Other references · 52

2017Proximal Policy Optimization Algorithmscited by 15,215
2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
2017Constrained Policy Optimizationcited by 983
2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
2016RL2: Fast Reinforcement Learning via Slow Reinforcement Learningcited by 503
2017Safe Reinforcement Learning via Shieldingcited by 449
2016Reinforcement Learning with Unsupervised Auxiliary Taskscited by 404
2016Learning to reinforcement learncited by 377
2020Projection-Based Constrained Policy Optimizationcited by 203
2022In-context Reinforcement Learning with Algorithm Distillationcited by 190
2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 170

38 more

2022Prompting Decision Transformer for Few-Shot Policy Generalizationcited by 146
2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 143
2023Supervised Pretraining Can Learn In-Context Reinforcement Learningcited by 139
2022Safe Planning in Dynamic Environments using Conformal Predictioncited by 119
2020Safe Reinforcement Learning in Constrained Markov Decision Processescited by 115
2021Pretraining Representations for Data-Efficient Reinforcement Learningcited by 105
2024Can large language models explore in-context?cited by 76
2020Reinforcement Learning for Safety-Critical Control under Model Uncertainty, using Control Lyapunov Functions and Control Barrier Functionscited by 68
2022Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environmentscited by 60
2022Constrained Update Projection Approach to Safe Policy Optimizationcited by 58
2022Safe Reinforcement Learning via Shielding under Partial Observabilitycited by 56
2022Contextualize Me -- The Case for Context in Reinforcement Learningcited by 46
2025Generalizing Safety Beyond Collision-Avoidance via Latent-Space Reachability Analysiscited by 45
2023Safe Reinforcement Learning via Probabilistic Logic Shieldscited by 42
2023Generalization to New Sequential Decision Making Tasks with In-Context Learningcited by 39
2025A Survey of In-Context Reinforcement Learningcited by 39
2023Iterative Reachability Estimation for Safe Reinforcement Learningcited by 37
2021Context-Aware Safe Reinforcement Learning for Non-Stationary Environmentscited by 33
2025Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failurescited by 33
2023In-Context Reinforcement Learning for Variable Action Spacescited by 24
2024Transformers Can Learn Temporal Difference Methods for In-Context Reinforcement Learningcited by 23
2019Single Episode Policy Transfer in Reinforcement Learningcited by 18
2024Zero-Shot Reinforcement Learning via Function Encoderscited by 18
2023Simplified Temporal Consistency Reinforcement Learningcited by 17
2024In-context Exploration-Exploitation for Reinforcement Learningcited by 16
2024Dreaming of Many Worlds: Learning Contextual World Models Aids Zero-Shot Generalizationcited by 16
2022An Adaptive Deep RL Method for Non-Stationary Environments with Piecewise Stable Contextcited by 15
2024AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformerscited by 15
2025Distilling Reinforcement Learning Algorithms for In-Context Model-Based Planningcited by 13
2024A CMDP-within-online framework for Meta-Safe Reinforcement Learningcited by 12
2022Safe and Efficient Reinforcement Learning Using Disturbance-Observer-Based Control Barrier Functionscited by 11
2024Zero-Shot Transfer of Neural ODEscited by 10
2025Vintix: Action Model via In-Context Reinforcement Learningcited by 10
2021MESA: Offline Meta-RL for Safe Adaptation and Fault Tolerancecited by 8
2023Safe POMDP Online Planning via Shieldingcited by 8
2024Safe POMDP Online Planning among Dynamic Agents via Adaptive Conformal Predictioncited by 8
2025AnySafe: Adapting Latent Safety Filters at Runtime via Safety Constraint Parameterization in the Latent Spacecited by 7
2025Scalable In-Context Q-Learningcited by 5

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

2026In-Context Reinforcement Learning under Non-Stationarity: A Surveycited by 1

Similar, not cited

Close in wording, no citation either way