# Safe on Average, Unsafe in the Tail: When Is the Episodic-Cost Tail Controllable?

arXiv: https://arxiv.org/abs/2610.09508 · HTML: https://mlsanity.com/paper/2610.09508/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 2 of its 22 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 2 of 2)

- [Safe Distributional Reinforcement Learning](https://mlsanity.com/paper/2102.13446/) (2021)
- [Safe Langevin Soft Actor Critic](https://mlsanity.com/paper/2602.00587/) (2026)

## Abstract (arXiv)

Safe reinforcement learning seeks policies that maximize return while satisfying constraints on cumulative cost. Most methods impose these constraints on expected episodic cost. Consequently, standard evaluations report mean episodic cost without characterizing how cost is distributed across episodes. A policy that satisfies the mean-cost criterion may therefore remain unsafe in its worst episodes. Mean-cost reporting neither identifies this tail violation nor shows whether it can be brought within budget while preserving return. In this work, we measure the episodic-cost tail using CVaR0.1, the average cost of the worst 10% of episodes. We classify a policy as tail-safe when CVaR0.1 is within the safety budget. This allows us first to identify policies that are safe on average but unsafe in the tail and then to study whether their tail violations can be controlled while preserving return. To identify tail-unsafe policies, we evaluate five standard algorithms on three Safety-Gymnasium navigation tasks. We then examine four constraint families on dense-hazard navigation and assess tail control across four navigation and four locomotion tasks.