mlsanityData to 14 Oct 2026

Tail-Safe Hedging: Explainable Risk-Sensitive Reinforcement Learning with a White-Box CBF--QP Safety Layer in Arbitrage-Free Markets

arXiv 2510.04555 ↗First posted 6 Oct 2025cs.LG q-fin.TRcited by 0

On the map: Deep reinforcement learning methods → Adversarial robustness and RL in finance

Abstract (arXiv)

We introduce Tail-Safe, a deployability-oriented framework for derivatives hedging that unifies distributional, risk-sensitive reinforcement learning with a white-box control-barrier-function (CBF) quadratic-program (QP) safety layer tailored to financial constraints. The learning component combines an IQN-based distributional critic with a CVaR objective (IQN--CVaR--PPO) and a Tail-Coverage Controller that regulates quantile sampling through temperature tilting and tail boosting to stabilize small-α estimation. The safety component enforces discrete-time CBF inequalities together with domain-specific constraints -- ellipsoidal no-trade bands, box and rate limits, and a sign-consistency gate -- solved as a convex QP whose telemetry (active sets, tightness, rate utilization, gate scores, slack, and solver status) forms an auditable trail for governance. We provide guarantees of robust forward invariance of the safe set under bounded model mismatch, a minimal-deviation projection interpretation of the QP, a KL-to-DRO upper bound linking per-state KL regularization to worst-case CVaR, concentration and sample-complexity results for the temperature-tilted CVaR estimator, and a CVaR trust-region improvement inequality under KL limits, together with feasibility persistence under expiry-aware tightening. Empirically, in arbitrage-free, microstructure-aware synthetic markets (SSVI → Dupire → VIX with ABIDES/MockLOB execution), Tail-Safe improves left-tail risk without degrading central performance and yields zero hard-constraint violations whenever the QP is feasible with zero slack. Telemetry is mapped to governance dashboards and incident workflows to support explainability and auditability. Limitations include reliance on synthetic data and simplified execution to isolate methodological contributions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Adversarial robustness and RL in finance, one of 15 directions in Deep reinforcement learning methods. Builds on 35 of its 73 references that are in the map. First posted on arXiv 6 Oct 2025.

Builds on

35 of 73 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024The Evolution of Reinforcement Learning in Quantitative Finance: A Survey20 Aug

Other references · 34

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Trust Region Policy Optimizationcited by 2,580
  3. 2021Convex optimizationcited by 2,515
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  6. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  7. 2017Constrained Policy Optimizationcited by 983
  8. 2018A Survey Of Methods For Explaining Black Box Modelscited by 973
  9. 2017A Distributional Perspective on Reinforcement Learningcited by 765
  10. 2017Deep Reinforcement Learning that Matterscited by 754
  11. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 599
  12. 2019Differentiable Convex Optimization Layerscited by 465
22 more
  1. 2017Robust Adversarial Reinforcement Learningcited by 451
  2. 2017Safe Reinforcement Learning via Shieldingcited by 449
  3. 2019Control Barrier Functions: Theory and Applicationscited by 372
  4. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 369
  5. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 337
  6. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 287
  7. 2018Safe Exploration in Continuous Action Spacescited by 247
  8. 2017OSQP: An Operator Splitting Solver for Quadratic Programscited by 161
  9. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 143
  10. 2021Recent Advances in Reinforcement Learning in Financecited by 110
  11. 2018Deep Hedgingcited by 108
  12. 2014Algorithms for CVaR Optimization in MDPscited by 99
  13. 2015Policy Gradient for Coherent Risk Measurescited by 76
  14. 2023State-wise Safe Reinforcement Learning: A Surveycited by 55
  15. 2021Robust Control Barrier-Value Functions for Safety-Critical Controlcited by 51
  16. 2021ABIDES-Gym: Gym Environments for Multi-Agent Discrete Event Simulation and Application to Financial Marketscited by 36
  17. 2019ABIDES: Towards High-Fidelity Market Simulation for AI Researchcited by 32
  18. 2016Robustness of Control Barrier Functions for Safety Critical Controlcited by 30
  19. 2023Distributionally Robust Model-based Reinforcement Learning with Large State Spacescited by 17
  20. 2022Differentiable Safe Controller Design through Control Barrier Functionscited by 8
  21. 2023Advances in the Theory of Control Barrier Functions: Addressing Practical Challenges in Safe Control Synthesis for Autonomous and Robotic Systemscited by 6
  22. 2022Disturbance Observer-based Robust Control Barrier Functionscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SteinGate: Tail-Sensitive Safe Reinforcement Learning via Stein Discrepancy
  2. 2025Risk-sensitive Reinforcement Learning Based on Convex Scoring Functions
  3. 2026Reinforcement Learning for Option Hedging: Static Implied-Volatility Fit versus Shortfall-Aware Performance
  4. 2025Learning to Hedge Swaptions
  5. 2026Autonomous AI Agents for Option Hedging: Enhancing Financial Stability through Shortfall Aware Reinforcement Learning
  6. 2024EX-DRL: Hedging Against Heavy Losses with EXtreme Distributional Reinforcement Learning
  7. 2026Reinforcement Learning for Risk-Sensitive Investment Management: a Free Energy--Entropy Duality Approach
  8. 2026Reward Redistribution for CVaR MDPs using a Bellman Operator on L-infinity