# Robust Control under Stationary Ambiguity

arXiv: https://arxiv.org/abs/2608.04832 · HTML: https://mlsanity.com/paper/2608.04832/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 88 references that are in the map.
- First posted on arXiv 5 Aug 2026.

## Builds on (most cited first, 20 of 20)

- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Designing and Interpreting Probes with Control Tasks](https://mlsanity.com/paper/1909.03368/) (2019)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [Learning Continuous Control Policies by Stochastic Value Gradients](https://mlsanity.com/paper/1510.09142/) (2015)
- [EPOpt: Learning Robust Neural Network Policies Using Model Ensembles](https://mlsanity.com/paper/1610.01283/) (2016)
- [Deep Hedging](https://mlsanity.com/paper/1802.03042/) (2018)
- [Robot Learning from Randomized Simulations: A Review](https://mlsanity.com/paper/2111.00956/) (2021)
- [Deep Learning Approximation for Stochastic Control Problems](https://mlsanity.com/paper/1611.07422/) (2016)
- [A Data-driven Market Simulator for Small Data Environments](https://mlsanity.com/paper/2006.14498/) (2020)
- [Multi-Period Trading via Convex Optimization](https://mlsanity.com/paper/1705.00109/) (2017)
- [Multi-Asset Spot and Option Market Simulation](https://mlsanity.com/paper/2112.06823/) (2021)
- [Gamma and Vega Hedging Using Deep Distributional Reinforcement Learning](https://mlsanity.com/paper/2205.05614/) (2022)
- [Robust Risk-Aware Option Hedging](https://mlsanity.com/paper/2303.15216/) (2023)
- [Fast Deep Hedging with Second-Order Optimization](https://mlsanity.com/paper/2410.22568/) (2024)
- [Model Uncertainty, Recalibration, and the Emergence of Delta-Vega Hedging](https://mlsanity.com/paper/1704.04524/) (2017)
- [Distributional Adversarial Attacks and Training in Deep Hedging](https://mlsanity.com/paper/2508.14757/) (2025)
- [Hedging with Small Uncertainty Aversion](https://mlsanity.com/paper/1605.06429/) (2016)
- [Uncertainty-Aware Strategies: A Model-Agnostic Framework for Robust Financial Optimization through Subsampling](https://mlsanity.com/paper/2506.07299/) (2025)
- [Generating Financial Time Series by Matching Random Convolutional Features](https://mlsanity.com/paper/2606.05138/) (2026)

## Abstract (arXiv)

Control policies optimized in simulation can perform poorly in the real system when the parameters x of the simulator are estimated from limited data but the resulting parameter uncertainty is not represented inside the simulation. A common way to incorporate such ambiguity is to simulate each trajectory of the system under a randomly drawn value for x. Since the policy cannot observe the drawn value, it must initially choose controls that perform well across many possible parameter values. However, if the policy progressively observes the system, it can often gradually infer the value of x, so that ambiguity vanishes. Over time, the policy then specializes to its estimate of x and loses its robustness. This is undesirable in many real systems, where latent factors are expected to shift. In financial markets, for example, a policy hedging a derivative payoff should remain robust to changes in the volatility regime. To induce such continual robustness, we propose training policies in simulators where ambiguity varies with the system's state but does not systematically decay over time. We formalize this requirement as stationary ambiguity: the simulator should induce a stationary filter process over the latent state. We show how to construct such simulators and demonstrate, on hedging problems, that policies trained under stationary ambiguity preserve robustness to latent factors over time, leading to strong performance on real market data. As a modeling principle, stationary ambiguity informs many simulator design decisions: which models make realistic simulators, how their parameters should be randomized, and how simulator and policy should be initialized. While our experiments focus on hedging, stationary ambiguity may also be useful for other sequential control problems driven by exogenous stochastic processes with shifting latent structure.