# Stable Multi-Step Rollouts via Uncertainty-Guided Hybrid Dynamics

arXiv: https://arxiv.org/abs/2608.16431 · HTML: https://mlsanity.com/paper/2608.16431/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 17 Aug 2026.

## Placed in

- Deep reinforcement learning methods / [RL theory and sample complexity](https://mlsanity.com/deeprl/rl-theory-and-sample-complexity/)

## Builds on (most cited first, 12 of 12)

- [When to Trust Your Model: Model-Based Policy Optimization](https://mlsanity.com/paper/1906.08253/) (2019)
- [Deep learning for universal linear embeddings of nonlinear dynamics](https://mlsanity.com/paper/1712.09707/) (2017)
- [Safe Model-based Reinforcement Learning with Stability Guarantees](https://mlsanity.com/paper/1705.08551/) (2017)
- [Learning Stable Deep Dynamics Models](https://mlsanity.com/paper/2001.06116/) (2020)
- [Koopman Neural Forecaster for Time Series with Temporal Distribution Shifts](https://mlsanity.com/paper/2210.03675/) (2022)
- [Multi-Agent Reachability Calibration with Conformal Prediction](https://mlsanity.com/paper/2304.00432/) (2023)
- [Plan To Predict: Learning an Uncertainty-Foreseeing Model for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2301.08502/) (2023)
- [Learning Switching Port-Hamiltonian Systems with Uncertainty Quantification](https://mlsanity.com/paper/2305.09689/) (2023)
- [Deep Learning based Uncertainty Decomposition for Real-time Control](https://mlsanity.com/paper/2010.02613/) (2020)
- [Learning Piecewise Residuals of Control Barrier Functions for Safety of Switching Systems using Multi-Output Gaussian Processes](https://mlsanity.com/paper/2403.18041/) (2024)
- [A Model-Based Approach to Imitation Learning through Multi-Step Predictions](https://mlsanity.com/paper/2504.13413/) (2025)
- [KIND: A Kalman-Inspired Adaptive Estimator for SRF Cavity Detuning](https://mlsanity.com/paper/2605.11957/) (2026)

## Abstract (arXiv)

Multi-step rollouts are essential for model-based reinforcement learning (RL) and predictive control, yet learned dynamics models often become unstable when recursively applied, leading to divergence and unreliable policy updates. This paper proposes a model-agnostic hybrid dynamics framework that blends a provably contracting nominal model with a flexible excursion model through an uncertainty-guided switching law. The switching signal is derived from calibrated epistemic uncertainty and activates only when the system leaves the nominal region, ensuring that each model operates within its reliability regime. Under clearly stated smoothness and boundedness assumptions, we show that the resulting hybrid predictor yields globally bounded recursive multi-step rollouts: trajectories remain Lyapunov-stable in the nominal region and exhibit at most affine growth during excursions. To illustrate the theory in practice, we instantiate the hybrid dynamics framework within a model-based RL scheme that uses real one-step transitions for value learning and hybrid rollouts for policy improvement. Experiments on a nonlinear Duffing oscillator demonstrate stable long-horizon prediction and improved cost-effort trade-offs relative to a stabilizing baseline.