mlsanityData to 14 Oct 2026

Stable Multi-Step Rollouts via Uncertainty-Guided Hybrid Dynamics

arXiv 2608.16431 ↗First posted 17 Aug 2026eess.SY cs.SYcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Multi-step rollouts are essential for model-based reinforcement learning (RL) and predictive control, yet learned dynamics models often become unstable when recursively applied, leading to divergence and unreliable policy updates. This paper proposes a model-agnostic hybrid dynamics framework that blends a provably contracting nominal model with a flexible excursion model through an uncertainty-guided switching law. The switching signal is derived from calibrated epistemic uncertainty and activates only when the system leaves the nominal region, ensuring that each model operates within its reliability regime. Under clearly stated smoothness and boundedness assumptions, we show that the resulting hybrid predictor yields globally bounded recursive multi-step rollouts: trajectories remain Lyapunov-stable in the nominal region and exhibit at most affine growth during excursions. To illustrate the theory in practice, we instantiate the hybrid dynamics framework within a model-based RL scheme that uses real one-step transitions for value learning and hybrid rollouts for policy improvement. Experiments on a nonlinear Duffing oscillator demonstrate stable long-horizon prediction and improved cost-effort trade-offs relative to a stabilizing baseline.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 19 references that are in the map. First posted on arXiv 17 Aug 2026.

Builds on

12 of 19 references.

References · 12

  1. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  2. 2017Deep learning for universal linear embeddings of nonlinear dynamicscited by 494
  3. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  4. 2020Learning Stable Deep Dynamics Modelscited by 111
  5. 2022Koopman Neural Forecaster for Time Series with Temporal Distribution Shiftscited by 37
  6. 2023Multi-Agent Reachability Calibration with Conformal Predictioncited by 25
  7. 2023Plan To Predict: Learning an Uncertainty-Foreseeing Model for Model-Based Reinforcement Learningcited by 17
  8. 2023Learning Switching Port-Hamiltonian Systems with Uncertainty Quantificationcited by 3
  9. 2020Deep Learning based Uncertainty Decomposition for Real-time Controlcited by 2
  10. 2024Learning Piecewise Residuals of Control Barrier Functions for Safety of Switching Systems using Multi-Output Gaussian Processescited by 1
  11. 2025A Model-Based Approach to Imitation Learning through Multi-Step Predictionscited by 1
  12. 2026KIND: A Kalman-Inspired Adaptive Estimator for SRF Cavity Detuningcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023COPlanner: Plan to Roll Out Conservatively but to Explore Optimistically for Model-Based RL
  2. 2024Solving robust MDPs as a sequence of static RL problems
  3. 2021COMBO: Conservative Offline Model-Based Policy Optimization
  4. 2024The Edge-of-Reach Problem in Offline Model-Based Reinforcement Learning
  5. 2025Sample-efficient and Scalable Exploration in Continuous-Time RL
  6. 2025Learning with Imperfect Models: When Multi-step Prediction Mitigates Compounding Error
  7. 2026Adaptive Rollout Truncation Based on Epistemic Uncertainty for Efficient Offline World Model Training
  8. 2024A Multi-step Loss Function for Robust Learning of the Dynamics in Model-based Reinforcement Learning