mlsanityData to 7 Oct 2026

Learning with Imperfect Models: When Multi-step Prediction Mitigates Compounding Error

arXiv 2504.01766 ↗First posted 2 Apr 2025eess.SY cs.LG cs.SYcited by 5

Not on the map. Why?

Abstract (arXiv)

Compounding error, where small prediction mistakes accumulate over time, presents a major challenge in learning-based control. For example, this issue often limits the performance of model-based reinforcement learning and imitation learning. One common approach to mitigate compounding error is to train multi-step predictors directly, rather than relying on autoregressive rollout of a single-step model. However, it is not well understood when the benefits of multi-step prediction outweigh the added complexity of learning a more complicated model. In this work, we provide a rigorous analysis of this trade-off in the context of linear dynamical systems. We show that when the model class is well-specified and accurately captures the system dynamics, single-step models achieve lower asymptotic prediction error. On the other hand, when the model class is misspecified due to partial observability, direct multi-step predictors can significantly reduce bias and thus outperform single-step approaches. These theoretical results are supported by numerical experiments, wherein we also (a) empirically evaluate an intermediate strategy which trains a single-step model using a multi-step loss and (b) evaluate performance of single step and multi-step predictors in a closed loop control setting.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 15 of its 22 references that are in the map. First posted on arXiv 2 Apr 2025.

Builds on

15 of 22 references.

References · 15

  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  3. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  4. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  5. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  6. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  7. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 365
  8. 2024Reinforcement learningcited by 236
  9. 2019Combating the Compounding-Error Problem with a Multi-step Modelcited by 52
  10. 2022Investigating Compounding Prediction Errors in Learned Dynamics Modelscited by 47
  11. 2023Provable Guarantees for Generative Behavior Cloning: Bridging Low-Level Stability and High-Level Behaviorcited by 33
  12. 2021Evaluation of deep learning models for multi-step ahead time series predictioncited by 25
3 more
  1. 2020Learning Accurate Long-term Dynamics for Model-based Reinforcement Learningcited by 21
  2. 2022State space models vs. multi-step predictors in predictive control: Are state space models complicating safe data-driven designs?cited by 6
  3. 2024Stochastic Data-Driven Predictive Control: Chance-Constraint Satisfaction with Identified Multi-step Predictorscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2026WIMLE: Uncertainty-Aware World Models with IMLE for Sample-Efficient Continuous Controlcited by 3
  2. 2025Logarithmic Regret and Polynomial Scaling in Online Multi-step-ahead Predictioncited by 1
  3. 2026EXHOLD: Experience-Aware Real-Time Hold Control for Large-Scale Ride-Hailing Matching at DiDicited by 1
  4. 2026Flow map learning in nonlinear vector autoregressive models: influence of the feature-library structure on the training errorcited by 0
  5. 2026RTS Smoother-Guided Learning of Physics-Based Neural Differential Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024A Multi-step Loss Function for Robust Learning of the Dynamics in Model-based Reinforcement Learning
  2. 2024Any-step Dynamics Model Improves Future Predictions for Online and Offline Reinforcement Learning
  3. 2025A Model-Based Approach to Imitation Learning through Multi-Step Predictions
  4. 2023Multi-timestep models for Model-based Reinforcement Learning
  5. 2018Learning the Reward Function for a Misspecified Model
  6. 2025The Pitfalls of Imitation Learning when Actions are Continuous
  7. 2025Action Chunking and Exploratory Data Collection Yield Exponential Improvements in Behavior Cloning for Continuous Control
  8. 2023A Connection between One-Step Regularization and Critic Regularization in Reinforcement Learning