mlsanityData to 7 Oct 2026

Bisimulation metric for Model Predictive Control

arXiv 2410.04553 ↗First posted 6 Oct 2024cs.LG cs.SY eess.SYcited by 6

Not on the map. Why?

Abstract (arXiv)

Model-based reinforcement learning has shown promise for improving sample efficiency and decision-making in complex environments. However, existing methods face challenges in training stability, robustness to noise, and computational efficiency. In this paper, we propose Bisimulation Metric for Model Predictive Control (BS-MPC), a novel approach that incorporates bisimulation metric loss in its objective function to directly optimize the encoder. This time-step-wise direct optimization enables the learned encoder to extract intrinsic information from the original state space while discarding irrelevant details and preventing the gradients and errors from diverging. BS-MPC improves training stability, robustness against input noise, and computational efficiency by reducing training time. We evaluate BS-MPC on both continuous control and image-based tasks from the DeepMind Control Suite, demonstrating superior performance and robustness compared to state-of-the-art baseline methods.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 33 of its 48 references that are in the map. First posted on arXiv 6 Oct 2024.

Builds on

33 of 48 references.

References · 33

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2017The Kinetics Human Action Video Datasetcited by 1,894
  6. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  7. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  8. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  9. 2023Mastering Diverse Domains through World Modelscited by 1,193
  10. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  11. 2020Mastering Atari with Discrete World Modelscited by 990
  12. 2018DeepMind Control Suitecited by 935
21 more
  1. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  2. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 753
  3. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  4. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  5. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  6. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  7. 2020Learning Invariant Representations for Reinforcement Learning without Reconstructioncited by 424
  8. 2022Temporal Difference Learning for Model Predictive Controlcited by 375
  9. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
  10. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 228
  11. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  12. 2018Learning to Walk via Deep Reinforcement Learningcited by 153
  13. 2018Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guaranteescited by 138
  14. 2019Scalable methods for computing state similarity in deterministic Markov Decision Processescited by 133
  15. 2019Exploring Model-based Planning with Policy Networkscited by 90
  16. 2017Information Theoretic Model Predictive Control: Theory and Applications to Autonomous Drivingcited by 81
  17. 2023Finetuning Offline World Models in the Real Worldcited by 39
  18. 2018Natural Environment Benchmarks for Reinforcement Learningcited by 32
  19. 2023MoDem-V2: Visuo-Motor World Models for Real-World Robot Manipulationcited by 27
  20. 2023Simplified Temporal Consistency Reinforcement Learningcited by 17
  21. 2023IQL-TD-MPC: Implicit Q-Learning for Hierarchical Model Predictive Controlcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 2; 2026: 4.

6 papers.

  1. 2026Learning Invariant Visual Representations for Planning with Joint-Embedding Predictive World Modelscited by 11
  2. 2025Offline Goal-Conditioned Reinforcement Learning with Projective Quasimetric Planningcited by 5
  3. 2026Diagnosing JEPA World Models with Action-Conditioned Predictive Consistencycited by 2
  4. 2025A Theoretical Analysis of State Similarity Between Markov Decision Processescited by 0
  5. 2026Adaptive Linear Path Model-Based Diffusioncited by 0
  6. 2026Scaling World-Model Reinforcement Learning Through Diffusion Policy Optimizationcited by 0