mlsanityData to 14 Oct 2026

A Generalist Dynamics Model for Control

arXiv 2305.10912 ↗First posted 18 May 2023cs.AI cs.ROcited by 24

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

We investigate the use of transformer sequence models as dynamics models (TDMs) for control. We find that TDMs exhibit strong generalization capabilities to unseen environments, both in a few-shot setting, where a generalist TDM is fine-tuned with small amounts of data from the target environment, and in a zero-shot setting, where a generalist TDM is applied to an unseen environment without any further training. Here, we demonstrate that generalizing system dynamics can work much better than generalizing optimal behavior directly as a policy. Additional results show that TDMs also perform well in a single-environment learning setting when compared to a number of baseline models. These properties make TDMs a promising ingredient for a foundation model of control.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 44 of its 61 references that are in the map. First posted on arXiv 18 May 2023.

Builds on

44 of 61 references.

Closest ancestors · 8

Same direction, newest first

  1. 2023Transformer-based World Models Are Happy With 100k Interactions13 Mar
  2. 2022Transformers are Sample-Efficient World Models1 Sep
  3. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problem3 Jun
  4. 2020Mastering Atari with Discrete World Models5 Oct
  5. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  6. 2018Recurrent World Models Facilitate Policy Evolution4 Sep
  7. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models30 May
  8. 2018World Models27 Mar

Other references · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  5. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  6. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  7. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  8. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
  9. 2018Relational inductive biases, deep learning, and graph networkscited by 1,360
  10. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  11. 2018DeepMind Control Suitecited by 956
  12. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
24 more
  1. 2022A Generalist Agentcited by 808
  2. 2018Learning Dexterous In-Hand Manipulationcited by 687
  3. 2019Model-Based Reinforcement Learning for Ataricited by 502
  4. 2020dm_control: Software and Tasks for Continuous Controlcited by 375
  5. 2018Maximum a Posteriori Policy Optimisationcited by 328
  6. 2015Embed to Control: A Locally Linear Latent Dynamics Model for Control from Raw Imagescited by 274
  7. 2019Stabilizing Transformers for Reinforcement Learningcited by 270
  8. 2018Graph networks as learnable physics engines for inference and controlcited by 244
  9. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
  10. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 164
  11. 2015Learning Continuous Control Policies by Stochastic Value Gradientscited by 162
  12. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
  13. 2020One Policy to Control Them All: Shared Modular Policies for Agent-Agnostic Controlcited by 121
  14. 2022MetaMorph: Learning Universal Controllers with Transformerscited by 89
  15. 2020My Body is a Cage: the Role of Morphology in Graph-Based Incompatible Controlcited by 70
  16. 2022Efficient Planning in a Compact Latent Action Spacecited by 57
  17. 2023SMART: Self-supervised Multi-task pretrAining with contRol Transformerscited by 44
  18. 2021Learning Dynamics Models for Model Predictive Agentscited by 22
  19. 2022Planning for Sample Efficient Imitation Learningcited by 20
  20. 2021Evaluating model-based planning and planner amortization for continuous controlcited by 15
  21. 2022A System for Morphology-Task Generalization via Unified Representation and Behavior Distillationcited by 13
  22. 2023Predictable MDP Abstraction for Unsupervised Model-Based RLcited by 8
  23. 2021Snowflake: Scaling GNNs to High-Dimensional Continuous Control via Parameter Freezingcited by 7
  24. 2023Leveraging Jumpy Models for Planning and Fast Learning in Robotic Domainscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 11; 2025: 6; 2026: 2.

24 papers.

Sort:

By area:

  1. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  2. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 347
  3. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  4. 2024Bigger, Regularized, Optimistic: scaling for compute and sample-efficient continuous controlcited by 108
  5. 2024Real-World Robot Applications of Foundation Models: A Reviewcited by 61
  6. 2025RLVR-World: Training World Models with Reinforcement Learningcited by 40
  7. 2023Model-Based Runtime Monitoring with Interactive Imitation Learningcited by 38
  8. 2024Diffusion Model Predictive Controlcited by 36
  9. 2023World Models via Policy-Guided Trajectory Diffusioncited by 34
  10. 2023Dynamics Generalisation in Reinforcement Learning via Adaptive Context-Aware Policiescited by 20
  11. 2025Bootstrapped Model Predictive Controlcited by 16
  12. 2025Learning Massively Multitask World Models for Continuous Controlcited by 12
12 more
  1. 2023H-GAP: Humanoid Control with a Generalist Plannercited by 11
  2. 2024WHALE: Towards Generalizable and Scalable World Models for Embodied Decision-makingcited by 9
  3. 2024Chameleon: A Data-Efficient Generalist for Dense Visual Prediction in the Wildcited by 6
  4. 2025Simulus: Combining Improvements in Sample-Efficient World Model Agentscited by 6
  5. 2024QT-TDM: Planning With Transformer Dynamics Model and Autoregressive Q-Learningcited by 4
  6. 2024On Foundation Models for Dynamical Systems from Purely Synthetic Datacited by 4
  7. 2024Meta-Controller: Few-Shot Imitation of Unseen Embodiments and Tasks in Continuous Controlcited by 4
  8. 2025Trajectory World Models for Heterogeneous Environmentscited by 4
  9. 2025Adversarial Diffusion for Robust Reinforcement Learningcited by 4
  10. 2024Learning Off-policy with Model-based Intrinsic Motivation For Active Online Explorationcited by 1
  11. 2026WestWorld: A Knowledge-Encoded Scalable Trajectory World Model for Diverse Robotic Systemscited by 1
  12. 2026Advantage-Guided Diffusion for Model-Based Reinforcement Learningcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Decision Transformer as a Foundation Model for Partially Observable Continuous Control
  2. 2026Generalist AI Control: Towards Multi-purpose Adaptive Algorithms
  3. 2023ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioning
  4. 2026Dual-Axis Policy Optimization for LLM Agents: Bayesian Feedback Attribution and Trajectory Mass Normalization
  5. 2025Much Ado About Noising: Dispelling the Myths of Generative Robotic Control
  6. 2024PWM: Policy Learning with Multi-Task World Models
  7. 2026On Sample-Efficient Generalized Planning via Learned Transition Models
  8. 2026From Gradients to Riccati Geometry: Kalman World Models for Single-Pass Learning