mlsanityData to 7 Oct 2026

Efficient World Models with Context-Aware Tokenization

arXiv 2406.19320 ↗First posted 27 Jun 2024cs.LG cs.AI cs.CVcited by 33

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Scaling up deep Reinforcement Learning (RL) methods presents a significant challenge. Following developments in generative modelling, model-based RL positions itself as a strong contender. Recent advances in sequence modelling have led to effective transformer-based world models, albeit at the price of heavy computations due to the long sequences of tokens required to accurately simulate environments. In this work, we propose Delta-IRIS, a new agent with a world model architecture composed of a discrete autoencoder that encodes stochastic deltas between time steps and an autoregressive transformer that predicts future deltas by summarizing the current state of the world with continuous tokens. In the Crafter benchmark, Delta-IRIS sets a new state of the art at multiple frame budgets, while being an order of magnitude faster to train than previous attention-based approaches. We release our code and models at https://github.com/vmicheli/delta-iris.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 37 of its 50 references that are in the map. First posted on arXiv 27 Jun 2024.

Builds on

37 of 50 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023STORM: Efficient Stochastic Transformer based World Models for Reinforcement Learning14 Oct
  2. 2023Mastering Diverse Domains through World Models10 Jan
  3. 2022Transformers are Sample-Efficient World Models1 Sep
  4. 2022TransDreamer: Reinforcement Learning with Transformer World Models19 Feb
  5. 2021Procedural Generalization by Planning with Self-Supervised World Models2 Nov
  6. 2021Mastering Atari Games with Limited Data30 Oct
  7. 2020Mastering Atari with Discrete World Models5 Oct
  8. 2020Planning to Explore via Self-Supervised World Models12 May
  9. 2019Dream to Control: Learning Behaviors by Latent Imagination3 Dec
  10. 2018Recurrent World Models Facilitate Policy Evolution4 Sep

Other references · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2017Neural Discrete Representation Learningcited by 4,650
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  7. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  8. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  9. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  10. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  11. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,221
  12. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
15 more
  1. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  2. 2018DeepMind Control Suitecited by 935
  3. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
  4. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  5. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  6. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  7. 2019Model-Based Reinforcement Learning for Ataricited by 495
  8. 2022Phenaki: Variable Length Video Generation From Open Domain Textual Descriptioncited by 475
  9. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 309
  10. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 193
  11. 2023Bigger, Better, Faster: Human-level Atari with human-level efficiencycited by 145
  12. 2018Improving Regression Performance with Distributional Lossescited by 66
  13. 2021Vector Quantized Models for Planningcited by 47
  14. 2022MineRL Diamond 2021 Competition: Overview, Results, and Lessons Learnedcited by 26
  15. 2023Curious Replay for Model-based Adaptationcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 17; 2026: 16.

33 papers.

Sort:

By area:

  1. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 237
  2. 2025A Comprehensive Survey on World Models for Embodied AIcited by 65
  3. 2025Learning Transformer-based World Models with Contrastive Predictive Codingcited by 20
  4. 2025FlowDreamer: A RGB-D World Model with Flow-based Motion Representations for Robot Manipulationcited by 17
  5. 2025NeuralOS: Towards Simulating Operating Systems via Neural Generative Modelscited by 17
  6. 2025Improving Transformer World Models for Data-Efficient RLcited by 13
  7. 2025Dyn-O: Building Structured World Models with Object-Centric Representationscited by 12
  8. 2026Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modelingcited by 11
  9. 2025DMWM: Dual-Mind World Model with Long-Term Imaginationcited by 10
  10. 2026R2-Dreamer: Redundancy-Reduced World Models without Decoders or Augmentationcited by 10
  11. 2025Object-Centric World Models from Few-Shot Annotations for Sample-Efficient Reinforcement Learningcited by 9
  12. 2026A Frame is Worth One Token: Efficient Generative World Modeling with Delta Tokenscited by 9
21 more
  1. 2025Accurate and Efficient World Modeling with Masked Latent Transformerscited by 7
  2. 2025Simulus: Combining Improvements in Sample-Efficient World Model Agentscited by 6
  3. 2026Planning in 8 Tokens: A Compact Discrete Tokenizer for Latent World Modelcited by 6
  4. 2026DDP-WM: Disentangled Dynamics Prediction for Efficient World Modelscited by 5
  5. 2025Curious Causality-Seeking Agents Learn Meta Causal Worldcited by 4
  6. 2025EDELINE: Enhancing Memory in Diffusion-based World Models via Linear-Time Sequence Modelingcited by 3
  7. 2026Morphology-Conditioned World Model for Cross-Embodiment Quadrupedal Locomotioncited by 3
  8. 2026World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applicationscited by 3
  9. 2025DyMoDreamer: World Modeling with Dynamic Modulationcited by 2
  10. 2026Pixels to Proofs: Probabilistically-Safe Latent World Model Control via Parallel Conformal Robust MPCcited by 2
  11. 2026Looped World Modelscited by 2
  12. 2025CrafterDojo: A Suite of Foundation Models for Building Open-Ended Embodied Agents in Craftercited by 1
  13. 2025Stochastic Siamese MAE Pretraining for Longitudinal Medical Imagescited by 1
  14. 2026Mixture-of-World Models: Scaling Multi-Task Reinforcement Learning with Modular Latent Dynamicscited by 1
  15. 2025Thinking agents for zero-shot generalization to qualitatively novel taskscited by 0
  16. 2025Can We Predict Your Next Move Without Breaking Your Privacy?cited by 0
  17. 2026Probing the Impact of Scale on Data-Efficient, Generalist Transformer World Models for Ataricited by 0
  18. 2026Transferable Delay-Aware Reinforcement Learning via Implicit Causal Graph Modelingcited by 0
  19. 2026The FIL Hypothesis: Inductive Biases Help with Kernel Engineeringcited by 0
  20. 2026World-Time Compute with Verified Code World Modelscited by 0
  21. 2026Beyond Static Graph World Models: Learning Stochastic Latent Dynamics over Evolving Topologiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Transformer-based World Models Are Happy With 100k Interactions
  2. 2026Token-World: World Modeling in Vision-Language Model Token Space for Robot Manipulation
  3. 2025RLVR-World: Training World Models with Reinforcement Learning
  4. 2026Identifiable Token Correspondence for World Models
  5. 2020Smaller World Models for Reinforcement Learning
  6. 2024Masked Generative Priors Improve World Models Sequence Modelling Capabilities
  7. 2024Pandora: Towards General World Model with Natural Language Actions and Video States
  8. 2024Improving Token-Based World Models with Parallel Observation Prediction