mlsanityData to 14 Oct 2026

Compete and Compose: Learning Independent Mechanisms for Modular World Models

arXiv 2404.15109 ↗First posted 23 Apr 2024cs.LGcited by 4

On the map: World models → World models for language agents

Abstract (arXiv)

We present COmpetitive Mechanisms for Efficient Transfer (COMET), a modular world model which leverages reusable, independent mechanisms across different environments. COMET is trained on multiple environments with varying dynamics via a two-step process: competition and composition. This enables the model to recognise and learn transferable mechanisms. Specifically, in the competition phase, COMET is trained with a winner-takes-all gradient allocation, encouraging the emergence of independent mechanisms. These are then re-used in the composition phase, where COMET learns to re-compose learnt mechanisms in ways that capture the dynamics of intervened environments. In so doing, COMET explicitly reuses prior knowledge, enabling efficient and interpretable adaptation. We evaluate COMET on environments with image-based observations. In contrast to competitive baselines, we demonstrate that COMET captures recognisable mechanisms without supervision. Moreover, we show that COMET is able to adapt to new environments with varying numbers of objects with improved sample efficiency compared to more conventional finetuning approaches.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 38 of its 50 references that are in the map. First posted on arXiv 23 Apr 2024.

Builds on

38 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Variational Causal Dynamics: Discovering Modular World Models from Interventions22 Jun

Other references · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  4. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  5. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  6. 2020Object-Centric Learning with Slot Attentioncited by 876
  7. 2018Recurrent World Models Facilitate Policy Evolutioncited by 785
  8. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
  9. 2016Expert Gate: Lifelong Learning with a Network of Expertscited by 372
  10. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 360
  11. 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 275
  12. 2019Recurrent Independent Mechanismscited by 219
25 more
  1. 2019Contrastive Learning of Structured World Modelscited by 209
  2. 2021Towards Causal Representation Learningcited by 209
  3. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  4. 2022SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videoscited by 166
  5. 2022CITRIS: Causal Identifiability from Temporal Intervened Sequencescited by 133
  6. 2018Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objectscited by 128
  7. 2019Entity Abstraction in Visual Model-Based Reinforcement Learningcited by 110
  8. 2021GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinementcited by 109
  9. 2019SCALOR: Generative World Models with Scalable Object Representationscited by 108
  10. 2017Learning Independent Causal Mechanismscited by 105
  11. 2018Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactionscited by 100
  12. 2019High Fidelity Video Prediction with Large Stochastic Recurrent Neural Networkscited by 88
  13. 2021Neural Production Systems: Learning Rule-Governed Visual Dynamicscited by 75
  14. 2020Improving Generative Imagination in Object-Centric World Modelscited by 71
  15. 2021Continual Learning via Local Module Compositioncited by 63
  16. 2019COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Explorationcited by 57
  17. 2020Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systemscited by 48
  18. 2021AdaRL: What, Where, and How to Adapt in Transfer Reinforcement Learningcited by 48
  19. 2020Towards causal generative scene models via competition of expertscited by 37
  20. 2022Curious Exploration via Structured World Models Yields Zero-Shot Object Manipulationcited by 31
  21. 2018Relational Forward Models for Multi-Agent Learningcited by 21
  22. 2021Structured World Belief for Reinforcement Learning in POMDPcited by 21
  23. 2019Switching Linear Dynamics for Variational Bayes Filteringcited by 17
  24. 2021APEX: Unsupervised, Object-Centric Scene Segmentation and Tracking for Robot Manipulationcited by 16
  25. 2021GATSBI: Generative Agent-centric Spatio-temporal Object Interactioncited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Dreamweaver: Learning Compositional World Models from Pixelscited by 4
  2. 2026From Observation to Insight: Mechanistic World Models and the Quest for Autonomous Discoverycited by 4
  3. 2026The Thinking Pixel: Recursive Sparse Reasoning in Multimodal Diffusion Latentscited by 2
  4. 2026BRICKS-WM: Building Reusability via Interface Composition Kinetics for Structured World Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026verdi: retrieval is not transfer for continual world model optimization
  2. 2026Benchmarking World Models for Continual Learning on Compositional Tasks
  3. 2025Context and Diversity Matter: The Emergence of In-Context Learning in World Models
  4. 2026Learning Transferable Dynamics Priors from Action to World Modeling
  5. 2026UniW(A)M: A Unified Mixture-of-Experts Framework for World and World-Action Models
  6. 2026OneWorld: Learning Consistent Physics Across Actions in World Models
  7. 2026Sandwich-Residuals: Parameter-Efficient Test-time Adaptation of World Models
  8. 2025AdaWorld: Learning Adaptable World Models with Latent Actions