# SOLD: Slot Object-Centric Latent Dynamics Models for Relational Manipulation Learning from Pixels

arXiv: https://arxiv.org/abs/2410.08822 · HTML: https://mlsanity.com/paper/2410.08822/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Oct 2024.

## Built on by (most cited first, 24 of 24)

- [Causal-JEPA: Learning World Models through Object-Level Latent Masking](https://mlsanity.com/paper/2602.11389/) (2026)
- [OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulation](https://mlsanity.com/paper/2605.06481/) (2026)
- [Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modeling](https://mlsanity.com/paper/2603.04553/) (2026)
- [PlaySlot: Learning Inverse Latent Dynamics for Controllable Object-Centric Video Prediction and Planning](https://mlsanity.com/paper/2502.07600/) (2025)
- [PEEK: Guiding and Minimal Image Representations for Zero-Shot Generalization of Robot Manipulation Policies](https://mlsanity.com/paper/2509.18282/) (2025)
- [Hierarchical Entity-centric Reinforcement Learning with Factored Subgoal Diffusion](https://mlsanity.com/paper/2602.02722/) (2026)
- [Object-Centric World Models for Causality-Aware Reinforcement Learning](https://mlsanity.com/paper/2511.14262/) (2025)
- [From monoliths to modules: Decomposing transducers for efficient world modelling](https://mlsanity.com/paper/2512.02193/) (2025)
- [Physically Interpretable World Models via Weakly Supervised Representation Learning](https://mlsanity.com/paper/2412.12870/) (2024)
- [PH-Dreamer: A Physics-Driven World Model via Port-Hamiltonian Generative Dynamics](https://mlsanity.com/paper/2605.18303/) (2026)
- [Object-centric proto-symbolic behavioural reasoning from pixels](https://mlsanity.com/paper/2411.17438/) (2024)
- [Unifying Object-Centric World Models and Diffusion Policy: A Hierarchical Framework for Multi-Stage Robotic Tasks](https://mlsanity.com/paper/2606.08775/) (2026)
- [Dream-MPC: Gradient-Based Model Predictive Control with Latent Imagination](https://mlsanity.com/paper/2605.04568/) (2026)
- [World-as-Graph: Relational World Modeling Through Latent Space Graphs](https://mlsanity.com/paper/2609.38927/) (2026)
- [When Object-Centric World Models Meet Policy Learning: From Pixels to Policies, and Where It Breaks](https://mlsanity.com/paper/2511.06136/) (2025)
- [MetaWorld-X: Hierarchical World Modeling via VLM-Orchestrated Experts for Humanoid Loco-Manipulation](https://mlsanity.com/paper/2603.08572/) (2026)
- [GRAIL: Autonomous Concept Grounding for Neuro-Symbolic Reinforcement Learning](https://mlsanity.com/paper/2604.16871/) (2026)
- [Causal Object-Centric Models for Planning with Monte Carlo Tree Search](https://mlsanity.com/paper/2606.14418/) (2026)
- [BRICKS-WM: Building Reusability via Interface Composition Kinetics for Structured World Models](https://mlsanity.com/paper/2606.16489/) (2026)
- [Grounding Spatial Relations in a Compact World Model: Instruction Leakage and a Goal-Free Dynamics Fix](https://mlsanity.com/paper/2607.06925/) (2026)
- [Learning visual representations for compositional analysis of artworks and photographs](https://mlsanity.com/paper/2608.06142/) (2026)
- [TaskSense: Focusing on What Matters in World Models](https://mlsanity.com/paper/2608.06544/) (2026)
- [Weakly supervised concept Bottleneck Learning for Robust Two stage Object centric visual reasoning](https://mlsanity.com/paper/2608.22584/) (2026)
- [Beyond Static Graph World Models: Learning Stochastic Latent Dynamics over Evolving Topologies](https://mlsanity.com/paper/2609.28670/) (2026)

## Abstract (arXiv)

Learning a latent dynamics model provides a task-agnostic representation of an agent's understanding of its environment. Leveraging this knowledge for model-based reinforcement learning (RL) holds the potential to improve sample efficiency over model-free methods by learning from imagined rollouts. Furthermore, because the latent space serves as input to behavior models, the informative representations learned by the world model facilitate efficient learning of desired skills. Most existing methods rely on holistic representations of the environment's state. In contrast, humans reason about objects and their interactions, predicting how actions will affect specific parts of their surroundings. Inspired by this, we propose Slot-Attention for Object-centric Latent Dynamics (SOLD), a novel model-based RL algorithm that learns object-centric dynamics models in an unsupervised manner from pixel inputs. We demonstrate that the structured latent space not only improves model interpretability but also provides a valuable input space for behavior models to reason over. Our results show that SOLD outperforms DreamerV3 and TD-MPC2 - state-of-the-art model-based RL algorithms - across a range of benchmark robotic environments that require relational reasoning and manipulation capabilities. Videos are available at https://slot-latent-dynamics.github.io/.