HCLSM: Hierarchical Causal Latent State Machines for Object-Centric World Modeling
On the map: World models → Latent predictive world models
Abstract (arXiv)
World models that predict future states from video remain limited by flat latent representations that entangle objects, ignore causal structure, and collapse temporal dynamics into a single scale. We present HCLSM, a world model architecture that operates on three interconnected principles: object-centric decomposition via slot attention with spatial broadcast decoding, hierarchical temporal dynamics through a three-level engine combining selective state space models for continuous physics, sparse transformers for discrete events, and compressed transformers for abstract goals, and causal structure learning through graph neural network interaction patterns. HCLSM introduces a two-stage training protocol where spatial reconstruction forces slot specialization before dynamics prediction begins. We train a 68M-parameter model on the PushT robotic manipulation benchmark from the Open X-Embodiment dataset, achieving 0.008 MSE next-state prediction loss with emerging spatial decomposition (SBD loss: 0.0075) and learned event boundaries. A custom Triton kernel for the SSM scan delivers 38x speedup over sequential PyTorch. The full system spans 8,478 lines of Python across 51 modules with 171 unit tests. Code: https://github.com/rightnow-ai/hclsm
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 18 of its 19 references that are in the map. First posted on arXiv 31 Mar 2026.
Builds on
18 of 19 references.
Closest ancestors · 1
Same direction, newest first
Other references · 17
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2020Object-Centric Learning with Slot Attentioncited by 865
- 2018DAGs with NO TEARS: Continuous Optimization for Structure Learningcited by 745
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
- 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
- 2019DAG-GNN: DAG Structure Learning with Graph Neural Networkscited by 329
- 2022Bridging the Gap to Real-World Object-Centric Learningcited by 219
- 2021Towards Causal Representation Learningcited by 207
5 more
- 2022SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videoscited by 165
- 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 146
- 2024Adaptive Slot Attention: Object Discovery with Dynamic Slot Numbercited by 34
- 2024Slot State Space Modelscited by 17
- 2025MetaSlot: Break Through the Fixed Number of Slots in Object-Centric Learningcited by 11
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026DexWorldModel: Causal Latent World Modeling towards Automated Learning of Embodied Tasks
- 2026MOSH-WM: Mask-Grounded Soft-Hamiltonian Dynamics for Object-Centric World Models
- 2025Learning Local Causal World Models with State Space Models and Attention
- 2026DDP-WM: Disentangled Dynamics Prediction for Efficient World Models
- 2026Event-Conditioned Diagnostics of Kinematic, Contact, and Object-Permanence Structure in Passive Object-State World Models
- 2026From World Models to World Action Models: Rethinking Next-State Prediction
- 2026LaWM: Least Action World Models for Long-Horizon Physical Consistency from Visual Observations
- 2026OpenWAM: An Open Framework for Composable World-Action Models