mlsanityData to 7 Oct 2026

Slot State Space Models

arXiv 2406.12272 ↗First posted 18 Jun 2024cs.AIcited by 17

On the map: State-space sequence models (S4, Mamba) → Core SSM architectures for language

Abstract (arXiv)

Recent State Space Models (SSMs) such as S4, S5, and Mamba have shown remarkable computational benefits in long-range temporal dependency modeling. However, in many sequence modeling problems, the underlying process is inherently modular and it is of interest to have inductive biases that mimic this modular structure. In this paper, we introduce SlotSSMs, a novel framework for incorporating independent mechanisms into SSMs to preserve or encourage separation of information. Unlike conventional SSMs that maintain a monolithic state vector, SlotSSMs maintains the state as a collection of multiple vectors called slots. Crucially, the state transitions are performed independently per slot with sparse interactions across slots implemented via the bottleneck of self-attention. In experiments, we evaluate our model in object-centric learning, 3D visual reasoning, and long-context video understanding tasks, which involve modeling multiple objects and their long-range temporal dependencies. We find that our proposed design offers substantial performance gains over existing sequence modeling methods. Project page is available at https://slotssms.github.io/

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Core SSM architectures for language, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 66 of its 74 references that are in the map. First posted on arXiv 18 Jun 2024.

Builds on

66 of 74 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spaces1 Dec
  2. 2023Structured State Space Models for In-Context Reinforcement Learning7 Mar
  3. 2022Simplified State Space Layers for Sequence Modeling9 Aug
  4. 2022Diagonal State Spaces are as Effective as Structured State Spaces27 Mar
  5. 2021Efficiently Modeling Long Sequences with Structured State Spaces31 Oct

Other references · 61

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2016Layer Normalizationcited by 5,579
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  5. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  6. 2022BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformerscited by 1,213
  7. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,051
  8. 2020Object-Centric Learning with Slot Attentioncited by 865
  9. 2020Long Range Arena: A Benchmark for Efficient Transformerscited by 696
  10. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
  11. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  12. 2020HiPPO: Recurrent Memory with Optimal Polynomial Projectionscited by 600
38 more
  1. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 495
  2. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 418
  3. 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 397
  4. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  5. 2022Kubric: A scalable dataset generatorcited by 373
  6. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  7. 2022Long Range Language Modeling via Gated State Spacescited by 265
  8. 2021Conditional Object-Centric Learning from Videocited by 247
  9. 2024Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Modelscited by 232
  10. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  11. 2019Recurrent Independent Mechanismscited by 219
  12. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 219
  13. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  14. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  15. 2022It's Raw! Audio Generation with State-Space Modelscited by 171
  16. 2022SAVi++: Towards End-to-End Object-Centric Learning from Real-World Videoscited by 165
  17. 2021Illiterate DALL-E Learns to Composecited by 147
  18. 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 146
  19. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  20. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  21. 2019CATER: A diagnostic dataset for Compositional Actions and TEmporal Reasoningcited by 137
  22. 2018Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objectscited by 128
  23. 2017Neural Expectation Maximizationcited by 123
  24. 2019Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEscited by 116
  25. 2021GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinementcited by 109
  26. 2019SCALOR: Generative World Models with Scalable Object Representationscited by 108
  27. 2022Long Movie Clip Classification with State-Space Video Modelscited by 106
  28. 2023Selective Structured State-Spaces for Long-Form Video Understandingcited by 105
  29. 2018Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactionscited by 100
  30. 2023Object-Centric Slot Diffusioncited by 90
  31. 2023SlotDiffusion: Object-Centric Generative Modeling with Diffusion Modelscited by 85
  32. 2020Improving Generative Imagination in Object-Centric World Modelscited by 70
  33. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
  34. 2022Improving Object-centric Learning with Query Optimizationcited by 60
  35. 2020Generative Neurosymbolic Machinescited by 57
  36. 2019Exploiting Spatial Invariance for Scalable Unsupervised Object Trackingcited by 40
  37. 2024Mastering Memory Tasks with World Modelscited by 38
  38. 2020Towards causal generative scene models via competition of expertscited by 37

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 11; 2026: 4.

17 papers.

Sort:

By area:

  1. 2025Dyn-O: Building Structured World Models with Object-Centric Representationscited by 12
  2. 2025Learning Interactive World Model for Object-Centric Reinforcement Learningcited by 11
  3. 2025SlotVLA: Towards Modeling of Object-Relation Representations in Robotic Manipulationcited by 9
  4. 2024FACTS: A Factored State-Space Framework For World Modellingcited by 4
  5. 2025Dreamweaver: Learning Compositional World Models from Pixelscited by 4
  6. 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challengescited by 4
  7. 2025AXIOM: Learning to Play Games in Minutes with Expanding Object-Centric Modelscited by 3
  8. 2026Disentangling Dynamical Systems: Causal Representation Learning Meets Local Sparse Attentioncited by 3
  9. 2025Learning Local Causal World Models with State Space Models and Attentioncited by 2
  10. 2025Discrete JEPA: Learning Discrete Token Representations without Reconstructioncited by 2
  11. 2024Advancing Efficient Brain Tumor Multi-Class Classification -- New Insights from the Vision Mamba Model in Transfer Learningcited by 1
  12. 2025Neural Force Field: Few-shot Learning of Generalized Physical Reasoningcited by 1
5 more
  1. 2025UNO: Unifying One-stage Video Scene Graph Generation via Object-Centric Visual Representation Learningcited by 1
  2. 2025Context-Selective State Space Models: Feedback is All You Needcited by 1
  3. 2026HCLSM: Hierarchical Causal Latent State Machines for Object-Centric World Modelingcited by 0
  4. 2026BRICKS-WM: Building Reusability via Interface Composition Kinetics for Structured World Modelscited by 0
  5. 2026iSEE: Object Permanence Through Self-Supervisioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Slot-VLM: SlowFast Slots for Video-Language Modeling
  2. 2025Long-Context State-Space Video World Models
  3. 2024Slot Structured World Models
  4. 2024Reasoning-Enhanced Object-Centric Learning for Videos
  5. 2026SlotVTG: Object-Centric Adapter for Generalizable Video Temporal Grounding
  6. 2025StateSpaceDiffuser: Bringing Long Context to Diffusion World Models
  7. 2026MuonSSM: Orthogonalizing State Space Models for Sequence Modeling
  8. 2023Convolutional State Space Models for Long-Range Spatiotemporal Modeling