mlsanityData to 7 Oct 2026

Unsupervised Object-Based Transition Models for 3D Partially Observable Environments

arXiv 2103.04693 ↗First posted 8 Mar 2021cs.CV cs.AIcited by 19

Not on the map. Why?

Abstract (arXiv)

We present a slot-wise, object-based transition model that decomposes a scene into objects, aligns them (with respect to a slot-wise object memory) to maintain a consistent order across time, and predicts how those objects evolve over successive frames. The model is trained end-to-end without supervision using losses at the level of the object-structured representation rather than pixels. Thanks to its alignment module, the model deals properly with two issues that are not handled satisfactorily by other transition models, namely object persistence and object identity. We show that the combination of an object-level loss and correct object alignment over time enables the model to outperform a state-of-the-art baseline, and allows it to deal well with object occlusion and re-appearance in partially observable environments.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Builds on 24 of its 40 references that are in the map. First posted on arXiv 8 Mar 2021.

Builds on

24 of 40 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2017Non-local Neural Networkscited by 2,290
  4. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  5. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  6. 2018World Modelscited by 1,444
  7. 2017Deep Setscited by 1,394
  8. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 388
  9. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 359
  10. 2019Contrastive Learning of Structured World Modelscited by 206
  11. 2017Imagination-Augmented Agents for Deep Reinforcement Learningcited by 137
  12. 2018Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objectscited by 128
12 more
  1. 2018Learning to Decompose and Disentangle Representations for Video Predictioncited by 120
  2. 2019Entity Abstraction in Visual Model-Based Reinforcement Learningcited by 110
  3. 2018Relational Neural Expectation Maximization: Unsupervised Discovery of Objects and their Interactionscited by 100
  4. 2016Tagger: Deep Unsupervised Perceptual Groupingcited by 69
  5. 2020Human Instruction-Following with Deep Reinforcement Learning via Transfer-Learning from Textcited by 59
  6. 2019COBRA: Data-Efficient Model-Based RL through Unsupervised Object Discovery and Curiosity-Driven Explorationcited by 55
  7. 2019Environmental drivers of systematicity and generalization in a situated agentcited by 50
  8. 2020Object Files and Schemata: Factorizing Declarative and Procedural Knowledge in Dynamical Systemscited by 48
  9. 2016Towards Deep Symbolic Reinforcement Learningcited by 42
  10. 2018Tracking by Animation: Unsupervised Learning of Multi-Object Attentive Trackerscited by 22
  11. 2020Learning Object-Centric Video Models by Contrasting Setscited by 19
  12. 2020AlignNet: Unsupervised Entity Alignmentcited by 6

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 9; 2023: 3; 2024: 2; 2025: 2.

19 papers.

Sort:

By area:

  1. 2021Conditional Object-Centric Learning from Videocited by 247
  2. 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 146
  3. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  4. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  5. 2023SlotDiffusion: Object-Centric Generative Modeling with Diffusion Modelscited by 85
  6. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 82
  7. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
  8. 2022Neural Systematic Bindercited by 37
  9. 2021Generative Video Transformer: Can Objects be the Words?cited by 24
  10. 2022Toward Compositional Generalization in Object-Oriented World Modelingcited by 22
  11. 2022Abstraction for Deep Reinforcement Learningcited by 19
  12. 2023Object-Centric Video Prediction via Decoupling of Object Dynamics and Interactionscited by 19
7 more
  1. 2022Learning to Generalize with Object-centric Agents in the Open World Survival Game Craftercited by 13
  2. 2025PlaySlot: Learning Inverse Latent Dynamics for Controllable Object-Centric Video Prediction and Planningcited by 13
  3. 2024VONet: Unsupervised Video Object Learning With Parallel U-Net Attention and Object-wise Sequential VAEcited by 2
  4. 2022Towards Improving the Generation Quality of Autoregressive Slot VAEscited by 1
  5. 2024Learning Disentangled Representation in Object-Centric Models for Visual Dynamics Prediction via Transformerscited by 1
  6. 2025TextOCVP: Object-Centric Video Prediction with Language Guidancecited by 1
  7. 2022Efficient entity-based reinforcement learningcited by 0