mlsanityData to 14 Oct 2026

SlotDiffusion: Object-Centric Generative Modeling with Diffusion Models

arXiv 2305.11281 ↗First posted 18 May 2023cs.CV cs.LGcited by 85

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Object-centric learning aims to represent visual data with a set of object entities (a.k.a. slots), providing structured representations that enable systematic generalization. Leveraging advanced architectures like Transformers, recent approaches have made significant progress in unsupervised object discovery. In addition, slot-based representations hold great potential for generative modeling, such as controllable image generation and object manipulation in image editing. However, current slot-based methods often produce blurry images and distorted objects, exhibiting poor generative modeling capabilities. In this paper, we focus on improving slot-to-image decoding, a crucial aspect for high-quality visual generation. We introduce SlotDiffusion -- an object-centric Latent Diffusion Model (LDM) designed for both image and video data. Thanks to the powerful modeling capacity of LDMs, SlotDiffusion surpasses previous slot models in unsupervised object segmentation and visual generation across six datasets. Furthermore, our learned object features can be utilized by existing object-centric dynamics models, improving video prediction quality and downstream temporal reasoning tasks. Finally, we demonstrate the scalability of SlotDiffusion to unconstrained real-world datasets such as PASCAL VOC and COCO, when integrated with self-supervised pre-trained image encoders.

Key facts. Cited by 85 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 102 of its 118 references that are in the map. First posted on arXiv 18 May 2023.

Builds on

102 of 118 references.

Closest ancestors · 11

Same direction, newest first

  1. 2023Your Diffusion Model is Secretly a Zero-Shot Classifier28 Mar
  2. 2023Visual Chain-of-Thought Diffusion Models28 Mar
  3. 2023Object-Centric Slot Diffusion20 Mar
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  5. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  6. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  7. 2022Compositional Visual Generation with Composable Diffusion Models3 Jun
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  9. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  11. 2021Palette: Image-to-Image Diffusion Models10 Nov

Other references · 91

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  6. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  8. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  9. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  10. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  11. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  12. 2023Segment Anythingcited by 8,234
38 more
  1. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  4. 2022Classifier-Free Diffusion Guidancecited by 5,847
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  6. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  8. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  9. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  10. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  11. 2022Video Diffusion Modelscited by 2,305
  12. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  13. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  14. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  15. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  16. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  17. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  18. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  19. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  20. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
  21. 2020Object-Centric Learning with Slot Attentioncited by 876
  22. 2019MaskGAN: Towards Diverse and Interactive Facial Image Manipulationcited by 628
  23. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 612
  24. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 499
  25. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 449
  26. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
  27. 2022Kubric: A scalable dataset generatorcited by 381
  28. 2023Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Modelscited by 375
  29. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 360
  30. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  31. 2021Conditional Object-Centric Learning from Videocited by 248
  32. 2020On the Binding Problem in Artificial Neural Networkscited by 233
  33. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  34. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 220
  35. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  36. 2019Contrastive Learning of Structured World Modelscited by 209
  37. 2023MeshDiffusion: Score-based Generative 3D Mesh Modelingcited by 199
  38. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190

50 most cited of 91 All 91 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 25; 2025: 32; 2026: 24.

85 papers.

Sort:

By area:

  1. 2023Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similaritiescited by 78
  2. 2023SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformerscited by 49
  3. 2024Artificial Kuramoto Oscillatory Neuronscited by 38
  4. 2024Temporally Consistent Object-Centric Learning by Contrasting Slotscited by 31
  5. 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Modelscited by 30
  6. 2026OA-WAM: Object-Addressable World Action Model for Robust Robot Manipulationcited by 28
  7. 2024Diffusion Model with Cross Attention as an Inductive Bias for Disentanglementcited by 27
  8. 2024Zero-Shot Object-Centric Representation Learningcited by 19
  9. 2024Slot State Space Modelscited by 17
  10. 2023Unsupervised Object Localization in the Era of Self-Supervised ViTs: A Surveycited by 16
  11. 2024Layout Agnostic Scene Text Image Synthesis with Diffusion Modelscited by 14
  12. 2024Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Modelscited by 14
38 more
  1. 2024GLASS: Guided Latent Slot Diffusion for Object-Centric Learningcited by 14
  2. 2025PlaySlot: Learning Inverse Latent Dynamics for Controllable Object-Centric Video Prediction and Planningcited by 14
  3. 2025Slot Attention with Re-Initialization and Self-Distillationcited by 14
  4. 2024Learning to Compose: Improving Object Centric Learning by Injecting Compositionalitycited by 13
  5. 2025Slot-Guided Adaptation of Pre-trained Diffusion Models for Object-Centric Learning and Compositional Generationcited by 13
  6. 2025Dyn-O: Building Structured World Models with Object-Centric Representationscited by 13
  7. 2024Compositional Image Decomposition with Diffusion Modelscited by 12
  8. 2025Object-centric Binding in Contrastive Language-Image Pretrainingcited by 12
  9. 2025CTRL-O: Language-Controllable Object-Centric Visual Representation Learningcited by 12
  10. 2025Predicting Video Slot Attention Queries from Random Slot-Feature Pairscited by 12
  11. 2024Slot Abstractors: Toward Scalable Abstract Visual Reasoningcited by 11
  12. 2025Vector-Quantized Vision Foundation Models for Object-Centric Learningcited by 11
  13. 2025MetaSlot: Break Through the Fixed Number of Slots in Object-Centric Learningcited by 11
  14. 2025Learning Interactive World Model for Object-Centric Reinforcement Learningcited by 11
  15. 2024View-Centric Multi-Object Tracking with Homographic Matching in Moving UAVcited by 10
  16. 2023Guided Diffusion from Self-Supervised Diffusion Featurescited by 9
  17. 2026Flow Equivariant World Models: Memory for Partially Observed Dynamic Environmentscited by 8
  18. 2024Multi-Scale Fusion for Object Representationcited by 7
  19. 2024Bootstrapping Top-down Information for Self-modulating Slot Attentioncited by 6
  20. 2025Object-Centric Pretraining via Target Encoder Bootstrappingcited by 6
  21. 2025Object-Centric Representations Improve Policy Generalization in Robot Manipulationcited by 6
  22. 2025Smoothing Slot Attention Iterations and Recurrencescited by 6
  23. 2024Object-Centric Temporal Consistency via Conditional Autoregressive Inductive Biasescited by 5
  24. 2024Grouped Discrete Representation for Object-Centric Learningcited by 5
  25. 2025Slot-BERT: Self-supervised Object Discovery in Surgical Videocited by 4
  26. 2025Dreamweaver: Learning Compositional World Models from Pixelscited by 4
  27. 2025Motion-Refined DINOSAUR for Unsupervised Multi-Object Discoverycited by 4
  28. 2025Object-Centric World Models for Causality-Aware Reinforcement Learningcited by 4
  29. 2025FORLA: Federated Object-centric Representation Learning with Slot Attentioncited by 3
  30. 2025Is an object-centric representation beneficial for robotic manipulation ?cited by 3
  31. 2026Improved Object-Centric Diffusion Learning with Registers and Contrastive Alignmentcited by 3
  32. 2026Unsupervised Decomposition and Recombination with Discriminator-Driven Diffusion Modelscited by 3
  33. 2024Parallelized Spatiotemporal Bindingcited by 2
  34. 2024Neural Slot Interpreters: Grounding Object Semantics in Emergent Slot Representationscited by 2
  35. 2024Grouped Discrete Representation Guides Object-Centric Learningcited by 2
  36. 2025From Missing Pieces to Masterpieces: Image Completion with Context-Adaptive Diffusioncited by 2
  37. 2025Future Slot Prediction for Unsupervised Object Discovery in Surgical Videocited by 2
  38. 2025Neural USD: An object-centric framework for iterative editing and controlcited by 2

50 most cited of 85 All 85 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning Object-Centric Representations Based on Slots in Real World Scenarios
  2. 2023Grounded Object Centric Learning
  3. 2023SlotGNN: Unsupervised Discovery of Multi-Object Representations and Visual Dynamics
  4. 2025SlotMatch: Distilling Object-Centric Representations for Unsupervised Video Segmentation
  5. 2025Conditional Latent Diffusion Models for Zero-Shot Instance Segmentation
  6. 2024Reasoning-Enhanced Object-Centric Learning for Videos
  7. 2023Object-centric Learning with Cyclic Walks between Parts and Whole
  8. 2023State of the Art on Diffusion Models for Visual Computing