mlsanityData to 14 Oct 2026

Illiterate DALL-E Learns to Compose

arXiv 2110.11405 ↗First posted 17 Oct 2021cs.CV cs.LGcited by 147

Not on the map. Why?

Abstract (arXiv)

Although DALL-E has shown an impressive ability of composition-based systematic generalization in image generation, it requires the dataset of text-image pairs and the compositionality is provided by the text. In contrast, object-centric representation models like the Slot Attention model learn composable representations without the text prompt. However, unlike DALL-E its ability to systematically generalize for zero-shot generation is significantly limited. In this paper, we propose a simple but novel slot-based autoencoding architecture, called SLATE, for combining the best of both worlds: learning object-centric representations that allows systematic generalization in zero-shot image generation without text. As such, this model can also be seen as an illiterate DALL-E model. Unlike the pixel-mixture decoders of existing object-centric representation models, we propose to use the Image GPT decoder conditioned on the slots for capturing complex interactions among the slots and pixels. In experiments, we show that this simple and easy-to-implement architecture not requiring a text prompt achieves significant improvement in in-distribution and out-of-distribution (zero-shot) image generation and qualitatively comparable or better slot-attention structure than the models based on mixture decoders.

Key facts. Cited by 147 papers in the map (data through 14 Oct 2026). Builds on 65 of its 87 references that are in the map. First posted on arXiv 17 Oct 2021.

Builds on

65 of 87 references.

References · 65

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  8. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  9. 2017Neural Discrete Representation Learningcited by 4,712
  10. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  12. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
38 more
  1. 2013Describing Textures in the Wildcited by 2,125
  2. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
  3. 2020Object-Centric Learning with Slot Attentioncited by 876
  4. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 860
  5. 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
  6. 2016Generative Adversarial Text to Image Synthesiscited by 735
  7. 2018Disentangling by Factorisingcited by 616
  8. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
  9. 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 360
  10. 2016Adversarial Feature Learningcited by 350
  11. 2018Image Generation from Scene Graphscited by 324
  12. 2017Variational Inference of Disentangled Latent Concepts from Unlabeled Observationscited by 241
  13. 2019Large Scale Adversarial Representation Learningcited by 239
  14. 2020On the Binding Problem in Artificial Neural Networkscited by 233
  15. 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
  16. 2019Recurrent Independent Mechanismscited by 219
  17. 2016Attend, Infer, Repeat: Fast Scene Understanding with Generative Modelscited by 209
  18. 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
  19. 2020BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Imagescited by 166
  20. 2021Self-supervised Video Object Segmentation by Motion Groupingcited by 147
  21. 2018Sequential Attend, Infer, Repeat: Generative Modelling of Moving Objectscited by 128
  22. 2017Neural Expectation Maximizationcited by 123
  23. 2021How to represent part-whole hierarchies in a neural networkcited by 123
  24. 2015Deep Convolutional Inverse Graphics Networkcited by 121
  25. 2019Spatial Broadcast Decoder: A Simple Architecture for Learning Disentangled Representations in VAEscited by 117
  26. 2019Object-driven Text-to-Image Synthesis via Adversarial Trainingcited by 117
  27. 2021GENESIS-V2: Inferring Unordered Object Representations without Iterative Refinementcited by 109
  28. 2019SCALOR: Generative World Models with Scalable Object Representationscited by 108
  29. 2016Learning What and Where to Drawcited by 105
  30. 2019Towards Unsupervised Learning of Generative Models for 3D Controllable Image Synthesiscited by 104
  31. 2015Attribute2Image: Conditional Image Generation from Visual Attributescited by 93
  32. 2019Specifying Object Attributes and Relations in Interactive Scene Generationcited by 76
  33. 2019Unsupervised Object Segmentation by Redrawingcited by 74
  34. 2020Improving Generative Imagination in Object-Centric World Modelscited by 71
  35. 2021SIMONe: View-Invariant, Temporally-Abstracted Object Representations via Unsupervised Video Decompositioncited by 69
  36. 2019Learning Canonical Representations for Scene Graph to Image Generationcited by 60
  37. 2019Emergence of Object Segmentation in Perturbed Generative Modelscited by 57
  38. 2020Generative Neurosymbolic Machinescited by 57

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 14; 2023: 37; 2024: 39; 2025: 30; 2026: 27.

147 papers.

Sort:

By area:

  1. 2022Bridging the Gap to Real-World Object-Centric Learningcited by 220
  2. 2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 147
  3. 2022Simple Unsupervised Object-Centric Learning for Complex and Naturalistic Videoscited by 139
  4. 2023Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Taskcited by 93
  5. 2022Object Scene Representation Transformercited by 90
  6. 2023Object-Centric Slot Diffusioncited by 90
  7. 2023SlotDiffusion: Object-Centric Generative Modeling with Diffusion Modelscited by 85
  8. 2023Object-Centric Learning for Real-World Videos by Predicting Temporal Feature Similaritiescited by 78
  9. 2022Improving Object-centric Learning with Query Optimizationcited by 60
  10. 2022Unsupervised Semantic Segmentation with Self-supervised Object-centric Representationscited by 54
  11. 2023Invariant Slot Attention: Object Discovery with Slot-Centric Reference Framescited by 51
  12. 2023An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learningcited by 49
38 more
  1. 2023Self-supervised Object-Centric Learning for Videoscited by 49
  2. 2023SPOT: Self-Training with Patch-Order Permutation for Object-Centric Learning with Autoregressive Transformerscited by 49
  3. 2022Object Representations as Fixed Points: Training Iterative Refinement Algorithms with Implicit Differentiationcited by 46
  4. 2023Provably Learning Object-Centric Representationscited by 45
  5. 2022Guess What Moves: Unsupervised Video and Image Segmentation by Anticipating Motioncited by 42
  6. 2023Provable Compositional Generalization for Object-Centric Learningcited by 41
  7. 2022Complex-Valued Autoencoders for Object Discoverycited by 40
  8. 2024Artificial Kuramoto Oscillatory Neuronscited by 38
  9. 2022Neural Systematic Bindercited by 37
  10. 2024Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Spacecited by 35
  11. 2024Adaptive Slot Attention: Object Discovery with Dynamic Slot Numbercited by 34
  12. 2023OCAtari: Object-Centric Atari 2600 Reinforcement Learning Environmentscited by 30
  13. 2024Neural Assets: 3D-Aware Multi-Object Scene Synthesis with Image Diffusion Modelscited by 30
  14. 2023Rotating Features for Object Discoverycited by 29
  15. 2024Entity-Centric Reinforcement Learning for Object Manipulation from Pixelscited by 27
  16. 2023Object Discovery from Motion-Guided Tokenscited by 25
  17. 2023Shatter and Gather: Learning Referring Image Segmentation with Text Supervisioncited by 25
  18. 2023Slot-VAE: Object-Centric Scene Generation with Slot Attentioncited by 20
  19. 2024Interaction Asymmetry: A General Principle for Learning Composable Abstractionscited by 20
  20. 2023DDLP: Unsupervised Object-Centric Video Prediction with Deep Dynamic Latent Particlescited by 19
  21. 2024Zero-Shot Object-Centric Representation Learningcited by 19
  22. 2022Boosting Object Representation Learning via Motion and Object Continuitycited by 17
  23. 2023Shepherding Slots to Objects: Towards Stable and Robust Object-Centric Learningcited by 17
  24. 2023Grounded Object Centric Learningcited by 17
  25. 2024Slot State Space Modelscited by 17
  26. 2023Object-centric Learning with Cyclic Walks between Parts and Wholecited by 16
  27. 2023DORSal: Diffusion for Object-centric Representations of Scenes et alcited by 16
  28. 2025Object-Centric World Model for Language-Guided Manipulationcited by 16
  29. 2023Learning Dynamic Attribute-factored World Models for Efficient Multi-object Reinforcement Learningcited by 15
  30. 2023Object-Centric Multiple Object Trackingcited by 15
  31. 2026Latent Particle World Models: Self-supervised Object-centric Stochastic Dynamics Modelingcited by 15
  32. 2023Ten Years of Generative Adversarial Nets (GANs): A survey of the state-of-the-artcited by 14
  33. 2024Neural Concept Bindercited by 14
  34. 2024Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Modelscited by 14
  35. 2024GLASS: Guided Latent Slot Diffusion for Object-Centric Learningcited by 14
  36. 2025PlaySlot: Learning Inverse Latent Dynamics for Controllable Object-Centric Video Prediction and Planningcited by 14
  37. 2025Slot Attention with Re-Initialization and Self-Distillationcited by 14
  38. 2024Learning to Compose: Improving Object Centric Learning by Injecting Compositionalitycited by 13

50 most cited of 147 All 147 (JSON)