Data to 14 Oct 2026

Attention over learned object embeddings enables complex visual reasoning

arXiv 2012.08508 ↗First posted 15 Dec 2020cs.CV cs.AI cs.CLcited by 59

Not on the map. Why?

Abstract (arXiv)

Neural networks have achieved success in a wide array of perceptual tasks but often fail at tasks involving both perception and higher-level reasoning. On these more challenging tasks, bespoke approaches (such as modular symbolic components, independent dynamics models or semantic parsers) targeted towards that specific type of task have typically performed better. The downside to these targeted approaches, however, is that they can be more brittle than general-purpose neural networks, requiring significant modification or even redesign according to the particular task at hand. Here, we propose a more general neural-network-based approach to dynamic visual reasoning problems that obtains state-of-the-art performance on three different domains, in each case outperforming bespoke modular approaches tailored specifically to the task. Our method relies on learned object-centric representations, self-attention and self-supervised dynamics learning, and all three elements together are required for strong performance to emerge. The success of this combination suggests that there may be no need to trade off flexibility for performance on problems involving spatio-temporal or causal-style reasoning. With the right soft biases and learning objectives in a neural network we may be able to attain the best of both worlds.

Key facts. Cited by 59 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 13; 2023: 20; 2024: 11; 2025: 9; 2026: 3.

59 papers.

Sort:

By area:

2021Conditional Object-Centric Learning from Videocited by 248
2022SlotFormer: Unsupervised Visual Dynamics Simulation with Object-Centric Modelscited by 147
2022Multimodal Masked Autoencoders Learn Transferable Representationscited by 109
2023SlotDiffusion: Object-Centric Generative Modeling with Diffusion Modelscited by 85
2025Grounded Reinforcement Learning for Visual Reasoningcited by 83
2022ComPhy: Compositional Physical Reasoning of Objects and Events from Videoscited by 63
2022Video Graph Transformer for Video Question Answeringcited by 63
2023Vamos: Versatile Action Models for Video Understandingcited by 40
2023Contrastive Video Question Answering via Video Graph Transformercited by 32
2026Causal-JEPA: Learning World Models through Object-Level Latent Maskingcited by 29
2022OGC: Unsupervised 3D Object Segmentation from Rigid Dynamics of Point Cloudscited by 25
2023Systematic Visual Reasoning through Object-Centric Relational Abstractioncited by 20

38 more

2023Learning to reason over visual objectscited by 19
2023GENOME: GenerativE Neuro-symbOlic visual reasoning by growing and reusing ModulEscited by 18
2023MEWL: Few-shot multimodal word learning with referential uncertaintycited by 17
2023Contrastive Training of Complex-Valued Autoencoders for Object Discoverycited by 16
2022Unsupervised Learning of Temporal Abstractions with Slot-based Transformerscited by 14
2023Look, Remember and Reason: Grounded reasoning in videos with language modelscited by 14
2024Exploring the Effectiveness of Object-Centric Representations in Visual Question Answering: Comparative Insights with Foundation Modelscited by 14
2021Understanding the computational demands underlying visual reasoningcited by 13
2022CRIPP-VQA: Counterfactual Reasoning about Implicit Physical Properties via Video Question Answeringcited by 13
2024Compositional Physical Reasoning of Objects and Events from Videoscited by 12
2025Predicting Video Slot Attention Queries from Random Slot-Feature Pairscited by 12
2022Unsupervised Multi-object Segmentation Using Attention and Soft-argmaxcited by 11
2023CLEVRER-Humans: Describing Physical and Causal Events the Human Waycited by 11
2024Slot Abstractors: Toward Scalable Abstract Visual Reasoningcited by 11
2022Reasoning about Actions over Visual and Linguistic Modalities: A Surveycited by 10
2024Physics Context Builders: A Modular Framework for Physical Reasoning in Vision-Language Modelscited by 10
2022Interactive Visual Reasoning under Uncertaintycited by 8
2023Intrinsic Physical Concepts Discovery with Object-Centric Predictive Modelscited by 8
2023Active Reasoning in an Open-World Environmentcited by 8
2023Benchmarks for Physical Reasoning AIcited by 8
2024Recurrent Complex-Weighted Autoencoders for Unsupervised Object Discoverycited by 8
2025Smoothing Slot Attention Iterations and Recurrencescited by 6
2021Topos and Stacks of Deep Neural Networkscited by 5
2022Pretraining on Interactions for Learning Grounded Affordance Representationscited by 5
2024Object-Centric Temporal Consistency via Conditional Autoregressive Inductive Biasescited by 5
2024Integrating Physics and Topology in Neural Networks for Learning Rigid Body Dynamicscited by 5
2025Evaluating Compositional Scene Understanding in Multimodal Generative Modelscited by 5
2023Reusable Slotwise Mechanismscited by 4
2023Learning Object Permanence from Videos via Latent Imaginationscited by 4
2024Human-Like Geometric Abstraction in Large Pre-trained Neural Networkscited by 4
2025Dreamweaver: Learning Compositional World Models from Pixelscited by 4
2023Laser: Latent Set Representations for 3D Generative Modelingcited by 3
2023X-VoE: Measuring eXplanatory Violation of Expectation in Physical Eventscited by 3
2024OLViT: Multi-Modal State Tracking via Attention-Based Embeddings for Video-Grounded Dialogcited by 3
2024Multi-granularity Contrastive Cross-modal Collaborative Generation for End-to-End Long-term Video Question Answeringcited by 3
2025Aligning Perception, Reasoning, Modeling and Interaction: A Survey on Physical AIcited by 3
2023A Reminder of its Brittleness: Language Reward Shaping May Hinder Learning for Instruction Following Agentscited by 2
2022Solving Reasoning Tasks with a Slot Transformercited by 1

50 most cited of 59 All 59 (JSON)