# Learning Perceptual Inference by Contrasting

arXiv: https://arxiv.org/abs/1912.00086 · HTML: https://mlsanity.com/paper/1912.00086/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Nov 2019.

## Built on by (most cited first, 30 of 57)

- [Abstraction and Analogy-Making in Artificial Intelligence](https://mlsanity.com/paper/2102.10717/) (2021)
- [A Neuro-vector-symbolic Architecture for Solving Raven's Progressive Matrices](https://mlsanity.com/paper/2203.04571/) (2022)
- [Abstract Spatial-Temporal Reasoning via Probabilistic Abduction and Execution](https://mlsanity.com/paper/2103.14230/) (2021)
- [Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matrices](https://mlsanity.com/paper/2201.12382/) (2022)
- [A Review of Emerging Research Directions in Abstract Visual Reasoning](https://mlsanity.com/paper/2202.10284/) (2022)
- [ACRE: Abstract Causal REasoning Beyond Covariation](https://mlsanity.com/paper/2103.14232/) (2021)
- [Learning Algebraic Representation for Systematic Generalization in Abstract Reasoning](https://mlsanity.com/paper/2111.12990/) (2021)
- [Effective Abstract Reasoning with Dual-Contrast Network](https://mlsanity.com/paper/2205.13720/) (2022)
- [In-Context Analogical Reasoning with Pre-Trained Language Models](https://mlsanity.com/paper/2305.17626/) (2023)
- [Learning to reason over visual objects](https://mlsanity.com/paper/2303.02260/) (2023)
- [MEWL: Few-shot multimodal word learning with referential uncertainty](https://mlsanity.com/paper/2306.00503/) (2023)
- [Probabilistic Analogical Mapping with Semantic Relation Networks](https://mlsanity.com/paper/2103.16704/) (2021)
- [Calibrating Concepts and Operations: Towards Symbolic Reasoning on Real Images](https://mlsanity.com/paper/2110.00519/) (2021)
- [Evaluating Understanding on Conceptual Abstraction Benchmarks](https://mlsanity.com/paper/2206.14187/) (2022)
- [Selective Replay Enhances Learning in Online Continual Analogical Reasoning](https://mlsanity.com/paper/2103.03987/) (2021)
- [Unsupervised Abstract Reasoning for Raven's Problem Matrices](https://mlsanity.com/paper/2109.10011/) (2021)
- [Visual analogy: Deep learning versus compositional models](https://mlsanity.com/paper/2105.07065/) (2021)
- [Slot Abstractors: Toward Scalable Abstract Visual Reasoning](https://mlsanity.com/paper/2403.03458/) (2024)
- [Raven's Progressive Matrices Completion with Latent Gaussian Process Priors](https://mlsanity.com/paper/2103.12045/) (2021)
- [SAViR-T: Spatially Attentive Visual Reasoning with Transformers](https://mlsanity.com/paper/2206.09265/) (2022)
- [Abstract Visual Reasoning: An Algebraic Approach for Solving Raven's Progressive Matrices](https://mlsanity.com/paper/2303.11730/) (2023)
- [Interactive Visual Reasoning under Uncertainty](https://mlsanity.com/paper/2206.09203/) (2022)
- [Abstract Reasoning via Logic-guided Generation](https://mlsanity.com/paper/2107.10493/) (2021)
- [Two-stage Rule-induction Visual Reasoning on RPMs with an Application to Video Prediction](https://mlsanity.com/paper/2111.12301/) (2021)
- [Multi-Viewpoint and Multi-Evaluation with Felicitous Inductive Bias Boost Machine Abstract Reasoning Ability](https://mlsanity.com/paper/2210.14914/) (2022)
- [A Minimalist Dataset for Systematic Generalization of Perception, Syntax, and Semantics](https://mlsanity.com/paper/2103.01403/) (2021)
- [Abstracting Concept-Changing Rules for Solving Raven's Progressive Matrix Problems](https://mlsanity.com/paper/2307.07734/) (2023)
- [One Self-Configurable Model to Solve Many Abstract Visual Reasoning Problems](https://mlsanity.com/paper/2312.09997/) (2023)
- [Integration of cognitive tasks into artificial general intelligence test for large models](https://mlsanity.com/paper/2402.02547/) (2024)
- [DETR++: Taming Your Multi-Scale Detection Transformer](https://mlsanity.com/paper/2206.02977/) (2022)

## Abstract (arXiv)

"Thinking in pictures," [1] i.e., spatial-temporal reasoning, effortless and instantaneous for humans, is believed to be a significant ability to perform logical induction and a crucial factor in the intellectual history of technology development. Modern Artificial Intelligence (AI), fueled by massive datasets, deeper models, and mighty computation, has come to a stage where (super-)human-level performances are observed in certain specific tasks. However, current AI's ability in "thinking in pictures" is still far lacking behind. In this work, we study how to improve machines' reasoning ability on one challenging task of this kind: Raven's Progressive Matrices (RPM). Specifically, we borrow the very idea of "contrast effects" from the field of psychology, cognition, and education to design and train a permutation-invariant model. Inspired by cognitive studies, we equip our model with a simple inference module that is jointly trained with the perception backbone. Combining all the elements, we propose the Contrastive Perceptual Inference network (CoPINet) and empirically demonstrate that CoPINet sets the new state-of-the-art for permutation-invariant models on two major datasets. We conclude that spatial-temporal reasoning depends on envisaging the possibilities consistent with the relations between objects and can be solved from pixel-level inputs.