# Zero-shot visual reasoning through probabilistic analogical mapping

arXiv: https://arxiv.org/abs/2209.15087 · HTML: https://mlsanity.com/paper/2209.15087/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 26 of its 59 references that are in the map.
- First posted on arXiv 29 Sep 2022.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Vision Transformers for Dense Prediction](https://mlsanity.com/paper/2103.13413/) (2021)
- [Relational inductive biases, deep learning, and graph networks](https://mlsanity.com/paper/1806.01261/) (2018)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understanding](https://mlsanity.com/paper/1810.02338/) (2018)
- [Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts](https://mlsanity.com/paper/1406.2031/) (2014)
- [RAVEN: A Dataset for Relational and Analogical Visual rEasoNing](https://mlsanity.com/paper/1903.02741/) (2019)
- [Contrastive Learning of Structured World Models](https://mlsanity.com/paper/1911.12247/) (2019)
- [Measuring abstract reasoning in neural networks](https://mlsanity.com/paper/1807.04225/) (2018)
- [Visual Attribute Transfer through Deep Image Analogy](https://mlsanity.com/paper/1705.01088/) (2017)
- [Abstraction and Analogy-Making in Artificial Intelligence](https://mlsanity.com/paper/2102.10717/) (2021)
- [Learning to Make Analogies by Contrasting Abstract Relational Structure](https://mlsanity.com/paper/1902.00120/) (2019)
- [Emergent Symbols through Binding in External Memory](https://mlsanity.com/paper/2012.14601/) (2020)
- [Learning Representations that Support Extrapolation](https://mlsanity.com/paper/2007.05059/) (2020)
- [On Neural Architecture Inductive Biases for Relational Tasks](https://mlsanity.com/paper/2206.05056/) (2022)
- [VISALOGY: Answering Visual Analogy Questions](https://mlsanity.com/paper/1510.08973/) (2015)
- [Probabilistic Analogical Mapping with Semantic Relation Networks](https://mlsanity.com/paper/2103.16704/) (2021)
- [Visual analogy: Deep learning versus compositional models](https://mlsanity.com/paper/2105.07065/) (2021)
- [Structured Set Matching Networks for One-Shot Part Labeling](https://mlsanity.com/paper/1712.01867/) (2017)

## Built on by (most cited first, 8 of 8)

- [Emergent Analogical Reasoning in Large Language Models](https://mlsanity.com/paper/2212.09196/) (2022)
- [The Relational Bottleneck as an Inductive Bias for Efficient Abstraction](https://mlsanity.com/paper/2309.06629/) (2023)
- [Unfolding Spatial Cognition: Evaluating Multimodal Models on Visual Simulations](https://mlsanity.com/paper/2506.04633/) (2025)
- [Evidence from counterfactual tasks supports emergent analogical reasoning in large language models](https://mlsanity.com/paper/2404.13070/) (2024)
- [Nature's Insight: A Novel Framework and Comprehensive Analysis of Agentic Reasoning Through the Lens of Neuroscience](https://mlsanity.com/paper/2505.05515/) (2025)
- [Can LLMs Help Improve Analogical Reasoning For Strategic Decisions? Experimental Evidence from Humans and GPT-4](https://mlsanity.com/paper/2505.00603/) (2025)
- [Few-Shot Learning of Visual Compositional Concepts through Probabilistic Schema Induction](https://mlsanity.com/paper/2505.09859/) (2025)
- [Determinantal Point Process Attention Over Grid Cell Code Supports Out of Distribution Generalization](https://mlsanity.com/paper/2305.18417/) (2023)

## Abstract (arXiv)

Human reasoning is grounded in an ability to identify highly abstract commonalities governing superficially dissimilar visual inputs. Recent efforts to develop algorithms with this capacity have largely focused on approaches that require extensive direct training on visual reasoning tasks, and yield limited generalization to problems with novel content. In contrast, a long tradition of research in cognitive science has focused on elucidating the computational principles underlying human analogical reasoning; however, this work has generally relied on manually constructed representations. Here we present visiPAM (visual Probabilistic Analogical Mapping), a model of visual reasoning that synthesizes these two approaches. VisiPAM employs learned representations derived directly from naturalistic visual inputs, coupled with a similarity-based mapping operation derived from cognitive theories of human reasoning. We show that without any direct training, visiPAM outperforms a state-of-the-art deep learning model on an analogical mapping task. In addition, visiPAM closely matches the pattern of human performance on a novel task involving mapping of 3D objects across disparate categories.