mlsanityData to 7 Oct 2026

Learning Representations that Support Extrapolation

arXiv 2007.05059 ↗First posted 9 Jul 2020cs.CVcited by 36

Not on the map. Why?

Abstract (arXiv)

Extrapolation -- the ability to make inferences that go beyond the scope of one's experiences -- is a hallmark of human intelligence. By contrast, the generalization exhibited by contemporary neural network algorithms is largely limited to interpolation between data points in their training corpora. In this paper, we consider the challenge of learning representations that support extrapolation. We introduce a novel visual analogy benchmark that allows the graded evaluation of extrapolation as a function of distance from the convex domain defined by the training data. We also introduce a simple technique, temporal context normalization, that encourages representations that emphasize the relations between objects. We find that this technique enables a significant improvement in the ability to extrapolate, considerably outperforming a number of competitive techniques.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 9; 2023: 8; 2024: 7; 2025: 6; 2026: 2.

36 papers.

Sort:

By area:

  1. 2022Emergent Analogical Reasoning in Large Language Modelscited by 239
  2. 2021Abstraction and Analogy-Making in Artificial Intelligencecited by 103
  3. 2023Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolationcited by 56
  4. 2022Deep Learning Methods for Abstract Visual Reasoning: A Survey on Raven's Progressive Matricescited by 37
  5. 2022A Review of Emerging Research Directions in Abstract Visual Reasoningcited by 33
  6. 2024Reasoning Abilities of Large Language Models: In-Depth Analysis on the Abstraction and Reasoning Corpuscited by 32
  7. 2024MARVEL: Multidimensional Abstraction and Reasoning through Visual Evaluation and Learningcited by 30
  8. 2025MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal Modelscited by 28
  9. 2022On Neural Architecture Inductive Biases for Relational Taskscited by 26
  10. 2023When can transformers reason with abstract symbols?cited by 23
  11. 2023Learning to reason over visual objectscited by 19
  12. 2023Systematic Visual Reasoning through Object-Centric Relational Abstractioncited by 19
24 more
  1. 2024Slot Abstractors: Toward Scalable Abstract Visual Reasoningcited by 10
  2. 2022Zero-shot visual reasoning through probabilistic analogical mappingcited by 8
  3. 2022Extrapolation and Spectral Bias of Neural Nets with Hadamard Product: a Polynomial Net Studycited by 7
  4. 2024ARCLE: The Abstraction and Reasoning Corpus Learning Environment for Reinforcement Learningcited by 7
  5. 2025Bongard-RWR+: Real-World Representations of Fine-Grained Concepts in Bongard Problemscited by 7
  6. 2023Imagine the Unseen World: A Benchmark for Systematic Generalization in Visual World Modelscited by 6
  7. 2024COLUMBUS: Evaluating COgnitive Lateral Understanding through Multiple-choice reBUSescited by 5
  8. 2021Modelling the development of counting with memory-augmented neural networkscited by 4
  9. 2021Extrapolation Frameworks in Cognitive Psychology Suitable for Study of Image Classification Modelscited by 4
  10. 2024A-I-RAVEN and I-RAVEN-Mesh: Two New Benchmarks for Abstract Visual Reasoningcited by 4
  11. 2021A Self-Supervised Framework for Function Learning and Extrapolationcited by 2
  12. 2023Learning Hierarchical Relational Representations through Relational Convolutionscited by 2
  13. 2022To what extent should we trust AI models when they extrapolate?cited by 1
  14. 2023Deep Neural Networks Tend To Extrapolate Predictablycited by 1
  15. 2022Should Machine Learning Models Report to Us When They Are Clueless?cited by 0
  16. 2022Functional Indirection Neural Estimator for Better Out-of-distribution Generalizationcited by 0
  17. 2023Determinantal Point Process Attention Over Grid Cell Code Supports Out of Distribution Generalizationcited by 0
  18. 2024The Extrapolation Power of Implicit Modelscited by 0
  19. 2025Advancing Generalization Across a Variety of Abstract Visual Reasoning Taskscited by 0
  20. 2025A Vision Toward Energy-Efficient Domain-Specific Artificial Intelligence Models and Agentscited by 0
  21. 2025Foundations of Artificial Intelligence Frameworks: Notion and Limits of AGIcited by 0
  22. 2025Physics-guided Neural Network-based Shaft Power Prediction for Vesselscited by 0
  23. 2026Quo Vadis, Visual In-Context Learning? A Unified Benchmark Across Domains and Taskscited by 0
  24. 2026Categorical Internalisation of Environmental Groupoids for Generalisable POMDP Solvingcited by 0