mlsanityData to 14 Oct 2026

Can Large Language Models generalize analogy solving like children can?

arXiv 2411.02348 ↗First posted 4 Nov 2024cs.AI cs.CL cs.HCcited by 9

Not on the map. Why?

Abstract (arXiv)

In people, the ability to solve analogies such as "body : feet :: table : ?" emerges in childhood, and appears to transfer easily to other domains, such as the visual domain "( : ) :: < : ?". Recent research shows that large language models (LLMs) can solve various forms of analogies. However, can LLMs generalize analogy solving to new domains like people can? To investigate this, we had children, adults, and LLMs solve a series of letter-string analogies (e.g., a b : a c :: j k : ?) in the Latin alphabet, in a near transfer domain (Greek alphabet), and a far transfer domain (list of symbols). Children and adults easily generalized their knowledge to unfamiliar domains, whereas LLMs did not. This key difference between human and AI performance is evidence that these LLMs still struggle with robust human-like analogical transfer.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 18 of its 53 references that are in the map. First posted on arXiv 4 Nov 2024.

Builds on

18 of 53 references.

References · 18

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2022Emergent Abilities of Large Language Modelscited by 2,512
  4. 2024Gemma: Open Models Based on Gemini Research and Technologycited by 916
  5. 2022Towards Reasoning in Large Language Models: A Surveycited by 596
  6. 2023Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Taskscited by 246
  7. 2022Emergent Analogical Reasoning in Large Language Modelscited by 240
  8. 2022Impact of Pretraining Term Frequencies on Few-Shot Reasoningcited by 142
  9. 2021Abstraction and Analogy-Making in Artificial Intelligencecited by 103
  10. 2023The ConceptARC Benchmark: Evaluating Understanding and Generalization in the ARC Domaincited by 101
  11. 2023Large Language Models Are Not Strong Abstract Reasonerscited by 53
  12. 2024Using Counterfactual Tasks to Evaluate the Generality of Analogical Reasoning in Large Language Modelscited by 38
6 more
  1. 2024KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Modelscited by 24
  2. 2024Evidence from counterfactual tasks supports emergent analogical reasoning in large language modelscited by 14
  3. 2023Response: Emergent analogical reasoning in large language modelscited by 13
  4. 2023Do large language models solve verbal analogies like children do?cited by 13
  5. 2019A Theory of Relation Learning and Cross-domain Generalizationcited by 11
  6. 2024Do Large Language Models Solve ARC Visual Analogies Like People Do?cited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 6; 2026: 2.

9 papers.