# AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogies

arXiv: https://arxiv.org/abs/2402.12370 · HTML: https://mlsanity.com/paper/2402.12370/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 76 references that are in the map.
- First posted on arXiv 19 Feb 2024.

## Builds on (most cited first, 30 of 31)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://mlsanity.com/paper/2210.09261/) (2022)
- [Extending Context Window of Large Language Models via Positional Interpolation](https://mlsanity.com/paper/2306.15595/) (2023)
- [Zephyr: Direct Distillation of LM Alignment](https://mlsanity.com/paper/2310.16944/) (2023)
- [UnifiedQA: Crossing Format Boundaries With a Single QA System](https://mlsanity.com/paper/2005.00700/) (2020)
- [Large Language Models Struggle to Learn Long-Tail Knowledge](https://mlsanity.com/paper/2211.08411/) (2022)
- [Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2](https://mlsanity.com/paper/2311.10702/) (2023)
- [RAVEN: A Dataset for Relational and Analogical Visual rEasoNing](https://mlsanity.com/paper/1903.02741/) (2019)
- [Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context](https://mlsanity.com/paper/1805.04623/) (2018)
- [Large Language Models as Analogical Reasoners](https://mlsanity.com/paper/2310.01714/) (2023)
- [Towards Understanding Linear Word Analogies](https://mlsanity.com/paper/1810.04882/) (2018)
- [UnifiedQA-v2: Stronger Generalization via Broader Cross-Format Training](https://mlsanity.com/paper/2202.12359/) (2022)
- [BERT is to NLP what AlexNet is to CV: Can Pre-Trained Language Models Identify Analogies?](https://mlsanity.com/paper/2105.04949/) (2021)
- [Accelerating Innovation Through Analogy Mining](https://mlsanity.com/paper/1706.05585/) (2017)
- [Analogy Generation by Prompting Large Language Models: A Case Study of InstructGPT](https://mlsanity.com/paper/2210.04186/) (2022)
- [StoryAnalogy: Deriving Story-level Analogies from Large Language Models to Unlock Analogical Understanding](https://mlsanity.com/paper/2310.12874/) (2023)
- [Life is a Circus and We are the Clowns: Automatically Finding Analogies between Situations and Processes](https://mlsanity.com/paper/2210.12197/) (2022)
- [VASR: Visual Analogies of Situation Recognition](https://mlsanity.com/paper/2212.04542/) (2022)
- [VISALOGY: Answering Visual Analogy Questions](https://mlsanity.com/paper/1510.08973/) (2015)
- [ANALOGYKB: Unlocking Analogical Reasoning of Language Models with A Million-scale Knowledge Base](https://mlsanity.com/paper/2305.05994/) (2023)
- [ePiC: Employing Proverbs in Context as a Benchmark for Abstract Language Understanding](https://mlsanity.com/paper/2109.06838/) (2021)
- [Large Language Model Displays Emergent Ability to Interpret Novel Literary Metaphors](https://mlsanity.com/paper/2308.01497/) (2023)
- [Beneath Surface Similarity: Large Language Models Make Reasonable Scientific Analogies after Structure Abduction](https://mlsanity.com/paper/2305.12660/) (2023)
- [A Neural Approach for Detecting Morphological Analogies](https://mlsanity.com/paper/2108.03945/) (2021)
- [Textual Analogy Parsing: What's Shared and What's Compared among Analogous Facts](https://mlsanity.com/paper/1809.02700/) (2018)
- [Solving morphological analogies: from retrieval to generation](https://mlsanity.com/paper/2303.18062/) (2023)

## Built on by (most cited first, 13 of 13)

- [Efficient Large Multi-modal Models via Visual Context Compression](https://mlsanity.com/paper/2406.20092/) (2024)
- [Benchmarking Language Model Creativity: A Case Study on Code Generation](https://mlsanity.com/paper/2407.09007/) (2024)
- [LogiDynamics: Unraveling the Dynamics of Inductive, Abductive and Deductive Logical Inferences in LLM Reasoning](https://mlsanity.com/paper/2502.11176/) (2025)
- [Hidden in the Haystack: Smaller Needles are More Difficult for LLMs to Find](https://mlsanity.com/paper/2505.18148/) (2025)
- [Boosting Scientific Concepts Understanding: Can Analogy from Teacher Models Empower Student Models?](https://mlsanity.com/paper/2406.11375/) (2024)
- [Past Meets Present: Creating Historical Analogy with Large Language Models](https://mlsanity.com/paper/2409.14820/) (2024)
- [Structural Ranking of the Cognitive Plausibility of Computational Models of Analogy and Metaphors with the Minimal Cognitive Grid](https://mlsanity.com/paper/2605.01359/) (2026)
- [VisAnalog: A Diagnostic Suite for Visual Concept Transfer on Natural Images](https://mlsanity.com/paper/2605.23141/) (2026)
- [Teaching Through Analogies: A Modular Pipeline for Educational Analogy Generation](https://mlsanity.com/paper/2605.24211/) (2026)
- [Analogical Deep Research: Retrieving and Integrating Historical Analogies for Foresight Analysis](https://mlsanity.com/paper/2607.13602/) (2026)
- [On the Diversity of Analogy Making in Large Language Models](https://mlsanity.com/paper/2608.03233/) (2026)
- [RATIO: A Benchmark for Retrieval Across Typed Ideation Operations in Scientific Literature](https://mlsanity.com/paper/2608.27394/) (2026)
- [PRISM: A Category-Theoretic Framework for Measuring and Refining Multimodal Analogies](https://mlsanity.com/paper/2610.01383/) (2026)

## Abstract (arXiv)

Humans regularly engage in analogical thinking, relating personal experiences to current situations (X is analogous to Y because of Z). Analogical thinking allows humans to solve problems in creative ways, grasp difficult concepts, and articulate ideas more effectively. Can language models (LMs) do the same? To answer this question, we propose AnaloBench, a benchmark to determine analogical reasoning ability in LMs. Our benchmarking approach focuses on aspects of this ability that are common among humans: (i) recalling related experiences from a large amount of information, and (ii) applying analogical reasoning to complex and lengthy scenarios. We test a broad collection of proprietary models (e.g., GPT family, Claude V2) and open source models such as LLaMA2. As in prior results, scaling up LMs results in some performance boosts. Surprisingly, scale offers minimal gains when, (i) analogies involve lengthy scenarios, or (ii) recalling relevant scenarios from a large pool of information, a process analogous to finding a needle in a haystack. We hope these observations encourage further research in this field.