# On the Diversity of Analogy Making in Large Language Models

arXiv: https://arxiv.org/abs/2608.03233 · HTML: https://mlsanity.com/paper/2608.03233/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 40 references that are in the map.
- First posted on arXiv 4 Aug 2026.

## Builds on (most cited first, 30 of 31)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [MAUVE: Measuring the Gap Between Neural Text and Human Text using Divergence Frontiers](https://mlsanity.com/paper/2102.01454/) (2021)
- [Function Vectors in Large Language Models](https://mlsanity.com/paper/2310.15213/) (2023)
- [Emergent Analogical Reasoning in Large Language Models](https://mlsanity.com/paper/2212.09196/) (2022)
- [Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models](https://mlsanity.com/paper/2401.06102/) (2024)
- [Time Travel in LLMs: Tracing Data Contamination in Large Language Models](https://mlsanity.com/paper/2308.08493/) (2023)
- [Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputs](https://mlsanity.com/paper/2407.01082/) (2024)
- [On Decoding Strategies for Neural Text Generators](https://mlsanity.com/paper/2203.15721/) (2022)
- [Are Large Language Models Capable of Generating Human-Level Narratives?](https://mlsanity.com/paper/2407.13248/) (2024)
- [A Text-guided Protein Design Framework](https://mlsanity.com/paper/2302.04611/) (2023)
- [Modifying Large Language Model Post-Training for Diverse Creative Writing](https://mlsanity.com/paper/2503.17126/) (2025)
- [Metaphor Understanding Challenge Dataset for LLMs](https://mlsanity.com/paper/2403.11810/) (2024)
- [Entropy-Gated Branching for Efficient Test-Time Reasoning](https://mlsanity.com/paper/2503.21961/) (2025)
- [Guiding Generative Models to Uncover Diverse and Novel Crystals via Reinforcement Learning](https://mlsanity.com/paper/2511.07158/) (2025)
- [G2: Guided Generation for Enhanced Output Diversity in LLMs](https://mlsanity.com/paper/2511.00432/) (2025)
- [Domain and Function: A Dual-Space Model of Semantic Relations and Compositions](https://mlsanity.com/paper/1309.4035/) (2013)
- [AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogies](https://mlsanity.com/paper/2402.12370/) (2024)
- [Bridging the Gap Between Latent and Explicit Reasoning with Looped Transformers](https://mlsanity.com/paper/2606.31779/) (2026)
- [Min-k Sampling: Decoupling Truncation from Temperature Scaling via Relative Logit Dynamics](https://mlsanity.com/paper/2604.11012/) (2026)
- [From Weights to Activations: Is Steering the Next Frontier of Adaptation?](https://mlsanity.com/paper/2604.14090/) (2026)
- [Unlocking LLM Creativity in Science through Analogical Reasoning](https://mlsanity.com/paper/2605.11258/) (2026)
- [Automatic Extraction of Metaphoric Analogies from Literary Texts: Task Formulation, Dataset Construction, and Evaluation](https://mlsanity.com/paper/2412.15375/) (2024)

## Abstract (arXiv)

Large Language Models (LLMs) have demonstrated remarkable potential for analogy making, a core cognitive capability that drives novelty and creativity. While prior research has extensively investigated the applications and underlying mechanisms of LLM-based analogy making, its output diversity remains largely unexplored, despite being essential for broadening cross-domain connections and fostering scientific innovation. In this work, we present a comprehensive evaluation of analogy diversity across ten state-of-the-art open- and closed-source LLMs. Our findings highlight a concerning issue of domain homogeneity, a prevalent tendency for LLMs to generate analogies from a narrow set of target domains, limiting both inter-query and intra-model diversity. Furthermore, our analysis reveals a fundamental trade-off in existing LLM diversity-enhancement methods: increasing output diversity often comes at the expense of output quality. Finally, our causal analysis of LLM information flow reveals substantial differences in the model-sensitive regions governing analogy diversity across LLMs, suggesting a potential mechanism for the observed diversity-quality trade-off. To our knowledge, this is among the first studies to systematically investigate output diversity in LLM-based analogy making.