PRISM: A Category-Theoretic Framework for Measuring and Refining Multimodal Analogies
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
Analogical reasoning involves identifying and preserving relational structures across domains. However, existing approaches to AI-driven multimodal analogy generation lack an interpretable measure of whether this structure is understood and maintained in the generated output. We address this gap with Pullback Refinement via Interpretable Structural Mapping (PRISM), a modality-agnostic framework for measuring and improving relational alignment in multimodal analogies, evaluated on visual metaphor generation. PRISM represents analogies as explicit relational mappings grounded in category theory and uses VLMs to instantiate these structures across modalities. Its first component, the pullback score, quantifies relational alignment from the resulting graph representation. On the AnaloBench benchmark, selecting the correct analogy purely by pullback score achieves 82.5% accuracy, demonstrating that the score captures meaningful relational information. PRISM's second component is an iterative refinement loop that uses the pullback score as an in-context feedback signal to iteratively revise the generated image towards greater relational depth. VLM-as-a-judge and human evaluations show that PRISM consistently improves metaphor consistency and analogy appropriateness over zero- shot generation, with human participants preferring the refined output in 57.65% of pairwise comparisons. However, a qualitative analysis reveals that refinement can favour visually crowded compositions rather than genuinely deeper relational correspondences.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 34 of its 58 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
34 of 58 references.
Closest ancestors · 6
Same direction, newest first
- 2026CARV: A Diagnostic Benchmark for Compositional Analogical Reasoning in Multimodal LLMs30 Mar
- 2025VOILA: Evaluation of MLLMs For Perceptual Understanding and Analogical Reasoning25 Feb
- 2024KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Models25 Jul
- 2022Visual Programming: Compositional visual reasoning without training18 Nov
- 2022Multimodal Analogical Reasoning over Knowledge Graphs1 Oct
- 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models8 Feb
Other references · 28
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
- 2025GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learningcited by 449
- 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 385
- 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
- 2018Neural-Symbolic VQA: Disentangling Reasoning from Vision and Language Understandingcited by 304
- 2017Learning Explanatory Rules from Noisy Datacited by 210
- 2023LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Proverscited by 161
- 2021Abstraction and Analogy-Making in Artificial Intelligencecited by 103
16 more
- 2017Automatic Understanding of Image and Video Advertisementscited by 67
- 2023MemeCap: A Dataset for Captioning and Interpreting Memescited by 46
- 2023I Spy a Metaphor: Large Language Models and Diffusion Models Co-Create Visual Metaphorscited by 31
- 2024Evaluating the Robustness of Analogical Reasoning in Large Language Modelscited by 31
- 2022MetaCLUE: Towards Comprehensive Visual Metaphors Researchcited by 28
- 2015VISALOGY: Answering Visual Analogy Questionscited by 17
- 2024LLMs as Models for Analogical Reasoningcited by 16
- 2024AnaloBench: Benchmarking the Identification of Abstract and Long-context Analogiescited by 13
- 2020Generating Correct Answers for Progressive Matrices Intelligence Testscited by 12
- 2023Beneath Surface Similarity: Large Language Models Make Reasonable Scientific Analogies after Structure Abductioncited by 11
- 2025Analogical Reasoning Inside Large Language Models: Concept Vectors and the Limits of Abstractioncited by 11
- 2024Relevant or Random: Can LLMs Truly Perform Analogical Reasoning?cited by 8
- 2022DeepGAR: Deep Graph Learning for Analogical Reasoningcited by 5
- 2025The Mind's Eye: A Multi-Faceted Reward Framework for Guiding Visual Metaphor Generationcited by 4
- 2026Beyond Pixels: Visual Metaphor Transfer via Schema-Driven Agentic Reasoningcited by 4
- 2026Enhancing Structural Mapping with LLM-derived Abstractions for Analogical Reasoning in Narrativescited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Enhancing multimodal analogical reasoning with Logic Augmented Generation
- 2023ARN: Analogical Reasoning on Narratives
- 2025The Curious Case of Analogies: Investigating Analogical Reasoning in Large Language Models
- 2026Reasoning Beyond Literal: Cross-style Multimodal Reasoning for Figurative Language Understanding
- 2026M3R-Bench: A Unified Benchmark for Evidence-Grounded Multimodal Metaphor Understanding
- 2022Scientific and Creative Analogies in Pretrained Language Models
- 2025ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation
- 2025Modeling Understanding of Story-Based Analogies Using Large Language Models