# Beyond Pixels: Visual Metaphor Transfer via Schema-Driven Agentic Reasoning

arXiv: https://arxiv.org/abs/2602.01335 · HTML: https://mlsanity.com/paper/2602.01335/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 28 of its 51 references that are in the map.
- First posted on arXiv 1 Feb 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 28 of 28)

- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [Emerging Properties in Unified Multimodal Pretraining](https://mlsanity.com/paper/2505.14683/) (2025)
- [Inversion-Based Style Transfer with Diffusion Models](https://mlsanity.com/paper/2211.13203/) (2022)
- [Seedream 4.0: Toward Next-generation Multimodal Image Generation](https://mlsanity.com/paper/2509.20427/) (2025)
- [Break-A-Scene: Extracting Multiple Concepts from a Single Image](https://mlsanity.com/paper/2305.16311/) (2023)
- [Style Aligned Image Generation via Shared Attention](https://mlsanity.com/paper/2312.02133/) (2023)
- [HunyuanImage 3.0 Technical Report](https://mlsanity.com/paper/2509.23951/) (2025)
- [DreamO: A Unified Framework for Image Customization](https://mlsanity.com/paper/2504.16915/) (2025)
- [I Spy a Metaphor: Large Language Models and Diffusion Models Co-Create Visual Metaphors](https://mlsanity.com/paper/2305.14724/) (2023)
- [SketchAgent: Language-Driven Sequential Sketch Generation](https://mlsanity.com/paper/2411.17673/) (2024)
- [MetaCLUE: Towards Comprehensive Visual Metaphors Research](https://mlsanity.com/paper/2212.09898/) (2022)
- [Personalized Image Generation with Deep Generative Models: A Decade Survey](https://mlsanity.com/paper/2502.13081/) (2025)
- [Text-to-Image Generation for Abstract Concepts](https://mlsanity.com/paper/2309.14623/) (2023)
- [EditDuet: A Multi-Agent System for Video Non-Linear Editing](https://mlsanity.com/paper/2509.10761/) (2025)
- [A Unified Arbitrary Style Transfer Framework via Adaptive Contrastive Learning](https://mlsanity.com/paper/2303.12710/) (2023)
- [Prompt Optimizer of Text-to-Image Diffusion Models for Abstract Concept Understanding](https://mlsanity.com/paper/2404.11589/) (2024)
- [In-Context Brush: Zero-shot Customized Subject Insertion with Context-Aware Latent Space Manipulation](https://mlsanity.com/paper/2505.20271/) (2025)
- [TAG-MoE: Task-Aware Gating for Unified Generative Mixture-of-Experts](https://mlsanity.com/paper/2601.08881/) (2026)
- [The Mind's Eye: A Multi-Faceted Reward Framework for Guiding Visual Metaphor Generation](https://mlsanity.com/paper/2508.18569/) (2025)
- [IIDM: Image-to-Image Diffusion Model for Semantic Image Synthesis](https://mlsanity.com/paper/2403.13378/) (2024)
- [IP-Prompter: Training-Free Theme-Specific Image Generation via Dynamic Visual Prompting](https://mlsanity.com/paper/2501.15641/) (2025)
- [Concept Drift Guided LayerNorm Tuning for Efficient Multimodal Metaphor Identification](https://mlsanity.com/paper/2505.11237/) (2025)

## Built on by (most cited first, 4 of 4)

- [What Can I Edit? Open-Ended Strategy Discovery and the Emotion Editability Landscape](https://mlsanity.com/paper/2607.23920/) (2026)
- [FontCrafter: High-Fidelity Element-Driven Artistic Font Creation with Visual In-Context Generation](https://mlsanity.com/paper/2603.22054/) (2026)
- [Blending Concepts: Benchmarking Visual Metaphor Generation in Text-to-Image Models](https://mlsanity.com/paper/2609.02502/) (2026)
- [PRISM: A Category-Theoretic Framework for Measuring and Refining Multimodal Analogies](https://mlsanity.com/paper/2610.01383/) (2026)

## Abstract (arXiv)

A visual metaphor constitutes a high-order form of human creativity, employing cross-domain semantic fusion to transform abstract concepts into impactful visual rhetoric. Despite the remarkable progress of generative AI, existing models remain largely confined to pixel-level instruction alignment and surface-level appearance preservation, failing to capture the underlying abstract logic necessary for genuine metaphorical generation. To bridge this gap, we introduce the task of Visual Metaphor Transfer (VMT), which challenges models to autonomously decouple the "creative essence" from a reference image and re-materialize that abstract logic onto a user-specified target subject. We propose a cognitive-inspired, multi-agent framework that operationalizes Conceptual Blending Theory (CBT) through a novel Schema Grammar ("G"). This structured representation decouples relational invariants from specific visual entities, providing a rigorous foundation for cross-domain logic re-instantiation. Our pipeline executes VMT through a collaborative system of specialized agents: a perception agent that distills the reference into a schema, a transfer agent that maintains generic space invariance to discover apt carriers, a generation agent for high-fidelity synthesis and a hierarchical diagnostic agent that mimics a professional critic, performing closed-loop backtracking to identify and rectify errors across abstract logic, component selection, and prompt encoding. Extensive experiments and human evaluations demonstrate that our method significantly outperforms SOTA baselines in metaphor consistency, analogy appropriateness, and visual creativity, paving the way for automated high-impact creative applications in advertising and media. Project page with source code and self-contained skills is at https://yuci-gpt.github.io/Beyond-Pixels/.