# mEOL: Training-Free Instruction-Guided Multimodal Embedder for Vector Graphics and Image Retrieval

arXiv: https://arxiv.org/abs/2604.17054 · HTML: https://mlsanity.com/paper/2604.17054/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 45 references that are in the map.
- First posted on arXiv 18 Apr 2026.

## Builds on (most cited first, 30 of 37)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://mlsanity.com/paper/2405.17428/) (2024)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://mlsanity.com/paper/2404.05961/) (2024)
- [Generative Representational Instruction Tuning](https://mlsanity.com/paper/2402.09906/) (2024)
- [VLM2Vec: Training Vision-Language Models for Massive Multimodal Embedding Tasks](https://mlsanity.com/paper/2410.05160/) (2024)
- [DeepSVG: A Hierarchical Generative Network for Vector Graphics Animation](https://mlsanity.com/paper/2007.11301/) (2020)
- [DiffCSE: Difference-based Contrastive Learning for Sentence Embeddings](https://mlsanity.com/paper/2204.10298/) (2022)
- [VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Models](https://mlsanity.com/paper/2211.11319/) (2022)
- [A Learned Representation for Scalable Vector Graphics](https://mlsanity.com/paper/1904.02632/) (2019)
- [Scaling Sentence Embeddings with Large Language Models](https://mlsanity.com/paper/2307.16645/) (2023)
- [SVGDreamer: Text Guided SVG Generation with Diffusion Model](https://mlsanity.com/paper/2312.16476/) (2023)
- [PromptBERT: Improving BERT Sentence Embeddings with Prompts](https://mlsanity.com/paper/2201.04337/) (2022)
- [OmniSVG: A Unified Scalable Vector Graphics Generation Model](https://mlsanity.com/paper/2504.06263/) (2025)
- [LLaVE: Large Language and Vision Embedding Models with Hardness-Weighted Contrastive Learning](https://mlsanity.com/paper/2503.04812/) (2025)
- [LayerTracer: Cognitive-Aligned Layered SVG Synthesis via Diffusion Transformer](https://mlsanity.com/paper/2502.01105/) (2025)
- [Chat2SVG: Vector Graphics Generation with Large Language Models and Image Diffusion Models](https://mlsanity.com/paper/2411.16602/) (2024)
- [Can Large Language Models Understand Symbolic Graphics Programs?](https://mlsanity.com/paper/2408.08313/) (2024)
- [StrokeNUWA: Tokenizing Strokes for Vector Graphic Synthesis](https://mlsanity.com/paper/2401.17093/) (2024)
- [Simple Techniques for Enhancing Sentence Embeddings in Generative Language Models](https://mlsanity.com/paper/2404.03921/) (2024)
- [Meta-Task Prompting Elicits Embeddings from Large Language Models](https://mlsanity.com/paper/2402.18458/) (2024)

## Built on by (most cited first, 1 of 1)

- [SVHalluc: Benchmarking Speech-Vision Hallucination in Audio-Visual Large Language Models](https://mlsanity.com/paper/2606.02642/) (2026)

## Abstract (arXiv)

Scalable Vector Graphics (SVGs) function both as visual images and as structured code that encode rich geometric and layout information, yet most methods rasterize them and discard this symbolic organization. At the same time, recent sentence embedding methods produce strong text representations but do not naturally extend to visual or structured modalities. We propose a training-free, instruction-guided multimodal embedding framework that uses a Multimodal Large Language Model (MLLM) to map text, raster images, and SVG code into an aligned embedding space. We control the direction of embeddings through modality-specific instructions and structural SVG cues, eliminating the need for learned projection heads or contrastive training. Our method has two key components: (1) Multimodal Explicit One-word Limitation (mEOL), which instructs the MLLM to summarize any multimodal input into a single token whose hidden state serves as a compact semantic embedding. (2) A semantic SVG rewriting module that assigns meaningful identifiers and simplifies nested SVG elements through visual reasoning over the rendered image, exposing geometric and relational cues hidden in raw code. Using a repurposed VGBench, we build the first text-to-SVG retrieval benchmark and show that our training-free embeddings outperform encoder-based and training-based multimodal baselines. These results highlight prompt-level control as an effective alternative to parameter-level training for structure-aware multimodal retrieval. Project page: https://scene-the-ella.github.io/meol/