The Unreasonable Effectiveness of Text Embedding Interpolation for Continuous Image Steering
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
We present a training-free framework for continuous and controllable image editing at test time for text-conditioned generative models. In contrast to prior approaches that rely on additional training or manual user intervention, we find that a simple steering in the text-embedding space is sufficient to produce smooth edit control. Given a target concept (e.g., enhancing photorealism or changing facial expression), we use a large language model to automatically construct a small set of debiased contrastive prompt pairs, from which we compute a steering vector in the generator's text-encoder space. We then add this vector directly to the input prompt representation to control generation along the desired semantic axis. To obtain a continuous control, we propose an elastic range search procedure that automatically identifies an effective interval of steering magnitudes, avoiding both under-steering (no-edit) and over-steering (changing other attributes). Adding the scaled versions of the same vector within this interval yields smooth and continuous edits. Since our method modifies only textual representations, it naturally generalizes across text-conditioned modalities, including image and video generation. To quantify the steering continuity, we introduce a new evaluation metric that measures the uniformity of semantic change across edit strengths. We compare the continuous editing behavior across methods and find that, despite its simplicity and lightweight design, our approach is comparable to training-based alternatives, outperforming other training-free methods.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 39 of its 44 references that are in the map. First posted on arXiv 18 Mar 2026.
Builds on
39 of 44 references.
References · 39
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2025Qwen3 Technical Reportcited by 8,195
- 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2022Video Diffusion Modelscited by 2,280
27 more
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
- 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,107
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
- 2025Qwen-Image Technical Reportcited by 973
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2023Steering Language Models With Activation Engineeringcited by 824
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
- 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
- 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 507
- 2025VACE: All-in-One Video Creation and Editingcited by 467
- 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
- 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 322
- 2023Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Modelscited by 148
- 2023LEDITS++: Limitless Image Editing using Text-to-Image Modelscited by 147
- 2024CogView3: Finer and Faster Text-to-Image Generation via Relay Diffusioncited by 72
- 2025Goku: Flow Based Video Generative Foundation Modelscited by 67
- 2024Spectral Editing of Activations for Large Language Model Alignmentcited by 54
- 2025Angular Steering: Behavior Control via Rotation in Activation Spacecited by 45
- 2024Continuous, Subject-Specific Attribute Control in T2I Models by Identifying Semantic Directionscited by 39
- 2024Interpreting the Weight Space of Customized Diffusion Modelscited by 35
- 2024Controlling Large Language Model Agents with Entropic Activation Steeringcited by 29
- 2024FluxSpace: Disentangled Semantic Editing in Rectified Flow Transformerscited by 25
- 2025Kontinuous Kontext: Continuous Strength Control for Instruction-based Image Editingcited by 15
- 2025SliderEdit: Continuous Image Editing with Fine-Grained Instruction Controlcited by 12
- 2025SAEdit: Token-level control for continuous image editing via Sparse AutoEncodercited by 8
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Steering Robustness into World Action Models via Mechanistic Interpretability and Optimal Controlcited by 3
- 2026Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Controlcited by 2
- 2026Token-to-Token Alignment of Text Embeddings for Semantic Blendingcited by 0
- 2026SemanticSlider3D: Training-Free Continuous Semantic Editing for 3D Objectscited by 0
- 2026SemanticSlider3D: Training-Free Continuous Semantic Editing for 3D Objects19 Aug 2026
- 2026Steering Robustness into World Action Models via Mechanistic Interpretability and Optimal Control16 Jul 2026
- 2026Token-to-Token Alignment of Text Embeddings for Semantic Blending22 Jun 2026
- 2026Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Control3 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2024Prompt-Softbox-Prompt: A Free-Text Embedding Control for Image Editing
- 2024Learn Once, Edit Anywhere: Visual Direction Transfer for Diffusion Models
- 2026Language-Free Generative Editing from One Visual Example
- 2023AdapEdit: Spatio-Temporal Guided Adaptive Editing Algorithm for Text-Based Continuity-Sensitive Image Editing
- 2023iEdit: Localised Text-guided Image Editing with Weak Supervision
- 2025EditInfinity: Image Editing with Binary-Quantized Generative Models
- 2023Manipulating Embeddings of Stable Diffusion Prompts
- 2025Controllable-Continuous Color Editing in Diffusion Model via Color Mapping