# Salvation Lies Within: Eliciting Inherent Style Transfer in Step-Distilled Diffusion Models

arXiv: https://arxiv.org/abs/2610.05066 · HTML: https://mlsanity.com/paper/2610.05066/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 38 of its 45 references that are in the map.
- First posted on arXiv 4 Oct 2026.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 38)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://mlsanity.com/paper/2310.04378/) (2023)
- [Adversarial Diffusion Distillation](https://mlsanity.com/paper/2311.17042/) (2023)
- [Improved Distribution Matching Distillation for Fast Image Synthesis](https://mlsanity.com/paper/2405.14867/) (2024)
- [Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer](https://mlsanity.com/paper/2511.22699/) (2025)
- [SDXL-Lightning: Progressive Adversarial Diffusion Distillation](https://mlsanity.com/paper/2402.13929/) (2024)
- [Style Aligned Image Generation via Shared Attention](https://mlsanity.com/paper/2312.02133/) (2023)
- [StyleDrop: Text-to-Image Generation in Any Style](https://mlsanity.com/paper/2306.00983/) (2023)
- [InstantStyle: Free Lunch towards Style-Preserving in Text-to-Image Generation](https://mlsanity.com/paper/2404.02733/) (2024)
- [Controllable Generation with Text-to-Image Diffusion Models: A Survey](https://mlsanity.com/paper/2403.04279/) (2024)
- [GenAI Arena: An Open Evaluation Platform for Generative Models](https://mlsanity.com/paper/2406.04485/) (2024)
- [StyleShot: A Snapshot on Any Style](https://mlsanity.com/paper/2407.01414/) (2024)
- [Learning Few-Step Diffusion Models by Trajectory Distribution Matching](https://mlsanity.com/paper/2503.06674/) (2025)
- [Distribution Matching Distillation Meets Reinforcement Learning](https://mlsanity.com/paper/2511.13649/) (2025)
- [Decoupled DMD: CFG Augmentation as the Spear, Distribution Matching as the Shield](https://mlsanity.com/paper/2511.22677/) (2025)
- [DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models](https://mlsanity.com/paper/2503.01645/) (2025)
- [SenseFlow: Scaling Distribution Matching for Flow-based Text-to-Image Distillation](https://mlsanity.com/paper/2506.00523/) (2025)
- [Tuning Timestep-Distilled Diffusion Model Using Pairwise Sample Optimization](https://mlsanity.com/paper/2410.03190/) (2024)
- [FreeStyle: Free Lunch for Text-guided Style Transfer using Diffusion Models](https://mlsanity.com/paper/2401.15636/) (2024)

## Abstract (arXiv)

Adapting step-distilled text-to-image (T2I) models through post-training incurs additional computational costs and affects native few-step generation behavior. This motivates a complementary route beyond style-specific adaptation: drawing on the visual knowledge already encoded in step-distilled T2I models to elicit stylistic capabilities through language. Pursuing this direction requires textual guidance that captures how visual attributes jointly define a style and remain applicable as the depicted content changes. To explore this approach, we introduce StyleForge, a fully automatic, training-free framework that expresses reference styles as reusable rendering instructions. By integrating overall rendering characteristics with local color and lighting behavior, StyleForge organizes visual evidence from reference images into a coherent specification of how the target style should be expressed. The specification is then compiled into textual guidance that can be reused across content prompts, enabling frozen step-distilled T2I models to render different subjects and scenes in the reference style while retaining native few-step generation. Extensive experiments show relative gains of up to 29.47% in generation quality scores over the strongest baseline, while Pareto analysis indicates that improved stylization is accompanied by strong adherence to the requested content. Code is available at https://github.com/sunshy-1/StyleForge.