mlsanityData to 7 Oct 2026

FreeText: Training-Free Text Rendering in Diffusion Transformers via Attention Localization and Spectral Glyph Injection

arXiv 2601.00535 ↗First posted 2 Jan 2026cs.CVcited by 4

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Large-scale text-to-image (T2I) diffusion models excel at open-domain synthesis but still struggle with precise text rendering, especially for multi-line layouts, dense typography, and long-tailed scripts such as Chinese. Prior solutions typically require costly retraining or rigid external layout constraints, which can degrade aesthetics and limit flexibility. We propose FreeText, a training-free, plug-and-play framework that improves text rendering by exploiting intrinsic mechanisms of Diffusion Transformer (DiT) models. FreeText decomposes the problem into where to write and what to write. For where to write, we localize writing regions by reading token-wise spatial attribution from endogenous image-to-text attention, using sink-like tokens as stable spatial anchors and topology-aware refinement to produce high-confidence masks. For what to write, we introduce Spectral-Modulated Glyph Injection (SGMI), which injects a noise-aligned glyph prior with frequency-domain band-pass modulation to strengthen glyph structure and suppress semantic leakage (rendering the concept instead of the word). Extensive experiments on Qwen-Image, FLUX.1-dev, and SD3 variants across longText-Benchmark, CVTG, and our CLT-Bench show consistent gains in text readability while largely preserving semantic alignment and aesthetic quality, with modest inference overhead.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 23 of its 27 references that are in the map. First posted on arXiv 2 Jan 2026.

Builds on

23 of 27 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space17 Jun
  2. 2023AnyText: Multilingual Visual Text Generation And Editing6 Nov
  3. 2023GlyphControl: Glyph Conditional Control for Visual Text Generation29 May
  4. 2023TextDiffuser: Diffusion Models as Text Painters18 May
  5. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
  6. 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attention10 Oct
  7. 2022Blended Latent Diffusion6 Jun

Other references · 16

  1. 2022Scalable Diffusion Models with Transformerscited by 5,931
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  3. 2025Qwen-Image Technical Reportcited by 973
  4. 2023Vision Transformers Need Registerscited by 844
  5. 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 233
  6. 2023Linear Representations of Sentiment in Large Language Modelscited by 157
  7. 2025See What You Are Told: Visual Attention Sink in Large Multimodal Modelscited by 127
  8. 2025X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Againcited by 105
  9. 2025FLUX-Reason-6M & PRISM-Bench: A Million-Scale Text-to-Image Reasoning Dataset and Comprehensive Benchmarkcited by 37
  10. 2023GlyphDraw: Seamlessly Rendering Text with Intricate Spatial Structures in Text-to-Image Generationcited by 34
  11. 2021Zero-Shot Chinese Character Recognition with Stroke-Level Decompositioncited by 26
  12. 2025Investigating Text Insulation and Attention Mechanisms for Complex Visual Text Generationcited by 22
4 more
  1. 2025Attention Sinks: A 'Catch, Tag, Release' Mechanism for Embeddingscited by 16
  2. 2025LeX-Art: Rethinking Text Generation via Scalable High-Quality Data Synthesiscited by 15
  3. 2025LLM-Microscope: Uncovering the Hidden Role of Punctuation in Context Memory of Transformerscited by 12
  4. 2025Punctuation and Predicates in Language Modelscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Precise Parameter Localization for Textual Generation in Diffusion Models
  2. 2025TextGuider: Training-Free Guidance for Text Rendering via Attention Alignment
  3. 2025FLUX-Text: A Simple and Advanced Diffusion Transformer Baseline for Scene Text Editing
  4. 2025Rare Text Semantics Were Always There in Your Diffusion Transformer
  5. 2025EasyText: Controllable Diffusion Transformer for Multilingual Text Rendering
  6. 2023Masked-Attention Diffusion Guidance for Spatially Controlling Text-to-Image Generation
  7. 2025OmniText: A Training-Free Generalist for Controllable Text-Image Manipulation
  8. 2025FreeCus: Free Lunch Subject-driven Customization in Diffusion Transformers