# Self-Prompting Diffusion Transformer for Open-Vocabulary Scene Text Editing via In-Context Learning

arXiv: https://arxiv.org/abs/2605.15523 · HTML: https://mlsanity.com/paper/2605.15523/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 28 of its 33 references that are in the map.
- First posted on arXiv 15 May 2026.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 28 of 28)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [Image Inpainting for Irregular Holes Using Partial Convolutions](https://mlsanity.com/paper/1804.07723/) (2018)
- [TextDiffuser: Diffusion Models as Text Painters](https://mlsanity.com/paper/2305.10855/) (2023)
- [BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion](https://mlsanity.com/paper/2403.06976/) (2024)
- [AnyText: Multilingual Visual Text Generation And Editing](https://mlsanity.com/paper/2311.03054/) (2023)
- [A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting](https://mlsanity.com/paper/2312.03594/) (2023)
- [PP-OCR: A Practical Ultra Lightweight OCR System](https://mlsanity.com/paper/2009.09941/) (2020)
- [ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019](https://mlsanity.com/paper/1907.00945/) (2019)
- [LongCat-Image Technical Report](https://mlsanity.com/paper/2512.07584/) (2025)
- [HD-Painter: High-Resolution and Prompt-Faithful Text-Guided Image Inpainting with Diffusion Models](https://mlsanity.com/paper/2312.14091/) (2023)
- [AnyText2: Visual Text Generation and Editing With Customizable Attributes](https://mlsanity.com/paper/2411.15245/) (2024)
- [SwapText: Image Based Texts Transfer in Scenes](https://mlsanity.com/paper/2003.08152/) (2020)
- [FLUX-Text: A Simple and Advanced Diffusion Transformer Baseline for Scene Text Editing](https://mlsanity.com/paper/2505.03329/) (2025)
- [TextCtrl: Diffusion-based Scene Text Editing with Prior Guidance Control](https://mlsanity.com/paper/2410.10133/) (2024)
- [Exploring Stroke-Level Modifications for Scene Text Editing](https://mlsanity.com/paper/2212.01982/) (2022)
- [FireRed-Image-Edit-1.0 Technical Report](https://mlsanity.com/paper/2602.13344/) (2026)
- [Metadata Conditioning Accelerates Language Model Pre-training](https://mlsanity.com/paper/2501.01956/) (2025)
- [DreamText: High Fidelity Scene Text Synthesis](https://mlsanity.com/paper/2405.14701/) (2024)
- [TextFlux: An OCR-Free DiT Model for High-Fidelity Multilingual Scene Text Synthesis](https://mlsanity.com/paper/2505.17778/) (2025)
- [GlyphMastero: A Glyph Encoder for High-Fidelity Scene Text Editing](https://mlsanity.com/paper/2505.04915/) (2025)
- [ThaiOCRBench: A Task-Diverse Benchmark for Vision-Language Understanding in Thai](https://mlsanity.com/paper/2511.04479/) (2025)
- [RusTitW: Russian Language Text Dataset for Visual Text in-the-Wild Recognition](https://mlsanity.com/paper/2303.16531/) (2023)
- [The First Swahili Language Scene Text Detection and Recognition Dataset](https://mlsanity.com/paper/2405.11437/) (2024)

## Built on by (most cited first, 1 of 1)

- [Efficient Scene-Text Refinement via Pixel-Space Grouped Image Generation](https://mlsanity.com/paper/2610.18284/) (2026)

## Abstract (arXiv)

Scene text editing aims to modify text in a target region of an image while preserving surrounding background style and texture. Existing methods rely solely on image background information while neglecting the visual details of target regions, which discards stylistic features in the original text and essentially degrades the task to text rendering. Moreover, the conditions imposed by pre-trained glyph encoder limit the scope of editable text. To address these issues, this paper proposes a self-prompting scene text editing method that constructs style and glyph prompts directly from the original image, without introducing additional style or glyph encoders. We employ a two-stage training strategy: the diffusion transformer is first trained on large-scale self-supervised data and then refined using a small set of paired images. By leveraging the in-context learning capability of the Multi-Modal Diffusion Transformer (MM-DiT), it achieves open-vocabulary and style-consistent text editing. Experimental results on various languages demonstrate that our method achieves the state-of-the-art performance in both text accuracy and style consistency. Our project page: hongxiii.github.io/mstedit.