WeEdit: A Dataset, Benchmark and Glyph-Guided Framework for Text-centric Image Editing
Not on the map. Why?
Abstract (arXiv)
Instruction-based image editing aims to modify specific content within existing images according to user-provided instructions while preserving non-target regions. Beyond traditional object- and style-centric manipulation, text-centric image editing focuses on modifying, translating, or rearranging textual elements embedded within images. However, existing leading models often struggle to execute complex text editing precisely, frequently producing blurry or hallucinated characters. We attribute these failures primarily to the lack of specialized training paradigms tailored for text-centric editing, as well as the absence of large-scale datasets and standardized benchmarks necessary for a closed-loop training and evaluation system. To address these limitations, we present WeEdit, a systematic solution encompassing a scalable data construction pipeline, two benchmarks, and a tailored two-stage training strategy. Specifically, we propose a novel HTML-based automatic editing pipeline, which generates 330K training pairs covering diverse editing operations and 15 languages, accompanied by standardized bilingual and multilingual benchmarks for comprehensive evaluation. On the algorithmic side, we employ glyph-guided supervised fine-tuning to inject explicit spatial and content priors, followed by a multi-objective reinforcement learning stage to align generation with instruction adherence, text clarity, and background preservation. Extensive experiments demonstrate that WeEdit outperforms previous open-source models by a clear margin across diverse editing operations.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 79 of its 89 references that are in the map. First posted on arXiv 12 Mar 2026.
Builds on
79 of 89 references.
References · 79
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
38 more
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
- 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
- 2025Qwen-Image Technical Reportcited by 973
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
- 2024OmniGen: Unified Image Generationcited by 388
- 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 388
- 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
- 2024What matters when building vision-language models?cited by 321
- 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 294
- 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
- 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
- 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
- 2023TextDiffuser: Diffusion Models as Text Painterscited by 223
- 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 204
- 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199
- 2024HQ-Edit: A High-Quality Dataset for Instruction-based Image Editingcited by 191
- 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 168
- 2023AnyText: Multilingual Visual Text Generation And Editingcited by 165
- 2024In-Context LoRA for Diffusion Transformerscited by 155
- 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 153
- 2025Seedream 3.0 Technical Reportcited by 146
- 2024OmniEdit: Building Image Editing Generalist Models Through Specialist Supervisioncited by 145
- 2025HunyuanImage 3.0 Technical Reportcited by 138
- 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
- 2023TextDiffuser-2: Unleashing the Power of Language Models for Text Renderingcited by 136
- 2025HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformercited by 131
- 2023GlyphControl: Glyph Conditional Control for Visual Text Generationcited by 127
- 2024Unlocking the conversion of Web Screenshots into HTML Code with the WebSight Datasetcited by 126
- 2024MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesiscited by 123
50 most cited of 79 All 79 (JSON)
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026GMO-E2DIT: Grounded Multi-Operation Editing for E-Commerce Imagescited by 3
- 2026SenseNova-U1.5: Towards Native Unified Visual Intelligencecited by 3
- 2026TextSculptor: Training and Benchmarking Scene Text Editingcited by 1
- 2026WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editingcited by 0
- 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 0
- 2026UltraText Bench: A Comprehensive Bilingual Benchmark for Evaluating Visual Text Rendering in Image Generationcited by 0
- 2026UltraText Bench: A Comprehensive Bilingual Benchmark for Evaluating Visual Text Rendering in Image Generation7 Oct 2026
- 2026SenseNova-U1.5: Towards Native Unified Visual Intelligence10 Sep 2026
- 2026Agentic Visual Generation: From Generative Models to Agentic Control6 Sep 2026
- 2026WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing4 Sep 2026
- 2026GMO-E2DIT: Grounded Multi-Operation Editing for E-Commerce Images1 Jul 2026
- 2026TextSculptor: Training and Benchmarking Scene Text Editing20 May 2026