# Choose What You Need: Disentangled Representation Learning for Scene Text Recognition, Removal and Editing

arXiv: https://arxiv.org/abs/2405.04377 · HTML: https://mlsanity.com/paper/2405.04377/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 57 references that are in the map.
- First posted on arXiv 7 May 2024.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [TextDiffuser: Diffusion Models as Text Painters](https://mlsanity.com/paper/2305.10855/) (2023)
- [Synthetic Data and Artificial Neural Networks for Natural Scene Text Recognition](https://mlsanity.com/paper/1406.2227/) (2014)
- [AnyText: Multilingual Visual Text Generation And Editing](https://mlsanity.com/paper/2311.03054/) (2023)
- [TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering](https://mlsanity.com/paper/2311.16465/) (2023)
- [SVTR: Scene Text Recognition with a Single Visual Model](https://mlsanity.com/paper/2205.00159/) (2022)
- [MomentDiff: Generative Video Moment Retrieval from Random to Real](https://mlsanity.com/paper/2307.02869/) (2023)
- [DiffUTE: Universal Text Editing Diffusion Model](https://mlsanity.com/paper/2305.10825/) (2023)
- [MaskOCR: Text Recognition with Masked Encoder-Decoder Pretraining](https://mlsanity.com/paper/2206.00311/) (2022)
- [Towards Balanced Alignment: Modal-Enhanced Semantic Modeling for Video Moment Retrieval](https://mlsanity.com/paper/2312.12155/) (2023)
- [Linguistic More: Taking a Further Step toward Efficient and Accurate Scene Text Recognition](https://mlsanity.com/paper/2305.05140/) (2023)
- [ViTEraser: Harnessing the Power of Vision Transformers for Scene Text Removal with SegMIM Pretraining](https://mlsanity.com/paper/2306.12106/) (2023)

## Built on by (most cited first, 18 of 18)

- [Hybrid-Level Instruction Injection for Video Token Compression in Multi-modal Large Language Models](https://mlsanity.com/paper/2503.16036/) (2025)
- [Visual Text Processing: A Comprehensive Review and Unified Evaluation](https://mlsanity.com/paper/2504.21682/) (2025)
- [Graph-based Unsupervised Disentangled Representation Learning via Multimodal Large Language Models](https://mlsanity.com/paper/2407.18999/) (2024)
- [How Control Information Influences Multilingual Text Image Generation and Editing?](https://mlsanity.com/paper/2407.11502/) (2024)
- [Self-Supervised Pre-training with Symmetric Superimposition Modeling for Scene Text Recognition](https://mlsanity.com/paper/2405.05841/) (2024)
- [ConText: Driving In-context Learning for Text Removal and Segmentation](https://mlsanity.com/paper/2506.03799/) (2025)
- [OmniText: A Training-Free Generalist for Controllable Text-Image Manipulation](https://mlsanity.com/paper/2510.24093/) (2025)
- [Mitigating Translationese Bias in Multilingual LLM-as-a-Judge via Disentangled Information Bottleneck](https://mlsanity.com/paper/2603.10351/) (2026)
- [CLEAR: Context-Aware Learning with End-to-End Mask-Free Inference for Adaptive Video Subtitle Removal](https://mlsanity.com/paper/2603.21901/) (2026)
- [Exploring the Performance Frontier of Compact Unified Image Generation Models](https://mlsanity.com/paper/2608.20334/) (2026)
- [Prompt-guided Disentangled Representation for Action Recognition](https://mlsanity.com/paper/2509.21783/) (2025)
- [MULTITEXTEDIT: Benchmarking Cross-Lingual Degradation in Text-in-Image Editing](https://mlsanity.com/paper/2605.08163/) (2026)
- [Towards Spoken Language Understanding via Multi-level Multi-grained Contrastive Learning](https://mlsanity.com/paper/2405.20852/) (2024)
- [SceneTextStylizer: A Training-Free Scene Text Style Transfer Framework with Diffusion Model](https://mlsanity.com/paper/2510.10910/) (2025)
- [TripleFDS: Triple Feature Disentanglement and Synthesis for Scene Text Editing](https://mlsanity.com/paper/2511.13399/) (2025)
- [DocRevive: A Unified Pipeline for Document Text Restoration](https://mlsanity.com/paper/2604.10077/) (2026)
- [Consistent Feature Transport for Image Relighting](https://mlsanity.com/paper/2607.17833/) (2026)
- [Knowledge-guided Disentanglement with Atomic Actions for Action Recognition](https://mlsanity.com/paper/2607.26097/) (2026)

## Abstract (arXiv)

Scene text images contain not only style information (font, background) but also content information (character, texture). Different scene text tasks need different information, but previous representation learning methods use tightly coupled features for all tasks, resulting in sub-optimal performance. We propose a Disentangled Representation Learning framework (DARLING) aimed at disentangling these two types of features for improved adaptability in better addressing various downstream tasks (choose what you really need). Specifically, we synthesize a dataset of image pairs with identical style but different content. Based on the dataset, we decouple the two types of features by the supervision design. Clearly, we directly split the visual representation into style and content features, the content features are supervised by a text recognition loss, while an alignment loss aligns the style features in the image pairs. Then, style features are employed in reconstructing the counterpart image via an image decoder with a prompt that indicates the counterpart's content. Such an operation effectively decouples the features based on their distinctive properties. To the best of our knowledge, this is the first time in the field of scene text that disentangles the inherent properties of the text images. Our method achieves state-of-the-art performance in Scene Text Recognition, Removal, and Editing.