# TextVBench: Evaluating Text Rendering and Preservation in Video Generation

arXiv: https://arxiv.org/abs/2610.13412 · HTML: https://mlsanity.com/paper/2610.13412/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 9 Oct 2026.

## Builds on (most cited first, 18 of 18)

- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [EvalCrafter: Benchmarking and Evaluating Large Video Generation Models](https://mlsanity.com/paper/2310.11440/) (2023)
- [Character Region Awareness for Text Detection](https://mlsanity.com/paper/1904.01941/) (2019)
- [Real-time Scene Text Detection with Differentiable Binarization](https://mlsanity.com/paper/1911.08947/) (2019)
- [AnyText: Multilingual Visual Text Generation And Editing](https://mlsanity.com/paper/2311.03054/) (2023)
- [Cosmos 3: Omnimodal World Models for Physical AI](https://mlsanity.com/paper/2606.02800/) (2026)
- [PaddleOCR 3.0 Technical Report](https://mlsanity.com/paper/2507.05595/) (2025)
- [TextDiffuser-2: Unleashing the Power of Language Models for Text Rendering](https://mlsanity.com/paper/2311.16465/) (2023)
- [GlyphControl: Glyph Conditional Control for Visual Text Generation](https://mlsanity.com/paper/2305.18259/) (2023)
- [LongCat-Video Technical Report](https://mlsanity.com/paper/2510.22200/) (2025)
- [Investigating Text Insulation and Attention Mechanisms for Complex Visual Text Generation](https://mlsanity.com/paper/2503.23461/) (2025)
- [End-to-End Video Text Spotting with Transformer](https://mlsanity.com/paper/2203.10539/) (2022)
- [AVGen-Bench: A Task-Driven Benchmark for Multi-Granular Evaluation of Text-to-Audio-Video Generation](https://mlsanity.com/paper/2604.08540/) (2026)
- [T2VTextBench: A Human Evaluation Benchmark for Textual Control in Video Generation Models](https://mlsanity.com/paper/2505.04946/) (2025)
- [MSAVBench: Towards Comprehensive and Reliable Evaluation of Multi-Shot Audio-Video Generation](https://mlsanity.com/paper/2605.20183/) (2026)
- [TextInVision: Text and Prompt Complexity Driven Visual Text Generation Benchmark](https://mlsanity.com/paper/2503.13730/) (2025)

## Abstract (arXiv)

Modern video generators can synthesize convincing scenes while rendering a few characters incorrectly, omitting requested text, or changing it across frames. Measuring these failures is harder than applying OCR independently to each frame: when a scene contains multiple text targets and multiple noisy OCR regions, the evaluator must first determine which observation belongs to which intended text. We introduce TextVBench, a benchmark and evaluation framework that formulates visual-text evaluation as a multi-target, multi-instance spatiotemporal correspondence problem. TextVBench contains two complementary tracks: TextVBench-T2V measures the creation and maintenance of prompt-specified text, while TextVBench-Ref2V measures preservation of text from reference images. Both tracks use target-conditioned candidate selection rather than unconstrained VLM transcription, support one target mapping to multiple OCR regions, and separate frame-level spatial fidelity from cross-frame temporal fidelity. A length-weighted harmonic score summarizes these factors, while extra-text diagnostics expose failure modes hidden by a single OCR score. The benchmark is organized around controlled text, layout, carrier, and motion attributes so that aggregate scores can be traced back to specific capability limits. Across the evaluated model runs, MiniMax-H3 reaches a T2V TextVScore of 68.32, while the strongest Ref2V score is 56.51; attribute slices reveal a pronounced drop as the number of simultaneous reference text regions increases.