mlsanityData to 14 Oct 2026

Towards End-to-End Unified Scene Text Detection and Layout Analysis

arXiv 2203.15143 ↗First posted 28 Mar 2022cs.CVcited by 97

Not on the map. Why?

Abstract (arXiv)

Scene text detection and document layout analysis have long been treated as two separate tasks in different image domains. In this paper, we bring them together and introduce the task of unified scene text detection and layout analysis. The first hierarchical scene text dataset is introduced to enable this novel research task. We also propose a novel method that is able to simultaneously detect scene text and form text clusters in a unified way. Comprehensive experiments show that our unified model achieves better performance than multiple well-designed baseline methods. Additionally, this model achieves state-of-the-art results on multiple scene text detection datasets without the need of complex post-processing. Dataset and code: https://github.com/google-research-datasets/hiertext and https://github.com/tensorflow/models/tree/master/official/projects/unified_detector.

Key facts. Cited by 97 papers in the map (data through 14 Oct 2026). Builds on 43 of its 64 references that are in the map. First posted on arXiv 28 Mar 2022.

Builds on

43 of 64 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,361
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2017Mask R-CNNcited by 7,277
  6. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  7. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,241
  8. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,292
  9. 2019Towards VQA Models That Can Readcited by 1,718
  10. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 915
  11. 2018Panoptic Segmentationcited by 802
  12. 2018The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scalecited by 544
31 more
  1. 2020MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformerscited by 355
  2. 2019PubLayNet: largest dataset ever for document layout analysiscited by 351
  3. 2019Scene Text Visual Question Answeringcited by 331
  4. 2019Character Region Awareness for Text Detectioncited by 278
  5. 2021DocFormer: End-to-End Transformer for Document Understandingcited by 256
  6. 2017EAST: An Efficient and Accurate Scene Text Detectorcited by 248
  7. 2021TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene textcited by 171
  8. 2017Total-Text: A Comprehensive Dataset for Scene Text Detection and Recognitioncited by 162
  9. 2019Shape Robust Text Detection with Progressive Scale Expansion Networkcited by 147
  10. 2019ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT)cited by 147
  11. 2019ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019cited by 139
  12. 2019ICDAR 2019 Competition on Large-scale Street View Text with Partial Labeling -- RRC-LSVTcited by 122
  13. 2018TextSnake: A Flexible Representation for Detecting Text of Arbitrary Shapescited by 114
  14. 2018Scene Text Detection and Recognition: The Deep Learning Eracited by 114
  15. 2020Mask TextSpotter v3: Segmentation Proposal Network for Robust Scene Text Spottingcited by 114
  16. 2017Focusing Attention: Towards Accurate Text Recognition in Natural Imagescited by 110
  17. 2017ICDAR2017 Competition on Reading Chinese Text in the Wild (RCTW-17)cited by 105
  18. 2017Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Networkcited by 92
  19. 2017Detecting Curve Text in the Wild: New Dataset and New Solutioncited by 83
  20. 2021Fourier Contour Embedding for Arbitrary-Shaped Text Detectioncited by 80
  21. 2019MASTER: Multi-Aspect Non-local Network for Scene Text Recognitioncited by 65
  22. 2019Towards Unconstrained End-to-End Text Spottingcited by 57
  23. 2021Adaptive Boundary Proposal Network for Arbitrary Shape Text Detectioncited by 47
  24. 2021Open Images V5 Text Annotation and Yet Another Mask Text Spottercited by 46
  25. 2021MOST: A Multi-Oriented Scene Text Detector with Localization Refinementcited by 33
  26. 2021Towards Accurate Text-based Image Captioning with Content Diversity Explorationcited by 25
  27. 2021DeepLab2: A TensorFlow Library for Deep Labelingcited by 22
  28. 2021Post-OCR Paragraph Recognition by Graph Convolutional Networkscited by 16
  29. 2021CentripetalText: An Efficient Text Instance Representation for Scene Text Detectioncited by 13
  30. 2019Tightness-aware Evaluation Protocol for Scene Text Detectioncited by 8
  31. 2019TedEval: A Fair Evaluation Metric for Scene Text Detectorscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 8; 2023: 22; 2024: 29; 2025: 25; 2026: 13.

97 papers.

Sort:

By area:

  1. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 733
  2. 2023LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agentscited by 193
  3. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Documentcited by 174
  4. 2024OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoningcited by 173
  5. 2022DETRs with Hybrid Matchingcited by 150
  6. 2024VisionLLM v2: An End-to-End Generalist Multimodal Large Language Model for Hundreds of Vision-Language Taskscited by 137
  7. 2024T-Rex2: Towards Generic Object Detection via Text-Visual Prompt Synergycited by 94
  8. 2024OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognitioncited by 74
  9. 2025Detect Anything via Next Point Predictioncited by 73
  10. 2024GROUNDHOG: Grounding Large Language Models to Holistic Segmentationcited by 70
  11. 2023ChatSpot: Bootstrapping Multimodal LLMs via Precise Referring Instruction Tuningcited by 65
  12. 2025Visual Agentic Reinforcement Fine-Tuningcited by 55
38 more
  1. 2024FreeCustom: Tuning-Free Customized Image Generation for Multi-Concept Compositioncited by 49
  2. 2023Revisiting Scene Text Recognition: A Data Perspectivecited by 45
  3. 2023Turning a CLIP Model into a Scene Text Detectorcited by 44
  4. 2025Visual Large Language Models for Generalized and Specialized Applicationscited by 36
  5. 2025YOLOE: Real-Time Seeing Anythingcited by 36
  6. 2026LongCat-Next: Lexicalizing Modalities as Discrete Tokenscited by 36
  7. 2024Hi-SAM: Marrying Segment Anything Model for Hierarchical Text Segmentationcited by 30
  8. 2024ChatRex: Taming Multimodal LLM for Joint Perception and Understandingcited by 30
  9. 2023Towards Unified Scene Text Spotting based on Sequence Generationcited by 27
  10. 2023ICDAR 2023 Competition on Hierarchical Text Detection and Recognitioncited by 23
  11. 2025OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Modelscited by 23
  12. 2023Less is More: Removing Text-regions Improves CLIP Training Efficiency and Robustnesscited by 19
  13. 2025M2-omni: Advancing Omni-MLLM for Comprehensive Modality Support with Competitive Performancecited by 19
  14. 2023CLIPTER: Looking at the Bigger Picture in Scene Text Recognitioncited by 16
  15. 2024Detect-Order-Construct: A Tree Construction based Approach for Hierarchical Document Structure Analysiscited by 15
  16. 2024Exploring the Capabilities of Large Multimodal Models on Dense Textcited by 14
  17. 2025A Token-level Text Image Foundation Model for Document Understandingcited by 14
  18. 2024RoDLA: Benchmarking the Robustness of Document Layout Analysis Modelscited by 13
  19. 2025UniRec-0.1B: Unified Text and Formula Recognition with 0.1B Parameterscited by 13
  20. 2023Towards Robust Real-Time Scene Text Detection: From Semantic to Instance Representation Learningcited by 12
  21. 2025Handwritten Text Recognition: A Surveycited by 12
  22. 2022Out-of-Vocabulary Challenge Reportcited by 11
  23. 2024DLAFormer: An End-to-End Transformer For Document Layout Analysiscited by 11
  24. 2024StrucTexTv3: An Efficient Vision-Language Model for Text-rich Image Perception, Comprehension, and Beyondcited by 11
  25. 2024Out of Length Text Recognition with Sub-String Matchingcited by 11
  26. 2023Turning a CLIP Model into a Scene Text Spottercited by 9
  27. 2024Bridging the Gap Between End-to-End and Two-Step Text Spottingcited by 9
  28. 2023End-to-End Page-Level Assessment of Handwritten Text Recognitioncited by 8
  29. 2023Hierarchical Text Spotter for Joint Text Spotting and Layout Analysiscited by 8
  30. 2024Lumos : Empowering Multimodal LLMs with Scene Text Recognitioncited by 8
  31. 2024InstructOCR: Instruction Boosting Scene Text Spottingcited by 7
  32. 2025Visual Template Inference for Data Extraction from Documentscited by 7
  33. 2026Vision as Unified Multimodal Generationcited by 7
  34. 2022Contextual Text Block Detection towards Scene Text Understandingcited by 6
  35. 2022CBNet: A Plug-and-Play Network for Segmentation-Based Scene Text Detectioncited by 6
  36. 2023On Manipulating Scene Text in the Wild with Diffusion Modelscited by 6
  37. 2025SO-Bench: A Structural Output Evaluation of Multimodal LLMscited by 6
  38. 2022On Web-based Visual Corpus Construction for Visual Document Understandingcited by 5

50 most cited of 97 All 97 (JSON)