# First-place Solution for Streetscape Shop Sign Recognition Competition

arXiv: https://arxiv.org/abs/2501.02811 · HTML: https://mlsanity.com/paper/2501.02811/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 61 of its 97 references that are in the map.
- First posted on arXiv 6 Jan 2025.

## Builds on (most cited first, 30 of 61)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [Real-time Scene Text Detection with Differentiable Binarization](https://mlsanity.com/paper/1911.08947/) (2019)
- [ABCNet: Real-time Scene Text Spotting with Adaptive Bezier-Curve Network](https://mlsanity.com/paper/2002.10200/) (2020)
- [Shape Robust Text Detection with Progressive Scale Expansion Network](https://mlsanity.com/paper/1903.12473/) (2019)
- [ICDAR2019 Robust Reading Challenge on Multi-lingual Scene Text Detection and Recognition -- RRC-MLT-2019](https://mlsanity.com/paper/1907.00945/) (2019)
- [Context-aware Biaffine Localizing Network for Temporal Sentence Grounding](https://mlsanity.com/paper/2103.11555/) (2021)
- [DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understanding](https://mlsanity.com/paper/2311.11810/) (2023)
- [Advancing Pose-Guided Image Synthesis with Progressive Conditional Diffusion Models](https://mlsanity.com/paper/2310.06313/) (2023)
- [IMAGDressing-v1: Customizable Virtual Dressing](https://mlsanity.com/paper/2407.12705/) (2024)
- [A Survey of Attacks on Large Vision-Language Models: Resources, Advances, and Future Trends](https://mlsanity.com/paper/2407.07403/) (2024)
- [Fourier Contour Embedding for Arbitrary-Shaped Text Detection](https://mlsanity.com/paper/2104.10442/) (2021)
- [Jointly Cross- and Self-Modal Graph Attention Network for Query-Based Moment Localization](https://mlsanity.com/paper/2008.01403/) (2020)
- [UniDoc: A Universal Large Multimodal Model for Simultaneous Text Detection, Recognition, Spotting and Understanding](https://mlsanity.com/paper/2308.11592/) (2023)
- [TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergy](https://mlsanity.com/paper/2406.01326/) (2024)
- [Sequence-to-Sequence Contrastive Learning for Text Recognition](https://mlsanity.com/paper/2012.10873/) (2020)
- [SPTS v2: Single-Point Scene Text Spotting](https://mlsanity.com/paper/2301.01635/) (2023)
- [Few Could Be Better Than All: Feature Sampling and Grouping for Scene Text Detection](https://mlsanity.com/paper/2203.15221/) (2022)
- [Attentive Eraser: Unleashing Diffusion Model's Object Removal Potential via Self-Attention Redirection Guidance](https://mlsanity.com/paper/2412.12974/) (2024)
- [TextSquare: Scaling up Text-Centric Visual Instruction Tuning](https://mlsanity.com/paper/2404.12803/) (2024)
- [Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learning](https://mlsanity.com/paper/2407.01531/) (2024)
- [A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understanding](https://mlsanity.com/paper/2407.01976/) (2024)
- [Boosting Consistency in Story Visualization with Rich-Contextual Conditional Diffusion Models](https://mlsanity.com/paper/2407.02482/) (2024)
- [Harmonizing Visual Text Comprehension and Generation](https://mlsanity.com/paper/2407.16364/) (2024)
- [Multi-modal In-Context Learning Makes an Ego-evolving Scene Text Recognizer](https://mlsanity.com/paper/2311.13120/) (2023)
- [Seeing Clearly by Layer Two: Enhancing Attention Heads to Alleviate Hallucination in LVLMs](https://mlsanity.com/paper/2411.09968/) (2024)

## Abstract (arXiv)

Text recognition technology applied to street-view storefront signs is increasingly utilized across various practical domains, including map navigation, smart city planning analysis, and business value assessments in commercial districts. This technology holds significant research and commercial potential. Nevertheless, it faces numerous challenges. Street view images often contain signboards with complex designs and diverse text styles, complicating the text recognition process. A notable advancement in this field was introduced by our team in a recent competition. We developed a novel multistage approach that integrates multimodal feature fusion, extensive self-supervised training, and a Transformer-based large model. Furthermore, innovative techniques such as BoxDQN, which relies on reinforcement learning, and text rectification methods were employed, leading to impressive outcomes. Comprehensive experiments have validated the effectiveness of these methods, showcasing our potential to enhance text recognition capabilities in complex urban environments.