# Tightness-aware Evaluation Protocol for Scene Text Detection

arXiv: https://arxiv.org/abs/1904.00813 · HTML: https://mlsanity.com/paper/1904.00813/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2019.

## Built on by (most cited first, 8 of 8)

- [Towards End-to-End Unified Scene Text Detection and Layout Analysis](https://mlsanity.com/paper/2203.15143/) (2022)
- [TPSNet: Reverse Thinking of Thin Plate Splines for Arbitrary Shape Scene Text Representation](https://mlsanity.com/paper/2110.12826/) (2021)
- [Don't Forget Me: Accurate Background Recovery for Text Removal via Modeling Local-Global Context](https://mlsanity.com/paper/2207.10273/) (2022)
- [PERT: A Progressively Region-based Network for Scene Text Removal](https://mlsanity.com/paper/2106.13029/) (2021)
- [Visual Text Meets Low-level Vision: A Comprehensive Survey on Visual Text Processing](https://mlsanity.com/paper/2402.03082/) (2024)
- [LAME: Layout Aware Metadata Extraction Approach for Research Articles](https://mlsanity.com/paper/2112.12353/) (2021)
- [Weakly Supervised Scene Text Detection using Deep Reinforcement Learning](https://mlsanity.com/paper/2201.04866/) (2022)
- [Aggregated Text Transformer for Scene Text Detection](https://mlsanity.com/paper/2211.13984/) (2022)

## Abstract (arXiv)

Evaluation protocols play key role in the developmental progress of text detection methods. There are strict requirements to ensure that the evaluation methods are fair, objective and reasonable. However, existing metrics exhibit some obvious drawbacks: 1) They are not goal-oriented; 2) they cannot recognize the tightness of detection methods; 3) existing one-to-many and many-to-one solutions involve inherent loopholes and deficiencies. Therefore, this paper proposes a novel evaluation protocol called Tightness-aware Intersect-over-Union (TIoU) metric that could quantify completeness of ground truth, compactness of detection, and tightness of matching degree. Specifically, instead of merely using the IoU value, two common detection behaviors are properly considered; meanwhile, directly using the score of TIoU to recognize the tightness. In addition, we further propose a straightforward method to address the annotation granularity issue, which can fairly evaluate word and text-line detections simultaneously. By adopting the detection results from published methods and general object detection frameworks, comprehensive experiments on ICDAR 2013 and ICDAR 2015 datasets are conducted to compare recent metrics and the proposed TIoU metric. The comparison demonstrated some promising new prospects, e.g., determining the methods and frameworks for which the detection is tighter and more beneficial to recognize. Our method is extremely simple; however, the novelty is none other than the proposed metric can utilize simplest but reasonable improvements to lead to many interesting and insightful prospects and solving most the issues of the previous metrics. The code is publicly available at https://github.com/Yuliang-Liu/TIoU-metric .