# Bridging the Performance Gap between DETR and R-CNN for Graphical Object Detection in Document Images

arXiv: https://arxiv.org/abs/2306.13526 · HTML: https://mlsanity.com/paper/2306.13526/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 72 references that are in the map.
- First posted on arXiv 23 Jun 2023.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [DN-DETR: Accelerate DETR Training by Introducing Query DeNoising](https://mlsanity.com/paper/2203.01305/) (2022)
- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [Cascade R-CNN: High Quality Object Detection and Instance Segmentation](https://mlsanity.com/paper/1906.09756/) (2019)
- [Conditional DETR for Fast Training Convergence](https://mlsanity.com/paper/2108.06152/) (2021)
- [PubLayNet: largest dataset ever for document layout analysis](https://mlsanity.com/paper/1908.07836/) (2019)
- [Rethinking Transformer-based Set Prediction for Object Detection](https://mlsanity.com/paper/2011.10881/) (2020)
- [DocBank: A Benchmark Dataset for Document Layout Analysis](https://mlsanity.com/paper/2006.01038/) (2020)
- [DiT: Self-supervised Pre-training for Document Image Transformer](https://mlsanity.com/paper/2203.02378/) (2022)
- [Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Context](https://mlsanity.com/paper/2005.00589/) (2020)
- [Efficient DETR: Improving End-to-End Object Detector with Dense Prior](https://mlsanity.com/paper/2104.01318/) (2021)
- [PubTables-1M: Towards comprehensive table extraction from unstructured documents](https://mlsanity.com/paper/2110.00061/) (2021)
- [CascadeTabNet: An approach for end to end table detection and structure recognition from image-based documents](https://mlsanity.com/paper/2004.12629/) (2020)
- [Learning to Extract Semantic Structure from Documents Using Multimodal Fully Convolutional Neural Network](https://mlsanity.com/paper/1706.02337/) (2017)
- [Unified Pretraining Framework for Document Understanding](https://mlsanity.com/paper/2204.10939/) (2022)

## Built on by (most cited first, 7 of 7)

- [A Hybrid Approach for Document Layout Analysis in Document images](https://mlsanity.com/paper/2404.17888/) (2024)
- [Towards End-to-End Semi-Supervised Table Detection with Semantic Aligned Matching Transformer](https://mlsanity.com/paper/2405.00187/) (2024)
- [End-to-End Semi-Supervised approach with Modulated Object Queries for Table Detection in Documents](https://mlsanity.com/paper/2405.04971/) (2024)
- [Semi-Supervised Object Detection: A Survey on Progress from CNN to Transformer](https://mlsanity.com/paper/2407.08460/) (2024)
- [Reading the unreadable: Creating a dataset of 19th century English newspapers using image-to-text language models](https://mlsanity.com/paper/2502.14901/) (2025)
- [UnSupDLA: Towards Unsupervised Document Layout Analysis](https://mlsanity.com/paper/2406.06236/) (2024)
- [Enhanced Bank Check Security: Introducing a Novel Dataset and Transformer-Based Approach for Detection and Verification](https://mlsanity.com/paper/2406.14370/) (2024)

## Abstract (arXiv)

This paper takes an important step in bridging the performance gap between DETR and R-CNN for graphical object detection. Existing graphical object detection approaches have enjoyed recent enhancements in CNN-based object detection methods, achieving remarkable progress. Recently, Transformer-based detectors have considerably boosted the generic object detection performance, eliminating the need for hand-crafted features or post-processing steps such as Non-Maximum Suppression (NMS) using object queries. However, the effectiveness of such enhanced transformer-based detection algorithms has yet to be verified for the problem of graphical object detection. Essentially, inspired by the latest advancements in the DETR, we employ the existing detection transformer with few modifications for graphical object detection. We modify object queries in different ways, using points, anchor boxes and adding positive and negative noise to the anchors to boost performance. These modifications allow for better handling of objects with varying sizes and aspect ratios, more robustness to small variations in object positions and sizes, and improved image discrimination between objects and non-objects. We evaluate our approach on the four graphical datasets: PubTables, TableBank, NTable and PubLaynet. Upon integrating query modifications in the DETR, we outperform prior works and achieve new state-of-the-art results with the mAP of 96.9%, 95.7% and 99.3% on TableBank, PubLaynet, PubTables, respectively. The results from extensive ablations show that transformer-based methods are more effective for document analysis analogous to other applications. We hope this study draws more attention to the research of using detection transformers in document image analysis.