# CornerFormer: Boosting Corner Representation for Fine-Grained Structured Reconstruction

arXiv: https://arxiv.org/abs/2304.07072 · HTML: https://mlsanity.com/paper/2304.07072/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 39 references that are in the map.
- First posted on arXiv 14 Apr 2023.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detection](https://mlsanity.com/paper/2203.03605/) (2022)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [Conditional DETR for Fast Training Convergence](https://mlsanity.com/paper/2108.06152/) (2021)
- [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016)
- [Human Pose Regression with Residual Log-likelihood Estimation](https://mlsanity.com/paper/2107.11291/) (2021)
- [Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation](https://mlsanity.com/paper/1406.2984/) (2014)
- [Efficient Interactive Annotation of Segmentation Datasets with Polygon-RNN++](https://mlsanity.com/paper/1803.09693/) (2018)
- [2D/3D Pose Estimation and Action Recognition using Multitask Deep Learning](https://mlsanity.com/paper/1802.09232/) (2018)
- [CenterFormer: Center-based Transformer for 3D Object Detection](https://mlsanity.com/paper/2209.05588/) (2022)
- [Lite-HRNet: A Lightweight High-Resolution Network](https://mlsanity.com/paper/2104.06403/) (2021)
- [Annotating Object Instances with a Polygon-RNN](https://mlsanity.com/paper/1704.05548/) (2017)
- [Line Segment Detection Using Transformers without Edges](https://mlsanity.com/paper/2101.01909/) (2021)
- [End-to-End Wireframe Parsing](https://mlsanity.com/paper/1905.03246/) (2019)
- [Learning to Parse Wireframes in Images of Man-Made Environments](https://mlsanity.com/paper/2007.07527/) (2020)
- [Holistically-Attracted Wireframe Parsing](https://mlsanity.com/paper/2003.01663/) (2020)
- [PPGNet: Learning Point-Pair Graph for Line Segment Detection](https://mlsanity.com/paper/1905.03415/) (2019)
- [Floor-SP: Inverse CAD for Floorplans by Sequential Room-wise Shortest Path](https://mlsanity.com/paper/1908.06702/) (2019)
- [HEAT: Holistic Edge Attention Transformer for Structured Reconstruction](https://mlsanity.com/paper/2111.15143/) (2021)
- [MonteFloor: Extending MCTS for Reconstructing Accurate Large-Scale Floor Plans](https://mlsanity.com/paper/2103.11161/) (2021)
- [Conv-MPN: Convolutional Message Passing Neural Network for Structured Outdoor Architecture Reconstruction](https://mlsanity.com/paper/1912.01756/) (2019)
- [Vectorizing World Buildings: Planar Graph Reconstruction by Primitive Detection and Relationship Inference](https://mlsanity.com/paper/1912.05135/) (2019)
- [Localization with Sampling-Argmax](https://mlsanity.com/paper/2110.08825/) (2021)

## Abstract (arXiv)

Structured reconstruction is a non-trivial dense prediction problem, which extracts structural information (eg, building corners and edges) from a raster image, then reconstructs it to a 2D planar graph accordingly. Compared with common segmentation or detection problems, it significantly relays on the capability that leveraging holistic geometric information for structural reasoning. Current transformer-based approaches tackle this challenging problem in a two-stage manner, which detect corners in the first model and classify the proposed edges (corner-pairs) in the second model. However, they separate two-stage into different models and only share the backbone encoder. Unlike the existing modeling strategies, we present an enhanced corner representation method: 1) It fuses knowledge between the corner detection and edge prediction by sharing feature in different granularity; 2) Corner candidates are proposed in four heatmap channels w.r.t its direction. Both qualitative and quantitative evaluations demonstrate that our proposed method can better reconstruct fine-grained structures, such as adjacent corners and tiny edges. Consequently, it outperforms the state-of-the-art model by +1.9%@F-1 on Corner and +3.0%@F-1 on Edge.