# ECT: Fine-grained Edge Detection with Learned Cause Tokens

arXiv: https://arxiv.org/abs/2308.03092 · HTML: https://mlsanity.com/paper/2308.03092/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 39 of its 65 references that are in the map.
- First posted on arXiv 6 Aug 2023.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [RefineNet: Multi-Path Refinement Networks for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/1611.06612/) (2016)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [K-Net: Towards Unified Image Segmentation](https://mlsanity.com/paper/2106.14855/) (2021)
- [Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal](https://mlsanity.com/paper/1712.02478/) (2017)
- [Richer Convolutional Features for Edge Detection](https://mlsanity.com/paper/1612.02103/) (2016)
- [Dense Extreme Inception Network: Towards a Robust CNN Model for Edge Detection](https://mlsanity.com/paper/1909.01955/) (2019)
- [Learning to predict crisp boundaries](https://mlsanity.com/paper/1807.10097/) (2018)
- [Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection](https://mlsanity.com/paper/1901.06340/) (2019)
- [AET vs. AED: Unsupervised Representation Learning by Auto-Encoding Transformations rather than Data](https://mlsanity.com/paper/1901.04596/) (2019)
- [Bi-Directional Cascade Network for Perceptual Edge Detection](https://mlsanity.com/paper/1902.10903/) (2019)
- [InverseForm: A Loss Function for Structured Boundary-Aware Segmentation](https://mlsanity.com/paper/2104.02745/) (2021)
- [DeepEdge: A Multi-Scale Bifurcated Deep Network for Top-Down Contour Detection](https://mlsanity.com/paper/1412.1123/) (2014)
- [Predicting Sharp and Accurate Occlusion Boundaries in Monocular Depth Estimation Using Displacement Fields](https://mlsanity.com/paper/2002.12730/) (2020)
- [CASENet: Deep Category-Aware Semantic Edge Detection](https://mlsanity.com/paper/1705.09759/) (2017)
- [EDTER: Edge Detection with Transformer](https://mlsanity.com/paper/2203.08566/) (2022)
- [Pushing the Boundaries of Boundary Detection using Deep Learning](https://mlsanity.com/paper/1511.07386/) (2015)
- [RINDNet: Edge Detection for Discontinuity in Reflectance, Illumination, Normal and Depth](https://mlsanity.com/paper/2108.00616/) (2021)
- [Devil is in the Edges: Learning Semantic Boundaries from Noisy Annotations](https://mlsanity.com/paper/1904.07934/) (2019)
- [Learning Deep Structured Multi-Scale Features using Attention-Gated CRFs for Contour Prediction](https://mlsanity.com/paper/1801.00524/) (2018)
- [Physics Inspired Optimization on Semantic Transfer Features: An Alternative Method for Room Layout Estimation](https://mlsanity.com/paper/1707.00383/) (2017)
- [PQ-Transformer: Jointly Parsing 3D Objects and Layouts from Point Clouds](https://mlsanity.com/paper/2109.05566/) (2021)
- [High-for-Low and Low-for-High: Efficient Boundary Detection from Deep Object Features and its Applications to High-Level Vision](https://mlsanity.com/paper/1504.06201/) (2015)
- [Simultaneous Edge Alignment and Learning](https://mlsanity.com/paper/1808.01992/) (2018)

## Abstract (arXiv)

In this study, we tackle the challenging fine-grained edge detection task, which refers to predicting specific edges caused by reflectance, illumination, normal, and depth changes, respectively. Prior methods exploit multi-scale convolutional networks, which are limited in three aspects: (1) Convolutions are local operators while identifying the cause of edge formation requires looking at far away pixels. (2) Priors specific to edge cause are fixed in prediction heads. (3) Using separate networks for generic and fine-grained edge detection, and the constraint between them may be violated. To address these three issues, we propose a two-stage transformer-based network sequentially predicting generic edges and fine-grained edges, which has a global receptive field thanks to the attention mechanism. The prior knowledge of edge causes is formulated as four learnable cause tokens in a cause-aware decoder design. Furthermore, to encourage the consistency between generic edges and fine-grained edges, an edge aggregation and alignment loss is exploited. We evaluate our method on the public benchmark BSDS-RIND and several newly derived benchmarks, and achieve new state-of-the-art results. Our code, data, and models are publicly available at https://github.com/Daniellli/ECT.git.