# SAUGE: Taming SAM for Uncertainty-Aligned Multi-Granularity Edge Detection

arXiv: https://arxiv.org/abs/2412.12892 · HTML: https://mlsanity.com/paper/2412.12892/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 17 Dec 2024.

## Builds on (most cited first, 16 of 16)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [Inpaint Anything: Segment Anything Meets Image Inpainting](https://mlsanity.com/paper/2304.06790/) (2023)
- [Pixel Difference Networks for Efficient Edge Detection](https://mlsanity.com/paper/2108.07009/) (2021)
- [Richer Convolutional Features for Edge Detection](https://mlsanity.com/paper/1612.02103/) (2016)
- [Low-Light Image Enhancement via Structure Modeling and Guidance](https://mlsanity.com/paper/2305.05839/) (2023)
- [Bi-Directional Cascade Network for Perceptual Edge Detection](https://mlsanity.com/paper/1902.10903/) (2019)
- [Pushing the Boundaries of Boundary Detection using Deep Learning](https://mlsanity.com/paper/1511.07386/) (2015)
- [Dynamic Feature Integration for Simultaneous Detection of Salient Object, Edge and Skeleton](https://mlsanity.com/paper/2004.08595/) (2020)
- [Oriented Edge Forests for Boundary Detection](https://mlsanity.com/paper/1412.4181/) (2014)
- [The Treasure Beneath Multiple Annotations: An Uncertainty-aware Edge Detector](https://mlsanity.com/paper/2303.11828/) (2023)
- [RankED: Addressing Imbalance and Uncertainty in Edge Detection Using Ranking-based Losses](https://mlsanity.com/paper/2403.01795/) (2024)
- [Masked and Adaptive Transformer for Exemplar Based Image Translation](https://mlsanity.com/paper/2303.17123/) (2023)
- [Boosting Medical Image Classification with Segmentation Foundation Model](https://mlsanity.com/paper/2406.11026/) (2024)

## Built on by (most cited first, 7 of 7)

- [MatchED: Crisp Edge Detection Using End-to-End, Matching-based Supervision](https://mlsanity.com/paper/2602.20689/) (2026)
- [TexEditor: Structure-Preserving Text-Driven Texture Editing](https://mlsanity.com/paper/2603.18488/) (2026)
- [MEMO: Human-like Crisp Edge Detection Using Masked Edge Prediction](https://mlsanity.com/paper/2603.20782/) (2026)
- [PEdger++: Practical Edge Detection via Assembling Cross Information](https://mlsanity.com/paper/2508.11961/) (2025)
- [HeroGS: Hierarchical Guidance for Robust 3D Gaussian Splatting under Sparse Views](https://mlsanity.com/paper/2603.01099/) (2026)
- [From Boundaries to Semantics: Prompt-Guided Multi-Task Learning for Petrographic Thin-section Segmentation](https://mlsanity.com/paper/2604.14805/) (2026)
- [RealOOB: A Definition-Consistent Real-World Oriented Occlusion Boundary Benchmark](https://mlsanity.com/paper/2608.30820/) (2026)

## Abstract (arXiv)

Edge labels are typically at various granularity levels owing to the varying preferences of annotators, thus handling the subjectivity of per-pixel labels has been a focal point for edge detection. Previous methods often employ a simple voting strategy to diminish such label uncertainty or impose a strong assumption of labels with a pre-defined distribution, e.g., Gaussian. In this work, we unveil that the segment anything model (SAM) provides strong prior knowledge to model the uncertainty in edge labels. Our key insight is that the intermediate SAM features inherently correspond to object edges at various granularities, which reflects different edge options due to uncertainty. Therefore, we attempt to align uncertainty with granularity by regressing intermediate SAM features from different layers to object edges at multi-granularity levels. In doing so, the model can fully and explicitly explore diverse ``uncertainties'' in a data-driven fashion. Specifically, we inject a lightweight module (~ 1.5% additional parameters) into the frozen SAM to progressively fuse and adapt its intermediate features to estimate edges from coarse to fine. It is crucial to normalize the granularity level of human edge labels to match their innate uncertainty. For this, we simply perform linear blending to the real edge labels at hand to create pseudo labels with varying granularities. Consequently, our uncertainty-aligned edge detector can flexibly produce edges at any desired granularity (including an optimal one). Thanks to SAM, our model uniquely demonstrates strong generalizability for cross-dataset edge detection. Extensive experimental results on BSDS500, Muticue and NYUDv2 validate our model's superiority.