# Learning Affinity via Spatial Propagation Networks

arXiv: https://arxiv.org/abs/1710.01020 · HTML: https://mlsanity.com/paper/1710.01020/

## Key facts

- Cited by 79 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Oct 2017.

## Built on by (most cited first, 30 of 79)

- [PENet: Towards Precise and Efficient Image Guided Depth Completion](https://mlsanity.com/paper/2103.00783/) (2021)
- [CompletionFormer: Depth Completion with Convolutions and Vision Transformers](https://mlsanity.com/paper/2304.13030/) (2023)
- [3D Semantic Scene Completion: a Survey](https://mlsanity.com/paper/2103.07466/) (2021)
- [Representation Compensation Networks for Continual Semantic Segmentation](https://mlsanity.com/paper/2203.05402/) (2022)
- [Dynamic Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2202.09769/) (2022)
- [InverseForm: A Loss Function for Structured Boundary-Aware Segmentation](https://mlsanity.com/paper/2104.02745/) (2021)
- [High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network](https://mlsanity.com/paper/2105.09188/) (2021)
- [Bilateral Propagation Network for Depth Completion](https://mlsanity.com/paper/2403.11270/) (2024)
- [Look Before You Leap: Learning Landmark Features for One-Stage Visual Grounding](https://mlsanity.com/paper/2104.04386/) (2021)
- [Marigold-DC: Zero-Shot Monocular Depth Completion with Guided Diffusion](https://mlsanity.com/paper/2412.13389/) (2024)
- [LRRU: Long-short Range Recurrent Updating Networks for Depth Completion](https://mlsanity.com/paper/2310.08956/) (2023)
- [Towards Accurate Reconstruction of 3D Scene Shape from A Single Monocular Image](https://mlsanity.com/paper/2208.13241/) (2022)
- [Tri-Perspective View Decomposition for Geometry-Aware Depth Completion](https://mlsanity.com/paper/2403.15008/) (2024)
- [Open-World Instance Segmentation: Exploiting Pseudo Ground Truth From Learned Pairwise Affinity](https://mlsanity.com/paper/2204.06107/) (2022)
- [Domain-Aware Universal Style Transfer](https://mlsanity.com/paper/2108.04441/) (2021)
- [Deep Depth Completion from Extremely Sparse Data: A Survey](https://mlsanity.com/paper/2205.05335/) (2022)
- [ProtoSeg: Interpretable Semantic Segmentation with Prototypical Parts](https://mlsanity.com/paper/2301.12276/) (2023)
- [High Quality Segmentation for Ultra High-resolution Images](https://mlsanity.com/paper/2111.14482/) (2021)
- [OMNI-DC: Highly Robust Depth Completion with Multiresolution Depth Integration](https://mlsanity.com/paper/2411.19278/) (2024)
- [Sparsity Agnostic Depth Completion](https://mlsanity.com/paper/2212.00790/) (2022)
- [DELTAR: Depth Estimation from a Light-weight ToF Sensor and RGB Image](https://mlsanity.com/paper/2209.13362/) (2022)
- [Depth Prompting for Sensor-Agnostic Depth Estimation](https://mlsanity.com/paper/2405.11867/) (2024)
- [CAP-VSTNet: Content Affinity Preserved Versatile Style Transfer](https://mlsanity.com/paper/2303.17867/) (2023)
- [SemAttNet: Towards Attention-based Semantic Aware Guided Depth Completion](https://mlsanity.com/paper/2204.13635/) (2022)
- [GraphCSPN: Geometry-Aware Depth Completion via Dynamic GCNs](https://mlsanity.com/paper/2210.10758/) (2022)
- [RGB-Depth Fusion GAN for Indoor Depth Completion](https://mlsanity.com/paper/2203.10856/) (2022)
- [Revisiting Depth Completion from a Stereo Matching Perspective for Cross-domain Generalization](https://mlsanity.com/paper/2312.09254/) (2023)
- [Delving into Shape-aware Zero-shot Semantic Segmentation](https://mlsanity.com/paper/2304.08491/) (2023)
- [Segmenting Transparent Object in the Wild with Transformer](https://mlsanity.com/paper/2101.08461/) (2021)
- [SRFNet: Monocular Depth Estimation with Fine-grained Structure via Spatial Reliability-oriented Fusion of Frames and Events](https://mlsanity.com/paper/2309.12842/) (2023)

## Abstract (arXiv)

In this paper, we propose spatial propagation networks for learning the affinity matrix for vision tasks. We show that by constructing a row/column linear propagation model, the spatially varying transformation matrix exactly constitutes an affinity matrix that models dense, global pairwise relationships of an image. Specifically, we develop a three-way connection for the linear propagation model, which (a) formulates a sparse transformation matrix, where all elements can be the output from a deep CNN, but (b) results in a dense affinity matrix that effectively models any task-specific pairwise similarity matrix. Instead of designing the similarity kernels according to image features of two points, we can directly output all the similarities in a purely data-driven manner. The spatial propagation network is a generic framework that can be applied to many affinity-related tasks, including but not limited to image matting, segmentation and colorization, to name a few. Essentially, the model can learn semantically-aware affinity values for high-level vision tasks due to the powerful learning capability of the deep neural network classifier. We validate the framework on the task of refinement for image segmentation boundaries. Experiments on the HELEN face parsing and PASCAL VOC-2012 semantic segmentation tasks show that the spatial propagation network provides a general, effective and efficient solution for generating high-quality segmentation results.