# Adaptive Affinity Fields for Semantic Segmentation

arXiv: https://arxiv.org/abs/1803.10335 · HTML: https://mlsanity.com/paper/1803.10335/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Mar 2018.

## Built on by (most cited first, 30 of 48)

- [Exploring Cross-Image Pixel Contrast for Semantic Segmentation](https://mlsanity.com/paper/2101.11939/) (2021)
- [Rethinking Semantic Segmentation: A Prototype View](https://mlsanity.com/paper/2203.15102/) (2022)
- [Learning Affinity from Attention: End-to-End Weakly-Supervised Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.02664/) (2022)
- [Enhanced Boundary Learning for Glass-like Object Segmentation](https://mlsanity.com/paper/2103.15734/) (2021)
- [Mining Contextual Information Beyond Image for Semantic Segmentation](https://mlsanity.com/paper/2108.11819/) (2021)
- [CTNet: Context-based Tandem Network for Semantic Segmentation](https://mlsanity.com/paper/2104.09805/) (2021)
- [Active Boundary Loss for Semantic Segmentation](https://mlsanity.com/paper/2102.02696/) (2021)
- [Unsupervised Hierarchical Semantic Segmentation with Multiview Cosegmentation and Clustering Transformers](https://mlsanity.com/paper/2204.11432/) (2022)
- [Learning Implicit Feature Alignment Function for Semantic Segmentation](https://mlsanity.com/paper/2206.08655/) (2022)
- [Adaptive Affinity Loss and Erroneous Pseudo-Label Refinement for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2108.01344/) (2021)
- [ACDC: The Adverse Conditions Dataset with Correspondences for Robust Semantic Driving Scene Perception](https://mlsanity.com/paper/2104.13395/) (2021)
- [Contextrast: Contextual Contrastive Learning for Semantic Segmentation](https://mlsanity.com/paper/2404.10633/) (2024)
- [Dynamic Neural Representational Decoders for High-Resolution Semantic Segmentation](https://mlsanity.com/paper/2107.14428/) (2021)
- [Fully Attentional Network for Semantic Segmentation](https://mlsanity.com/paper/2112.04108/) (2021)
- [Shallow Features Guide Unsupervised Domain Adaptation for Semantic Segmentation at Class Boundaries](https://mlsanity.com/paper/2110.02833/) (2021)
- [MCIBI++: Soft Mining Contextual Information Beyond Image for Semantic Segmentation](https://mlsanity.com/paper/2209.04471/) (2022)
- [Global Aggregation then Local Distribution for Scene Parsing](https://mlsanity.com/paper/2107.13154/) (2021)
- [Multi-scale and Cross-scale Contrastive Learning for Semantic Segmentation](https://mlsanity.com/paper/2203.13409/) (2022)
- [PLGAN: Generative Adversarial Networks for Power-Line Segmentation in Aerial Images](https://mlsanity.com/paper/2204.07243/) (2022)
- [IDRNet: Intervention-Driven Relation Network for Semantic Segmentation](https://mlsanity.com/paper/2310.10755/) (2023)
- [Beyond mAP: Towards better evaluation of instance segmentation](https://mlsanity.com/paper/2207.01614/) (2022)
- [Exploiting Shape Cues for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2208.04286/) (2022)
- [Positive-Negative Equal Contrastive Loss for Semantic Segmentation](https://mlsanity.com/paper/2207.01417/) (2022)
- [BoundarySqueeze: Image Segmentation as Boundary Squeezing](https://mlsanity.com/paper/2105.11668/) (2021)
- [Distribution-aware Margin Calibration for Semantic Segmentation in Images](https://mlsanity.com/paper/2112.11554/) (2021)
- [Automatic Reconstruction of Semantic 3D Models from 2D Floor Plans](https://mlsanity.com/paper/2306.01642/) (2023)
- [Semantic-Aware Contrastive Learning for Multi-object Medical Image Segmentation](https://mlsanity.com/paper/2106.01596/) (2021)
- [U-HRNet: Delving into Improving Semantic Representation of High Resolution Network for Dense Prediction](https://mlsanity.com/paper/2210.07140/) (2022)
- [Affinity Feature Strengthening for Accurate, Complete and Robust Vessel Segmentation](https://mlsanity.com/paper/2211.06578/) (2022)
- [Variational Structured Attention Networks for Deep Visual Representation Learning](https://mlsanity.com/paper/2103.03510/) (2021)

## Abstract (arXiv)

Semantic segmentation has made much progress with increasingly powerful pixel-wise classifiers and incorporating structural priors via Conditional Random Fields (CRF) or Generative Adversarial Networks (GAN). We propose a simpler alternative that learns to verify the spatial structure of segmentation during training only. Unlike existing approaches that enforce semantic labels on individual pixels and match labels between neighbouring pixels, we propose the concept of Adaptive Affinity Fields (AAF) to capture and match the semantic relations between neighbouring pixels in the label space. We use adversarial learning to select the optimal affinity field size for each semantic category. It is formulated as a minimax problem, optimizing our segmentation neural network in a best worst-case learning scenario. AAF is versatile for representing structures as a collection of pixel-centric relations, easier to train than GAN and more efficient than CRF without run-time inference. Our extensive evaluations on PASCAL VOC 2012, Cityscapes, and GTA5 datasets demonstrate its above-par segmentation performance and robust generalization across domains.