# Learning Implicit Feature Alignment Function for Semantic Segmentation

arXiv: https://arxiv.org/abs/2206.08655 · HTML: https://mlsanity.com/paper/2206.08655/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 67 references that are in the map.
- First posted on arXiv 17 Jun 2022.

## Builds on (most cited first, 30 of 54)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Dual Attention Network for Scene Segmentation](https://mlsanity.com/paper/1809.02983/) (2018)
- [Segmenter: Transformer for Semantic Segmentation](https://mlsanity.com/paper/2105.05633/) (2021)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016)
- [Segmentation Transformer: Object-Contextual Representations for Semantic Segmentation](https://mlsanity.com/paper/1909.11065/) (2019)

## Built on by (most cited first, 30 of 31)

- [FeatUp: A Model-Agnostic Framework for Features at Any Resolution](https://mlsanity.com/paper/2403.10516/) (2024)
- [Frequency-aware Feature Fusion for Dense Image Prediction](https://mlsanity.com/paper/2408.12879/) (2024)
- [X-Align: Cross-Modal Cross-View Alignment for Bird's-Eye-View Segmentation](https://mlsanity.com/paper/2210.06778/) (2022)
- [Multi-view Aggregation Network for Dichotomous Image Segmentation](https://mlsanity.com/paper/2404.07445/) (2024)
- [Blind Video Deflickering by Neural Filtering with a Flawed Atlas](https://mlsanity.com/paper/2303.08120/) (2023)
- [I-MedSAM: Implicit Medical Image Segmentation with Segment Anything](https://mlsanity.com/paper/2311.17081/) (2023)
- [4D Panoptic Segmentation as Invariant and Equivariant Field Prediction](https://mlsanity.com/paper/2303.15651/) (2023)
- [AnyFlow: Arbitrary Scale Optical Flow with Implicit Neural Representation](https://mlsanity.com/paper/2303.16493/) (2023)
- [Unite-Divide-Unite: Joint Boosting Trunk and Structure for High-accuracy Dichotomous Image Segmentation](https://mlsanity.com/paper/2307.14052/) (2023)
- [SFNet: Faster and Accurate Semantic Segmentation via Semantic Flow](https://mlsanity.com/paper/2207.04415/) (2022)
- [Bi-VLGM : Bi-Level Class-Severity-Aware Vision-Language Graph Matching for Text Guided Medical Image Segmentation](https://mlsanity.com/paper/2305.12231/) (2023)
- [Continuous Cross-resolution Remote Sensing Image Change Detection](https://mlsanity.com/paper/2305.14722/) (2023)
- [Imaging Interiors: An Implicit Solution to Electromagnetic Inverse Scattering Problems](https://mlsanity.com/paper/2407.09352/) (2024)
- [High-Precision Dichotomous Image Segmentation via Probing Diffusion Capacity](https://mlsanity.com/paper/2410.10105/) (2024)
- [SwIPE: Efficient and Robust Medical Image Segmentation with Implicit Patch Embeddings](https://mlsanity.com/paper/2307.12429/) (2023)
- [Neural Implicit Heart Coordinates: 3D cardiac shape reconstruction from sparse segmentations](https://mlsanity.com/paper/2512.19316/) (2025)
- [DPF: Learning Dense Prediction Fields with Weak Supervision](https://mlsanity.com/paper/2303.16890/) (2023)
- [Memory-Constrained Semantic Segmentation for Ultra-High Resolution UAV Imagery](https://mlsanity.com/paper/2310.04721/) (2023)
- [A Deep Semantic Segmentation Network with Semantic and Contextual Refinements](https://mlsanity.com/paper/2412.08671/) (2024)
- [Spatial Frequency Modulation for Semantic Segmentation](https://mlsanity.com/paper/2507.11893/) (2025)
- [Tsanet: Temporal and Scale Alignment for Unsupervised Video Object Segmentation](https://mlsanity.com/paper/2303.04376/) (2023)
- [Neural Field Conditioning Strategies for 2D Semantic Segmentation](https://mlsanity.com/paper/2304.14371/) (2023)
- [Cross-level Attention with Overlapped Windows for Camouflaged Object Detection](https://mlsanity.com/paper/2311.16618/) (2023)
- [Kernelized Back-Projection Networks for Blind Super Resolution](https://mlsanity.com/paper/2302.08478/) (2023)
- [Q2A: Querying Implicit Fully Continuous Feature Pyramid to Align Features for Medical Image Segmentation](https://mlsanity.com/paper/2404.09472/) (2024)
- [A feature refinement module for light-weight semantic segmentation network](https://mlsanity.com/paper/2412.08670/) (2024)
- [Benchmarking Feature Upsampling Methods for Vision Foundation Models using Interactive Segmentation](https://mlsanity.com/paper/2505.02075/) (2025)
- [X-Align++: cross-modal cross-view alignment for Bird's-eye-view segmentation](https://mlsanity.com/paper/2306.03810/) (2023)
- [A Decoding Scheme with Successive Aggregation of Multi-Level Features for Light-Weight Semantic Segmentation](https://mlsanity.com/paper/2402.11201/) (2024)
- [HSDA: High-frequency Shuffle Data Augmentation for Bird's-Eye-View Map Segmentation](https://mlsanity.com/paper/2412.06127/) (2024)

## Abstract (arXiv)

Integrating high-level context information with low-level details is of central importance in semantic segmentation. Towards this end, most existing segmentation models apply bilinear up-sampling and convolutions to feature maps of different scales, and then align them at the same resolution. However, bilinear up-sampling blurs the precise information learned in these feature maps and convolutions incur extra computation costs. To address these issues, we propose the Implicit Feature Alignment function (IFA). Our method is inspired by the rapidly expanding topic of implicit neural representations, where coordinate-based neural networks are used to designate fields of signals. In IFA, feature vectors are viewed as representing a 2D field of information. Given a query coordinate, nearby feature vectors with their relative coordinates are taken from the multi-level feature maps and then fed into an MLP to generate the corresponding output. As such, IFA implicitly aligns the feature maps at different levels and is capable of producing segmentation maps in arbitrary resolutions. We demonstrate the efficacy of IFA on multiple datasets, including Cityscapes, PASCAL Context, and ADE20K. Our method can be combined with improvement on various architectures, and it achieves state-of-the-art computation-accuracy trade-off on common benchmarks. Code will be made available at https://github.com/hzhupku/IFA.