# When Semantic Segmentation Meets Frequency Aliasing arXiv: https://arxiv.org/abs/2403.09065 ยท HTML: https://mlsanity.com/paper/2403.09065/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Builds on 56 of its 81 references that are in the map. - First posted on arXiv 14 Mar 2024. ## Builds on (most cited first, 30 of 56) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021) - [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021) - [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021) - [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016) - [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014) - [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016) - [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022) - [Masked-attention Mask Transformer for Universal Image Segmentation](https://mlsanity.com/paper/2112.01527/) (2021) - [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017) - [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018) - [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018) - [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021) - [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021) - [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022) - [Training Region-based Object Detectors with Online Hard Example Mining](https://mlsanity.com/paper/1604.03540/) (2016) - [Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement](https://mlsanity.com/paper/2001.06826/) (2020) - [RepVGG: Making VGG-style ConvNets Great Again](https://mlsanity.com/paper/2101.03697/) (2021) - [InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutions](https://mlsanity.com/paper/2211.05778/) (2022) - [EnlightenGAN: Deep Light Enhancement without Paired Supervision](https://mlsanity.com/paper/1906.06972/) (2019) - [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018) - [PointRend: Image Segmentation as Rendering](https://mlsanity.com/paper/1912.08193/) (2019) - [Making Convolutional Networks Shift-Invariant Again](https://mlsanity.com/paper/1904.11486/) (2019) - [Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions](https://mlsanity.com/paper/2003.01826/) (2020) - [Learning in the Frequency Domain](https://mlsanity.com/paper/2002.12416/) (2020) - [Boundary IoU: Improving Object-Centric Image Segmentation Evaluation](https://mlsanity.com/paper/2103.16562/) (2021) - [FcaNet: Frequency Channel Attention Networks](https://mlsanity.com/paper/2012.11879/) (2020) - [Pattern-Affinitive Propagation across Depth, Surface Normal and Semantic Segmentation](https://mlsanity.com/paper/1906.03525/) (2019) - [A Physics-based Noise Formation Model for Extreme Low-light Raw Denoising](https://mlsanity.com/paper/2003.12751/) (2020) ## Built on by (most cited first, 10 of 10) - [Frequency-aware Feature Fusion for Dense Image Prediction](https://mlsanity.com/paper/2408.12879/) (2024) - [Frequency Dynamic Convolution for Dense Image Prediction](https://mlsanity.com/paper/2503.18783/) (2025) - [Frequency-Dynamic Attention Modulation for Dense Prediction](https://mlsanity.com/paper/2507.12006/) (2025) - [HFI: A unified framework for training-free detection and implicit watermarking of latent diffusion model generated images](https://mlsanity.com/paper/2412.20704/) (2024) - [BFANet: Revisiting 3D Semantic Segmentation with Boundary Feature Analysis](https://mlsanity.com/paper/2503.12539/) (2025) - [FreqU-FNet: Frequency-Aware U-Net for Imbalanced Medical Image Segmentation](https://mlsanity.com/paper/2505.17544/) (2025) - [Spatial Frequency Modulation for Semantic Segmentation](https://mlsanity.com/paper/2507.11893/) (2025) - [Optimizing against Infeasible Inclusions from Data for Semantic Segmentation through Morphology](https://mlsanity.com/paper/2408.14672/) (2024) - [To Blend In, First Decouple: Rethinking Camouflage Image Generation via Context-Decoupled Representations](https://mlsanity.com/paper/2607.17768/) (2026) - [FreqTrack: Frequency Learning based Vision Transformer for RGB-Event Object Tracking](https://mlsanity.com/paper/2604.14526/) (2026) ## Abstract (arXiv) Despite recent advancements in semantic segmentation, where and what pixels are hard to segment remains largely unexplored. Existing research only separates an image into easy and hard regions and empirically observes the latter are associated with object boundaries. In this paper, we conduct a comprehensive analysis of hard pixel errors, categorizing them into three types: false responses, merging mistakes, and displacements. Our findings reveal a quantitative association between hard pixels and aliasing, which is distortion caused by the overlapping of frequency components in the Fourier domain during downsampling. To identify the frequencies responsible for aliasing, we propose using the equivalent sampling rate to calculate the Nyquist frequency, which marks the threshold for aliasing. Then, we introduce the aliasing score as a metric to quantify the extent of aliasing. While positively correlated with the proposed aliasing score, three types of hard pixels exhibit different patterns. Here, we propose two novel de-aliasing filter (DAF) and frequency mixing (FreqMix) modules to alleviate aliasing degradation by accurately removing or adjusting frequencies higher than the Nyquist frequency. The DAF precisely removes the frequencies responsible for aliasing before downsampling, while the FreqMix dynamically selects high-frequency components within the encoder block. Experimental results demonstrate consistent improvements in semantic segmentation and low-light instance segmentation tasks. The code is available at: https://github.com/Linwei-Chen/Seg-Aliasing.