mlsanityData to 14 Oct 2026

Improving Feature Stability during Upsampling -- Spectral Artifacts and the Importance of Spatial Context

arXiv 2311.17524 ↗First posted 29 Nov 2023cs.CVcited by 8

Not on the map. Why?

Abstract (arXiv)

Pixel-wise predictions are required in a wide variety of tasks such as image restoration, image segmentation, or disparity estimation. Common models involve several stages of data resampling, in which the resolution of feature maps is first reduced to aggregate information and then increased to generate a high-resolution output. Previous works have shown that resampling operations are subject to artifacts such as aliasing. During downsampling, aliases have been shown to compromise the prediction stability of image classifiers. During upsampling, they have been leveraged to detect generated content. Yet, the effect of aliases during upsampling has not yet been discussed w.r.t. the stability and robustness of pixel-wise predictions. While falling under the same term (aliasing), the challenges for correct upsampling in neural networks differ significantly from those during downsampling: when downsampling, some high frequencies can not be correctly represented and have to be removed to avoid aliases. However, when upsampling for pixel-wise predictions, we actually require the model to restore such high frequencies that can not be encoded in lower resolutions. The application of findings from signal processing is therefore a necessary but not a sufficient condition to achieve the desirable output. In contrast, we find that the availability of large spatial context during upsampling allows to provide stable, high-quality pixel-wise predictions, even when fully learning all filter weights.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 67 of its 96 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

67 of 96 references.

References · 67

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  6. 2016Layer Normalizationcited by 5,669
  7. 2022Generative Adversarial Networkscited by 5,627
  8. 2013Intriguing properties of neural networkscited by 5,155
  9. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  10. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  11. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  12. 2022A ConvNet for the 2020scited by 3,579
38 more
  1. 2016Gaussian Error Linear Units (GELUs)cited by 3,502
  2. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  3. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  4. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  5. 2016Pyramid Scene Parsing Networkcited by 2,479
  6. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,415
  7. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,054
  8. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,627
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  10. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,487
  11. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  12. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,204
  13. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  14. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
  15. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,059
  16. 2016Adversarial Machine Learning at Scalecited by 936
  17. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 881
  18. 2018Robustness May Be at Odds with Accuracycited by 880
  19. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 784
  20. 2016Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurringcited by 727
  21. 2022Simple Baselines for Image Restorationcited by 708
  22. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 499
  23. 2018Pyramid Stereo Matching Networkcited by 489
  24. 2019Making Convolutional Networks Shift-Invariant Againcited by 391
  25. 2015Learning Deconvolution Network for Semantic Segmentationcited by 365
  26. 2020Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributionscited by 280
  27. 2022SegNeXt: Rethinking Convolutional Attention Design for Semantic Segmentationcited by 252
  28. 2016Generating Images with Perceptual Similarity Metrics based on Deep Networkscited by 222
  29. 2017Large Kernel Matters -- Improve Semantic Segmentation by Global Convolutional Networkcited by 208
  30. 2020Revisiting Stereo Depth Estimation From a Sequence-to-Sequence Perspective with Transformerscited by 190
  31. 2016A guide to convolution arithmetic for deep learningcited by 185
  32. 2015Inverting Visual Representations with Convolutional Networkscited by 170
  33. 2022More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsitycited by 142
  34. 2022SegPGD: An Effective and Efficient Adversarial Attack for Evaluating and Boosting Segmentation Robustnesscited by 74
  35. 2021A Closer Look at Fourier Spectrum Discrepancies for CNN-generated Images Detectioncited by 68
  36. 2020Delving Deeper into Anti-aliasing in ConvNetscited by 60
  37. 2021Beyond the Spectrum: Detecting Deepfakes via Re-Synthesiscited by 44
  38. 2017Checkerboard artifact free sub-pixel convolution: A note on sub-pixel convolution, resize convolution and convolution resizecited by 37

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 2.

8 papers.

  1. 2024HFI: A unified framework for training-free detection and implicit watermarking of latent diffusion model generated imagescited by 9
  2. 2025RobustSpring: Benchmarking Robustness to Image Corruptions for Optical Flow, Scene Flow and Stereocited by 7
  3. 2024FAIR-TAT: Improving Model Fairness Using Targeted Adversarial Trainingcited by 5
  4. 2024How Do Training Methods Influence the Utilization of Vision Models?cited by 3
  5. 2026GeomPrompt: Geometric Prompt Learning for RGB-D Semantic Segmentation Under Missing and Degraded Depthcited by 1
  6. 2025TRIX- Trading Adversarial Fairness via Mixed Adversarial Trainingcited by 0
  7. 2025Fourier-Guided Attention Upsampling for Image Super-Resolutioncited by 0
  8. 2026Failure or Drift? Evaluating Monocular SLAM under Synthetic and Real-World Corruptionscited by 0