mlsanityData to 7 Oct 2026

How Much Position Information Do Convolutional Neural Networks Encode?

arXiv 2001.08248 ↗First posted 22 Jan 2020cs.CV cs.LGcited by 171

Not on the map. Why?

Abstract (arXiv)

In contrast to fully connected networks, Convolutional Neural Networks (CNNs) achieve efficiency by learning weights associated with local filters with a finite spatial extent. An implication of this is that a filter may know what it is looking at, but not where it is positioned in the image. Information concerning absolute position is inherently useful, and it is reasonable to assume that deep CNNs may implicitly learn to encode this information if there is a means to do so. In this paper, we test this hypothesis revealing the surprising degree of absolute position information that is encoded in commonly used neural networks. A comprehensive set of experiments show the validity of this hypothesis and shed light on how and where this information is represented while offering clues to where positional information is derived from in deep CNNs.

Key facts. Cited by 171 papers in the map (data through 7 Oct 2026). First posted on arXiv 22 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 45; 2022: 38; 2023: 33; 2024: 27; 2025: 17; 2026: 11.

171 papers.

Sort:

By area:

  1. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  2. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,923
  3. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  4. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  5. 2021TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classificationcited by 549
  6. 2021TS2Vec: Towards Universal Representation of Time Seriescited by 463
  7. 2021Conditional Positional Encodings for Vision Transformerscited by 402
  8. 2021A Survey of Transformerscited by 399
  9. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 331
  10. 2021Graph Neural Networks with Learnable Structural and Positional Representationscited by 326
  11. 2021CMT: Convolutional Neural Networks Meet Vision Transformerscited by 310
  12. 2022Vision Transformers for Single Image Dehazingcited by 255
38 more
  1. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  2. 2021StyTr2: Image Style Transfer with Transformerscited by 186
  3. 2022UniFormer: Unified Transformer for Efficient Spatiotemporal Representation Learningcited by 182
  4. 2021StyleSwin: Transformer-based GAN for High-resolution Image Generationcited by 152
  5. 2022AiATrack: Attention in Attention for Transformer Visual Trackingcited by 137
  6. 2021MPViT: Multi-Path Vision Transformer for Dense Predictioncited by 122
  7. 2021A Survey of Visual Transformerscited by 117
  8. 2021MISSFormer: An Effective Medical Image Segmentation Transformercited by 103
  9. 2022A graph-transformer for whole slide image classificationcited by 84
  10. 2022Incremental Transformer Structure Enhanced Image Inpainting with Masking Positional Encodingcited by 83
  11. 2021ResT: An Efficient Transformer for Visual Recognitioncited by 79
  12. 2022Fast Vision Transformers with HiLo Attentioncited by 78
  13. 2024EMCAD: Efficient Multi-scale Convolutional Attention Decoding for Medical Image Segmentationcited by 64
  14. 2023TransNeXt: Robust Foveal Visual Perception for Vision Transformerscited by 61
  15. 2021Medical Image Segmentation Using Squeeze-and-Expansion Transformerscited by 56
  16. 2023MLIC++: Linear Complexity Multi-Reference Entropy Modeling for Learned Image Compressioncited by 56
  17. 2022MVSFormer: Multi-View Stereo by Learning Robust Image Features and Temperature-based Depthcited by 50
  18. 2024MVSFormer++: Revealing the Devil in Transformer's Details for Multi-View Stereocited by 49
  19. 2023Super-Resolution Neural Operatorcited by 48
  20. 2022Shift-tolerant Perceptual Similarity Metriccited by 44
  21. 2021MS-TCT: Multi-Scale Temporal ConvTransformer for Action Detectioncited by 42
  22. 2024Changen2: Multi-Temporal Remote Sensing Generative Change Foundation Modelcited by 37
  23. 2022ScalableViT: Rethinking the Context-oriented Generalization of Vision Transformercited by 36
  24. 2021SOLO: A Simple Framework for Instance Segmentationcited by 32
  25. 2021PLADE-Net: Towards Pixel-Level Accuracy for Self-Supervised Single-View Depth Estimation with Neural Positional Encoding and Distilled Matting Losscited by 31
  26. 2024SegFormer3D: an Efficient Transformer for 3D Medical Image Segmentationcited by 31
  27. 2021Position, Padding and Predictions: A Deeper Look at Position Information in CNNscited by 26
  28. 2022Locality-Aware Inter-and Intra-Video Reconstruction for Self-Supervised Correspondence Learningcited by 26
  29. 2024HTR-VT: Handwritten Text Recognition with Vision Transformercited by 24
  30. 2021UniNet: Unified Architecture Search with Convolution, Transformer, and MLPcited by 23
  31. 2022High-fidelity GAN Inversion with Padding Spacecited by 23
  32. 2024CSTA: CNN-based Spatiotemporal Attention for Video Summarizationcited by 23
  33. 2022ZITS++: Image Inpainting by Improving the Incremental Transformer on Structural Priorscited by 21
  34. 2023FormerTime: Hierarchical Multi-Scale Representations for Multivariate Time Series Classificationcited by 21
  35. 2023G-CASCADE: Efficient Cascaded Graph Convolutional Decoding for 2D Medical Image Segmentationcited by 21
  36. 2024GreedyViG: Dynamic Axial Graph Construction for Efficient Vision GNNscited by 21
  37. 2021GANSeg: Learning to Segment by Unsupervised Hierarchical Image Generationcited by 19
  38. 2021Global Transport for Fluid Reconstruction with Learned Self-Supervisioncited by 18

50 most cited of 171 All 171 (JSON)