mlsanityData to 14 Oct 2026

Spooky Action at a Distance: Normalization Layers Enable Side-Channel Spatial Communication

arXiv 2507.04709 ↗First posted 7 Jul 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

This work shows that normalization layers can facilitate a surprising degree of communication across the spatial dimensions of an input tensor. We study a toy localization task with a convolutional architecture and show that normalization layers enable an iterative message passing procedure, allowing information aggregation from well outside the local receptive field. Our results suggest that normalization layers should be employed with caution in applications such as diffusion-based trajectory generation, where maintaining a spatially limited receptive field is crucial.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 28 references that are in the map. First posted on arXiv 7 Jul 2025.

Builds on

27 of 28 references.

References · 27

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2017Mask R-CNNcited by 7,277
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  8. 2016Layer Normalizationcited by 5,669
  9. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  10. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  11. 2018Group Normalizationcited by 1,524
  12. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,198
15 more
  1. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  2. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 516
  3. 2023Analyzing and Improving the Training Dynamics of Diffusion Modelscited by 402
  4. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 307
  5. 2018On the Generalization of Equivariance and Convolution in Neural Networks to the Action of Compact Groupscited by 300
  6. 2020How Much Position Information Do Convolutional Neural Networks Encode?cited by 172
  7. 2020On Translation Invariance in CNNs: Convolutional Layers can Exploit Absolute Spatial Locationcited by 142
  8. 2017Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Modelscited by 128
  9. 2024An analytic theory of creativity in convolutional diffusion modelscited by 104
  10. 2023PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Playcited by 50
  11. 2021Rethinking "Batch" in BatchNormcited by 45
  12. 2019EvalNorm: Estimating Batch Normalization Statistics for Evaluationcited by 20
  13. 2020Towards Stabilizing Batch Statistics in Backward Propagation of Batch Normalizationcited by 19
  14. 2025Learning Coordinated Bimanual Manipulation Policies using State Diffusion and Inverse Dynamics Modelscited by 9
  15. 2025Extendable Planning via Multiscale Diffusioncited by 2

Built on by

Not cited yet.