Training-Time Batch Normalization Reshapes Local Partition Geometry in Piecewise-Affine Networks
Not on the map. Why?
Abstract (arXiv)
Batch normalization (BN) is central to modern deep networks, but its effect on the realized function during training remains less understood than its optimization benefits. We study training-time BN in continuous piecewise-affine (CPA) networks through the geometry of switching hyperplanes and the induced affine-region partition. Conditioned on a mini-batch, we show that BN defines for each neuron a reference hyperplane through the batch centroid, and that breakpoint-switching hyperplanes are parallel translates whose offsets are expressed in batch-standardized coordinates and are independent of the raw bias. This yields an exact criterion for when a switching hyperplane intersects a local ℓ_∞ window and motivates a local region-density functional based on exact affine-region counts. Under explicit sufficient conditions, we show that BN increases expected local partition refinement in ReLU and more general piecewise-affine networks, and that this mechanism transfers locally through depth inside parent affine regions where the upstream representation map is an affine embedding. These results provide a function-level geometric account of training-time BN as a batch-conditional recentering mechanism near the data.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 36 of its 50 references that are in the map. First posted on arXiv 6 May 2026.
Builds on
36 of 50 references.
References · 36
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
- 2016Densely Connected Convolutional Networkscited by 6,720
- 2016Weight Normalization: A Simple Reparameterization to Accelerate Training of Deep Neural Networkscited by 543
- 2014On the Number of Linear Regions of Deep Neural Networkscited by 381
- 2018How Does Batch Normalization Help Optimization?cited by 357
- 2016Understanding Deep Neural Networks with Rectified Linear Unitscited by 257
- 2020Scaled-YOLOv4: Scaling Cross Stage Partial Networkcited by 211
- 2019Complexity of Linear Regions in Deep Networkscited by 167
- 2019Deep ReLU Networks Have Surprisingly Few Activation Patternscited by 161
- 2018Understanding Batch Normalizationcited by 159
24 more
- 2019A Mean Field Theory of Batch Normalizationcited by 58
- 2021Towards Lower Bounds on the Depth of ReLU Neural Networkscited by 56
- 2019The Geometry of Deep Networks: Power Diagram Subdivisioncited by 53
- 2020On the Number of Linear Regions of Convolutional Neural Networkscited by 49
- 2021Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sumscited by 37
- 2020Empirical Studies on the Properties of Linear Regions in Deep Neural Networkscited by 36
- 2023Lower Bounds on the Depth of Integral ReLU Neural Networks via Lattice Polytopescited by 36
- 2023SplineCam: Exact Visualization and Characterization of Deep Network Geometry and Decision Boundariescited by 31
- 2018Exponential convergence rates for Batch Normalization: The power of length-direction decoupling in non-convex optimizationcited by 27
- 2022Improved Bounds on Neural Complexity for Representing Piecewise Linear Functionscited by 26
- 2022On the Number of Regions of Piecewise Linear Neural Networkscited by 21
- 2025On the Expressiveness of Rational ReLU Neural Networks With Bounded Depthcited by 18
- 2021Deep ReLU Networks Preserve Expected Lengthcited by 17
- 2021On the Expected Complexity of Maxout Networkscited by 15
- 2022GhostNets on Heterogeneous Devices via Cheap Operationscited by 15
- 2023Polyhedral Complex Extraction from ReLU Networks using Edge Subdivisioncited by 14
- 2022Effects of Data Geometry in Early Deep Learningcited by 12
- 2023Exact Count of Boundary Pieces of ReLU Classifiers: Towards the Proper Complexity Measure for Classificationcited by 11
- 2018ChannelNets: Compact and Efficient Convolutional Neural Networks via Channel-Wise Convolutionscited by 10
- 2022Lower and Upper Bounds for Numbers of Linear Regions of Graph Convolutional Networkscited by 9
- 2021Using activation histograms to bound the number of affine regions in ReLU feed-forward neural networkscited by 7
- 2025The Computational Complexity of Counting Linear Regions in ReLU Neural Networkscited by 7
- 2023Compelling ReLU Networks to Exhibit Exponentially Many Linear Regions at Initialization and During Trainingcited by 4
- 2023The Evolution of the Interplay Between Input Distributions and Linear Regions in Networkscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.