# Weight and Gradient Centralization in Deep Neural Networks

arXiv: https://arxiv.org/abs/2010.00866 · HTML: https://mlsanity.com/paper/2010.00866/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2020.

## Built on by (most cited first, 11 of 11)

- [TEyeD: Over 20 million real-world eye images with Pupil, Eyelid, and Iris 2D and 3D Segmentations, 2D and 3D Landmarks, 3D Eyeball, Gaze Vector, and Eye Movement Types](https://mlsanity.com/paper/2102.02115/) (2021)
- [The Gaze and Mouse Signal as additional Source for User Fingerprints in Browser Applications](https://mlsanity.com/paper/2101.03793/) (2021)
- [A Multimodal Eye Movement Dataset and a Multimodal Eye Movement Segmentation Analysis](https://mlsanity.com/paper/2101.04318/) (2021)
- [1000 Pupil Segmentations in a Second using Haar Like Features and Statistical Learning](https://mlsanity.com/paper/2102.01921/) (2021)
- [Maximum and Leaky Maximum Propagation](https://mlsanity.com/paper/2105.10277/) (2021)
- [Tensor Normalization and Full Distribution Training](https://mlsanity.com/paper/2109.02345/) (2021)
- [GroupGazer: A Tool to Compute the Gaze per Participant in Groups with integrated Calibration to Map the Gaze Online to a Screen or Beamer Projection](https://mlsanity.com/paper/2201.07692/) (2022)
- [GCSAM: Gradient Centralized Sharpness Aware Minimization](https://mlsanity.com/paper/2501.11584/) (2025)
- [Technical Report: Combining knowledge from Transfer Learning during training and Wide Resnets](https://mlsanity.com/paper/2206.09697/) (2022)
- [Improving the Accuracy and Interpretability of Neural Networks for Wind Power Forecasting](https://mlsanity.com/paper/2312.15741/) (2023)
- [ESNERA: Empirical and semantic named entity alignment for named entity dataset merging](https://mlsanity.com/paper/2508.06877/) (2025)

## Abstract (arXiv)

Batch normalization is currently the most widely used variant of internal normalization for deep neural networks. Additional work has shown that the normalization of weights and additional conditioning as well as the normalization of gradients further improve the generalization. In this work, we combine several of these methods and thereby increase the generalization of the networks. The advantage of the newer methods compared to the batch normalization is not only increased generalization, but also that these methods only have to be applied during training and, therefore, do not influence the running time during use. Link to CUDA code https://atreus.informatik.uni-tuebingen.de/seafile/d/8e2ab8c3fdd444e1a135/