# MMA Regularization: Decorrelating Weights of Neural Networks by Maximizing the Minimal Angles

arXiv: https://arxiv.org/abs/2006.06527 · HTML: https://mlsanity.com/paper/2006.06527/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2020.

## Built on by (most cited first, 15 of 15)

- [Inducing Neural Collapse in Deep Long-tailed Learning](https://mlsanity.com/paper/2302.12453/) (2023)
- [Maximum Class Separation as Inductive Bias in One Matrix](https://mlsanity.com/paper/2206.08704/) (2022)
- [Improving GANs for Long-Tailed Data through Group Spectral Regularization](https://mlsanity.com/paper/2208.09932/) (2022)
- [Can We Evaluate Domain Adaptation Models Without Target-Domain Labels?](https://mlsanity.com/paper/2305.18712/) (2023)
- [FedLoGe: Joint Local and Generic Federated Learning under Long-tailed Data](https://mlsanity.com/paper/2401.08977/) (2024)
- [OrtSAE: Orthogonal Sparse Autoencoders Uncover Atomic Features](https://mlsanity.com/paper/2509.22033/) (2025)
- [Adversarial Parameter Attack on Deep Neural Networks](https://mlsanity.com/paper/2203.10502/) (2022)
- [Robust and Information-theoretically Safe Bias Classifier against Adversarial Attacks](https://mlsanity.com/paper/2111.04404/) (2021)
- [TIMA: Text-Image Mutual Awareness for Balancing Zero-Shot Adversarial Robustness and Generalization Ability](https://mlsanity.com/paper/2405.17678/) (2024)
- [Preventing Model Collapse in Deep Canonical Correlation Analysis by Noise Regularization](https://mlsanity.com/paper/2411.00383/) (2024)
- [Keep your distance: learning dispersed embeddings on mathbbSm](https://mlsanity.com/paper/2502.08231/) (2025)
- [Angular Dispersion Accelerates k-Nearest Neighbors Machine Translation](https://mlsanity.com/paper/2509.16729/) (2025)
- [Robust Ensemble Person Re-Identification via Orthogonal Fusion with Occlusion Handling](https://mlsanity.com/paper/2404.00107/) (2024)
- [Beyond topography: Topographic regularization improves robustness and reshapes representations in convolutional neural networks](https://mlsanity.com/paper/2508.00043/) (2025)
- [Representation Collapse in Machine Translation Through the Lens of Angular Dispersion](https://mlsanity.com/paper/2602.17287/) (2026)

## Abstract (arXiv)

The strong correlation between neurons or filters can significantly weaken the generalization ability of neural networks. Inspired by the well-known Tammes problem, we propose a novel diversity regularization method to address this issue, which makes the normalized weight vectors of neurons or filters distributed on a hypersphere as uniformly as possible, through maximizing the minimal pairwise angles (MMA). This method can easily exert its effect by plugging the MMA regularization term into the loss function with negligible computational overhead. The MMA regularization is simple, efficient, and effective. Therefore, it can be used as a basic regularization method in neural network training. Extensive experiments demonstrate that MMA regularization is able to enhance the generalization ability of various modern models and achieves considerable performance improvements on CIFAR100 and TinyImageNet datasets. In addition, experiments on face verification show that MMA regularization is also effective for feature learning. Code is available at: https://github.com/wznpub/MMA_Regularization.