# Game-Theoretic Gradient Control for Robust Neural Network Training

arXiv: https://arxiv.org/abs/2507.19143 · HTML: https://mlsanity.com/paper/2507.19143/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 14 references that are in the map.
- First posted on arXiv 25 Jul 2025.

## Builds on (most cited first, 9 of 9)

- [Entropy-SGD: Biasing Gradient Descent Into Wide Valleys](https://mlsanity.com/paper/1611.01838/) (2016)
- [Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropout](https://mlsanity.com/paper/2010.06808/) (2020)
- [How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Model](https://mlsanity.com/paper/2307.02129/) (2023)
- [Interpreting and Boosting Dropout from a Game-Theoretic View](https://mlsanity.com/paper/2009.11729/) (2020)
- [Regularizing Meta-Learning via Gradient Dropout](https://mlsanity.com/paper/2004.05859/) (2020)
- [On The Specialization of Neural Modules](https://mlsanity.com/paper/2409.14981/) (2024)
- [Compositional game theory](https://mlsanity.com/paper/1603.04641/) (2016)
- [Deep Learning with Parametric Lenses](https://mlsanity.com/paper/2404.00408/) (2024)
- [Semantics, Representations and Grammars for Deep Learning](https://mlsanity.com/paper/1509.08627/) (2015)

## Abstract (arXiv)

Feed-forward neural networks (FFNNs) are vulnerable to input noise, reducing prediction performance. Existing regularization methods like dropout often alter network architecture or overlook neuron interactions. This study aims to enhance FFNN noise robustness by modifying backpropagation, interpreted as a multi-agent game, and exploring controlled target variable noising. Our "gradient dropout" selectively nullifies hidden layer neuron gradients with probability 1 - p during backpropagation, while keeping forward passes active. This is framed within compositional game theory. Additionally, target variables were perturbed with white noise or stable distributions. Experiments on ten diverse tabular datasets show varying impacts: improvement or diminishing of robustness and accuracy, depending on dataset and hyperparameters. Notably, on regression tasks, gradient dropout (p = 0.9) combined with stable distribution target noising significantly increased input noise robustness, evidenced by flatter MSE curves and more stable SMAPE values. These results highlight the method's potential, underscore the critical role of adaptive parameter tuning, and open new avenues for analyzing neural networks as complex adaptive systems exhibiting emergent behavior within a game-theoretic framework.