# Dynamic DropConnect: Enhancing Neural Network Robustness through Adaptive Edge Dropping Strategies

arXiv: https://arxiv.org/abs/2502.19948 · HTML: https://mlsanity.com/paper/2502.19948/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 15 references that are in the map.
- First posted on arXiv 27 Feb 2025.

## Builds on (most cited first, 4 of 4)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [Survey of Dropout Methods for Deep Neural Networks](https://mlsanity.com/paper/1904.13310/) (2019)

## Built on by (most cited first, 4 of 4)

- [μ-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Experts](https://mlsanity.com/paper/2505.18451/) (2025)
- [Deep Neural Networks Inspired by Differential Equations](https://mlsanity.com/paper/2510.09685/) (2025)
- [Label-Noise Resistant Learning via Optimal Brain Damage Masking](https://mlsanity.com/paper/2508.09697/) (2025)
- [CeRA: Breaking the Linear Ceiling of Low-Rank Adaptation with Non-linearity Retained at Inference](https://mlsanity.com/paper/2602.22911/) (2026)

## Abstract (arXiv)

Dropout and DropConnect are well-known techniques that apply a consistent drop rate to randomly deactivate neurons or edges in a neural network layer during training. This paper introduces a novel methodology that assigns dynamic drop rates to each edge within a layer, uniquely tailoring the dropping process without incorporating additional learning parameters. We perform experiments on synthetic and openly available datasets to validate the effectiveness of our approach. The results demonstrate that our method outperforms Dropout, DropConnect, and Standout, a classic mechanism known for its adaptive dropout capabilities. Furthermore, our approach improves the robustness and generalization of neural network training without increasing computational complexity. The complete implementation of our methodology is publicly accessible for research and replication purposes at https://github.com/ericabd888/Adjusting-the-drop-probability-in-DropConnect-based-on-the-magnitude-of-the-gradient/.