# AutoClip: Adaptive Gradient Clipping for Source Separation Networks

arXiv: https://arxiv.org/abs/2007.14469 · HTML: https://mlsanity.com/paper/2007.14469/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2020.

## Built on by (most cited first, 12 of 12)

- [Masked Discrimination for Self-Supervised Learning on Point Clouds](https://mlsanity.com/paper/2203.11183/) (2022)
- [An Experimental Study of Byzantine-Robust Aggregation Schemes in Federated Learning](https://mlsanity.com/paper/2302.07173/) (2023)
- [ZClip: Adaptive Spike Mitigation for LLM Pre-Training](https://mlsanity.com/paper/2504.02507/) (2025)
- [JIGMARK: A Black-Box Approach for Enhancing Image Watermarks against Diffusion Model Edits](https://mlsanity.com/paper/2406.03720/) (2024)
- [DeepLSS: breaking parameter degeneracies in large scale structure with deep learning analysis of combined probes](https://mlsanity.com/paper/2203.09616/) (2022)
- [Introducing Self-Attention to Target Attentive Graph Neural Networks](https://mlsanity.com/paper/2107.01516/) (2021)
- [Robust Stochastic Optimization via Gradient Quantile Clipping](https://mlsanity.com/paper/2309.17316/) (2023)
- [Robust Federated Learning Over the Air: Combating Heavy-Tailed Noise with Median Anchored Clipping](https://mlsanity.com/paper/2409.15100/) (2024)
- [Advances in Electron Microscopy with Deep Learning](https://mlsanity.com/paper/2101.01178/) (2021)
- [Switchable Lightweight Anti-symmetric Processing (SLAP) with CNN Outspeeds Data Augmentation by Smaller Sample -- Application in Gomoku Reinforcement Learning](https://mlsanity.com/paper/2301.04746/) (2023)
- [SAFT: Shape and Appearance of Fabrics from Template via Differentiable Physical Simulations from Monocular Video](https://mlsanity.com/paper/2509.08828/) (2025)
- [Towards Universal Calibration for In-Context Classification](https://mlsanity.com/paper/2610.17096/) (2026)

## Abstract (arXiv)

Clipping the gradient is a known approach to improving gradient descent, but requires hand selection of a clipping threshold hyperparameter. We present AutoClip, a simple method for automatically and adaptively choosing a gradient clipping threshold, based on the history of gradient norms observed during training. Experimental results show that applying AutoClip results in improved generalization performance for audio source separation networks. Observation of the training dynamics of a separation network trained with and without AutoClip show that AutoClip guides optimization into smoother parts of the loss landscape. AutoClip is very simple to implement and can be integrated readily into a variety of applications across multiple domains.