# Why Smooth Stability Assumptions Fail for ReLU Learning

arXiv: https://arxiv.org/abs/2512.22055 · HTML: https://mlsanity.com/paper/2512.22055/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 8 references that are in the map.
- First posted on arXiv 26 Dec 2025.

## Builds on (most cited first, 2 of 2)

- [Train faster, generalize better: Stability of stochastic gradient descent](https://mlsanity.com/paper/1509.01240/) (2015)
- [A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networks](https://mlsanity.com/paper/1707.09564/) (2017)

## Abstract (arXiv)

Stability analyses of modern learning systems are frequently derived under smoothness assumptions that are violated by ReLU-type nonlinearities. In this note, we isolate a minimal obstruction by showing that no uniform smoothness-based stability proxy such as gradient Lipschitzness or Hessian control can hold globally for ReLU networks, even in simple settings where training trajectories appear empirically stable. We give a concrete counterexample demonstrating the failure of classical stability bounds and identify a minimal generalized derivative condition under which stability statements can be meaningfully restored. The result clarifies why smooth approximations of ReLU can be misleading and motivates nonsmooth-aware stability frameworks.