# Branch Geometry and Finite-Radius Sensitivity of Hard-ReLU Training

arXiv: https://arxiv.org/abs/2608.30960 · HTML: https://mlsanity.com/paper/2608.30960/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 3 of its 27 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 3 of 3)

- [A mathematical model for automatic differentiation in machine learning](https://mlsanity.com/paper/2006.02080/) (2020)
- [Continuous-Time Piecewise-Linear Recurrent Neural Networks](https://mlsanity.com/paper/2602.15649/) (2026)
- [Correcting Auto-Differentiation in Neural-ODE Training](https://mlsanity.com/paper/2306.02192/) (2023)

## Abstract (arXiv)

Outer-learning algorithms use infinitesimal sensitivities to propose finite changes to initialization or training parameters. For hard-ReLU training, the derivative of the finite program and the derivative of its flow limit do not by themselves specify the response at a chosen radius. We characterize the intervening regime in which the perturbation radius is proportional to the GD step. Integer event rounding then survives at leading order: smooth Euler bias shifts each discrete phase, and upstream rounding moves downstream branch boundaries. We derive the crossing indices and a uniform endpoint expansion for finitely many separated transverse events in piecewise-C2 dynamics, away from recursive phase boundaries. In contractive affine regions, an explicit remainder and complete branch verification certify finite candidate comparisons. Scalar phase frequencies and a coupled feedback ablation test the mechanism; frozen nonlinear-network experiments show radius-dependent prediction accuracy, including incomplete branch matches and failed-word tails. Together with local AD and uniform flow consistency, the result identifies sufficient response regimes: differentiating training is a choice of perturbation resolution as well as a choice of derivative.