Activation Saturation and Floquet Spectrum Collapse in Neural ODEs

We prove that activation saturation imposes a structural dynamical limitation on autonomous Neural ODEs doth=fθ(h) with saturating activations (tanh, sigmoid, etc.): if q hidden layers of the MLP fθ satisfy |σ'|≤δ on a region~U, the input Jacobian is attenuated as normDfθ(x)≤ C(U) (for activations with supx|σ'(x)|≤ 1, e.g. tanh and sigmoid, this reduces to CWδq), forcing every Floquet (Lyapunov) exponen along any T-periodic orbit γsubset U into the interval [-C(U),;C(U)]. This is a collapse of the Floquet spectrum: as saturation deepens (δ→ 0), all exponents are driven to zero, limiting both strong contraction and chaotic sensitivity. The obstruction is structural -- it constrains the learned vector field at inference time, independent of training quality. As a secondary contribution, for activations with σ'>0, a saturation-weighted spectral factorisation yields a refined bound widetildeC(U)≤ C(U) whose improvement is amplified exponentially in~T at the flow level. All results are numerically illustrated on the Stuart--Landau oscillator; the bounds provide a theoretical explanation for the empirically observed failure of tanh-NODEs on the Morris--Lecar neuron model.