Avoiding Premature Collapse: Adaptive Annealing for Entropy-Regularized Structural Inference
Not on the map. Why?
Abstract (arXiv)
Differentiable matching layers and residual connection paradigms, often implemented via entropy-regularized Optimal Transport (OT), serve as critical mechanisms in structural prediction and architectural scaling. However, recovering discrete permutations or maintaining identity mappings via annealing ε → 0 is notoriously unstable. In this work, we identify a fundamental mechanism for this failure: Premature Mode Collapse. By analyzing the non-normal dynamics of the Sinkhorn fixed-point map, we reveal a theoretical thermodynamic speed limit: standard exponential cooling outpaces the contraction rate of the inference operator, which degrades as O(1/ε). To address this, we propose Efficient Piecewise Hybrid Adaptive Stability Control (EPH-ASC), an adaptive scheduling algorithm that monitors the stability of the inference process. We demonstrate that EPH-ASC is essential for stabilizing Manifold-Constrained Hyper-Connections (mHC) during large-scale training on the FineWeb-Edu dataset, effectively preventing late-stage gradient explosions by enforcing a linear stability law.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 9 of its 14 references that are in the map. First posted on arXiv 30 Jan 2026.
Builds on
9 of 14 references.
References · 9
- 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
- 2017Learning Generative Models with Sinkhorn Divergencescited by 310
- 2017Near-linear time approximation algorithms for optimal transport via Sinkhorn iterationcited by 219
- 2017Smooth and Sparse Optimal Transportcited by 83
- 2024Hyper-Connectionscited by 81
- 2025mHC: Manifold-Constrained Hyper-Connectionscited by 76
- 2022A Unified Framework for Implicit Sinkhorn Differentiationcited by 28
- 2018An explicit analysis of the entropic penalty in linear programmingcited by 23
- 2024Annealed Sinkhorn for Optimal Transport: convergence, regularization path and debiasingcited by 6
Built on by
Not cited yet.