mlsanityData to 7 Oct 2026

Avoiding Premature Collapse: Adaptive Annealing for Entropy-Regularized Structural Inference

arXiv 2601.23039 ↗First posted 30 Jan 2026cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Differentiable matching layers and residual connection paradigms, often implemented via entropy-regularized Optimal Transport (OT), serve as critical mechanisms in structural prediction and architectural scaling. However, recovering discrete permutations or maintaining identity mappings via annealing ε → 0 is notoriously unstable. In this work, we identify a fundamental mechanism for this failure: Premature Mode Collapse. By analyzing the non-normal dynamics of the Sinkhorn fixed-point map, we reveal a theoretical thermodynamic speed limit: standard exponential cooling outpaces the contraction rate of the inference operator, which degrades as O(1/ε). To address this, we propose Efficient Piecewise Hybrid Adaptive Stability Control (EPH-ASC), an adaptive scheduling algorithm that monitors the stability of the inference process. We demonstrate that EPH-ASC is essential for stabilizing Manifold-Constrained Hyper-Connections (mHC) during large-scale training on the FineWeb-Edu dataset, effectively preventing late-stage gradient explosions by enforcing a linear stability law.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 9 of its 14 references that are in the map. First posted on arXiv 30 Jan 2026.

Builds on

9 of 14 references.

References · 9

  1. 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,277
  2. 2017Learning Generative Models with Sinkhorn Divergencescited by 310
  3. 2017Near-linear time approximation algorithms for optimal transport via Sinkhorn iterationcited by 219
  4. 2017Smooth and Sparse Optimal Transportcited by 83
  5. 2024Hyper-Connectionscited by 81
  6. 2025mHC: Manifold-Constrained Hyper-Connectionscited by 76
  7. 2022A Unified Framework for Implicit Sinkhorn Differentiationcited by 28
  8. 2018An explicit analysis of the entropic penalty in linear programmingcited by 23
  9. 2024Annealed Sinkhorn for Optimal Transport: convergence, regularization path and debiasingcited by 6

Built on by

Not cited yet.