Support Topology and Gradient Mixing in Sinkhorn Layers
Not on the map. Why?
Abstract (arXiv)
Sparse Sinkhorn layers use a fixed support graph to restrict transport between tokens. How does this graph control gradient propagation through the scaling iterations. We develop a fixed-support calculus showing that each row-column cycle induces a row-stochastic operator on column-potential perturbations modulo constants. Its transpose propagates zero-mass reverse-mode cotangents. The finite-cycle operator uses two distinct half-step transport plans; at a balanced fixed point it reduces to a two-step walk determined by a single plan. We derive the accompanying score and marginal source terms and use Dobrushin contraction and minorization to bound homogeneous and source-driven tail cotangents. Our main result characterizes when support and marginals guarantee one-step contraction uniformly over finite scores: every feasible face of the transportation polytope must have pairwise two-hop column overlap. Otherwise, suitable score directions make the contraction coefficient arbitrarily close to one. We extend this analysis to ordered support schedules and derive certificates for partition heat-bath layers, coordinate sweeps, forced shared mass, and register-augmented supports. These results provide mathematical criteria for support design in differentiable transport layers, with guarantees restricted to the fixed-support quotient-gradient component.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 5 of its 15 references that are in the map. First posted on arXiv 7 Sep 2026.
Builds on
5 of 15 references.
References · 5
- 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
- 2022A Unified Framework for Implicit Sinkhorn Differentiationcited by 28
- 2018An explicit analysis of the entropic penalty in linear programmingcited by 24
- 2022The derivatives of Sinkhorn-Knopp convergecited by 9
- 2013Dobrushin ergodicity coefficient for Markov operators on cones, and beyondcited by 1
Built on by
Not cited yet.