The Matching Principle: When Does a Training Penalty Cover Deployment Shift?
Not on the map. Why?
Abstract (arXiv)
Ordinary training optimises the task loss and then stops. It never pays for internal representation energy: Jacobians can stay large in directions that never helped the label, so even small label-preserving noise throws the model off---a design gap that classical noise-injection theory fixes at second order, but only when applied as default regularisation, which current practice does not do. We make that precise with a Matching Principle: name deployment directions (Sigma_task) and the training penalty Sigma', and ask whether the second covers the first. The no-thinking default is even-spread / isotropic penalty (Sigma' proportional to I)---classical Gaussian / Tikhonov at second order: no axis estimate, no architecture change, and---in a simple linear ridge model---strictly less deployment drift than task-only training, with no coverage miss by construction. When axes are known, matching is sharper; when they are missed, a residual floor remains. Across seven domains a named second-moment penalty beats unregularised training; a controlled illustration recovers match > even-spread > wrong-axis when axes are forced. The ridge theorems are proved; deep nets remain experiments under a specified perturbation. Design rule: fix internal energy by default (even-spread); match when axes are known; treat losses that control representation sensitivity as first-class design.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 3 of its 28 references that are in the map. First posted on arXiv 21 May 2026.
Builds on
3 of 28 references.
References · 3
- 2019Invariant Risk Minimizationcited by 1,854
- 2014Deep Domain Confusion: Maximizing for Domain Invariancecited by 600
- 2017Domain Adaptation for Visual Applications: A Comprehensive Surveycited by 128
Built on by
Citing papers per year: 2026: 1.
1 paper.