Asymptotic and Finite-Time Guarantees for Langevin-Based Temperature Annealing in InfoNCE
Not on the map. Why?
Abstract (arXiv)
The InfoNCE loss in contrastive learning depends critically on a temperature parameter, yet its dynamics under fixed versus annealed schedules remain poorly understood. We provide a theoretical analysis by modeling embedding evolution under Langevin dynamics on a compact Riemannian manifold. Under mild smoothness and energy-barrier assumptions, we show that classical simulated annealing guarantees extend to this setting: slow logarithmic inverse-temperature schedules ensure convergence in probability to a set of globally optimal representations, while faster schedules risk becoming trapped in suboptimal minima. Our results establish a link between contrastive learning and simulated annealing, providing a principled basis for understanding and tuning temperature schedules.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 17 of its 37 references that are in the map. First posted on arXiv 13 Mar 2026.
Builds on
17 of 37 references.
References · 17
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,393
- 2019A Theoretical Analysis of Contrastive Unsupervised Representation Learningcited by 530
- 2020Understanding the Behaviour of Contrastive Losscited by 397
- 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
- 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 298
- 2017A Conceptual Introduction to Hamiltonian Monte Carlocited by 207
- 2022Chaos is a Ladder: A New Theoretical Understanding of Contrastive Learning via Augmentation Overlapcited by 105
- 2023Temperature Schedules for Self-Supervised Contrastive Methods on Long-Tail Datacited by 44
5 more
- 2024InfoNCE: Identifying the Gap Between Theory and Practicecited by 39
- 2020On Learning Rates and Schr"odinger Operatorscited by 33
- 2023Not All Semantics are Created Equal: Contrastive Self-supervised Learning with Automatic Temperature Individualizationcited by 25
- 2022What shapes the loss landscape of self-supervised learning?cited by 19
- 2025Temperature-Free Loss Function for Contrastive Learningcited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.