# Temperature-Free Loss Function for Contrastive Learning

arXiv: https://arxiv.org/abs/2501.17683 · HTML: https://mlsanity.com/paper/2501.17683/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jan 2025.

## Built on by (most cited first, 7 of 7)

- [ReScene4D: Temporally Consistent Semantic Instance Segmentation of Evolving Indoor 3D Scenes](https://mlsanity.com/paper/2601.11508/) (2026)
- [Cross-Lingual IPA Contrastive Learning for Zero-Shot NER](https://mlsanity.com/paper/2503.07214/) (2025)
- [MM-TS: Multi-Modal Temperature and Margin Schedules for Contrastive Learning with Long-Tail Data](https://mlsanity.com/paper/2603.08202/) (2026)
- [Asymptotic and Finite-Time Guarantees for Langevin-Based Temperature Annealing in InfoNCE](https://mlsanity.com/paper/2603.12552/) (2026)
- [Enhancing Language Models for Robust Greenwashing Detection](https://mlsanity.com/paper/2601.21722/) (2026)
- [MaCo-GAN: Manifold-Contrastive Adversarial Learning for Single Image Super-Resolution](https://mlsanity.com/paper/2606.05068/) (2026)
- [TeMo: Temperature Modulation for Multimodal Contrastive Learning](https://mlsanity.com/paper/2609.07540/) (2026)

## Abstract (arXiv)

As one of the most promising methods in self-supervised learning, contrastive learning has achieved a series of breakthroughs across numerous fields. A predominant approach to implementing contrastive learning is applying InfoNCE loss: By capturing the similarities between pairs, InfoNCE loss enables learning the representation of data. Albeit its success, adopting InfoNCE loss requires tuning a temperature, which is a core hyperparameter for calibrating similarity scores. Despite its significance and sensitivity to performance being emphasized by several studies, searching for a valid temperature requires extensive trial-and-error-based experiments, which increases the difficulty of adopting InfoNCE loss. To address this difficulty, we propose a novel method to deploy InfoNCE loss without temperature. Specifically, we replace temperature scaling with the inverse hyperbolic tangent function, resulting in a modified InfoNCE loss. In addition to hyperparameter-free deployment, we observed that the proposed method even yielded a performance gain in contrastive learning. Our detailed theoretical analysis discovers that the current practice of temperature scaling in InfoNCE loss causes serious problems in gradient descent, whereas our method provides desirable gradient properties. The proposed method was validated on five benchmarks on contrastive learning, yielding satisfactory results without temperature tuning.