# Fine-tune your Classifier: Finding Correlations With Temperature

arXiv: https://arxiv.org/abs/2210.09715 · HTML: https://mlsanity.com/paper/2210.09715/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 25 references that are in the map.
- First posted on arXiv 18 Oct 2022.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020)
- [Toward Controlled Generation of Text](https://mlsanity.com/paper/1703.00955/) (2017)
- [Language GANs Falling Short](https://mlsanity.com/paper/1811.02549/) (2018)
- [L2-constrained Softmax Loss for Discriminative Face Verification](https://mlsanity.com/paper/1703.09507/) (2017)
- [How Complex is your classification problem? A survey on measuring classification complexity](https://mlsanity.com/paper/1808.03591/) (2018)
- [Improving Generalization via Scalable Neighborhood Component Analysis](https://mlsanity.com/paper/1808.04699/) (2018)
- [Contextual Temperature for Language Modeling](https://mlsanity.com/paper/2012.13575/) (2020)
- [Heated-Up Softmax Embedding](https://mlsanity.com/paper/1809.04157/) (2018)
- [Rethinking Feature Discrimination and Polymerization for Large-scale Recognition](https://mlsanity.com/paper/1710.00870/) (2017)
- [Softmax Q-Distribution Estimation for Structured Prediction: A Theoretical Interpretation for RAML](https://mlsanity.com/paper/1705.07136/) (2017)
- [What can we Learn by Predicting Accuracy?](https://mlsanity.com/paper/2208.01358/) (2022)
- [Learning an Adaptation Function to Assess Image Visual Similarities](https://mlsanity.com/paper/2206.01417/) (2022)

## Abstract (arXiv)

Temperature is a widely used hyperparameter in various tasks involving neural networks, such as classification or metric learning, whose choice can have a direct impact on the model performance. Most of existing works select its value using hyperparameter optimization methods requiring several runs to find the optimal value. We propose to analyze the impact of temperature on classification tasks by describing a dataset as a set of statistics computed on representations on which we can build a heuristic giving us a default value of temperature. We study the correlation between these extracted statistics and the observed optimal temperatures. This preliminary study on more than a hundred combinations of different datasets and features extractors highlights promising results towards the construction of a general heuristic for temperature.