# Two-temperature logistic regression based on the Tsallis divergence

arXiv: https://arxiv.org/abs/1705.07210 · HTML: https://mlsanity.com/paper/1705.07210/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2017.

## Built on by (most cited first, 14 of 14)

- [Learning with Feature-Dependent Label Noise: A Progressive Approach](https://mlsanity.com/paper/2103.07756/) (2021)
- [Clusterability as an Alternative to Anchor Points When Learning with Noisy Labels](https://mlsanity.com/paper/2102.05291/) (2021)
- [Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyond](https://mlsanity.com/paper/2104.01493/) (2021)
- [Searching for Robustness: Loss Learning for Noisy Classification Tasks](https://mlsanity.com/paper/2103.00243/) (2021)
- [q-Paths: Generalizing the Geometric Annealing Path using Power Means](https://mlsanity.com/paper/2107.00745/) (2021)
- [Being Properly Improper](https://mlsanity.com/paper/2106.09920/) (2021)
- [Multiplicative Reweighting for Robust Neural Network Optimization](https://mlsanity.com/paper/2102.12192/) (2021)
- [Rethinking Noisy Label Models: Labeler-Dependent Noise with Adversarial Awareness](https://mlsanity.com/paper/2105.14083/) (2021)
- [Proper losses regret at least 1/2-order](https://mlsanity.com/paper/2407.10417/) (2024)
- [Information-Theoretic Measures on Lattices for High-Order Interactions](https://mlsanity.com/paper/2408.07533/) (2024)
- [q-exponential family for policy optimization](https://mlsanity.com/paper/2408.07245/) (2024)
- [What killed the Convex Booster ?](https://mlsanity.com/paper/2205.09628/) (2022)
- [Implicit to Explicit Entropy Regularization: Benchmarking ViT Fine-tuning under Noisy Labels](https://mlsanity.com/paper/2410.04256/) (2024)
- [Perspectives on Tsallis Statistics for Artificial Intelligence](https://mlsanity.com/paper/2608.01223/) (2026)

## Abstract (arXiv)

We develop a variant of multiclass logistic regression that is significantly more robust to noise. The algorithm has one weight vector per class and the surrogate loss is a function of the linear activations (one per class). The surrogate loss of an example with linear activation vector a and class c has the form -logt1 expt2 (ac - Gt2(a)) where the two temperatures t1 and t2 ''temper'' the log and exp, respectively, and Gt2(a) is a scalar value that generalizes the log-partition function. We motivate this loss using the Tsallis divergence. Our method allows transitioning between non-convex and convex losses by the choice of the temperature parameters. As the temperature t1 of the logarithm becomes smaller than the temperature t2 of the exponential, the surrogate loss becomes ''quasi convex''. Various tunings of the temperatures recover previous methods and tuning the degree of non-convexity is crucial in the experiments. In particular, quasi-convexity and boundedness of the loss provide significant robustness to the outliers. We explain this by showing that t1 < 1 caps the surrogate loss and t2 >1 makes the predictive distribution have a heavy tail. We show that the surrogate loss is Bayes-consistent, even in the non-convex case. Additionally, we provide efficient iterative algorithms for calculating the log-partition value only in a few number of iterations. Our compelling experimental results on large real-world datasets show the advantage of using the two-temperature variant in the noisy as well as the noise free case.