mlsanityData to 7 Oct 2026

Robust Bi-Tempered Logistic Loss Based on Bregman Divergences

arXiv 1906.03361 ↗First posted 8 Jun 2019cs.LG stat.MLcited by 61

Not on the map. Why?

Abstract (arXiv)

We introduce a temperature into the exponential function and replace the softmax output layer of neural nets by a high temperature generalization. Similarly, the logarithm in the log loss we use for training is replaced by a low temperature logarithm. By tuning the two temperatures we create loss functions that are non-convex already in the single layer case. When replacing the last layer of the neural nets by our bi-temperature generalization of logistic loss, the training becomes more robust to noise. We visualize the effect of tuning the two temperatures in a simple setting and show the efficacy of our method on large data sets. Our methodology is based on Bregman divergences and is superior to a related two-temperature method using the Tsallis divergence.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 20; 2022: 15; 2023: 10; 2024: 5; 2025: 6; 2026: 5.

61 papers.

Sort:

By area:

  1. 2021Learning with Noisy Labels Revisited: A Study Using Real-World Human Annotationscited by 253
  2. 2021SCARF: Self-Supervised Contrastive Learning using Random Feature Corruptioncited by 151
  3. 2021Large-Scale Differentially Private BERTcited by 117
  4. 2021Learning with Feature-Dependent Label Noise: A Progressive Approachcited by 102
  5. 2022Robust Training under Label Noise by Over-parameterizationcited by 93
  6. 2021To Smooth or Not? When Label Smoothing Meets Noisy Labelscited by 70
  7. 2021Clusterability as an Alternative to Anchor Points When Learning with Noisy Labelscited by 65
  8. 2021Detecting Corrupted Labels Without Training a Model to Predictcited by 57
  9. 2022Self-Filtering: A Noise-Aware Sample Selection for Label Noise with Confidence Penalizationcited by 33
  10. 2022To Aggregate or Not? Learning with Separate Noisy Labelscited by 29
  11. 2022CMW-Net: Learning a Class-Aware Sample Weighting Mapping for Robust Deep Learningcited by 24
  12. 2022On Learning Contrastive Representations for Learning with Noisy Labelscited by 21
38 more
  1. 2022Identifying Hard Noise in Long-Tailed Sample Distributioncited by 19
  2. 2021Exponentiated Gradient Reweighting for Robust Training Under Label Noise and Beyondcited by 16
  3. 2022Detecting Label Errors by using Pre-Trained Language Modelscited by 16
  4. 2023Distributionally Robust Post-hoc Classifiers under Prior Shiftscited by 16
  5. 2021LocoProp: Enhancing BackProp via Local Loss Optimizationcited by 15
  6. 2022DC-Check: A Data-Centric AI checklist to guide the development of reliable machine learning systemscited by 14
  7. 2021Searching for Robustness: Loss Learning for Noisy Classification Taskscited by 13
  8. 2022Labeling Chaos to Learning Harmony: Federated Learning with Noisy Labelscited by 13
  9. 2021Constrained Instance and Class Reweighting for Robust Learning under Label Noisecited by 12
  10. 2021Being Properly Impropercited by 11
  11. 2022Non-convex online learning via algorithmic equivalencecited by 10
  12. 2024How to Make the Gradients Small Privately: Improved Rates for Differentially Private Non-Convex Optimizationcited by 10
  13. 2023SLaM: Student-Label Mixing for Distillation with Unlabeled Examplescited by 9
  14. 2022MuMIC -- Multimodal Embedding for Multi-label Image Classification with Tempered Sigmoidcited by 8
  15. 2023Dynamics-Aware Loss for Learning with Label Noisecited by 8
  16. 2021Locally Adaptive Label Smoothing for Predictive Churncited by 7
  17. 2022Robust PACm: Training Ensemble Models Under Misspecification and Outlierscited by 6
  18. 2022Clustering above Exponential Families with Tempered Exponential Measurescited by 6
  19. 2023The Conditional Cauchy-Schwarz Divergence with Applications to Time-Series Data and Sequential Decision Makingcited by 6
  20. 2023On Classification-Calibration of Gamma-Phi Lossescited by 6
  21. 2025Bias-variance decompositions: the exclusive privilege of Bregman divergencescited by 6
  22. 2025When VLMs Meet Image Classification: Test Sets Renovation via Missing Label Identificationcited by 5
  23. 2021Learning to Combat Noisy Labels via Classification Marginscited by 4
  24. 2021Multiplicative Reweighting for Robust Neural Network Optimizationcited by 4
  25. 2021Rethinking Noisy Label Models: Labeler-Dependent Noise with Adversarial Awarenesscited by 4
  26. 2024Improving Calibration by Relating Focal Loss, Temperature Scaling, and Propernesscited by 4
  27. 2023Logistic-Normal Likelihoods for Heteroscedastic Label Noisecited by 3
  28. 2023Optimal Transport with Tempered Exponential Measurescited by 3
  29. 2024Learning under Label Noise through Few-Shot Human-in-the-Loop Refinementcited by 3
  30. 2021The Phonexia VoxCeleb Speaker Recognition Challenge 2021 System Descriptioncited by 2
  31. 2025Robust Reward Alignment via Hypothesis Space Batch Cuttingcited by 2
  32. 2025SMRS: advocating a unified reporting standard for surrogate models in the artificial intelligence eracited by 2
  33. 2021Lower-Bounded Proper Losses for Weakly Supervised Classificationcited by 1
  34. 2022What killed the Convex Booster ?cited by 1
  35. 2022Layerwise Bregman Representation Learning with Applications to Knowledge Distillationcited by 1
  36. 2024NoisyAG-News: A Benchmark for Addressing Instance-Dependent Noise in Text Classificationcited by 1
  37. 2024Implicit to Explicit Entropy Regularization: Benchmarking ViT Fine-tuning under Noisy Labelscited by 1
  38. 2026Perspectives on Tsallis Statistics for Artificial Intelligencecited by 1

50 most cited of 61 All 61 (JSON)