mlsanityData to 14 Oct 2026

Focal-SAM: Focal Sharpness-Aware Minimization for Long-Tailed Classification

arXiv 2505.01660 ↗First posted 3 May 2025cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Real-world datasets often follow a long-tailed distribution, making generalization to tail classes difficult. Recent methods resorted to long-tail variants of Sharpness-Aware Minimization (SAM), such as ImbSAM and CC-SAM, to improve generalization by flattening the loss landscape. However, these attempts face a trade-off between computational efficiency and control over the loss landscape. On the one hand, ImbSAM is efficient but offers only coarse control as it excludes head classes from the SAM process. On the other hand, CC-SAM provides fine-grained control through class-dependent perturbations but at the cost of efficiency due to multiple backpropagations. Seeing this dilemma, we introduce Focal-SAM, which assigns different penalties to class-wise sharpness, achieving fine-grained control without extra backpropagations, thus maintaining efficiency. Furthermore, we theoretically analyze Focal-SAM's generalization ability and derive a sharper generalization bound. Extensive experiments on both traditional and foundation models validate the effectiveness of Focal-SAM.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Builds on 54 of its 75 references that are in the map. First posted on arXiv 3 May 2025.

Builds on

54 of 75 references.

References · 54

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  7. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  8. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
  9. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  10. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  11. 2019Class-Balanced Loss Based on Effective Number of Samplescited by 1,245
  12. 2019Learning Robust Global Representations by Penalizing Local Predictive Powercited by 1,013
38 more
  1. 2019Learning Imbalanced Datasets with Label-Distribution-Aware Margin Losscited by 989
  2. 2019Decoupling Representation and Classifier for Long-Tailed Recognitioncited by 809
  3. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 705
  4. 2020Long-tail learning via logit adjustmentcited by 618
  5. 2017A systematic study of the class imbalance problem in convolutional neural networkscited by 513
  6. 2019Fantastic Generalization Measures and Where to Find Themcited by 513
  7. 2021Deep Long-Tailed Learning: A Surveycited by 417
  8. 2020Balanced Meta-Softmax for Long-Tailed Visual Recognitioncited by 400
  9. 2019An Investigation into Neural Net Optimization via Hessian Eigenvalue Densitycited by 292
  10. 2023Self-regulating Prompts: Foundational Model Adaptation without Forgettingcited by 272
  11. 2020Long-tailed Recognition by Routing Diverse Distribution-Aware Expertscited by 271
  12. 2021Parametric Contrastive Learningcited by 217
  13. 2021Improving Calibration for Long-Tailed Recognitioncited by 212
  14. 2021Distribution Alignment: A Unified Framework for Long-tail Visual Recognitioncited by 198
  15. 2020Disentangling Label Distribution for Long-tailed Visual Recognitioncited by 172
  16. 2022Balanced Contrastive Learning for Long-Tailed Visual Recognitioncited by 144
  17. 2020M2m: Imbalanced Classification via Major-to-minor Translationcited by 134
  18. 2022Long-Tailed Recognition via Weight Balancingcited by 107
  19. 2021Self-Supervised Aggregation of Diverse Experts for Test-Agnostic Long-Tailed Recognitioncited by 95
  20. 2019Dynamic Curriculum Learning for Imbalanced Data Classificationcited by 93
  21. 2021Distributional Robustness Loss for Long-tail Learningcited by 84
  22. 2021Label-Imbalanced and Group-Sensitive Classification under Overparameterizationcited by 74
  23. 2021Distilling Virtual Examples for Long-tailed Recognitioncited by 72
  24. 2022LPT: Long-tailed Prompt Tuning for Image Classificationcited by 69
  25. 2022Nested Collaborative Learning for Long-Tailed Visual Recognitioncited by 63
  26. 2023Long-Tail Learning with Foundation Model: Heavy Fine-Tuning Hurtscited by 58
  27. 2022Generalized Parametric Contrastive Learningcited by 45
  28. 2021Learning with Multiclass AUC: Theory and Algorithmscited by 35
  29. 2023Exploring Vision-Language Models for Imbalanced Learningcited by 35
  30. 2022OpenAUC: Towards AUC-Oriented Open-Set Recognitioncited by 33
  31. 2023ImbSAM: A Closer Look at Sharpness-Aware Minimization in Class-Imbalanced Recognitioncited by 31
  32. 2022Escaping Saddle Points for Effective Generalization on Class-Imbalanced Datacited by 28
  33. 2024A Systematic Review on Long-Tailed Learningcited by 28
  34. 2024Prompt Learning via Meta-Regularizationcited by 26
  35. 2023A Unified Perspective for Loss-Oriented Imbalanced Learning via Localizationcited by 24
  36. 2022Balanced Product of Calibrated Experts for Long-Tailed Recognitioncited by 20
  37. 2024AUCSeg: AUC-oriented Pixel-level Long-tail Semantic Segmentationcited by 18
  38. 2024DirMixE: Harnessing Test Agnostic Long-tail Recognition with Hierarchical Label Variationscited by 15

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 11.

14 papers.

Sort:

By area:

  1. 2025Improved Balanced Classification with Theoretically Grounded Loss Functionscited by 9
  2. 2025Dual-Stage Reweighted MoE for Long-Tailed Egocentric Mistake Detectioncited by 3
  3. 2026Making Training-Free Diffusion Segmentors Scale with the Generative Powercited by 3
  4. 2025Quantifying the Potential to Escape Filter Bubbles: A Behavior-Aware Measure via Contrastive Simulationcited by 2
  5. 2026Mind the Way You Select Negative Texts: Pursuing the Distance Consistency in OOD Detection with VLMscited by 1
  6. 2026Reframing Long-Tailed Learning via Loss Landscape Geometrycited by 1
  7. 2026Fix the Loss, Not the Radius: Rethinking the Adversarial Perturbation of Sharpness-Aware Minimizationcited by 1
  8. 2026Revisiting Sharpness-Aware Minimization: A More Faithful and Effective Implementationcited by 0
  9. 2026Rethinking Loss Reweighting for Imbalance Learning as an Inverse Problem: A Neural Collapse Point of Viewcited by 0
  10. 2026From Perturbation Correction to Geometry-Aware Sampling: Sharpness-Guided Equilibrium Sampling for Balanced Flat Minima in Long-Tailed Learningcited by 0
  11. 2026Deep Convolutional Large-Margin ℓp-SVDD for Visual Anomaly Detectioncited by 0
  12. 2026Beyond Static Costs: Learning-Dynamics Aware Loss Functions for Long-Tailed Classificationcited by 0
2 more
  1. 2026Age of Learning: Temporal Persistence of Prediction Errors as a Learning Signalcited by 0
  2. 2026OFBD: Object-Focused Background Debiasing for Long-Tailed Learningcited by 0