mlsanityData to 7 Oct 2026

Cyclical Focal Loss

arXiv 2202.08978 ↗First posted 16 Feb 2022cs.CV cs.AI cs.LGcited by 8

Not on the map. Why?

Abstract (arXiv)

The cross-entropy softmax loss is the primary loss function used to train deep neural networks. On the other hand, the focal loss function has been demonstrated to provide improved performance when there is an imbalance in the number of training samples in each class, such as in long-tailed datasets. In this paper, we introduce a novel cyclical focal loss and demonstrate that it is a more universal loss function than cross-entropy softmax loss or focal loss. We describe the intuition behind the cyclical focal loss and our experiments provide evidence that cyclical focal loss provides superior performance for balanced, imbalanced, or long-tailed datasets. We provide numerous experimental results for CIFAR-10/CIFAR-100, ImageNet, balanced and imbalanced 4,000 training sample versions of CIFAR-10/CIFAR-100, and ImageNet-LT and Places-LT from the Open Long-Tailed Recognition (OLTR) challenge. Implementing the cyclical focal loss function requires only a few lines of code and does not increase training time. In the spirit of reproducibility, our code is available at urlhttps://github.com/lnsmith54/CFL.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 21 of its 28 references that are in the map. First posted on arXiv 16 Feb 2022.

Builds on

21 of 28 references.

References · 21

  1. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  2. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  3. 2017Accurate, Large Minibatch SGD: Training ImageNet in 1 Hourcited by 1,479
  4. 2017Focal Loss for Dense Object Detectioncited by 749
  5. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  6. 2015Cyclical Learning Rates for Training Neural Networkscited by 627
  7. 2015Towards Open Set Deep Networkscited by 533
  8. 2020Generalized Focal Loss: Learning Qualified and Distributed Bounding Boxes for Dense Object Detectioncited by 408
  9. 2020Asymmetric Loss For Multi-Label Classificationcited by 379
  10. 2020Calibrating Deep Neural Networks using Focal Losscited by 353
  11. 2017Don't Decay the Learning Rate, Increase the Batch Sizecited by 313
  12. 2017Super-Convergence: Very Fast Training of Neural Networks Using Large Learning Ratescited by 196
9 more
  1. 2020TResNet: High Performance GPU-Dedicated Architecturecited by 119
  2. 2020The Early Phase of Neural Network Trainingcited by 114
  3. 2018Three Mechanisms of Weight Decay Regularizationcited by 112
  4. 2019Time Matters in Regularizing Deep Networks: Weight Decay and Data Augmentation Affect Early Learning Dynamics, Matter Little Near Convergencecited by 54
  5. 2020On the training dynamics of deep networks with L2 regularizationcited by 38
  6. 2019Adaptive Weight Decay for Deep Neural Networkscited by 13
  7. 2020Understanding Decoupled and Early Weight Decaycited by 12
  8. 2018Focal Loss in 3D Object Detectioncited by 8
  9. 2021Contrastive Unpaired Translation using Focal Loss for Patch Classificationcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 3; 2024: 2; 2025: 1.

8 papers.

  1. 2022Are All Losses Created Equal: A Neural Collapse Perspectivecited by 78
  2. 2024A Short Survey on Importance Weighting for Machine Learningcited by 15
  3. 2023Neural Collapse in Multi-label Learning with Pick-all-label Losscited by 14
  4. 2023Skeleton-based Action Recognition through Contrasting Two-Stream Spatial-Temporal Networkscited by 9
  5. 2022General Cyclical Training of Neural Networkscited by 5
  6. 2025Building Age Estimation: A New Multi-Modal Benchmark Dataset and Community Challengecited by 2
  7. 2023A dual-branch model with inter- and intra-branch contrastive loss for long-tailed recognitioncited by 0
  8. 2024MIFI: MultI-camera Feature Integration for Roust 3D Distracted Driver Activity Recognitioncited by 0