mlsanityData to 7 Oct 2026

GradTail: Learning Long-Tailed Data Using Gradient-based Sample Weighting

arXiv 2201.05938 ↗First posted 16 Jan 2022cs.LG cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

We propose GradTail, an algorithm that uses gradients to improve model performance on the fly in the face of long-tailed training data distributions. Unlike conventional long-tail classifiers which operate on converged - and possibly overfit - models, we demonstrate that an approach based on gradient dot product agreement can isolate long-tailed data early on during model training and improve performance by dynamically picking higher sample weights for that data. We show that such upweighting leads to model improvements for both classification and regression models, the latter of which are relatively unexplored in the long-tail literature, and that the long-tail examples found by gradient alignment are consistent with our semantic expectations.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 20 of its 29 references that are in the map. First posted on arXiv 16 Jan 2022.

Builds on

20 of 29 references.

References · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  5. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  6. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
  7. 2017Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semanticscited by 1,345
  8. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  9. 2017GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networkscited by 816
  10. 2019Large-Scale Long-Tailed Recognition in an Open Worldcited by 704
  11. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 555
  12. 2019Learning Loss for Active Learningcited by 335
8 more
  1. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  2. 2020Equalization Loss for Long-Tailed Object Recognitioncited by 285
  3. 2020Just Pick a Sign: Optimizing Deep Multitask Models with Gradient Sign Dropoutcited by 199
  4. 2019Ensemble Distribution Distillationcited by 158
  5. 2017Multiplicative Normalizing Flows for Variational Bayesian Neural Networkscited by 157
  6. 2018Out-of-Distribution Detection Using an Ensemble of Self Supervised Leave-out Classifierscited by 126
  7. 2020Can weight sharing outperform random architecture search? An investigation with TuNAScited by 68
  8. 2019FastDraw: Addressing the Long Tail of Lane Detection by Adapting a Sequential Prediction Networkcited by 34

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 0; 2026: 1.

4 papers.

  1. 2023Revisiting Mobility Modeling with Graph: A Graph Transformer Model for Next Point-of-Interest Recommendationcited by 11
  2. 2024On the Role of Long-tail Knowledge in Retrieval Augmented Large Language Modelscited by 10
  3. 2026Long-Tail Knowledge in Large Language Models: Taxonomy, Mechanisms, Interventions and Implicationscited by 2
  4. 2023SHIFT3D: Synthesizing Hard Inputs For Tricking 3D Detectorscited by 1