mlsanityData to 7 Oct 2026

Optimizing Rank-based Metrics with Blackbox Differentiation

arXiv 1912.03500 ↗First posted 7 Dec 2019cs.LG stat.MLcited by 58

Not on the map. Why?

Abstract (arXiv)

Rank-based metrics are some of the most widely used criteria for performance evaluation of computer vision models. Despite years of effort, direct optimization for these metrics remains a challenge due to their non-differentiable and non-decomposable nature. We present an efficient, theoretically sound, and general method for differentiating rank-based metrics with mini-batch gradient descent. In addition, we address optimization instability and sparsity of the supervision signal that both arise from using rank-based metrics as optimization targets. Resulting losses based on recall and Average Precision are applied to image retrieval and object detection tasks. We obtain performance that is competitive with state-of-the-art on standard image retrieval datasets and consistently improve performance of near state-of-the-art object detectors. The code is available at https://github.com/martius-lab/blackbox-backprop

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 17; 2023: 13; 2024: 9; 2025: 3; 2026: 2.

58 papers.

Sort:

By area:

  1. 2023Decision-Focused Learning: Foundations, State of the Art, Benchmark and Future Opportunitiescited by 129
  2. 2023LVM-Med: Learning Large-Scale Self-Supervised Vision Models for Medical Imaging via Second-order Graph Matchingcited by 60
  3. 2022Ordered Subgraph Aggregation Networkscited by 57
  4. 2022RankSim: Ranking Similarity Regularization for Deep Imbalanced Regressioncited by 52
  5. 2021CombOptNet: Fit the Right NP-Hard Problem by Learning Integer Programming Constraintscited by 49
  6. 2021Recall@k Surrogate Loss with Large Batches and Similarity Mixupcited by 49
  7. 2021Stochastic Optimization of Areas Under Precision-Recall Curves with Provable Convergencecited by 34
  8. 2022SIMPLE: A Gradient Estimator for k-Subset Samplingcited by 34
  9. 2022Backpropagation through Combinatorial Algorithms: Identity with Projection Workscited by 29
  10. 2022Rank-based Decomposable Losses in Machine Learning: A Surveycited by 26
  11. 2023Fast, Differentiable and Sparse Top-k: a Convex Analysis Perspectivecited by 26
  12. 2021Rank & Sort Loss for Object Detection and Instance Segmentationcited by 22
38 more
  1. 2021Learning with Algorithmic Supervision via Continuous Relaxationscited by 21
  2. 2022SurCo: Learning Linear Surrogates For Combinatorial Nonlinear Optimization Problemscited by 21
  3. 2021Robust and Decomposable Average Precision for Image Retrievalcited by 19
  4. 2024RankED: Addressing Imbalance and Uncertainty in Edge Detection Using Ranking-based Lossescited by 18
  5. 2022Learning Where To Look -- Generative NAS is Surprisingly Efficientcited by 15
  6. 2023LibAUC: A Deep Learning Library for X-Risk Optimizationcited by 15
  7. 2021Self-Supervised Learning of Audio Representations from Permutations with Differentiable Rankingcited by 14
  8. 2021Momentum Accelerates the Convergence of Stochastic AUPRC Maximizationcited by 14
  9. 2023Long-tail Detection with Effective Class-Marginscited by 13
  10. 2021Neuro-algorithmic Policies enable Fast Combinatorial Generalizationcited by 11
  11. 2021Combinatorial Optimization for Panoptic Segmentation: A Fully Differentiable Approachcited by 10
  12. 2023Optimization of Rank Losses for Image Retrievalcited by 10
  13. 2021MetricOpt: Learning to Optimize Black-Box Evaluation Metricscited by 9
  14. 2023IKOL: Inverse kinematics optimization layer for 3D human pose and shape estimation via Gauss-Newton differentiationcited by 9
  15. 2024Not All Pairs are Equal: Hierarchical Learning for Average-Precision-Oriented Video Retrievalcited by 9
  16. 2022Total Variation Optimization Layers for Computer Visioncited by 8
  17. 2022Hierarchical Average Precision Training for Pertinent Image Retrievalcited by 8
  18. 2022Algorithmic Foundations of Empirical X-risk Minimizationcited by 7
  19. 2021Set-to-Sequence Methods in Machine Learning: a Reviewcited by 6
  20. 2021Deep AUC Maximization for Medical Image Classification: Challenges and Opportunitiescited by 6
  21. 2024LPGD: A General Framework for Backpropagation through Embedded Optimization Layerscited by 6
  22. 2024ExGra-Med: Extended Context Graph Alignment for Medical Vision-Language Modelscited by 6
  23. 2022Supervised Metric Learning to Rank for Retrieval via Contextual Similarity Optimizationcited by 5
  24. 2022A Solver-Free Framework for Scalable Learning in Neural ILP Architecturescited by 5
  25. 2021Rethinking the Optimization of Average Precision: Only Penalizing Negative Instances before Positive Ones is Enoughcited by 4
  26. 2023Center Contrastive Loss for Metric Learningcited by 4
  27. 2023Divide & Conquer for Entailment-aware Multi-hop Evidence Retrievalcited by 3
  28. 2022Intra-class Adaptive Augmentation with Neighbor Correction for Deep Metric Learningcited by 2
  29. 2023SimCon Loss with Multiple Views for Text Supervised Semantic Segmentationcited by 2
  30. 2024Evolutionary Retrofittingcited by 2
  31. 2024Newton Losses: Using Curvature Information for Learning with Differentiable Algorithmscited by 2
  32. 2025Exact Reformulation and Optimization for Direct Metric Optimization in Binary Imbalanced Classificationcited by 2
  33. 2026SoftJAX & SoftTorch: Empowering Automatic Differentiation Libraries with Informative Gradientscited by 2
  34. 2022Improving Generalization of Metric Learning via Listwise Self-distillationcited by 1
  35. 2023Ranking Regularization for Critical Rare Classes: Minimizing False Positives at a High True Positive Ratecited by 1
  36. 2024Optimizing Automatic Speech Assessment: W-RankSim Regularization and Hybrid Feature Fusion Strategiescited by 1
  37. 2025Instance-Level Generation for Representation Learningcited by 1
  38. 2022Segment Augmentation and Differentiable Ranking for Logo Retrievalcited by 0

50 most cited of 58 All 58 (JSON)