# SoDeep: a Sorting Deep net to learn ranking loss surrogates

arXiv: https://arxiv.org/abs/1904.04272 · HTML: https://mlsanity.com/paper/1904.04272/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 Apr 2019.

## Built on by (most cited first, 28 of 28)

- [RankSim: Ranking Similarity Regularization for Deep Imbalanced Regression](https://mlsanity.com/paper/2205.15236/) (2022)
- [Recall@k Surrogate Loss with Large Batches and Similarity Mixup](https://mlsanity.com/paper/2108.11179/) (2021)
- [ICON: Learning Regular Maps Through Inverse Consistency](https://mlsanity.com/paper/2105.04459/) (2021)
- [Robust and Decomposable Average Precision for Image Retrieval](https://mlsanity.com/paper/2110.01445/) (2021)
- [NTIRE 2021 Challenge on Perceptual Image Quality Assessment](https://mlsanity.com/paper/2105.03072/) (2021)
- [Brain Age Estimation From MRI Using Cascade Networks with Ranking Loss](https://mlsanity.com/paper/2106.03052/) (2021)
- [Optimization of Rank Losses for Image Retrieval](https://mlsanity.com/paper/2309.08250/) (2023)
- [How to Handle Sketch-Abstraction in Sketch-Based Image Retrieval?](https://mlsanity.com/paper/2403.07203/) (2024)
- [MetricOpt: Learning to Optimize Black-Box Evaluation Metrics](https://mlsanity.com/paper/2104.10631/) (2021)
- [TractoSCR: A Novel Supervised Contrastive Regression Framework for Prediction of Neurocognitive Measures Using Multi-Site Harmonized Diffusion MRI Tractography](https://mlsanity.com/paper/2210.07411/) (2022)
- [Memorability: An image-computable measure of information utility](https://mlsanity.com/paper/2104.00805/) (2021)
- [Exploring the Algorithm-Dependent Generalization of AUPRC Optimization with List Stability](https://mlsanity.com/paper/2209.13262/) (2022)
- [(ASNA) An Attention-based Siamese-Difference Neural Network with Surrogate Ranking Loss function for Perceptual Image Quality Assessment](https://mlsanity.com/paper/2105.02531/) (2021)
- [Set-to-Sequence Methods in Machine Learning: a Review](https://mlsanity.com/paper/2103.09656/) (2021)
- [Rethinking the Optimization of Average Precision: Only Penalizing Negative Instances before Positive Ones is Enough](https://mlsanity.com/paper/2102.04640/) (2021)
- [Fine-grained Correlation Loss for Regression](https://mlsanity.com/paper/2207.00347/) (2022)
- [Center Contrastive Loss for Metric Learning](https://mlsanity.com/paper/2308.00458/) (2023)
- [Learning semantic image quality for fetal ultrasound from noisy ranking annotation](https://mlsanity.com/paper/2402.08294/) (2024)
- [Beyond Score Changes: Adversarial Attack on No-Reference Image Quality Assessment from Two Perspectives](https://mlsanity.com/paper/2404.13277/) (2024)
- [QAMRO: Quality-aware Adaptive Margin Ranking Optimization for Human-aligned Assessment of Audio Generation Systems](https://mlsanity.com/paper/2508.08957/) (2025)
- [Content-Diverse Comparisons improve IQA](https://mlsanity.com/paper/2211.05215/) (2022)
- [Semi-Supervised Variational Adversarial Active Learning via Learning to Rank and Agreement-Based Pseudo Labeling](https://mlsanity.com/paper/2408.12774/) (2024)
- [Exact Reformulation and Optimization for Direct Metric Optimization in Binary Imbalanced Classification](https://mlsanity.com/paper/2507.15240/) (2025)
- [Language Model Metrics and Procrustes Analysis for Improved Vector Transformation of NLP Embeddings](https://mlsanity.com/paper/2106.02490/) (2021)
- [Searching Parameterized AP Loss for Object Detection](https://mlsanity.com/paper/2112.05138/) (2021)
- [Towards Practicality of Sketch-Based Visual Understanding](https://mlsanity.com/paper/2210.15146/) (2022)
- [DM3D: Dynamic Mamba via Offset-Guided Feature Resampling for Point Cloud Understanding](https://mlsanity.com/paper/2512.03424/) (2025)
- [Backpropagation-Free Test-Time Adaptation for Lightweight EEG-Based Brain-Computer Interfaces](https://mlsanity.com/paper/2601.07556/) (2026)

## Abstract (arXiv)

Several tasks in machine learning are evaluated using non-differentiable metrics such as mean average precision or Spearman correlation. However, their non-differentiability prevents from using them as objective functions in a learning framework. Surrogate and relaxation methods exist but tend to be specific to a given metric. In the present work, we introduce a new method to learn approximations of such non-differentiable objective functions. Our approach is based on a deep architecture that approximates the sorting of arbitrary sets of scores. It is trained virtually for free using synthetic data. This sorting deep (SoDeep) net can then be combined in a plug-and-play manner with existing deep architectures. We demonstrate the interest of our approach in three different tasks that require ranking: Cross-modal text-image retrieval, multi-label image classification and visual memorability ranking. Our approach yields very competitive results on these three tasks, which validates the merit and the flexibility of SoDeep as a proxy for sorting operation in ranking-based losses.