mlsanityData to 7 Oct 2026

Differentiable Sorting Networks for Scalable Sorting and Ranking Supervision

arXiv 2105.04019 ↗First posted 9 May 2021cs.LG cs.IRcited by 31

Not on the map. Why?

Abstract (arXiv)

Sorting and ranking supervision is a method for training neural networks end-to-end based on ordering constraints. That is, the ground truth order of sets of samples is known, while their absolute values remain unsupervised. For that, we propose differentiable sorting networks by relaxing their pairwise conditional swap operations. To address the problems of vanishing gradients and extensive blurring that arise with larger numbers of layers, we propose mapping activations to regions with moderate gradients. We consider odd-even as well as bitonic sorting networks, which outperform existing relaxations of the sorting operation. We show that bitonic sorting networks can achieve stable training on large input sets of up to 1024 elements.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 10 of its 26 references that are in the map. First posted on arXiv 9 May 2021.

Builds on

10 of 26 references.

References · 10

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2014Neural Turing Machinescited by 630
  5. 2015Order Matters: Sequence to sequence for setscited by 236
  6. 2020Fast Differentiable Sorting and Rankingcited by 196
  7. 2018Learning Latent Permutations with Gumbel-Sinkhorn Networkscited by 172
  8. 2019Differentiable Ranks and Sorting using Optimal Transportcited by 113
  9. 2019Stochastic Optimization of Sorting Networks via Continuous Relaxationscited by 111
  10. 2013Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networkscited by 84

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 5; 2023: 7; 2024: 9; 2025: 4; 2026: 5.

31 papers.

Sort:

By area:

  1. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 460
  2. 2022Deep Differentiable Logic Gate Networkscited by 67
  3. 2021Deep Neural Networks for Rank-Consistent Ordinal Regression Based On Conditional Probabilitiescited by 57
  4. 2023Fast, Differentiable and Sparse Top-k: a Convex Analysis Perspectivecited by 26
  5. 2022Monotonic Differentiable Sorting Networkscited by 25
  6. 2022Differentiable Top-k Classification Learningcited by 21
  7. 2024Near, far: Patch-ordering enhances vision foundation models' scene understandingcited by 15
  8. 2023Learning by Sorting: Self-supervised Learning with Group Ordering Constraintscited by 13
  9. 2024Data-Efficient Learning with Neural Programscited by 7
  10. 2023Beam Tree Recursive Cellscited by 6
  11. 2023Parallel Algorithms Align with Neural Executioncited by 6
  12. 2023Adaptive Neural Ranking Framework: Toward Maximized Business Goal for Cascade Ranking Systemscited by 6
19 more
  1. 2024Made to Order: Discovering monotonic temporal changes via self-supervised video orderingcited by 5
  2. 2024SymmetricDiffusers: Learning Discrete Diffusion on Finite Symmetric Groupscited by 5
  3. 2023Generalized Neural Sorting Networks with Error-Free Differentiable Swap Functionscited by 4
  4. 2024Generalizing Stochastic Smoothing for Differentiation and Gradient Estimationcited by 4
  5. 2022Relational Surrogate Loss Learningcited by 2
  6. 2024Holistic Utility Preference Learning for Listwise Alignmentcited by 2
  7. 2024Newton Losses: Using Curvature Information for Learning with Differentiable Algorithmscited by 2
  8. 2025Learning Cascade Ranking as One Networkcited by 2
  9. 2026SoftJAX & SoftTorch: Empowering Automatic Differentiation Libraries with Informative Gradientscited by 2
  10. 2023Diffsurv: Differentiable sorting for censored time-to-event datacited by 0
  11. 2024SAMSA: Efficient Transformer for Many Data Modalitiescited by 0
  12. 2024RoCP-GNN: Robust Conformal Prediction for Graph Neural Networks in Node-Classificationcited by 0
  13. 2025On the algorithmic construction of deep ReLU networkscited by 0
  14. 2025Conformal Correction for Efficiency May be at Odds with Entropycited by 0
  15. 2025Structural Incompatibility of Differentiable Sorting and Within-Vector Rank Normalizationcited by 0
  16. 2026Logic-Gated Time-Shared Feedforward Networks for Alternating Finite Automata: Exact Simulation and Learnabilitycited by 0
  17. 2026LaPrune: Controllable Differentiable Sparsity at Million Scalecited by 0
  18. 2026Fast LapSum: Exact Differentiable Top-k at Million Scalecited by 0
  19. 2026SPD: Single Pass Decoding for Generative Rerankingcited by 0