# Monotonic Differentiable Sorting Networks

arXiv: https://arxiv.org/abs/2203.09630 · HTML: https://mlsanity.com/paper/2203.09630/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 17 Mar 2022.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distances](https://mlsanity.com/paper/1306.0895/) (2013)
- [Fast Differentiable Sorting and Ranking](https://mlsanity.com/paper/2002.08871/) (2020)
- [Learning Latent Permutations with Gumbel-Sinkhorn Networks](https://mlsanity.com/paper/1802.08665/) (2018)
- [Differentiable Ranks and Sorting using Optimal Transport](https://mlsanity.com/paper/1905.11885/) (2019)
- [Stochastic Optimization of Sorting Networks via Continuous Relaxations](https://mlsanity.com/paper/1903.08850/) (2019)
- [Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks](https://mlsanity.com/paper/1312.6082/) (2013)
- [Differentiable Patch Selection for Image Recognition](https://mlsanity.com/paper/2104.03059/) (2021)
- [Differentiable Sorting Networks for Scalable Sorting and Ranking Supervision](https://mlsanity.com/paper/2105.04019/) (2021)
- [Learning with Algorithmic Supervision via Continuous Relaxations](https://mlsanity.com/paper/2110.05651/) (2021)
- [A Continuous Relaxation of Beam Search for End-to-end Training of Neural Sequence Models](https://mlsanity.com/paper/1708.00111/) (2017)

## Built on by (most cited first, 25 of 25)

- [Git Re-Basin: Merging Models modulo Permutation Symmetries](https://mlsanity.com/paper/2209.04836/) (2022)
- [LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics](https://mlsanity.com/paper/2511.08544/) (2025)
- [Deep Differentiable Logic Gate Networks](https://mlsanity.com/paper/2210.08277/) (2022)
- [Differentiable Top-k Classification Learning](https://mlsanity.com/paper/2206.07290/) (2022)
- [An Information Theoretic Perspective on Conformal Prediction](https://mlsanity.com/paper/2405.02140/) (2024)
- [Learning by Sorting: Self-supervised Learning with Group Ordering Constraints](https://mlsanity.com/paper/2301.02009/) (2023)
- [LapSum -- One Method to Differentiate Them All: Ranking, Sorting and Top-k Selection](https://mlsanity.com/paper/2503.06242/) (2025)
- [Uncertainty Quantification via Stable Distribution Propagation](https://mlsanity.com/paper/2402.08324/) (2024)
- [Beam Tree Recursive Cells](https://mlsanity.com/paper/2305.19999/) (2023)
- [Adaptive Neural Ranking Framework: Toward Maximized Business Goal for Cascade Ranking Systems](https://mlsanity.com/paper/2310.10462/) (2023)
- [SymmetricDiffusers: Learning Discrete Diffusion on Finite Symmetric Groups](https://mlsanity.com/paper/2410.02942/) (2024)
- [Made to Order: Discovering monotonic temporal changes via self-supervised video ordering](https://mlsanity.com/paper/2404.16828/) (2024)
- [Generalized Neural Sorting Networks with Error-Free Differentiable Swap Functions](https://mlsanity.com/paper/2310.07174/) (2023)
- [Generalizing Stochastic Smoothing for Differentiation and Gradient Estimation](https://mlsanity.com/paper/2410.08125/) (2024)
- [Differentiable Random Partition Models](https://mlsanity.com/paper/2305.16841/) (2023)
- [Hacking Generative Models with Differentiable Network Bending](https://mlsanity.com/paper/2310.04816/) (2023)
- [Holistic Utility Preference Learning for Listwise Alignment](https://mlsanity.com/paper/2410.18127/) (2024)
- [Newton Losses: Using Curvature Information for Learning with Differentiable Algorithms](https://mlsanity.com/paper/2410.19055/) (2024)
- [Learning Cascade Ranking as One Network](https://mlsanity.com/paper/2503.09492/) (2025)
- [Permutation Learning with Only N Parameters: From SoftSort to Self-Organizing Gaussians](https://mlsanity.com/paper/2503.13051/) (2025)
- [Learning Distributions over Permutations and Rankings with Factorized Representations](https://mlsanity.com/paper/2505.24664/) (2025)
- [Structural Incompatibility of Differentiable Sorting and Within-Vector Rank Normalization](https://mlsanity.com/paper/2512.22587/) (2025)
- [Reward Learning through Ranking Mean Squared Error](https://mlsanity.com/paper/2601.09236/) (2026)
- [Class Adaptive Conformal Training](https://mlsanity.com/paper/2601.09522/) (2026)
- [Learning Permutation Distributions via Reflected Diffusion on Ranks](https://mlsanity.com/paper/2603.17353/) (2026)

## Abstract (arXiv)

Differentiable sorting algorithms allow training with sorting and ranking supervision, where only the ordering or ranking of samples is known. Various methods have been proposed to address this challenge, ranging from optimal transport-based differentiable Sinkhorn sorting algorithms to making classic sorting networks differentiable. One problem of current differentiable sorting methods is that they are non-monotonic. To address this issue, we propose a novel relaxation of conditional swap operations that guarantees monotonicity in differentiable sorting networks. We introduce a family of sigmoid functions and prove that they produce differentiable sorting networks that are monotonic. Monotonicity ensures that the gradients always have the correct sign, which is an advantage in gradient-based optimization. We demonstrate that monotonic differentiable sorting networks improve upon previous differentiable sorting methods.