mlsanityData to 7 Oct 2026

SoftSort: A Continuous Relaxation for the argsort Operator

arXiv 2006.16038 ↗First posted 29 Jun 2020cs.LG stat.MLcited by 66

Not on the map. Why?

Abstract (arXiv)

While sorting is an important procedure in computer science, the argsort operator - which takes as input a vector and returns its sorting permutation - has a discrete image and thus zero gradients almost everywhere. This prohibits end-to-end, gradient-based learning of models that rely on the argsort operator. A natural way to overcome this problem is to replace the argsort operator with a continuous relaxation. Recent work has shown a number of ways to do this, but the relaxations proposed so far are computationally complex. In this work we propose a simple continuous relaxation for the argsort operator which has the following qualities: it can be implemented in three lines of code, achieves state-of-the-art performance, is easy to reason about mathematically - substantially simplifying proofs - and is faster than competing approaches. We open source the code to reproduce all of the experiments and results.

Key facts. Cited by 66 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 7; 2023: 8; 2024: 10; 2025: 18; 2026: 17.

66 papers.

Sort:

By area:

  1. 2022Git Re-Basin: Merging Models modulo Permutation Symmetriescited by 460
  2. 2021SPViT: Enabling Faster Vision Transformers via Soft Token Pruningcited by 165
  3. 2021Robustness of Graph Neural Networks at Scalecited by 100
  4. 2021GrooMeD-NMS: Grouped Mathematically Differentiable NMS for Monocular 3D Object Detectioncited by 66
  5. 2022On the Symmetries of Deep Learning Models and their Internal Representationscited by 63
  6. 2022Differentiable DAG Samplingcited by 46
  7. 2022HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformerscited by 46
  8. 2021Learning with Noisy Labels via Sparse Regularizationcited by 38
  9. 2022Unsupervised Learning of Group Invariant and Equivariant Representationscited by 38
  10. 2023TransFlow: Transformer as Flow Learnercited by 32
  11. 2023Fast, Differentiable and Sparse Top-k: a Convex Analysis Perspectivecited by 26
  12. 2023Learning Probabilistic Symmetrization for Architecture Agnostic Equivariancecited by 25
38 more
  1. 2021Permutation-Invariant Variational Autoencoder for Graph-Level Representation Learningcited by 23
  2. 2023Learned Thresholds Token Merging and Pruning for Vision Transformerscited by 22
  3. 2022Differentiable Top-k Classification Learningcited by 21
  4. 2023Learning by Sorting: Self-supervised Learning with Group Ordering Constraintscited by 13
  5. 2025LapSum -- One Method to Differentiate Them All: Ranking, Sorting and Top-k Selectioncited by 12
  6. 2026Learning to Evict from Key-Value Cachecited by 12
  7. 2025InfiGFusion: Graph-on-Logits Distillation via Efficient Gromov-Wasserstein for Model Fusioncited by 11
  8. 2025VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMscited by 11
  9. 2025ESPFormer: Doubly-Stochastic Attention with Expected Sliced Transport Planscited by 10
  10. 2023DAG Learning on the Permutahedroncited by 9
  11. 2022Learning to Hash Naturally Sortscited by 4
  12. 2024Permutation-based Inference for Variational Learning of Directed Acyclic Graphscited by 4
  13. 2024Generalizing Stochastic Smoothing for Differentiation and Gradient Estimationcited by 4
  14. 2023Differentiable Random Partition Modelscited by 3
  15. 2023Fast Slate Policy Optimization: Going Beyond Plackett-Lucecited by 3
  16. 2025Learning Classifiers That Induce Marketscited by 3
  17. 2024BEACON: A Bayesian Optimization Inspired Strategy for Efficient Novelty Searchcited by 2
  18. 2024Newton Losses: Using Curvature Information for Learning with Differentiable Algorithmscited by 2
  19. 2025Learning Cascade Ranking as One Networkcited by 2
  20. 2025Efficient Token Compression for Vision Transformer with Spatial Information Preservedcited by 2
  21. 2025Constrained Sliced Wasserstein Embeddingcited by 2
  22. 2025Differentiable Fast Top-K Selection for Large-Scale Recommendationcited by 2
  23. 2026SoftJAX & SoftTorch: Empowering Automatic Differentiation Libraries with Informative Gradientscited by 2
  24. 2021A Deep Generative Model for Reordering Adjacency Matricescited by 1
  25. 2024Scalable Variational Causal Discovery Unconstrained by Acyclicitycited by 1
  26. 2024Demystifying the Token Dynamics of Deep Selective State Space Modelscited by 1
  27. 2025Creating Sorted Grid Layouts with Gradient-based Optimizationcited by 1
  28. 2025Deep greedy unfolding: Sorting out argsorting in greedy sparse recovery algorithmscited by 1
  29. 2024Downstream Task Guided Masking Learning in Masked Autoencoders Using Multi-Level Optimizationcited by 0
  30. 2024Learning Fair and Preferable Allocations through Neural Networkcited by 0
  31. 2024Learning Neural Strategy-Proof Matching Mechanism from Examplescited by 0
  32. 2024The Differentiable Feasibility Pumpcited by 0
  33. 2025Permutation Learning with Only N Parameters: From SoftSort to Self-Organizing Gaussianscited by 0
  34. 2025XicorAttention: Time Series Transformer Using Attention with Nonlinear Correlationcited by 0
  35. 2025RewardRank: Optimizing True Learning-to-Rank Utilitycited by 0
  36. 2025Gradient-Based Join Orderingcited by 0
  37. 2025DM3D: Dynamic Mamba via Offset-Guided Feature Resampling for Point Cloud Understandingcited by 0
  38. 2025Towards Unsupervised Causal Representation Learning via Latent Additive Noise Model Causal Autoencoderscited by 0

50 most cited of 66 All 66 (JSON)