mlsanityData to 7 Oct 2026

LapSum -- One Method to Differentiate Them All: Ranking, Sorting and Top-k Selection

arXiv 2503.06242 ↗First posted 8 Mar 2025cs.AIcited by 12

Not on the map. Why?

Abstract (arXiv)

We present a novel technique for constructing differentiable order-type operations, including soft ranking, soft top-k selection, and soft permutations. Our approach leverages an efficient closed-form formula for the inverse of the function LapSum, defined as the sum of Laplace distributions. This formulation ensures low computational and memory complexity in selecting the highest activations, enabling losses and gradients to be computed in O(nlogn) time. Through extensive experiments, we demonstrate that our method outperforms state-of-the-art techniques for high-dimensional vectors and large k values. Furthermore, we provide efficient implementations for both CPU and CUDA environments, underscoring the practicality and scalability of our method for large-scale ranking and differentiable ordering problems.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 21 of its 26 references that are in the map. First posted on arXiv 8 Mar 2025.

Builds on

21 of 26 references.

References · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  3. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 521
  4. 2023Learning A Sparse Transformer Network for Effective Image Derainingcited by 204
  5. 2020Fast Differentiable Sorting and Rankingcited by 196
  6. 2018Top-K Off-Policy Correction for a REINFORCE Recommender Systemcited by 181
  7. 2019Differentiable Ranks and Sorting using Optimal Transportcited by 113
  8. 2019Stochastic Optimization of Sorting Networks via Continuous Relaxationscited by 111
  9. 2020SoftSort: A Continuous Relaxation for the argsort Operatorcited by 66
  10. 2019Explicit Sparse Transformer: Concentrated Attention Through Explicit Selectioncited by 61
  11. 2018Smooth Loss Functions for Deep Top-k Classificationcited by 46
  12. 2015Loss Functions for Top-k Error: Analysis and Insightscited by 38
9 more
  1. 2021Learning with Noisy Labels via Sparse Regularizationcited by 38
  2. 2023Fast, Differentiable and Sparse Top-k: a Convex Analysis Perspectivecited by 26
  3. 2022Monotonic Differentiable Sorting Networkscited by 25
  4. 2024Cardinality-Aware Set Prediction and Top-k Classificationcited by 23
  5. 2022Differentiable Top-k Classification Learningcited by 21
  6. 2020Differentiable Top-k Operator with Optimal Transportcited by 17
  7. 2023Learning by Sorting: Self-supervised Learning with Group Ordering Constraintscited by 13
  8. 2022Stochastic smoothing of the top-K calibrated hinge loss for deep imbalanced classificationcited by 10
  9. 2024Top-k Classification and Cardinality-Aware Predictioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 8.

12 papers.

  1. 2025VisionSelector: End-to-End Learnable Visual Token Compression for Efficient Multimodal LLMscited by 11
  2. 2025Efficient Transferable Optimal Transport via Min-Sliced Transport Planscited by 4
  3. 2026SoftMoE: Soft Differentiable Routing for Mixture-of-Experts in LLMscited by 3
  4. 2025Differentiable Fast Top-K Selection for Large-Scale Recommendationcited by 2
  5. 2026SoftSAE: Dynamic Top-K Selection for Adaptive Sparse Autoencoderscited by 1
  6. 2026DiffPrune: differentiable information throttling for token pruning in vision-language modelscited by 1
  7. 2025Structural Incompatibility of Differentiable Sorting and Within-Vector Rank Normalizationcited by 0
  8. 2026Min Generalized Sliced Gromov Wasserstein: A Scalable Path to Gromov Wassersteincited by 0
  9. 2026Interpretability-Guided Soft Pruning of Attention Heads in Vision Transformerscited by 0
  10. 2026LaPrune: Controllable Differentiable Sparsity at Million Scalecited by 0
  11. 2026Fast LapSum: Exact Differentiable Top-k at Million Scalecited by 0
  12. 2026VLA-ACL: Action-Consistent Visual Token Pruning for Efficient Vision-Language-Action Modelscited by 0