# NeuralNDCG: Direct Optimisation of a Ranking Metric via Differentiable Relaxation of Sorting

arXiv: https://arxiv.org/abs/2102.07831 · HTML: https://mlsanity.com/paper/2102.07831/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2021.

## Built on by (most cited first, 30 of 41)

- [Efficient LLM Scheduling by Learning to Rank](https://mlsanity.com/paper/2408.15792/) (2024)
- [Ensuring User-side Fairness in Dynamic Recommender Systems](https://mlsanity.com/paper/2308.15651/) (2023)
- [RankCSE: Unsupervised Sentence Representations Learning via Learning to Rank](https://mlsanity.com/paper/2305.16726/) (2023)
- [Large-scale Stochastic Optimization of NDCG Surrogates for Deep Learning with Provable Convergence](https://mlsanity.com/paper/2202.12183/) (2022)
- [Metasql: A Generate-then-Rank Framework for Natural Language to SQL Translation](https://mlsanity.com/paper/2402.17144/) (2024)
- [LibAUC: A Deep Learning Library for X-Risk Optimization](https://mlsanity.com/paper/2306.03065/) (2023)
- [Towards Global Optimal Visual In-Context Learning Prompt Selection](https://mlsanity.com/paper/2405.15279/) (2024)
- [HLM-Cite: Hybrid Language Model Workflow for Text-based Scientific Citation Prediction](https://mlsanity.com/paper/2410.09112/) (2024)
- [Offline Model-Based Optimization by Learning to Rank](https://mlsanity.com/paper/2410.11502/) (2024)
- [Performative Recommendation: Diversifying Content via Strategic Incentives](https://mlsanity.com/paper/2302.04336/) (2023)
- [Risk Assessment Framework for Code LLMs via Leveraging Internal States](https://mlsanity.com/paper/2504.14640/) (2025)
- [Breaking the Top-K Barrier: Advancing Top-K Ranking Metrics Optimization in Recommender Systems](https://mlsanity.com/paper/2508.05673/) (2025)
- [Algorithmic Foundations of Empirical X-risk Minimization](https://mlsanity.com/paper/2206.00439/) (2022)
- [Rank-LIME: Local Model-Agnostic Feature Attribution for Learning to Rank](https://mlsanity.com/paper/2212.12722/) (2022)
- [Gradient-Boosted Decision Tree for Listwise Context Model in Multimodal Review Helpfulness Prediction](https://mlsanity.com/paper/2305.12678/) (2023)
- [Permutative Preference Alignment from Listwise Ranking of Human Judgments](https://mlsanity.com/paper/2410.04346/) (2024)
- [Unified Multimodal Model with Unlikelihood Training for Visual Dialog](https://mlsanity.com/paper/2211.13235/) (2022)
- [RankSHAP: Shapley Value Based Feature Attributions for Learning to Rank](https://mlsanity.com/paper/2405.01848/) (2024)
- [Enhancing Mobile "How-to" Queries with Automated Search Results Verification and Reranking](https://mlsanity.com/paper/2404.08860/) (2024)
- [Performative Debias with Fair-exposure Optimization Driven by Strategic Agents in Recommender Systems](https://mlsanity.com/paper/2406.17475/) (2024)
- [LLM-Guided Semantic Relational Reasoning for Multimodal Intent Recognition](https://mlsanity.com/paper/2509.01337/) (2025)
- [Holistic Utility Preference Learning for Listwise Alignment](https://mlsanity.com/paper/2410.18127/) (2024)
- [Learning Cascade Ranking as One Network](https://mlsanity.com/paper/2503.09492/) (2025)
- [LLMDistill4Ads: Using Cross-Encoders to Distill from LLM Signals for Advertiser Keyphrase Recommendations at eBay](https://mlsanity.com/paper/2508.03628/) (2025)
- [Ranking Before Serving: Low-Latency LLM Serving via Pairwise Learning-to-Rank](https://mlsanity.com/paper/2510.03243/) (2025)
- [Beyond Prediction: On-street Parking Recommendation using Heterogeneous Graph-based List-wise Ranking](https://mlsanity.com/paper/2305.00162/) (2023)
- [Post-hoc Selection of Pareto-Optimal Solutions in Search and Recommendation](https://mlsanity.com/paper/2306.12165/) (2023)
- [UBiSS: A Unified Framework for Bimodal Semantic Summarization of Videos](https://mlsanity.com/paper/2406.16301/) (2024)
- [ITEM: Improving Training and Evaluation of Message-Passing based GNNs for top-k recommendation](https://mlsanity.com/paper/2407.07912/) (2024)
- [Dialogue-Contextualized Re-ranking for Medical History-Taking](https://mlsanity.com/paper/2304.01974/) (2023)

## Abstract (arXiv)

Learning to Rank (LTR) algorithms are usually evaluated using Information Retrieval metrics like Normalised Discounted Cumulative Gain (NDCG) or Mean Average Precision. As these metrics rely on sorting predicted items' scores (and thus, on items' ranks), their derivatives are either undefined or zero everywhere. This makes them unsuitable for gradient-based optimisation, which is the usual method of learning appropriate scoring functions. Commonly used LTR loss functions are only loosely related to the evaluation metrics, causing a mismatch between the optimisation objective and the evaluation criterion. In this paper, we address this mismatch by proposing NeuralNDCG, a novel differentiable approximation to NDCG. Since NDCG relies on the non-differentiable sorting operator, we obtain NeuralNDCG by relaxing that operator using NeuralSort, a differentiable approximation of sorting. As a result, we obtain a new ranking loss function which is an arbitrarily accurate approximation to the evaluation metric, thus closing the gap between the training and the evaluation of LTR models. We introduce two variants of the proposed loss function. Finally, the empirical evaluation shows that our proposed method outperforms previous work aimed at direct optimisation of NDCG and is competitive with the state-of-the-art methods.