# Dual Correction Strategy for Ranking Distillation in Top-N Recommender System

arXiv: https://arxiv.org/abs/2109.03459 · HTML: https://mlsanity.com/paper/2109.03459/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for recommender systems, one of 12 directions in Knowledge distillation.
- Builds on 16 of its 30 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Placed in

- Knowledge distillation / [Distillation for recommender systems](https://mlsanity.com/distill/distillation-for-recommender-systems/)

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [Born Again Neural Networks](https://mlsanity.com/paper/1805.04770/) (2018)
- [Ranking Distillation: Learning Compact Ranking Models With High Performance for Recommender System](https://mlsanity.com/paper/1809.07428/) (2018)
- [DE-RRD: A Knowledge Distillation Framework for Recommender System](https://mlsanity.com/paper/2012.04357/) (2020)
- [Topology Distillation for Recommender System](https://mlsanity.com/paper/2106.08700/) (2021)
- [Bootstrapping User and Item Representations for One-Class Collaborative Filtering](https://mlsanity.com/paper/2105.06323/) (2021)
- [Bidirectional Distillation for Top-K Recommender System](https://mlsanity.com/paper/2106.02870/) (2021)
- [Candidate Generation with Binary Codes for Large-Scale Top-N Recommendation](https://mlsanity.com/paper/1909.05475/) (2019)
- [Collaborative Distillation for Top-N Recommendation](https://mlsanity.com/paper/1911.05276/) (2019)
- [Discrete Factorization Machines for Fast Feature-based Recommendation](https://mlsanity.com/paper/1805.02232/) (2018)
- [Binarized Collaborative Filtering with Distilling Graph Convolutional Networks](https://mlsanity.com/paper/1906.01829/) (2019)
- [Unsupervised Proxy Selection for Session-based Recommender Systems](https://mlsanity.com/paper/2107.03564/) (2021)

## Built on by (most cited first, 10 of 10)

- [Distillation from Heterogeneous Models for Top-K Recommendation](https://mlsanity.com/paper/2303.01130/) (2023)
- [Dynamic Sparse Learning: A Novel Paradigm for Efficient Recommendation](https://mlsanity.com/paper/2402.02855/) (2024)
- [RRADistill: Distilling LLMs' Passage Ranking Ability for Long-Tail Queries Document Re-Ranking on a Search Engine](https://mlsanity.com/paper/2410.18097/) (2024)
- [Active Large Language Model-based Knowledge Distillation for Session-based Recommendation](https://mlsanity.com/paper/2502.15685/) (2024)
- [Consensus Learning from Heterogeneous Objectives for One-Class Collaborative Filtering](https://mlsanity.com/paper/2202.13140/) (2022)
- [Exploring Feature-based Knowledge Distillation for Recommender System: A Frequency Perspective](https://mlsanity.com/paper/2411.10676/) (2024)
- [Debias the Black-box: A Fair Ranking Framework via Knowledge Distillation](https://mlsanity.com/paper/2208.11628/) (2022)
- [Rejuvenating Cross-Entropy Loss in Knowledge Distillation for Recommender Systems](https://mlsanity.com/paper/2509.20989/) (2025)
- [IRA: Adaptive Interest-aware Representation and Alignment for Personalized Multi-interest Retrieval](https://mlsanity.com/paper/2504.17529/) (2025)
- [GOD: Enhancing Generalization via Deep Grafting for Sequential Recommendation](https://mlsanity.com/paper/2608.16073/) (2026)

## Abstract (arXiv)

Knowledge Distillation (KD), which transfers the knowledge of a well-trained large model (teacher) to a small model (student), has become an important area of research for practical deployment of recommender systems. Recently, Relaxed Ranking Distillation (RRD) has shown that distilling the ranking information in the recommendation list significantly improves the performance. However, the method still has limitations in that 1) it does not fully utilize the prediction errors of the student model, which makes the training not fully efficient, and 2) it only distills the user-side ranking information, which provides an insufficient view under the sparse implicit feedback. This paper presents Dual Correction strategy for Distillation (DCD), which transfers the ranking information from the teacher model to the student model in a more efficient manner. Most importantly, DCD uses the discrepancy between the teacher model and the student model predictions to decide which knowledge to be distilled. By doing so, DCD essentially provides the learning guidance tailored to "correcting" what the student model has failed to accurately predict. This process is applied for transferring the ranking information from the user-side as well as the item-side to address sparse implicit user feedback. Our experiments show that the proposed method outperforms the state-of-the-art baselines, and ablation studies validate the effectiveness of each component.