# On-Device Next-Item Recommendation with Self-Supervised Knowledge Distillation

arXiv: https://arxiv.org/abs/2204.11091 · HTML: https://mlsanity.com/paper/2204.11091/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Apr 2022.

## Built on by (most cited first, 21 of 21)

- [Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models](https://mlsanity.com/paper/2503.06027/) (2025)
- [A Comprehensive Survey on Trustworthy Recommender Systems](https://mlsanity.com/paper/2209.10117/) (2022)
- [Graph Condensation for Inductive Node Representation Learning](https://mlsanity.com/paper/2307.15967/) (2023)
- [Distillation from Heterogeneous Models for Top-K Recommendation](https://mlsanity.com/paper/2303.01130/) (2023)
- [Unbiased Knowledge Distillation for Recommendation](https://mlsanity.com/paper/2211.14729/) (2022)
- [Adaptive Low-Precision Training for Embeddings in Click-Through Rate Prediction](https://mlsanity.com/paper/2212.05735/) (2022)
- [Training Machine Learning models at the Edge: A Survey](https://mlsanity.com/paper/2403.02619/) (2024)
- [Active Large Language Model-based Knowledge Distillation for Session-based Recommendation](https://mlsanity.com/paper/2502.15685/) (2024)
- [GPU-based Private Information Retrieval for On-Device Machine Learning Inference](https://mlsanity.com/paper/2301.10904/) (2023)
- [A Thorough Performance Benchmarking on Lightweight Embedding-based Recommender Systems](https://mlsanity.com/paper/2406.17335/) (2024)
- [LightGNN: Simple Graph Neural Network for Recommendation](https://mlsanity.com/paper/2501.03228/) (2025)
- [DIET: Customized Slimming for Incompatible Networks in Sequential Recommendation](https://mlsanity.com/paper/2406.08804/) (2024)
- [Forward Once for All: Structural Parameterized Adaptation for Efficient Cloud-coordinated On-device Recommendation](https://mlsanity.com/paper/2501.02837/) (2025)
- [On-device Content-based Recommendation with Single-shot Embedding Pruning: A Cooperative Game Perspective](https://mlsanity.com/paper/2411.13052/) (2024)
- [Spatial-Temporal Knowledge Distillation for Takeaway Recommendation](https://mlsanity.com/paper/2412.16502/) (2024)
- [Cloud-Device Collaborative Agents for Sequential Recommendation](https://mlsanity.com/paper/2509.01551/) (2025)
- [Unified Low-rank Compression Framework for Click-through Rate Prediction](https://mlsanity.com/paper/2405.18146/) (2024)
- [Coarse-to-Fine Lightweight Meta-Embedding for ID-Based Recommendation](https://mlsanity.com/paper/2501.11870/) (2025)
- [On-Device Large Language Models for Sequential Recommendation](https://mlsanity.com/paper/2601.09306/) (2026)
- [Efficient Content-based Recommendation Model Training via Noise-aware Coreset Selection](https://mlsanity.com/paper/2601.10067/) (2026)
- [Balanced Co-Clustering of Users and Items for Embedding Table Compression in Recommender Systems](https://mlsanity.com/paper/2604.18351/) (2026)

## Abstract (arXiv)

Modern recommender systems operate in a fully server-based fashion. To cater to millions of users, the frequent model maintaining and the high-speed processing for concurrent user requests are required, which comes at the cost of a huge carbon footprint. Meanwhile, users need to upload their behavior data even including the immediate environmental context to the server, raising the public concern about privacy. On-device recommender systems circumvent these two issues with cost-conscious settings and local inference. However, due to the limited memory and computing resources, on-device recommender systems are confronted with two fundamental challenges: (1) how to reduce the size of regular models to fit edge devices? (2) how to retain the original capacity? Previous research mostly adopts tensor decomposition techniques to compress the regular recommendation model with limited compression ratio so as to avoid drastic performance degradation. In this paper, we explore ultra-compact models for next-item recommendation, by loosing the constraint of dimensionality consistency in tensor decomposition. Meanwhile, to compensate for the capacity loss caused by compression, we develop a self-supervised knowledge distillation framework which enables the compressed model (student) to distill the essential information lying in the raw data, and improves the long-tail item recommendation through an embedding-recombination strategy with the original model (teacher). The extensive experiments on two benchmarks demonstrate that, with 30x model size reduction, the compressed model almost comes with no accuracy loss, and even outperforms its uncompressed counterpart in most cases.