# Tiny-NewsRec: Effective and Efficient PLM-based News Recommendation

arXiv: https://arxiv.org/abs/2112.00944 · HTML: https://mlsanity.com/paper/2112.00944/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Dec 2021.

## Built on by (most cited first, 16 of 16)

- [A Survey on Large Language Models for Recommendation](https://mlsanity.com/paper/2305.19860/) (2023)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities](https://mlsanity.com/paper/2307.16376/) (2023)
- [ReLLa: Retrieval-enhanced Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation](https://mlsanity.com/paper/2308.11131/) (2023)
- [RAH! RecSys-Assistant-Human: A Human-Centered Recommendation Framework with LLM Agents](https://mlsanity.com/paper/2308.09904/) (2023)
- [LLM-Enhanced User-Item Interactions: Leveraging Edge Information for Optimized Recommendations](https://mlsanity.com/paper/2402.09617/) (2024)
- [Lifelong Personalized Low-Rank Adaptation of Large Language Models for Recommendation](https://mlsanity.com/paper/2408.03533/) (2024)
- [LLMs for User Interest Exploration in Large-scale Recommendation Systems](https://mlsanity.com/paper/2405.16363/) (2024)
- [Large Language Models Make Sample-Efficient Recommender Systems](https://mlsanity.com/paper/2406.02368/) (2024)
- [AdaptSSR: Pre-training User Model with Augmentation-Adaptive Self-Supervised Ranking](https://mlsanity.com/paper/2310.09706/) (2023)
- [A Survey on LLM-based News Recommender Systems](https://mlsanity.com/paper/2502.09797/) (2025)
- [Efficient Pointwise-Pairwise Learning-to-Rank for News Recommendation](https://mlsanity.com/paper/2409.17711/) (2024)
- [Does Knowledge Distillation Matter for Large Language Model based Bundle Generation?](https://mlsanity.com/paper/2504.17220/) (2025)
- [Routing Distilled Knowledge via Mixture of LoRA Experts for Large Language Model based Bundle Generation](https://mlsanity.com/paper/2508.17250/) (2025)
- [Full-Stack Optimized Large Language Models for Lifelong Sequential Behavior Comprehension in Recommendation](https://mlsanity.com/paper/2501.13344/) (2025)
- [Is This News Still Interesting to You?: Lifetime-aware Interest Matching for News Recommendation](https://mlsanity.com/paper/2508.13064/) (2025)

## Abstract (arXiv)

News recommendation is a widely adopted technique to provide personalized news feeds for the user. Recently, pre-trained language models (PLMs) have demonstrated the great capability of natural language understanding and benefited news recommendation via improving news modeling. However, most existing works simply finetune the PLM with the news recommendation task, which may suffer from the known domain shift problem between the pre-training corpus and downstream news texts. Moreover, PLMs usually contain a large volume of parameters and have high computational overhead, which imposes a great burden on low-latency online services. In this paper, we propose Tiny-NewsRec, which can improve both the effectiveness and the efficiency of PLM-based news recommendation. We first design a self-supervised domain-specific post-training method to better adapt the general PLM to the news domain with a contrastive matching task between news titles and news bodies. We further propose a two-stage knowledge distillation method to improve the efficiency of the large PLM-based news recommendation model while maintaining its performance. Multiple teacher models originated from different time steps of our post-training procedure are used to transfer comprehensive knowledge to the student in both its post-training and finetuning stage. Extensive experiments on two real-world datasets validate the effectiveness and efficiency of our method.