# Empowering News Recommendation with Pre-trained Language Models

arXiv: https://arxiv.org/abs/2104.07413 · HTML: https://mlsanity.com/paper/2104.07413/

## Key facts

- Cited by 63 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2021.

## Built on by (most cited first, 30 of 63)

- [A Survey on Large Language Models for Recommendation](https://mlsanity.com/paper/2305.19860/) (2023)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [When Large Language Models Meet Personalization: Perspectives of Challenges and Opportunities](https://mlsanity.com/paper/2307.16376/) (2023)
- [Fastformer: Additive Attention Can Be All You Need](https://mlsanity.com/paper/2108.09084/) (2021)
- [A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)](https://mlsanity.com/paper/2404.00579/) (2024)
- [ONCE: Boosting Content-based Recommendation with Both Open- and Closed-source Large Language Models](https://mlsanity.com/paper/2305.06566/) (2023)
- [IDGenRec: LLM-RecSys Alignment with Textual ID Learning](https://mlsanity.com/paper/2403.19021/) (2024)
- [NoisyTune: A Little Noise Can Help You Finetune Pretrained Language Models Better](https://mlsanity.com/paper/2202.12024/) (2022)
- [ChatGPT Perpetuates Gender Bias in Machine Translation and Ignores Non-Gendered Pronouns: Findings across Bengali and Five other Low-Resource Languages](https://mlsanity.com/paper/2305.10510/) (2023)
- [RAH! RecSys-Assistant-Human: A Human-Centered Recommendation Framework with LLM Agents](https://mlsanity.com/paper/2308.09904/) (2023)
- [IISAN: Efficiently Adapting Multimodal Representation for Sequential Recommendation with Decoupled PEFT](https://mlsanity.com/paper/2404.02059/) (2024)
- [Prompt Learning for News Recommendation](https://mlsanity.com/paper/2304.05263/) (2023)
- [LLM-Enhanced User-Item Interactions: Leveraging Edge Information for Optimized Recommendations](https://mlsanity.com/paper/2402.09617/) (2024)
- [LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Application](https://mlsanity.com/paper/2405.03988/) (2024)
- [UniTRec: A Unified Text-to-Text Transformer and Joint Contrastive Learning Framework for Text-based Recommendation](https://mlsanity.com/paper/2305.15756/) (2023)
- [LLM-Generated Fake News Induces Truth Decay in News Ecosystem: A Case Study on Neural News Recommendation](https://mlsanity.com/paper/2504.20013/) (2025)
- [Knowledge Plugins: Enhancing Large Language Models for Domain-Specific Recommendations](https://mlsanity.com/paper/2311.10779/) (2023)
- [Knowledge Prompt-tuning for Sequential Recommendation](https://mlsanity.com/paper/2308.08459/) (2023)
- [Two Heads Are Better Than One: Integrating Knowledge from Knowledge Graphs and Large Language Models for Entity Alignment](https://mlsanity.com/paper/2401.16960/) (2024)
- [RA-Rec: An Efficient ID Representation Alignment Framework for LLM-based Recommendation](https://mlsanity.com/paper/2402.04527/) (2024)
- [AMEM4Rec: Leveraging Cross-User Similarity for Memory Evolution in Agentic LLM Recommenders](https://mlsanity.com/paper/2602.08837/) (2026)
- [SPAR: Personalized Content-Based Recommendation via Long Engagement Attention](https://mlsanity.com/paper/2402.10555/) (2024)
- [Rethinking the Privacy of Text Embeddings: A Reproducibility Study of "Text Embeddings Reveal (Almost) As Much As Text"](https://mlsanity.com/paper/2507.07700/) (2025)
- [Modeling User Viewing Flow Using Large Language Models for Article Recommendation](https://mlsanity.com/paper/2311.07619/) (2023)
- [EmbSum: Leveraging the Summarization Capabilities of Large Language Models for Content-Based Recommendations](https://mlsanity.com/paper/2405.11441/) (2024)
- [News Recommendation with Category Description by a Large Language Model](https://mlsanity.com/paper/2405.13007/) (2024)
- [News Without Borders: Domain Adaptation of Multilingual Sentence Embeddings for Cross-lingual News Recommendation](https://mlsanity.com/paper/2406.12634/) (2024)
- [GRAM: Fast Fine-tuning of Pre-trained Language Models for Content-based Collaborative Filtering](https://mlsanity.com/paper/2204.04179/) (2022)
- [Few-shot News Recommendation via Cross-lingual Transfer](https://mlsanity.com/paper/2207.14370/) (2022)
- [MIM: Multi-modal Content Interest Modeling Paradigm for User Behavior Modeling](https://mlsanity.com/paper/2502.00321/) (2025)

## Abstract (arXiv)

Personalized news recommendation is an essential technique for online news services. News articles usually contain rich textual content, and accurate news modeling is important for personalized news recommendation. Existing news recommendation methods mainly model news texts based on traditional text modeling methods, which is not optimal for mining the deep semantic information in news texts. Pre-trained language models (PLMs) are powerful for natural language understanding, which has the potential for better news modeling. However, there is no public report that show PLMs have been applied to news recommendation. In this paper, we report our work on exploiting pre-trained language models to empower news recommendation. Offline experimental results on both monolingual and multilingual news recommendation datasets show that leveraging PLMs for news modeling can effectively improve the performance of news recommendation. Our PLM-empowered news recommendation models have been deployed to the Microsoft News platform, and achieved significant gains in terms of both click and pageview in both English-speaking and global markets.