# Peeling Back the Layers: An In-Depth Evaluation of Encoder Architectures in Neural News Recommenders

arXiv: https://arxiv.org/abs/2410.01470 · HTML: https://mlsanity.com/paper/2410.01470/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 65 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Similarity of Neural Network Representations Revisited](https://mlsanity.com/paper/1905.00414/) (2019)
- [Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1408.5882/) (2014)
- [DKN: Deep Knowledge-Aware Network for News Recommendation](https://mlsanity.com/paper/1801.08284/) (2018)
- [Similarity of Neural Network Models: A Survey of Functional and Representational Measures](https://mlsanity.com/paper/2305.06329/) (2023)
- [Empowering News Recommendation with Pre-trained Language Models](https://mlsanity.com/paper/2104.07413/) (2021)
- [Neural News Recommendation with Attentive Multi-View Learning](https://mlsanity.com/paper/1907.05576/) (2019)
- [FairRec: Fairness-aware News Recommendation with Decomposed Adversarial Learning](https://mlsanity.com/paper/2006.16742/) (2020)
- [Graph Enhanced Representation Learning for News Recommendation](https://mlsanity.com/paper/2003.14292/) (2020)
- [FeedRec: News Feed Recommendation with Various User Feedbacks](https://mlsanity.com/paper/2102.04903/) (2021)
- [PP-Rec: News Recommendation with Personalized User Interest and Time-aware News Popularity](https://mlsanity.com/paper/2106.01300/) (2021)
- [News Recommendation with Candidate-aware User Modeling](https://mlsanity.com/paper/2204.04726/) (2022)
- [Revisiting Word Embeddings in the LLM Era](https://mlsanity.com/paper/2402.11094/) (2024)
- [Understanding the Inner Workings of Language Models Through Representation Dissimilarity](https://mlsanity.com/paper/2310.14993/) (2023)
- [Beyond Benchmarks: Evaluating Embedding Model Similarity for Retrieval Augmented Generation Systems](https://mlsanity.com/paper/2407.08275/) (2024)
- [Why Do We Click: Visual Impression-aware News Recommendation](https://mlsanity.com/paper/2109.12651/) (2021)
- [Is News Recommendation a Sequential Recommendation Task?](https://mlsanity.com/paper/2108.08984/) (2021)
- [Train Once, Use Flexibly: A Modular Framework for Multi-Aspect Neural News Recommendation](https://mlsanity.com/paper/2307.16089/) (2023)
- [NewsRecLib: A PyTorch-Lightning Library for Neural News Recommendation](https://mlsanity.com/paper/2310.01146/) (2023)
- [Two Birds with One Stone: Unified Model Learning for Both Recall and Ranking in News Recommendation](https://mlsanity.com/paper/2104.07404/) (2021)
- [MM-Rec: Multimodal News Recommendation](https://mlsanity.com/paper/2104.07407/) (2021)
- [Neural News Recommendation with Negative Feedback](https://mlsanity.com/paper/2101.04328/) (2021)
- [News Without Borders: Domain Adaptation of Multilingual Sentence Embeddings for Cross-lingual News Recommendation](https://mlsanity.com/paper/2406.12634/) (2024)
- [Deep Dynamic Neural Network to trade-off between Accuracy and Diversity in a News Recommender System](https://mlsanity.com/paper/2103.08458/) (2021)
- [Simplifying Content-Based Neural News Recommendation: On User Modeling and Training Objectives](https://mlsanity.com/paper/2304.03112/) (2023)
- [Neural News Recommendation with Event Extraction](https://mlsanity.com/paper/2111.05068/) (2021)
- [Understanding the Relation of User and News Representations in Content-Based Neural News Recommendation](https://mlsanity.com/paper/2207.14704/) (2022)

## Built on by (most cited first, 1 of 1)

- [Optimizing Vision-Language Consistency via Cross-Layer Regional Attention Alignment](https://mlsanity.com/paper/2508.00945/) (2025)

## Abstract (arXiv)

Encoder architectures play a pivotal role in neural news recommenders by embedding the semantic and contextual information of news and users. Thus, research has heavily focused on enhancing the representational capabilities of news and user encoders to improve recommender performance. Despite the significant impact of encoder architectures on the quality of news and user representations, existing analyses of encoder designs focus only on the overall downstream recommendation performance. This offers a one-sided assessment of the encoders' similarity, ignoring more nuanced differences in their behavior, and potentially resulting in sub-optimal model selection. In this work, we perform a comprehensive analysis of encoder architectures in neural news recommender systems. We systematically evaluate the most prominent news and user encoder architectures, focusing on their (i) representational similarity, measured with the Central Kernel Alignment, (ii) overlap of generated recommendation lists, quantified with the Jaccard similarity, and (iii) the overall recommendation performance. Our analysis reveals that the complexity of certain encoding techniques is often empirically unjustified, highlighting the potential for simpler, more efficient architectures. By isolating the effects of individual components, we provide valuable insights for researchers and practitioners to make better informed decisions about encoder selection and avoid unnecessary complexity in the design of news recommenders.