# LLM-Enhanced Linear Autoencoders for Recommendation

arXiv: https://arxiv.org/abs/2508.13500 · HTML: https://mlsanity.com/paper/2508.13500/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 36 references that are in the map.
- First posted on arXiv 19 Aug 2025.

## Builds on (most cited first, 22 of 22)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Models](https://mlsanity.com/paper/2506.05176/) (2025)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation](https://mlsanity.com/paper/2305.00447/) (2023)
- [Self-supervised Graph Learning for Recommendation](https://mlsanity.com/paper/2010.10783/) (2020)
- [Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders](https://mlsanity.com/paper/2403.03952/) (2024)
- [Are Graph Augmentations Necessary? Simple Graph Contrastive Learning for Recommendation](https://mlsanity.com/paper/2112.08679/) (2021)
- [Representation Learning with Large Language Models for Recommendation](https://mlsanity.com/paper/2310.15950/) (2023)
- [Embarrassingly Shallow Autoencoders for Sparse Data](https://mlsanity.com/paper/1905.03375/) (2019)
- [Knowledge Distillation from Internal Representations](https://mlsanity.com/paper/1910.03723/) (2019)
- [How Powerful is Graph Convolution for Recommendation?](https://mlsanity.com/paper/2108.07567/) (2021)
- [E4SRec: An Elegant Effective Efficient Extensible Solution of Large Language Models for Sequential Recommendation](https://mlsanity.com/paper/2312.02443/) (2023)
- [Blurring-Sharpening Process Models for Collaborative Filtering](https://mlsanity.com/paper/2211.09324/) (2022)
- [Language Representations Can be What Recommenders Need: Findings and Potentials](https://mlsanity.com/paper/2407.05441/) (2024)
- [Toward a Better Understanding of Loss Functions for Collaborative Filtering](https://mlsanity.com/paper/2308.06091/) (2023)
- [How Powerful is Graph Filtering for Recommendation](https://mlsanity.com/paper/2406.08827/) (2024)
- [It's Enough: Relaxing Diagonal Constraints in Linear Autoencoders for Recommendation](https://mlsanity.com/paper/2305.12922/) (2023)
- [LLMInit: A Free Lunch from Large Language Models for Selective Initialization of Recommendation](https://mlsanity.com/paper/2503.01814/) (2025)
- [Session-aware Linear Item-Item Models for Session-based Recommendation](https://mlsanity.com/paper/2103.16104/) (2021)
- [SVD-AE: Simple Autoencoders for Collaborative Filtering](https://mlsanity.com/paper/2405.04746/) (2024)
- [Temporal Linear Item-Item Model for Sequential Recommendation](https://mlsanity.com/paper/2412.07382/) (2024)
- [Why is Normalization Necessary for Linear Recommenders?](https://mlsanity.com/paper/2504.05805/) (2025)

## Built on by (most cited first, 1 of 1)

- [MergeRec: Model Merging for Data-Isolated Cross-Domain Sequential Recommendation](https://mlsanity.com/paper/2601.01753/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have been widely adopted to enrich the semantic representation of textual item information in recommender systems. However, existing linear autoencoders (LAEs) that incorporate textual information rely on sparse word co-occurrence patterns, limiting their ability to capture rich textual semantics. To address this, we propose L3AE, the first integration of LLMs into the LAE framework. L3AE effectively integrates the heterogeneous knowledge of textual semantics and user-item interactions through a two-phase optimization strategy. (i) L3AE first constructs a semantic item-to-item correlation matrix from LLM-derived item representations. (ii) It then learns an item-to-item weight matrix from collaborative signals while distilling semantic item correlations as regularization. Notably, each phase of L3AE is optimized through closed-form solutions, ensuring global optimality and computational efficiency. Extensive experiments demonstrate that L3AE consistently outperforms state-of-the-art LLM-enhanced models on three benchmark datasets, achieving gains of 27.6% in Recall@20 and 39.3% in NDCG@20. The source code is available at https://github.com/jaewan7599/L3AE_CIKM2025.