# It's Enough: Relaxing Diagonal Constraints in Linear Autoencoders for Recommendation

arXiv: https://arxiv.org/abs/2305.12922 · HTML: https://mlsanity.com/paper/2305.12922/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 46 references that are in the map.
- First posted on arXiv 22 May 2023.

## Builds on (most cited first, 19 of 19)

- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [Variational Autoencoders for Collaborative Filtering](https://mlsanity.com/paper/1802.05814/) (2018)
- [Neural Attentive Session-based Recommendation](https://mlsanity.com/paper/1711.04725/) (2017)
- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [Revisiting Graph based Collaborative Filtering: A Linear Residual Graph Convolutional Network Approach](https://mlsanity.com/paper/2001.10167/) (2020)
- [Embarrassingly Shallow Autoencoders for Sparse Data](https://mlsanity.com/paper/1905.03375/) (2019)
- [A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research](https://mlsanity.com/paper/1911.07698/) (2019)
- [RecVAE: a New Variational Autoencoder for Top-N Recommendations with Implicit Feedback](https://mlsanity.com/paper/1912.11160/) (2019)
- [How Powerful is Graph Convolution for Recommendation?](https://mlsanity.com/paper/2108.07567/) (2021)
- [On the Difficulty of Evaluating Baselines: A Study on Recommender Systems](https://mlsanity.com/paper/1905.01395/) (2019)
- [LT-OCF: Learnable-Time ODE-based Collaborative Filtering](https://mlsanity.com/paper/2108.06208/) (2021)
- [Linear, or Non-Linear, That is the Question!](https://mlsanity.com/paper/2111.07265/) (2021)
- [Towards a Better Understanding of Linear Models for Recommendation](https://mlsanity.com/paper/2105.12937/) (2021)
- [Theoretical Understandings of Product Embedding for E-commerce Machine Learning](https://mlsanity.com/paper/2102.12029/) (2021)
- [Bilateral Self-unbiased Learning from Biased Implicit Feedback](https://mlsanity.com/paper/2207.12660/) (2022)

## Built on by (most cited first, 7 of 7)

- [SVD-AE: Simple Autoencoders for Collaborative Filtering](https://mlsanity.com/paper/2405.04746/) (2024)
- [Temporal Linear Item-Item Model for Sequential Recommendation](https://mlsanity.com/paper/2412.07382/) (2024)
- [PAC-Bayes Bounds for Multivariate Linear Regression and Linear Autoencoders](https://mlsanity.com/paper/2512.12905/) (2025)
- [LLM-Enhanced Linear Autoencoders for Recommendation](https://mlsanity.com/paper/2508.13500/) (2025)
- [Implicit ZCA Whitening Effects of Linear Autoencoders for Recommendation](https://mlsanity.com/paper/2308.13536/) (2023)
- [MergeRec: Model Merging for Data-Isolated Cross-Domain Sequential Recommendation](https://mlsanity.com/paper/2601.01753/) (2026)
- [Generalizing Linear Autoencoder Recommenders with Decoupled Expected Quadratic Loss](https://mlsanity.com/paper/2603.07402/) (2026)

## Abstract (arXiv)

Linear autoencoder models learn an item-to-item weight matrix via convex optimization with L2 regularization and zero-diagonal constraints. Despite their simplicity, they have shown remarkable performance compared to sophisticated non-linear models. This paper aims to theoretically understand the properties of two terms in linear autoencoders. Through the lens of singular value decomposition (SVD) and principal component analysis (PCA), it is revealed that L2 regularization enhances the impact of high-ranked PCs. Meanwhile, zero-diagonal constraints reduce the impact of low-ranked PCs, leading to performance degradation for unpopular items. Inspired by this analysis, we propose simple-yet-effective linear autoencoder models using diagonal inequality constraints, called Relaxed Linear AutoEncoder (RLAE) and Relaxed Denoising Linear AutoEncoder (RDLAE). We prove that they generalize linear autoencoders by adjusting the degree of diagonal constraints. Experimental results demonstrate that our models are comparable or superior to state-of-the-art linear and non-linear models on six benchmark datasets; they significantly improve the accuracy of long-tail items. These results also support our theoretical insights on regularization and diagonal constraints in linear autoencoders.