# Towards a Better Understanding of Linear Models for Recommendation

arXiv: https://arxiv.org/abs/2105.12937 · HTML: https://mlsanity.com/paper/2105.12937/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 36 references that are in the map.
- First posted on arXiv 27 May 2021.

## Builds on (most cited first, 10 of 10)

- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [Variational Autoencoders for Collaborative Filtering](https://mlsanity.com/paper/1802.05814/) (2018)
- [Are We Really Making Much Progress? A Worrying Analysis of Recent Neural Recommendation Approaches](https://mlsanity.com/paper/1907.06902/) (2019)
- [Deep Learning based Recommender System: A Survey and New Perspectives](https://mlsanity.com/paper/1707.07435/) (2017)
- [Embarrassingly Shallow Autoencoders for Sparse Data](https://mlsanity.com/paper/1905.03375/) (2019)
- [A Troubling Analysis of Reproducibility and Progress in Recommender Systems Research](https://mlsanity.com/paper/1911.07698/) (2019)
- [On the Difficulty of Evaluating Baselines: A Study on Recommender Systems](https://mlsanity.com/paper/1905.01395/) (2019)
- [Lecture notes on ridge regression](https://mlsanity.com/paper/1509.09169/) (2015)
- [Regularized Singular Value Decomposition and Application to Recommender System](https://mlsanity.com/paper/1804.05090/) (2018)
- [Post model-fitting exploration via a "Next-Door" analysis](https://mlsanity.com/paper/1806.01326/) (2018)

## Built on by (most cited first, 18 of 18)

- [Is Cosine-Similarity of Embeddings Really About Similarity?](https://mlsanity.com/paper/2403.05440/) (2024)
- [Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization](https://mlsanity.com/paper/2310.15080/) (2023)
- [Revisiting the Performance of iALS on Item Recommendation Benchmarks](https://mlsanity.com/paper/2110.14037/) (2021)
- [Bridging Symmetry and Robustness: On the Role of Equivariance in Enhancing Adversarial Robustness](https://mlsanity.com/paper/2510.16171/) (2025)
- [It's Enough: Relaxing Diagonal Constraints in Linear Autoencoders for Recommendation](https://mlsanity.com/paper/2305.12922/) (2023)
- [When Newer is Not Better: Does Deep Learning Really Benefit Recommendation From Implicit Feedback?](https://mlsanity.com/paper/2305.01801/) (2023)
- [Scalable Higher-Order Tensor Product Spline Models](https://mlsanity.com/paper/2402.01090/) (2024)
- [Factorized Structured Regression for Large-Scale Varying Coefficient Models](https://mlsanity.com/paper/2205.13080/) (2022)
- [PAC-Bayes Bounds for Multivariate Linear Regression and Linear Autoencoders](https://mlsanity.com/paper/2512.12905/) (2025)
- [Impression-Informed Multi-Behavior Recommender System: A Hierarchical Graph Attention Approach](https://mlsanity.com/paper/2309.03169/) (2023)
- [On the Regularization of Autoencoders](https://mlsanity.com/paper/2110.11402/) (2021)
- [Additive Higher-Order Factorization Machines](https://mlsanity.com/paper/2205.14515/) (2022)
- [Where Do We Go From Here? Guidelines For Offline Recommender Evaluation](https://mlsanity.com/paper/2211.01261/) (2022)
- [Of Spiky SVDs and Music Recommendation](https://mlsanity.com/paper/2307.01212/) (2023)
- [Implicit ZCA Whitening Effects of Linear Autoencoders for Recommendation](https://mlsanity.com/paper/2308.13536/) (2023)
- [(Debiased) Contrastive Learning Loss for Recommendation (Technical Report)](https://mlsanity.com/paper/2312.08517/) (2023)
- [ImplicitSLIM and How it Improves Embedding-based Collaborative Filtering](https://mlsanity.com/paper/2406.00198/) (2024)
- [Does Weighting Improve Matrix Factorization for Recommender Systems?](https://mlsanity.com/paper/2510.10440/) (2025)

## Abstract (arXiv)

Recently, linear regression models, such as EASE and SLIM, have shown to often produce rather competitive results against more sophisticated deep learning models. On the other side, the (weighted) matrix factorization approaches have been popular choices for recommendation in the past and widely adopted in the industry. In this work, we aim to theoretically understand the relationship between these two approaches, which are the cornerstones of model-based recommendations. Through the derivation and analysis of the closed-form solutions for two basic regression and matrix factorization approaches, we found these two approaches are indeed inherently related but also diverge in how they "scale-down" the singular values of the original user-item interaction matrix. This analysis also helps resolve the questions related to the regularization parameter range and model complexities. We further introduce a new learning algorithm in searching (hyper)parameters for the closed-form solution and utilize it to discover the nearby models of the existing solutions. The experimental results demonstrate that the basic models and their closed-form solutions are indeed quite competitive against the state-of-the-art models, thus, confirming the validity of studying the basic models. The effectiveness of exploring the nearby models are also experimentally validated.