# Efficient Mixed Dimension Embeddings for Matrix Factorization

arXiv: https://arxiv.org/abs/2205.11248 · HTML: https://mlsanity.com/paper/2205.11248/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 28 references that are in the map.
- First posted on arXiv 18 May 2022.

## Builds on (most cited first, 18 of 18)

- [Adaptive Input Representations for Neural Language Modeling](https://mlsanity.com/paper/1809.10853/) (2018)
- [Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems](https://mlsanity.com/paper/1909.02107/) (2019)
- [AutoEmb: Automated Embedding Dimensionality Search in Streaming Recommendations](https://mlsanity.com/paper/2002.11252/) (2020)
- [On the Difficulty of Evaluating Baselines: A Study on Recommender Systems](https://mlsanity.com/paper/1905.01395/) (2019)
- [Learnable Embedding Sizes for Recommender Systems](https://mlsanity.com/paper/2101.07577/) (2021)
- [Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems](https://mlsanity.com/paper/1909.11810/) (2019)
- [Neural Input Search for Large Scale Recommendation Models](https://mlsanity.com/paper/1907.04471/) (2019)
- [Model Size Reduction Using Frequency Based Double Hashing for Recommender Systems](https://mlsanity.com/paper/2007.14523/) (2020)
- [GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking](https://mlsanity.com/paper/1806.06950/) (2018)
- [Learning Multi-granular Quantized Embeddings for Large-Vocab Categorical Features in Recommender Systems](https://mlsanity.com/paper/2002.08530/) (2020)
- [Revisiting the Performance of iALS on Item Recommendation Benchmarks](https://mlsanity.com/paper/2110.14037/) (2021)
- [Differentiable Neural Input Search for Recommender Systems](https://mlsanity.com/paper/2006.04466/) (2020)
- [Memory-efficient Embedding for Recommendations](https://mlsanity.com/paper/2006.14827/) (2020)
- [Binary Code based Hash Embedding for Web-scale Applications](https://mlsanity.com/paper/2109.02471/) (2021)
- [Faster and Cheaper: Parallelizing Large-Scale Matrix Factorization on GPUs](https://mlsanity.com/paper/1603.03820/) (2016)
- [Random Offset Block Embedding Array (ROBE) for CriteoTB Benchmark MLPerf DLRM Model : 1000× Compression and 3.1× Faster Inference](https://mlsanity.com/paper/2108.02191/) (2021)
- [Matrix Factorization on GPUs with Memory Optimization and Approximate Computing](https://mlsanity.com/paper/1808.03843/) (2018)
- [ALX: Large Scale Matrix Factorization on TPUs](https://mlsanity.com/paper/2112.02194/) (2021)

## Built on by (most cited first, 1 of 1)

- [Mem-Rec: Memory Efficient Recommendation System using Alternative Representation](https://mlsanity.com/paper/2305.07205/) (2023)

## Abstract (arXiv)

Despite the prominence of neural network approaches in the field of recommender systems, simple methods such as matrix factorization with quadratic loss are still used in industry for several reasons. These models can be trained with alternating least squares, which makes them easy to implement in a massively parallel manner, thus making it possible to utilize billions of events from real-world datasets. Large-scale recommender systems need to account for severe popularity skew in the distributions of users and items, so a lot of research is focused on implementing sparse, mixed dimension or shared embeddings to reduce both the number of parameters and overfitting on rare users and items. In this paper we propose two matrix factorization models with mixed dimension embeddings, which can be optimized in a massively parallel fashion using the alternating least squares approach.