# Tensor Random Projection for Low Memory Dimension Reduction

arXiv: https://arxiv.org/abs/2105.00105 · HTML: https://mlsanity.com/paper/2105.00105/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 25 references that are in the map.
- First posted on arXiv 30 Apr 2021.

## Builds on (most cited first, 4 of 4)

- [Sketching as a Tool for Numerical Linear Algebra](https://mlsanity.com/paper/1411.4357/) (2014)
- [Fast and Guaranteed Tensor Decomposition via Sketching](https://mlsanity.com/paper/1506.04448/) (2015)
- [Toward a unified theory of sparse dimensionality reduction in Euclidean space](https://mlsanity.com/paper/1311.2542/) (2013)
- [Johnson-Lindenstrauss Compression with Neuroscience-Based Constraints](https://mlsanity.com/paper/1411.5383/) (2014)

## Built on by (most cited first, 7 of 7)

- [A survey of unsupervised learning methods for high-dimensional uncertainty quantification in black-box-type problems](https://mlsanity.com/paper/2202.04648/) (2022)
- [Random Fourier Signature Features](https://mlsanity.com/paper/2311.12214/) (2023)
- [More Efficient Sampling for Tensor Decomposition With Worst-Case Guarantees](https://mlsanity.com/paper/2110.07631/) (2021)
- [Tensorization of neural networks for improved privacy and interpretability](https://mlsanity.com/paper/2501.06300/) (2025)
- [Rademacher Random Projections with Tensor Networks](https://mlsanity.com/paper/2110.13970/) (2021)
- [Approximation and learning with compositional tensor trains](https://mlsanity.com/paper/2512.18059/) (2025)
- [Sample Efficient Learning of Factored Embeddings of Tensor Fields](https://mlsanity.com/paper/2209.00372/) (2022)

## Abstract (arXiv)

Random projections reduce the dimension of a set of vectors while preserving structural information, such as distances between vectors in the set. This paper proposes a novel use of row-product random matrices in random projection, where we call it Tensor Random Projection (TRP). It requires substantially less memory than existing dimension reduction maps. The TRP map is formed as the Khatri-Rao product of several smaller random projections, and is compatible with any base random projection including sparse maps, which enable dimension reduction with very low query cost and no floating point operations. We also develop a reduced variance extension. We provide a theoretical analysis of the bias and variance of the TRP, and a non-asymptotic error analysis for a TRP composed of two smaller maps. Experiments on both synthetic and MNIST data show that our method performs as well as conventional methods with substantially less storage.