# Learning with tree-based tensor formats

arXiv: https://arxiv.org/abs/1811.04455 · HTML: https://mlsanity.com/paper/1811.04455/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2018.

## Built on by (most cited first, 4 of 4)

- [Compositionality Unlocks Deep Interpretable Models](https://mlsanity.com/paper/2504.02667/) (2025)
- [Riemannian Optimization on Tree Tensor Networks with Application in Machine Learning](https://mlsanity.com/paper/2507.21726/) (2025)
- [TTML: tensor trains for general supervised machine learning](https://mlsanity.com/paper/2203.04352/) (2022)
- [Approximation and learning with compositional tensor trains](https://mlsanity.com/paper/2512.18059/) (2025)

## Abstract (arXiv)

This paper is concerned with the approximation of high-dimensional functions in a statistical learning setting, by empirical risk minimization over model classes of functions in tree-based tensor format. These are particular classes of rank-structured functions that can be seen as deep neural networks with a sparse architecture related to the tree and multilinear activation functions. For learning in a given model class, we exploit the fact that tree-based tensor formats are multilinear models and recast the problem of risk minimization over a nonlinear set into a succession of learning problems with linear models. Suitable changes of representation yield numerically stable learning problems and allow to exploit sparsity. For high-dimensional problems or when only a small data set is available, the selection of a good model class is a critical issue. For a given tree, the selection of the tuple of tree-based ranks that minimize the risk is a combinatorial problem. Here, we propose a rank adaptation strategy which provides in practice a good convergence of the risk as a function of the model class complexity. Finding a good tree is also a combinatorial problem, which can be related to the choice of a particular sparse architecture for deep neural networks. Here, we propose a stochastic algorithm for minimizing the complexity of the representation of a given function over a class of trees with a given arity, allowing changes in the topology of the tree. This tree optimization algorithm is then included in a learning scheme that successively adapts the tree and the corresponding tree-based ranks. Contrary to classical learning algorithms for nonlinear model classes, the proposed algorithms are numerically stable, reliable, and require only a low level expertise of the user.