# TTOpt: A Maximum Volume Quantized Tensor Train-based Optimization and its Application to Reinforcement Learning

arXiv: https://arxiv.org/abs/2205.00293 · HTML: https://mlsanity.com/paper/2205.00293/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 103 references that are in the map.
- First posted on arXiv 30 Apr 2022.

## Builds on (most cited first, 20 of 20)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Recurrent World Models Facilitate Policy Evolution](https://mlsanity.com/paper/1809.01999/) (2018)
- [Evolution Strategies as a Scalable Alternative to Reinforcement Learning](https://mlsanity.com/paper/1703.03864/) (2017)
- [Ab-Initio Solution of the Many-Electron Schr"odinger Equation with Deep Neural Networks](https://mlsanity.com/paper/1909.02487/) (2019)
- [Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning](https://mlsanity.com/paper/1712.06567/) (2017)
- [Improving Exploration in Evolution Strategies for Deep Reinforcement Learning via a Population of Novelty-Seeking Agents](https://mlsanity.com/paper/1712.06560/) (2017)
- [Evolution-Guided Policy Gradient in Reinforcement Learning](https://mlsanity.com/paper/1805.07917/) (2018)
- [A Literature Survey of Benchmark Functions For Global Optimization Problems](https://mlsanity.com/paper/1308.4008/) (2013)
- [AlphaStar: An Evolutionary Computation Perspective](https://mlsanity.com/paper/1902.01724/) (2019)
- [Structured Evolution with Compact Architectures for Scalable Policy Optimization](https://mlsanity.com/paper/1804.02395/) (2018)
- [Global Deterministic Optimization with Artificial Neural Networks Embedded](https://mlsanity.com/paper/1801.07114/) (2018)
- [Evolving Rewards to Automate Reinforcement Learning](https://mlsanity.com/paper/1905.07628/) (2019)
- [Tesseract: Tensorised Actors for Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2106.00136/) (2021)
- [ES Is More Than Just a Traditional Finite-Difference Approximator](https://mlsanity.com/paper/1712.06568/) (2017)
- [Particle Swarm Optimization for Generating Interpretable Fuzzy Reinforcement Learning Policies](https://mlsanity.com/paper/1610.05984/) (2016)
- [Black-Box Optimization Revisited: Improving Algorithm Selection Wizards through Massive Benchmarking](https://mlsanity.com/paper/2010.04542/) (2020)
- [High-Dimensional Stochastic Optimal Control using Continuous Tensor Decompositions](https://mlsanity.com/paper/1611.04706/) (2016)
- [Automatic differentiation for Riemannian optimization on low-rank matrix and tensor-train manifolds](https://mlsanity.com/paper/2103.14974/) (2021)

## Built on by (most cited first, 15 of 15)

- [MaxInfo: A Training-Free Key-Frame Selection Method Using Maximum Volume for Enhanced Video Understanding](https://mlsanity.com/paper/2502.03183/) (2025)
- [Alternating Local Enumeration (TnALE): Solving Tensor Network Structure Search with Fewer Evaluations](https://mlsanity.com/paper/2304.12875/) (2023)
- [Towards Practical Control of Singular Values of Convolutional Layers](https://mlsanity.com/paper/2211.13771/) (2022)
- [tntorch: Tensor Network Learning with PyTorch](https://mlsanity.com/paper/2206.11128/) (2022)
- [Are Quantum Computers Practical Yet? A Case for Feature Selection in Recommender Systems using Tensor Networks](https://mlsanity.com/paper/2205.04490/) (2022)
- [Decomposition Ascribed Synergistic Learning for Unified Image Restoration](https://mlsanity.com/paper/2308.00759/) (2023)
- [Tensor Train for Global Optimization Problems in Robotics](https://mlsanity.com/paper/2206.05077/) (2022)
- [Translate your gibberish: black-box adversarial attack on machine translation systems](https://mlsanity.com/paper/2303.10974/) (2023)
- [Fast gradient-free activation maximization for neurons in spiking neural networks](https://mlsanity.com/paper/2401.10748/) (2023)
- [Black-Box Approximation and Optimization with Hierarchical Tucker Decomposition](https://mlsanity.com/paper/2402.02890/) (2024)
- [T'ecnicas Quantum-Inspired en Tensor Networks para Contextos Industriales](https://mlsanity.com/paper/2404.17645/) (2024)
- [Quantum-based QoE Optimization in Advanced Cellular Networks: Integration and Cloud Gaming Use Case](https://mlsanity.com/paper/2509.01008/) (2025)
- [Tensor Computing Interface: An Application-Oriented, Lightweight Interface for Portable High-Performance Tensor Network Applications](https://mlsanity.com/paper/2512.23917/) (2025)
- [A Tensor-Train Framework for Bayesian Inference in High-Dimensional Systems: Applications to MIMO Detection and Channel Decoding](https://mlsanity.com/paper/2604.05890/) (2026)
- [Constraint-Aware Discrete Black-Box Optimization Using Tensor Decomposition](https://mlsanity.com/paper/2609.09370/) (2026)

## Abstract (arXiv)

We present a novel procedure for optimization based on the combination of efficient quantized tensor train representation and a generalized maximum matrix volume principle. We demonstrate the applicability of the new Tensor Train Optimizer (TTOpt) method for various tasks, ranging from minimization of multidimensional functions to reinforcement learning. Our algorithm compares favorably to popular evolutionary-based methods and outperforms them by the number of function evaluations or execution time, often by a significant margin.