# TriADA: Massively Parallel Trilinear Matrix-by-Tensor Multiply-Add Algorithm and Device Architecture for the Acceleration of 3D Discrete Transformations

arXiv: https://arxiv.org/abs/2506.22818 · HTML: https://mlsanity.com/paper/2506.22818/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 70 references that are in the map.
- First posted on arXiv 28 Jun 2025.

## Builds on (most cited first, 17 of 17)

- [Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks](https://mlsanity.com/paper/2102.00554/) (2021)
- [Tensor Decomposition for Signal Processing and Machine Learning](https://mlsanity.com/paper/1607.01668/) (2016)
- [TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddings](https://mlsanity.com/paper/2304.01433/) (2023)
- [Primer: Searching for Efficient Transformers for Language Modeling](https://mlsanity.com/paper/2109.08668/) (2021)
- [Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2109.14545/) (2021)
- [Benchmarking and Dissecting the Nvidia Hopper GPU Architecture](https://mlsanity.com/paper/2402.13499/) (2024)
- [HighLight: Efficient and Flexible DNN Acceleration with Hierarchical Structured Sparsity](https://mlsanity.com/paper/2305.12718/) (2023)
- [A Survey on Error-Bounded Lossy Compression for Scientific Datasets](https://mlsanity.com/paper/2404.02840/) (2024)
- [Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency](https://mlsanity.com/paper/2303.11525/) (2023)
- [Enabling Unstructured Sparse Acceleration on Structured Sparse Accelerators](https://mlsanity.com/paper/2403.07953/) (2024)
- [FFT-Based Deep Learning Deployment in Embedded Systems](https://mlsanity.com/paper/1712.04910/) (2017)
- [Wafer-scale Computing: Advancements, Challenges, and Future Perspectives](https://mlsanity.com/paper/2310.09568/) (2023)
- [S4: a High-sparsity, High-performance AI Accelerator](https://mlsanity.com/paper/2207.08006/) (2022)
- [Matrix Engines for High Performance Computing:A Paragon of Performance or Grasping at Straws?](https://mlsanity.com/paper/2010.14373/) (2020)
- [Breaking the Molecular Dynamics Timescale Barrier Using a Wafer-Scale System](https://mlsanity.com/paper/2405.07898/) (2024)
- [Scalability of 3D-DFT by block tensor-matrix multiplication on the JUWELS Cluster](https://mlsanity.com/paper/2303.13337/) (2023)
- [Tailors: Accelerating Sparse Tensor Algebra by Overbooking Buffer Capacity](https://mlsanity.com/paper/2310.00192/) (2023)

## Abstract (arXiv)

Multilinear transformations are key in high-performance computing (HPC) and artificial intelligence (AI) workloads, where data is represented as tensors. However, their high computational and memory demands, which grow with dimensionality, often slow down critical tasks. Moreover, scaling computation by enlarging the number of parallel processing units substantially increases energy consumption, limiting widespread adoption, especially for sparse data, which is common in HPC and AI applications. This paper introduces the Trilinear Algorithm and isomorphic to algorithm Device Architecture (TriADA) to address these challenges with the following innovations: (1) a massively parallel, low-rank algorithm for computing a family of trilinear (3D) discrete orthogonal transformations (3D-DXTs), which is a special case of the more general 3-mode matrix-by-tensor multiplication (3D-GEMT); (2) a new outer-product-based GEMM kernel with decoupled streaming active memory, specially designed to accelerate 3D-GEMT operation; (3) an isomorphic to the proposed algorithm, fully distributed 3D network of mesh interconnected processing elements or cells with a coordinate-free, data-driven local processing activity, which is independent of problem size; (4) an elastic sparse outer-product (ESOP) method that avoids unnecessary computing and communication operations with zero-valued operands, thereby enhancing energy efficiency, computational accuracy, and stability. TriADA is capable of performing a variety of trilinear transformations with hypercubic arithmetic complexity in a linear number of time-steps. The massively parallel, scalable, and energy-efficient architecture of TriADA is ideal for accelerating multilinear tensor operations, which are the most demanding parts of AI and HPC workloads.