# FLAASH: Flexible Accelerator Architecture for Sparse High-Order Tensor Contraction

arXiv: https://arxiv.org/abs/2404.16317 · HTML: https://mlsanity.com/paper/2404.16317/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 24 references that are in the map.
- First posted on arXiv 25 Apr 2024.

## Builds on (most cited first, 13 of 13)

- [To prune, or not to prune: exploring the efficacy of pruning for model compression](https://mlsanity.com/paper/1710.01878/) (2017)
- [Movement Pruning: Adaptive Sparsity by Fine-Tuning](https://mlsanity.com/paper/2005.07683/) (2020)
- [Accelerating Sparse Deep Neural Networks](https://mlsanity.com/paper/2104.08378/) (2021)
- [SGD with Large Step Sizes Learns Sparse Features](https://mlsanity.com/paper/2210.05337/) (2022)
- [SpArch: Efficient Architecture for Sparse Matrix Multiplication](https://mlsanity.com/paper/2002.08947/) (2020)
- [Tensor Regression Networks](https://mlsanity.com/paper/1707.08308/) (2017)
- [Supervised Learning with Quantum-Inspired Tensor Networks](https://mlsanity.com/paper/1605.05775/) (2016)
- [Dual-side Sparse Tensor Core](https://mlsanity.com/paper/2105.09564/) (2021)
- [Hardware Acceleration of Sparse and Irregular Tensor Computations of ML Models: A Survey and Insights](https://mlsanity.com/paper/2007.00864/) (2020)
- [Tensor Contraction Layers for Parsimonious Deep Nets](https://mlsanity.com/paper/1706.00439/) (2017)
- [Extending Sparse Tensor Accelerators to Support Multiple Compression Formats](https://mlsanity.com/paper/2103.10452/) (2021)
- [Barrier-Free Large-Scale Sparse Tensor Accelerator (BARISTA) For Convolutional Neural Networks](https://mlsanity.com/paper/2104.08734/) (2021)
- [OpSparse: a Highly Optimized Framework for Sparse General Matrix Multiplication on GPUs](https://mlsanity.com/paper/2206.07244/) (2022)

## Abstract (arXiv)

Tensors play a vital role in machine learning (ML) and often exhibit properties best explored while maintaining high-order. Efficiently performing ML computations requires taking advantage of sparsity, but generalized hardware support is challenging. This paper introduces FLAASH, a flexible and modular accelerator design for sparse tensor contraction that achieves over 25x speedup for a deep learning workload. Our architecture performs sparse high-order tensor contraction by distributing sparse dot products, or portions thereof, to numerous Sparse Dot Product Engines (SDPEs). Memory structure and job distribution can be customized, and we demonstrate a simple approach as a proof of concept. We address the challenges associated with control flow to navigate data structures, high-order representation, and high-sparsity handling. The effectiveness of our approach is demonstrated through various evaluations, showcasing significant speedup as sparsity and order increase.