# Extending Sparse Tensor Accelerators to Support Multiple Compression Formats

arXiv: https://arxiv.org/abs/2103.10452 · HTML: https://mlsanity.com/paper/2103.10452/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Mar 2021.

## Built on by (most cited first, 6 of 6)

- [Progressive Gradient Flow for Robust N:M Sparsity Training in Transformers](https://mlsanity.com/paper/2402.04744/) (2024)
- [Training Recipe for N:M Structured Sparsity with Decaying Pruning Mask](https://mlsanity.com/paper/2209.07617/) (2022)
- [FLAASH: Flexible Accelerator Architecture for Sparse High-Order Tensor Contraction](https://mlsanity.com/paper/2404.16317/) (2024)
- [SparseMap: A Sparse Tensor Accelerator Framework Based on Evolution Strategy](https://mlsanity.com/paper/2508.12906/) (2025)
- [A Multi-Engine Dataflow for MoE Decoding on Scratchpad-Based Tensor Accelerators](https://mlsanity.com/paper/2609.21137/) (2026)
- [SparseCraft: Agentic Hardware-Software Co-Optimization for Sparse Computing](https://mlsanity.com/paper/2610.05037/) (2026)

## Abstract (arXiv)

Sparsity, which occurs in both scientific applications and Deep Learning (DL) models, has been a key target of optimization within recent ASIC accelerators due to the potential memory and compute savings. These applications use data stored in a variety of compression formats. We demonstrate that both the compactness of different compression formats and the compute efficiency of the algorithms enabled by them vary across tensor dimensions and amount of sparsity. Since DL and scientific workloads span across all sparsity regions, there can be numerous format combinations for optimizing memory and compute efficiency. Unfortunately, many proposed accelerators operate on one or two fixed format combinations. This work proposes hardware extensions to accelerators for supporting numerous format combinations seamlessly and demonstrates ~4X speedup over performing format conversions in software.