# StreamTensor: Make Tensors Stream in Dataflow Accelerators for LLMs

arXiv: https://arxiv.org/abs/2509.13694 · HTML: https://mlsanity.com/paper/2509.13694/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2025.

## Built on by (most cited first, 3 of 3)

- [LUT-LLM: Efficient Large Language Model Inference with Memory-based Computations on FPGAs](https://mlsanity.com/paper/2511.06174/) (2025)
- [FlexLLM: Composable HLS Library for Flexible Hybrid LLM Accelerator Design](https://mlsanity.com/paper/2601.15710/) (2026)
- [AI+HW 2035: Shaping the Next Decade](https://mlsanity.com/paper/2603.05225/) (2026)

## Abstract (arXiv)

Efficient execution of deep learning workloads on dataflow architectures is crucial for overcoming memory bottlenecks and maximizing performance. While streaming intermediate results between computation kernels can significantly improve efficiency, existing approaches struggle with inter-kernel correlations, external memory access management, and buffer optimization. In this work, we propose StreamTensor, a compiler framework that automatically constructs and optimizes stream-based dataflow accelerators. StreamTensor introduces a novel iterative tensor type system to explicitly encode stream layouts, enabling seamless kernel fusion, buffer allocation, and memory optimization. By systematically exploring three hierarchical design spaces, including tensor tiling, kernel fusion, and resource allocation, StreamTensor balances computational intensity, memory efficiency, and data streaming to maximize performance. Based on FPGA evaluations on Large Language Models (LLM), StreamTensor achieves up to 0.76x and 0.64x lower latency compared to the state-of-the-art FPGA LLM accelerators and GPUs, and up to 1.99x higher energy efficiency compared to GPUs, making it a promising approach for scalable dataflow-based deep learning acceleration.