# Hardware-Aware Data and Instruction Mapping for AI Tasks: Balancing Parallelism, I/O and Memory Tradeoffs

arXiv: https://arxiv.org/abs/2509.03846 · HTML: https://mlsanity.com/paper/2509.03846/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 38 references that are in the map.
- First posted on arXiv 4 Sep 2025.

## Builds on (most cited first, 11 of 11)

- [AI and Memory Wall](https://mlsanity.com/paper/2403.14123/) (2024)
- [Evaluating Modern GPU Interconnect: PCIe, NVLink, NV-SLI, NVSwitch and GPUDirect](https://mlsanity.com/paper/1903.04611/) (2019)
- [An Evaluation of Edge TPU Accelerators for Convolutional Neural Networks](https://mlsanity.com/paper/2102.10423/) (2021)
- [Loom: Exploiting Weight and Activation Precisions to Accelerate Convolutional Neural Networks](https://mlsanity.com/paper/1706.07853/) (2017)
- [Neurostream: Scalable and Energy Efficient Deep Learning with Smart Memory Cubes](https://mlsanity.com/paper/1701.06420/) (2017)
- [Integrating NVIDIA Deep Learning Accelerator (NVDLA) with RISC-V SoC on FireSim](https://mlsanity.com/paper/1903.06495/) (2019)
- [A Survey on Coarse-Grained Reconfigurable Architectures from a Performance Perspective](https://mlsanity.com/paper/2004.04509/) (2020)
- [OCCAM: Optimal Data Reuse for Convolutional Neural Networks](https://mlsanity.com/paper/2106.14138/) (2021)
- [Messaging-based Adaptive Vector Computing (MAVeC) Accelerator for AI Workloads](https://mlsanity.com/paper/2410.09961/) (2024)
- [Accelerating PageRank Algorithmic Tasks with a new Programmable Hardware Architecture](https://mlsanity.com/paper/2502.00001/) (2024)
- [Demystifying the 7-D Convolution Loop Nest for Data and Instruction Streaming in Reconfigurable AI Accelerators](https://mlsanity.com/paper/2507.20420/) (2025)

## Abstract (arXiv)

We introduce a mapping framework for deep learning inference that takes advantage of predictable neural network behavior to plan both computation and communication ahead of time. The framework generates a unified stream of instructions and data, enabling the hardware to execute operations and route information on its own, without frequent involvement from the host and with minimal off-chip memory use. This naturally reduces reliance on I/O, off-chip memory, and host control. By leveraging fine-grained message passing on a programmable, message-based compute architecture, the framework keeps data movement local and coordinates computation across the array using techniques such as stationary-weight reuse, in-array multicasting, and staged reductions. Applied to VGG-19, the framework sustains high utilization (88 to 92 percent), with over 97 percent of messages generated internally and nearly 89 percent of time consumed on-chip transfers. Computation throughput scales beyond 1 TFLOP/s on larger arrays, while traffic reductions from reuse and local aggregation reach up to 100 MB per layer. Overall, the results highlight the effectiveness of streaming-based computation and show how our mapper enables this execution style by tightly coordinating data and instruction flow across the hardware.