Tempus: A Temporally Scalable Resource-Invariant GEMM Streaming Framework for Versal AI Edge
Not on the map. Why?
Abstract (arXiv)
Scaling laws for Large Language Models (LLMs) establish that model quality improves with computational scale, yet edge deployment imposes strict constraints on compute, memory, and power. Since General Matrix Multiplication (GEMM) accounts for up to 90% of inference time, efficient GEMM acceleration is critical for edge AI. The Adaptive Intelligent Engines available in the AMD Versal adaptive SoCs are well suited for this task, but existing state-of-the-art (SOTA) frameworks maximize performance through spatial scaling, distributing workloads across hundreds of cores -- an approach that fails on resource-limited edge SoCs due to physical implementation failures, bandwidth saturation, and excessive resource consumption. We propose Tempus, a Resource-Invariant Temporal GEMM framework for the AMD Versal AI Edge SoC. Rather than expanding hardware resources with matrix size, Tempus employs a fixed compute block of 16 AIE-ML cores, achieving scalability through iterative graph execution and algorithmic data tiling and replication in the Programmable Logic. High-speed cascade streaming ensures low-latency partial sum reduction at Initiation Interval (II) of 1, while a deadlock-free DATAFLOW protocol maximizes transfer-compute overlap and PLIO reuse. Evaluated on GEMM workloads, Tempus achieves 607 GOPS at 10.677 W total on-chip power. By characterizing system-level efficiency through the Platform-Aware Utility (PAU) metric, we prove that Tempus achieves a 211.2x higher prominence factor than the leading spatial SOTA (ARIES). Furthermore, the framework maintains a 0.00% utilization of URAM/DSP, yielding 22.0x core frugality, 7.1x power frugality, and a 6.3x reduction in I/O demand, establishing a sustainable, scalable foundation for edge LLM inference.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 16 of its 30 references that are in the map. First posted on arXiv 1 May 2026.
Builds on
16 of 30 references.
References · 16
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020Scaling Laws for Neural Language Modelscited by 6,411
- 2022Training Compute-Optimal Large Language Modelscited by 2,927
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
- 2024TinyLlama: An Open-Source Small Language Modelcited by 604
- 2021BERT: A Review of Applications in Natural Language Processing and Understandingcited by 109
- 2024Reconciling Kaplan and Chinchilla Scaling Lawscited by 36
- 2024A Survey: Collaborative Hardware and Software Design in the Era of Large Language Modelscited by 21
- 2023MaxEVA: Maximizing the Efficiency of Matrix Multiplication on Versal AI Enginecited by 7
- 2023CHARM: Composing Heterogeneous Accelerators for Matrix Multiply on Versal ACAP Architecturecited by 5
- 2025AIE4ML: An End-to-End Framework for Compiling Neural Networks for the Next Generation of AMD AI Enginescited by 5
4 more
- 2025GAMA: High-Performance GEMM Acceleration on AMD Versal ML-Optimized AI Enginescited by 3
- 2024Mapping Parallel Matrix Multiplication in GotoBLAS2 to the AMD Versal ACAP for Deep Learningcited by 1
- 2025SLIM: A Heterogeneous Accelerator for Edge Inference of Sparse Large Language Model via Adaptive Thresholdingcited by 1
- 2025Tiny but Mighty: A Software-Hardware Co-Design Approach for Efficient Multimodal Inference on Battery-Powered Small Devicescited by 1
Built on by
Not cited yet.