# Design Rules for Extreme-Edge Scientific Computing on AI Engines

arXiv: https://arxiv.org/abs/2604.19106 · HTML: https://mlsanity.com/paper/2604.19106/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 36 references that are in the map.
- First posted on arXiv 21 Apr 2026.

## Builds on (most cited first, 19 of 19)

- [FINN: A Framework for Fast, Scalable Binarized Neural Network Inference](https://mlsanity.com/paper/1612.07119/) (2016)
- [MLPerf Tiny Benchmark](https://mlsanity.com/paper/2106.07597/) (2021)
- [hls4ml: An Open-Source Codesign Workflow to Empower Scientific Low-Power Machine Learning Devices](https://mlsanity.com/paper/2103.05579/) (2021)
- [A reconfigurable neural network ASIC for detector front-end data compression at the HL-LHC](https://mlsanity.com/paper/2105.01683/) (2021)
- [FastML Science Benchmarks: Accelerating Real-Time Scientific Edge Machine Learning](https://mlsanity.com/paper/2207.07958/) (2022)
- [MaxEVA: Maximizing the Efficiency of Matrix Multiplication on Versal AI Engine](https://mlsanity.com/paper/2311.04980/) (2023)
- [Low latency optical-based mode tracking with machine learning deployed on FPGAs on a tokamak](https://mlsanity.com/paper/2312.00128/) (2023)
- [CHARM: Composing Heterogeneous Accelerators for Matrix Multiply on Versal ACAP Architecture](https://mlsanity.com/paper/2301.02359/) (2023)
- [AIE4ML: An End-to-End Framework for Compiling Neural Networks for the Next Generation of AMD AI Engines](https://mlsanity.com/paper/2512.15946/) (2025)
- [Low-latency machine learning FPGA accelerator for multi-qubit-state discrimination](https://mlsanity.com/paper/2407.03852/) (2024)
- [SPARTA: Spatial Acceleration for Efficient and Scalable Horizontal Diffusion Weather Stencil Computation](https://mlsanity.com/paper/2303.03509/) (2023)
- [AIM: Accelerating Arbitrary-precision Integer Multiplication on Heterogeneous Reconfigurable Computing Platform Versal ACAP](https://mlsanity.com/paper/2309.12275/) (2023)
- [Exploring the Versal AI engines for accelerating stencil-based atmospheric advection simulation](https://mlsanity.com/paper/2301.13016/) (2022)
- [Exploiting On-chip Heterogeneity of Versal Architecture for GNN Inference Acceleration](https://mlsanity.com/paper/2308.02749/) (2023)
- [Architectural Implications of Neural Network Inference for High Data-Rate, Low-Latency Scientific Applications](https://mlsanity.com/paper/2403.08980/) (2024)
- [Reliable edge machine learning hardware for scientific applications](https://mlsanity.com/paper/2406.19522/) (2024)
- [CAT: Customized Transformer Accelerator Framework on Versal ACAP](https://mlsanity.com/paper/2409.09689/) (2024)
- [Analysis of Hardware Synthesis Strategies for Machine Learning in Collider Trigger and Data Acquisition](https://mlsanity.com/paper/2411.11678/) (2024)
- [Machine Learning on Heterogeneous, Edge, and Quantum Hardware for Particle Physics (ML-HEQUPP)](https://mlsanity.com/paper/2602.22248/) (2026)

## Abstract (arXiv)

Extreme-edge scientific applications use machine learning models to analyze sensor data and make real-time decisions. Their stringent latency and throughput requirements demand small batch sizes and require that model weights remain fully on-chip. Spatial dataflow implementations are common for extreme-edge applications. Spatial dataflow works well for small networks, but it fails to scale to larger models due to inherent resource scaling limitations. AI Engines on modern FPGA SoCs offer a promising alternative with high compute density and additional on-chip memory. However, the architecture, programming model, and performance-scaling behavior of AI Engines differ fundamentally from those of the programmable logic, making direct comparison non-trivial and the benefits of using AI Engines unclear. This work addresses how and when extreme-edge scientific neural networks should be implemented on AI Engines versus programmable logic. We provide systematic architectural characterization and micro-benchmarking and introduce a latency-adjusted resource equivalence (LARE) metric that identifies when AI Engine implementations outperform programmable logic designs. We further propose spatial and API-level dataflow optimizations tailored to low-latency scientific inference. Finally, we demonstrate the successful deployment of end-to-end neural networks on AI Engines that cannot fit on programmable logic when using the hlsml toolchain.