# Automating Versatile Time-Series Analysis with Tiny Transformers on Embedded FPGAs

arXiv: https://arxiv.org/abs/2505.17662 · HTML: https://mlsanity.com/paper/2505.17662/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 23 May 2025.

## Builds on (most cited first, 11 of 11)

- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models](https://mlsanity.com/paper/2503.06027/) (2025)
- [Improving the Efficiency of Transformers for Resource-Constrained Devices](https://mlsanity.com/paper/2106.16006/) (2021)
- [Bioformers: Embedding Transformers for Ultra-Low Power sEMG-based Gesture Recognition](https://mlsanity.com/paper/2203.12932/) (2022)
- [Optimizing the Deployment of Tiny Transformers on Low-Power MCUs](https://mlsanity.com/paper/2404.02945/) (2024)
- [A survey on FPGA-based accelerator for ML models](https://mlsanity.com/paper/2412.15666/) (2024)
- [A Tiny Transformer for Low-Power Arrhythmia Classification on Microcontrollers](https://mlsanity.com/paper/2402.10748/) (2024)
- [Integer-only Quantized Transformers for Embedded FPGA-based Time-series Forecasting in AIoT](https://mlsanity.com/paper/2407.11041/) (2024)
- [ElasticAI: Creating and Deploying Energy-Efficient Deep Learning Accelerator for Pervasive Computing](https://mlsanity.com/paper/2409.09044/) (2024)
- [Optimizing TinyML: The Impact of Reduced Data Acquisition Rates for Time Series Classification on Microcontrollers](https://mlsanity.com/paper/2409.10942/) (2024)
- [Resource-aware Mixed-precision Quantization for Enhancing Deployability of Transformers for Time-series Forecasting on Embedded FPGAs](https://mlsanity.com/paper/2410.03294/) (2024)

## Built on by (most cited first, 4 of 4)

- [Automated Energy-Aware Time-Series Model Deployment on Embedded FPGAs for Resilient Combined Sewer Overflow Management](https://mlsanity.com/paper/2508.13905/) (2025)
- [Enabling Vibration-Based Gesture Recognition on Everyday Furniture via Energy-Efficient FPGA Implementation of 1D Convolutional Networks](https://mlsanity.com/paper/2510.23156/) (2025)
- [StrikeWatch: Wrist-worn Gait Recognition with Compact Time-series Models on Low-power FPGAs](https://mlsanity.com/paper/2510.24738/) (2025)
- [Q-DEQ: Discrete Solving and Quantization for Deep Equilibrium Models in Time Series Forecasting under Edge Deployment Coding Constraints](https://mlsanity.com/paper/2609.24042/) (2026)

## Abstract (arXiv)

Transformer-based models have shown strong performance across diverse time-series tasks, but their deployment on resource-constrained devices remains challenging due to high memory and computational demand. While prior work targeting Microcontroller Units (MCUs) has explored hardware-specific optimizations, such approaches are often task-specific and limited to 8-bit fixed-point precision. Field-Programmable Gate Arrays (FPGAs) offer greater flexibility, enabling fine-grained control over data precision and architecture. However, existing FPGA-based deployments of Transformers for time-series analysis typically focus on high-density platforms with manual configuration. This paper presents a unified and fully automated deployment framework for Tiny Transformers on embedded FPGAs. Our framework supports a compact encoder-only Transformer architecture across three representative time-series tasks (forecasting, classification, and anomaly detection). It combines quantization-aware training (down to 4 bits), hardware-aware hyperparameter search using Optuna, and automatic VHDL generation for seamless deployment. We evaluate our framework on six public datasets across two embedded FPGA platforms. Results show that our framework produces integer-only, task-specific Transformer accelerators achieving as low as 0.033 mJ per inference with millisecond latency on AMD Spartan-7, while also providing insights into deployment feasibility on Lattice iCE40. All source code will be released in the GitHub repository (https://github.com/Edwina1030/TinyTransformer4TS).