# RedMule: A Mixed-Precision Matrix-Matrix Operation Engine for Flexible and Energy-Efficient On-Chip Linear Algebra and TinyML Training Acceleration

arXiv: https://arxiv.org/abs/2301.03904 · HTML: https://mlsanity.com/paper/2301.03904/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 61 references that are in the map.
- First posted on arXiv 10 Jan 2023.

## Builds on (most cited first, 18 of 18)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computing](https://mlsanity.com/paper/1905.10083/) (2019)
- [FP8 Formats for Deep Learning](https://mlsanity.com/paper/2209.05433/) (2022)
- [Training Deep Neural Networks with 8-bit Floating Point Numbers](https://mlsanity.com/paper/1812.08011/) (2018)
- [MLPerf Tiny Benchmark](https://mlsanity.com/paper/2106.07597/) (2021)
- [TinyOL: TinyML with Online-Learning on Microcontrollers](https://mlsanity.com/paper/2103.08295/) (2021)
- [A TinyML Platform for On-Device Continual Learning with Quantized Latent Replays](https://mlsanity.com/paper/2110.10486/) (2021)
- [XNOR Neural Engine: a Hardware Accelerator IP for 21.6 fJ/op Binary Neural Network Inference](https://mlsanity.com/paper/1807.03010/) (2018)
- [8-bit Numerical Formats for Deep Neural Networks](https://mlsanity.com/paper/2206.02915/) (2022)
- [AI and ML Accelerator Survey and Trends](https://mlsanity.com/paper/2210.04055/) (2022)
- [Vega: A 10-Core SoC for IoT End-Nodes with DNN Acceleration and Cognitive Wake-Up From MRAM-Based State-Retentive Sleep Mode](https://mlsanity.com/paper/2110.09101/) (2021)
- [FPnew: An Open-Source Multi-Format Floating-Point Unit Architecture for Energy-Proportional Transprecision Computing](https://mlsanity.com/paper/2007.01530/) (2020)
- [RedMulE: A Compact FP16 Matrix-Multiplication Accelerator for Adaptive Deep Learning on RISC-V-Based Ultra-Low-Power SoCs](https://mlsanity.com/paper/2204.11192/) (2022)
- [A Transprecision Floating-Point Platform for Ultra-Low Power Computing](https://mlsanity.com/paper/1711.10374/) (2017)
- [Memory-Latency-Accuracy Trade-offs for Continual Learning on a RISC-V Extreme-Edge Node](https://mlsanity.com/paper/2007.13631/) (2020)
- [MiniFloat-NN and ExSdotp: An ISA Extension and a Modular Open Hardware Unit for Low-Precision Training on RISC-V cores](https://mlsanity.com/paper/2207.03192/) (2022)
- [SIMD2: A Generalized Matrix Instruction Set for Accelerating Tensor Computation beyond GEMM](https://mlsanity.com/paper/2205.01252/) (2022)

## Built on by (most cited first, 4 of 4)

- [A Survey on Deep Learning Hardware Accelerators for Heterogeneous HPC Platforms](https://mlsanity.com/paper/2306.15552/) (2023)
- [From Tiny Machine Learning to Tiny Deep Learning: A Survey](https://mlsanity.com/paper/2506.18927/) (2025)
- [OpenGeMM: A High-Utilization GeMM Accelerator Generator with Lightweight RISC-V Control and Tight Memory Coupling](https://mlsanity.com/paper/2411.09543/) (2024)
- [TrainDeeploy: Hardware-Accelerated Parameter-Efficient Fine-Tuning of Small Transformer Models at the Extreme Edge](https://mlsanity.com/paper/2603.09511/) (2026)

## Abstract (arXiv)

The increasing interest in TinyML, i.e., near-sensor machine learning on power budgets of a few tens of mW, is currently pushing toward enabling TinyML-class training as opposed to inference only. Current training algorithms, based on various forms of error and gradient backpropagation, rely on floating-point matrix operations to meet the precision and dynamic range requirements. So far, the energy and power cost of these operations has been considered too high for TinyML scenarios. This paper addresses the open challenge of near-sensor training on a few mW power budget and presents RedMulE - Reduced-Precision Matrix Multiplication Engine, a low-power specialized accelerator conceived for multi-precision floating-point General Matrix-Matrix Operations (GEMM-Ops) acceleration, supporting FP16, as well as hybrid FP8 formats, with {sign, exponent, mantissa}=({1,4,3}, {1,5,2}). We integrate RedMule into a Parallel Ultra-Low-Power (PULP) cluster containing eight energy-efficient RISC-V cores sharing a tightly-coupled data memory and implement the resulting system in a 22 nm technology. At its best efficiency point (@ 470 MHz, 0.65 V), the RedMulE-augmented PULP cluster achieves 755 GFLOPS/W and 920 GFLOPS/W during regular General Matrix-Matrix Multiplication (GEMM), and up to 1.19 TFLOPS/W and 1.67 TFLOPS/W when executing GEMM-Ops, respectively, for FP16 and FP8 input/output tensors. In its best performance point (@ 613 MHz, 0.8 V), RedMulE achieves up to 58.5 GFLOPS and 117 GFLOPS for FP16 and FP8, respectively, with 99.4% utilization of the array of Computing Elements and consuming less than 60 mW on average, thus enabling on-device training of deep learning models in TinyML application scenarios while retaining the flexibility to tackle other classes of common linear algebra problems efficiently.