# MaRVIn: A Cross-Layer Mixed-Precision RISC-V Framework for DNN Inference, from ISA Extension to Hardware Acceleration

arXiv: https://arxiv.org/abs/2509.15187 · HTML: https://mlsanity.com/paper/2509.15187/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 48 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Builds on (most cited first, 19 of 19)

- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [MCUNet: Tiny Deep Learning on IoT Devices](https://mlsanity.com/paper/2007.10319/) (2020)
- [CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUs](https://mlsanity.com/paper/1801.06601/) (2018)
- [Resnet in Resnet: Generalizing Residual Architectures](https://mlsanity.com/paper/1603.08029/) (2016)
- [DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs](https://mlsanity.com/paper/2008.07127/) (2020)
- [Visual Wake Words Dataset](https://mlsanity.com/paper/1906.05721/) (2019)
- [Hardware Approximate Techniques for Deep Neural Network Accelerators: A Survey](https://mlsanity.com/paper/2203.08737/) (2022)
- [Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead](https://mlsanity.com/paper/2012.11233/) (2020)
- [A Mixed-Precision RISC-V Processor for Extreme-Edge DNN Inference](https://mlsanity.com/paper/2010.04073/) (2020)
- [Cross-Layer Approximation For Printed Machine Learning Circuits](https://mlsanity.com/paper/2203.05915/) (2022)
- [Co-Design of Approximate Multilayer Perceptron for Ultra-Resource Constrained Printed Circuits](https://mlsanity.com/paper/2302.14576/) (2023)
- [Enabling Mixed-Precision Quantized Neural Networks in Extreme-Edge Devices](https://mlsanity.com/paper/2007.07759/) (2020)
- [Hardware-Aware DNN Compression via Diverse Pruning and Mixed-Precision Quantization](https://mlsanity.com/paper/2312.15322/) (2023)
- [BARVINN: Arbitrary Precision DNN Accelerator Controlled by a RISC-V CPU](https://mlsanity.com/paper/2301.00290/) (2022)
- [Mixed-precision Neural Networks on RISC-V Cores: ISA extensions for Multi-Pumped Soft SIMD Operations](https://mlsanity.com/paper/2407.14274/) (2024)
- [Approximate Computing Survey, Part I: Terminology and Software & Hardware Approximation Techniques](https://mlsanity.com/paper/2307.11124/) (2023)
- [An Ultra-low Power TinyML System for Real-time Visual Processing at Edge](https://mlsanity.com/paper/2207.04663/) (2022)
- [SPEED: A Scalable RISC-V Vector Processor Enabling Efficient Multi-Precision DNN Inference](https://mlsanity.com/paper/2409.14017/) (2024)

## Abstract (arXiv)

The evolution of quantization and mixed-precision techniques has unlocked new possibilities for enhancing the speed and energy efficiency of NNs. Several recent studies indicate that adapting precision levels across different parameters can maintain accuracy comparable to full-precision models while significantly reducing computational demands. However, existing embedded microprocessors lack sufficient architectural support for efficiently executing mixed-precision NNs, both in terms of ISA extensions and hardware design, resulting in inefficiencies such as excessive data packing/unpacking and underutilized arithmetic units. In this work, we propose novel ISA extensions and a micro-architecture implementation specifically designed to optimize mixed-precision execution, enabling energy-efficient deep learning inference on RISC-V architectures. We introduce MaRVIn, a cross-layer hardware-software co-design framework that enhances power efficiency and performance through a combination of hardware improvements, mixed-precision quantization, ISA-level optimizations, and cycle-accurate emulation. At the hardware level, we enhance the ALU with configurable mixed-precision arithmetic (2, 4, 8 bits) for weights/activations and employ multi-pumping to reduce execution latency while implementing soft SIMD for efficient 2-bit ops. At the software level, we integrate a pruning-aware fine-tuning method to optimize model compression and a greedy-based DSE approach to efficiently search for Pareto-optimal mixed-quantized models. Additionally, we incorporate voltage scaling to boost the power efficiency of our system. Our experimental evaluation over widely used DNNs and datasets, such as CIFAR10 and ImageNet, demonstrates that our framework can achieve, on average, 17.6x speedup for less than 1% accuracy loss and outperforms the ISA-agnostic state-of-the-art RISC-V cores, delivering up to 1.8 TOPs/W.