# MIVE: A Minimalist Integer Vector Engine for Softmax LayerNorm and RMSNorm Acceleration

arXiv: https://arxiv.org/abs/2606.17781 · HTML: https://mlsanity.com/paper/2606.17781/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 18 references that are in the map.
- First posted on arXiv 16 Jun 2026.

## Builds on (most cited first, 11 of 11)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016)
- [I-BERT: Integer-only BERT Quantization](https://mlsanity.com/paper/2101.01321/) (2021)
- [Softermax: Hardware/Software Co-Design of an Efficient Softmax for Transformers](https://mlsanity.com/paper/2103.09301/) (2021)
- [NN-LUT: Neural Approximation of Non-Linear Operations for Efficient Transformer Inference](https://mlsanity.com/paper/2112.02191/) (2021)
- [SOLE: Hardware-Software Co-design of Softmax and LayerNorm for Efficient Transformer Inference](https://mlsanity.com/paper/2510.17189/) (2025)
- [Reusing Softmax Hardware Unit for GELU Computation in Transformers](https://mlsanity.com/paper/2402.10118/) (2024)
- [Genetic Quantization-Aware Approximation for Non-Linear Operations in Transformers](https://mlsanity.com/paper/2403.19591/) (2024)

## Abstract (arXiv)

The rapid growth of Large Language Models (LLMs) has intensified the need for specialized hardware accelerators that can satisfy stringent inference latency and power constraints. Although matrix multiplications dominate the overall computational workload, non-linear vector normalization operations, such as LayerNorm, RMSNorm and Softmax can become critical hardware bottlenecks. Existing accelerators typically implement these functions using dedicated hardware blocks, leading to duplicated resources and inefficient silicon utilization. To address this limitation, we propose a Minimalist Integer Vector Engine (MIVE), a programmable architecture capable of executing all three operations within a unified datapath. By exploiting common computational patterns across LayerNorm, RMSNorm and Softmax the proposed vector engine maximizes hardware sharing while reducing implementation overhead. Physical ASIC implementation results show that MIVE provides comprehensive multi-function support while achieving higher area and hardware efficiency than most state-of-the-art standalone accelerators.