# FLARE: FP-Less PTQ and Low-ENOB ADC Based AMS-PiM for Error-Resilient, Fast, and Efficient Transformer Acceleration

arXiv: https://arxiv.org/abs/2411.14733 · HTML: https://mlsanity.com/paper/2411.14733/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 59 references that are in the map.
- First posted on arXiv 22 Nov 2024.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations](https://mlsanity.com/paper/1609.07061/) (2016)
- [I-ViT: Integer-only Quantization for Efficient Vision Transformer Inference](https://mlsanity.com/paper/2207.01405/) (2022)
- [FLAT: An Optimized Dataflow for Mitigating Attention Bottlenecks](https://mlsanity.com/paper/2107.06419/) (2021)
- [X-Former: In-Memory Acceleration of Transformers](https://mlsanity.com/paper/2303.07470/) (2023)
- [MoNDE: Mixture of Near-Data Experts for Large-Scale Sparse Models](https://mlsanity.com/paper/2405.18832/) (2024)
- [RAELLA: Reforming the Arithmetic for Efficient, Low-Resolution, and Low-Loss Analog PIM: No Retraining Required!](https://mlsanity.com/paper/2304.07935/) (2023)
- [MIMHD: Accurate and Efficient Hyperdimensional Inference Using Multi-Bit In-Memory Computing](https://mlsanity.com/paper/2106.12029/) (2021)
- [MRAM Co-designed Processing-in-Memory CNN Accelerator for Mobile and IoT Applications](https://mlsanity.com/paper/1811.12179/) (2018)
- [PIVOT- Input-aware Path Selection for Energy-efficient ViT Inference](https://mlsanity.com/paper/2404.15185/) (2024)
- [PIM-DRAM: Accelerating Machine Learning Workloads using Processing in Commodity DRAM](https://mlsanity.com/paper/2105.03736/) (2021)

## Abstract (arXiv)

Encoder-based transformers, powered by self-attention layers, have revolutionized machine learning with their context-aware representations. However, their quadratic growth in computational and memory demands presents significant bottlenecks. Analog-Mixed-Signal Process-in-Memory (AMS-PiM) architectures address these challenges by enabling efficient on-chip processing. Traditionally, AMS-PiM relies on Quantization-Aware Training (QAT), which is hardware-efficient but requires extensive retraining to adapt models to AMS-PiMs, making it increasingly impractical for transformer models. Post-Training Quantization (PTQ) mitigates this training overhead but introduces significant hardware inefficiencies. PTQ relies on dequantization-quantization (DQ-Q) processes, floating-point units (FPUs), and high-ENOB (Effective Number of Bits) analog-to-digital converters (ADCs). Particularly, High-ENOB ADCs scale exponentially in area and energy (2ENOB), reduce sensing margins, and increase susceptibility to process, voltage, and temperature (PVT) variations, further compounding PTQ's challenges in AMS-PiM systems. To overcome these limitations, we propose RAP, an AMS-PiM architecture that eliminates DQ-Q processes, introduces FPU- and division-free nonlinear processing, and employs a low-ENOB-ADC-based sparse Matrix Vector multiplication technique. Using the proposed techniques, RAP improves error resiliency, area/energy efficiency, and computational speed while preserving numerical stability. Experimental results demonstrate that RAP outperforms state-of-the-art GPUs and conventional PiM architectures in energy efficiency, latency, and accuracy, making it a scalable solution for the efficient deployment of transformers.