# Neural Precision Polarization: Simplifying Neural Network Inference with Dual-Level Precision

arXiv: https://arxiv.org/abs/2411.05845 · HTML: https://mlsanity.com/paper/2411.05845/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 23 references that are in the map.
- First posted on arXiv 6 Nov 2024.

## Builds on (most cited first, 12 of 12)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [LoRA+: Efficient Low Rank Adaptation of Large Models](https://mlsanity.com/paper/2402.12354/) (2024)
- [Training Neural Networks from Scratch with Parallel Low-Rank Adapters](https://mlsanity.com/paper/2402.16828/) (2024)
- [MF-Net: Compute-In-Memory SRAM for Multibit Precision Inference using Memory-immersed Data Conversion and Multiplication-free Operators](https://mlsanity.com/paper/2102.00035/) (2021)
- [Channel-wise Mixed-precision Assignment for DNN Inference on Constrained Edge Nodes](https://mlsanity.com/paper/2206.08852/) (2022)
- [Stochastic Layer-Wise Precision in Deep Neural Networks](https://mlsanity.com/paper/1807.00942/) (2018)
- [MC-CIM: Compute-in-Memory with Monte-Carlo Dropouts for Bayesian Edge Intelligence](https://mlsanity.com/paper/2111.07125/) (2021)
- [Accuracy and Resiliency of Analog Compute-in-Memory Inference Engines](https://mlsanity.com/paper/2008.02400/) (2020)
- [ADC/DAC-Free Analog Acceleration of Deep Neural Networks with Frequency Transformation](https://mlsanity.com/paper/2309.01771/) (2023)
- [Low Power Unsupervised Anomaly Detection by Non-Parametric Modeling of Sensor Statistics](https://mlsanity.com/paper/2003.10088/) (2020)
- [Enabling Deep Learning on Edge Devices](https://mlsanity.com/paper/2210.03204/) (2022)

## Built on by (most cited first, 2 of 2)

- [Beyond Confidence: Adaptive Abstention in Dual-Threshold Conformal Prediction for Autonomous System Perception](https://mlsanity.com/paper/2502.07255/) (2025)
- [Intelligent Sensing-to-Action for Robust Autonomy at the Edge: Opportunities and Challenges](https://mlsanity.com/paper/2502.02692/) (2025)

## Abstract (arXiv)

We introduce a precision polarization scheme for DNN inference that utilizes only very low and very high precision levels, assigning low precision to the majority of network weights and activations while reserving high precision paths for targeted error compensation. This separation allows for distinct optimization of each precision level, thereby reducing memory and computation demands without compromising model accuracy. In the discussed approach, a floating-point model can be trained in the cloud and then downloaded to an edge device, where network weights and activations are directly quantized to meet the edge devices' desired level, such as NF4 or INT8. To address accuracy loss from quantization, surrogate paths are introduced, leveraging low-rank approximations on a layer-by-layer basis. These paths are trained with a sensitivity-based metric on minimal training data to recover accuracy loss under quantization as well as due to process variability, such as when the main prediction path is implemented using analog acceleration. Our simulation results show that neural precision polarization enables approximately 464 TOPS per Watt MAC efficiency and reliability by integrating rank-8 error recovery paths with highly efficient, though potentially unreliable, bit plane-wise compute-in-memory processing.