# MCEL: Margin-Based Cross-Entropy Loss for Error-Tolerant Quantized Neural Networks

arXiv: https://arxiv.org/abs/2603.05048 · HTML: https://mlsanity.com/paper/2603.05048/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 34 references that are in the map.
- First posted on arXiv 5 Mar 2026.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [Hardware Approximate Techniques for Deep Neural Network Accelerators: A Survey](https://mlsanity.com/paper/2203.08737/) (2022)
- [ALWANN: Automatic Layer-Wise Approximation of Deep Neural Network Accelerators without Retraining](https://mlsanity.com/paper/1907.07229/) (2019)
- [EDEN: Enabling Energy-Efficient, High-Performance Deep Neural Network Inference Using Approximate DRAM](https://mlsanity.com/paper/1910.05340/) (2019)
- [SphereFace Revived: Unifying Hyperspherical Face Recognition](https://mlsanity.com/paper/2109.05565/) (2021)
- [MoBiNet: A Mobile Binary Network for Image Classification](https://mlsanity.com/paper/1907.12629/) (2019)
- [On the Universal Approximation Property and Equivalence of Stochastic Computing-based Neural Networks and Binary Neural Networks](https://mlsanity.com/paper/1803.05391/) (2018)
- [Implementing Binarized Neural Networks with Magnetoresistive RAM without Error Correction](https://mlsanity.com/paper/1908.04085/) (2019)
- [Binarized ResNet: Enabling Robust Automatic Modulation Classification at the resource-constrained Edge](https://mlsanity.com/paper/2110.14357/) (2021)
- [Bit Error Tolerance Metrics for Binarized Neural Networks](https://mlsanity.com/paper/2102.01344/) (2021)
- [Towards Explainable Bit Error Tolerance of Resistive RAM-Based Binarized Neural Networks](https://mlsanity.com/paper/2002.00909/) (2020)
- [AxOSyn: An Open-source Framework for Synthesizing Novel Approximate Arithmetic Operators](https://mlsanity.com/paper/2507.20007/) (2025)

## Built on by (most cited first, 1 of 1)

- [Quantized Reasoning Models Think They Need to Think Longer, but They Do Not](https://mlsanity.com/paper/2606.00206/) (2026)

## Abstract (arXiv)

Robustness to bit errors is a key requirement for the reliable use of neural networks (NNs) on emerging approximate computing platforms and error-prone memory technologies. A common approach to achieve bit error tolerance in NNs is injecting bit flips during training according to a predefined error model. While effective in certain scenarios, training-time bit flip injection introduces substantial computational overhead, often degrades inference accuracy at high error rates, and scales poorly for larger NN architectures. These limitations make error injection an increasingly impractical solution for ensuring robustness on future approximate computing platforms and error-prone memory technologies. In this work, we investigate the mechanisms that enable NNs to tolerate bit errors without relying on error-aware training. We establish a direct connection between bit error tolerance and classification margins at the output layer. Building on this insight, we propose a novel loss function, the Margin Cross-Entropy Loss (MCEL), which explicitly promotes logit-level margin separation while preserving the favorable optimization properties of the standard cross-entropy loss. Furthermore, MCEL introduces an interpretable margin parameter that allows robustness to be tuned in a principled manner. Extensive experimental evaluations across multiple datasets of varying complexity, diverse NN architectures, and a range of quantization schemes demonstrate that MCEL substantially improves bit error tolerance, up to 15 % in accuracy for an error rate of 1 %. Our proposed MCEL method is simple to implement, efficient, and can be integrated as a drop-in replacement for standard CEL. It provides a scalable and principled alternative to training-time bit flip injection, offering new insights into the origins of NN robustness and enabling more efficient deployment on approximate computing and memory systems.