# Exploring Model Invariance with Discrete Search for Ultra-Low-Bit Quantization

arXiv: https://arxiv.org/abs/2502.06844 · HTML: https://mlsanity.com/paper/2502.06844/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Feb 2025.

## Built on by (most cited first, 1 of 1)

- [LoRAQuant: Mixed-Precision Quantization of LoRA to Ultra-Low Bits](https://mlsanity.com/paper/2510.26690/) (2025)

## Abstract (arXiv)

Large language models have been increasing in size due to their success in a wide range of applications. This calls for a pressing need to reduce memory usage to make them more accessible. Post-training quantization is a popular technique which uses fewer bits (e.g., 4--8 bits) to represent the model without retraining it. However, it remains a challenging task to perform quantization in an ultra-low-bit setup (e.g., 2 bits). In this paper, we propose InvarExplore, a unified framework that systematically explores different model invariance at the same time, allowing us to take advantage of the synergy between each type of invariance. Importantly, InvarExplore features a discrete search algorithm that enables us to explore permutation invariance, which is under-studied as it cannot be optimized with gradient-based methods. Results show that InvarExplore is compatible with existing state-of-the-art methods, achieving an add-on performance improvement over strong competing methods.