# Q-DETR: An Efficient Low-Bit Quantized Detection Transformer

arXiv: https://arxiv.org/abs/2304.00253 · HTML: https://mlsanity.com/paper/2304.00253/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 49 references that are in the map.
- First posted on arXiv 1 Apr 2023.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [DN-DETR: Accelerate DETR Training by Introducing Query DeNoising](https://mlsanity.com/paper/2203.01305/) (2022)
- [Conditional DETR for Fast Training Convergence](https://mlsanity.com/paper/2108.06152/) (2021)
- [How Does Batch Normalization Help Optimization?](https://mlsanity.com/paper/1805.11604/) (2018)
- [Post-Training Quantization for Vision Transformer](https://mlsanity.com/paper/2106.14156/) (2021)
- [ZeroQ: A Novel Zero Shot Quantization Framework](https://mlsanity.com/paper/2001.00281/) (2020)
- [Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm](https://mlsanity.com/paper/1808.00278/) (2018)
- [Predicting Parameters in Deep Learning](https://mlsanity.com/paper/1306.0543/) (2013)
- [Trained Ternary Quantization](https://mlsanity.com/paper/1612.01064/) (2016)
- [Distilling Object Detectors with Fine-grained Feature Imitation](https://mlsanity.com/paper/1906.03609/) (2019)
- [LSQ+: Improving low-bit quantization through learnable offsets and better initialization](https://mlsanity.com/paper/2004.09576/) (2020)
- [ReActNet: Towards Precise Binary Neural Network with Generalized Activation Functions](https://mlsanity.com/paper/2003.03488/) (2020)
- [Fast Convergence of DETR with Spatially Modulated Co-Attention](https://mlsanity.com/paper/2101.07448/) (2021)
- [Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Loss](https://mlsanity.com/paper/1808.05779/) (2018)
- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [FQ-ViT: Post-Training Quantization for Fully Quantized Vision Transformer](https://mlsanity.com/paper/2111.13824/) (2021)

## Built on by (most cited first, 24 of 24)

- [QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuning](https://mlsanity.com/paper/2402.03666/) (2024)
- [A Survey on Transformer Compression](https://mlsanity.com/paper/2402.05964/) (2024)
- [Bi-ViT: Pushing the Limit of Vision Transformer Quantization](https://mlsanity.com/paper/2305.12354/) (2023)
- [Retraining-free Model Quantization via One-Shot Weight-Coupling Learning](https://mlsanity.com/paper/2401.01543/) (2024)
- [RSBuilding: Towards General Remote Sensing Image Building Extraction and Change Detection with Foundation Model](https://mlsanity.com/paper/2403.07564/) (2024)
- [Low-bit Model Quantization for Deep Neural Networks: A Survey](https://mlsanity.com/paper/2505.05530/) (2025)
- [IntLoRA: Integral Low-rank Adaptation of Quantized Diffusion Models](https://mlsanity.com/paper/2410.21759/) (2024)
- [P4Q: Learning to Prompt for Quantization in Visual-language Models](https://mlsanity.com/paper/2409.17634/) (2024)
- [Towards Incremental Learning in Large Language Models: A Critical Review](https://mlsanity.com/paper/2404.18311/) (2024)
- [Q-YOLO: Efficient Inference for Real-time Object Detection](https://mlsanity.com/paper/2307.04816/) (2023)
- [Entropy-based Guidance of Deep Neural Networks for Accelerated Convergence and Improved Performance](https://mlsanity.com/paper/2308.14938/) (2023)
- [An Analysis on Quantizing Diffusion Transformers](https://mlsanity.com/paper/2406.11100/) (2024)
- [Binary Event-Driven Spiking Transformer](https://mlsanity.com/paper/2501.05904/) (2025)
- [A General and Efficient Training for Transformer via Token Expansion](https://mlsanity.com/paper/2404.00672/) (2024)
- [FQ-PETR: Fully Quantized Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2502.15488/) (2025)
- [In-Distribution Consistency Regularization Improves the Generalization of Quantization-Aware Training](https://mlsanity.com/paper/2402.13497/) (2024)
- [DCP-NAS: Discrepant Child-Parent Neural Architecture Search for 1-bit CNNs](https://mlsanity.com/paper/2306.15390/) (2023)
- [SONIQ: System-Optimized Noise-Injected Ultra-Low-Precision Quantization with Full-Precision Parity](https://mlsanity.com/paper/2311.14114/) (2023)
- [MEC-Quant: Maximum Entropy Coding for Extremely Low Bit Quantization-Aware Training](https://mlsanity.com/paper/2509.15514/) (2025)
- [Q2: Quantization-Aware Gradient Balancing and Attention Alignment for Low-Bit Quantization](https://mlsanity.com/paper/2511.05898/) (2025)
- [FQ-PETR: Fully Quantized Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2511.09347/) (2025)
- [QATMA: Quantization-Aware Training with Multimodal Alignment for Open-Vocabulary Object Detection](https://mlsanity.com/paper/2603.05964/) (2026)
- [KinematicRL: A Sim-to-Real Reinforcement Learning Framework For Social Navigation With Kinodynamic Feasibility](https://mlsanity.com/paper/2606.12042/) (2026)
- [Enabling Fully Integer-Only Inference for Lightweight Detection Transformers](https://mlsanity.com/paper/2607.24981/) (2026)

## Abstract (arXiv)

The recent detection transformer (DETR) has advanced object detection, but its application on resource-constrained devices requires massive computation and memory resources. Quantization stands out as a solution by representing the network in low-bit parameters and operations. However, there is a significant performance drop when performing low-bit quantized DETR (Q-DETR) with existing quantization methods. We find that the bottlenecks of Q-DETR come from the query information distortion through our empirical analyses. This paper addresses this problem based on a distribution rectification distillation (DRD). We formulate our DRD as a bi-level optimization problem, which can be derived by generalizing the information bottleneck (IB) principle to the learning of Q-DETR. At the inner level, we conduct a distribution alignment for the queries to maximize the self-information entropy. At the upper level, we introduce a new foreground-aware query matching scheme to effectively transfer the teacher information to distillation-desired features to minimize the conditional information entropy. Extensive experimental results show that our method performs much better than prior arts. For example, the 4-bit Q-DETR can theoretically accelerate DETR with ResNet-50 backbone by 6.6x and achieve 39.4% AP, with only 2.6% performance gaps than its real-valued counterpart on the COCO dataset.