# Quantization Mimic: Towards Very Tiny CNN for Object Detection

arXiv: https://arxiv.org/abs/1805.02152 · HTML: https://mlsanity.com/paper/1805.02152/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 May 2018.

## Built on by (most cited first, 20 of 20)

- [Distilling Knowledge via Knowledge Review](https://mlsanity.com/paper/2104.09044/) (2021)
- [Distilling Object Detectors via Decoupled Features](https://mlsanity.com/paper/2103.14475/) (2021)
- [QueryDet: Cascaded Sparse Query for Accelerating High-Resolution Small Object Detection](https://mlsanity.com/paper/2103.09136/) (2021)
- [LiDAR Distillation: Bridging the Beam-Induced Domain Gap for 3D Object Detection](https://mlsanity.com/paper/2203.14956/) (2022)
- [FGR: Frustum-Aware Geometric Reasoning for Weakly Supervised 3D Vehicle Detection](https://mlsanity.com/paper/2105.07647/) (2021)
- [LiDAR-PTQ: Post-Training Quantization for Point Cloud 3D Object Detection](https://mlsanity.com/paper/2401.15865/) (2024)
- [LGD: Label-guided Self-distillation for Object Detection](https://mlsanity.com/paper/2109.11496/) (2021)
- [ERNIE-Tiny : A Progressive Distillation Framework for Pretrained Transformer Compression](https://mlsanity.com/paper/2106.02241/) (2021)
- [Decoupling Dark Knowledge via Block-wise Logit Distillation for Feature-level Alignment](https://mlsanity.com/paper/2411.01547/) (2024)
- [SIMPLE: SIngle-network with Mimicking and Point Learning for Bottom-up Human Pose Estimation](https://mlsanity.com/paper/2104.02486/) (2021)
- [An Empirical Study of Adder Neural Networks for Object Detection](https://mlsanity.com/paper/2112.13608/) (2021)
- [Knowledge Distillation in YOLOX-ViT for Side-Scan Sonar Object Detection](https://mlsanity.com/paper/2403.09313/) (2024)
- [Dynamic Multi-Scale Loss Optimization for Object Detection](https://mlsanity.com/paper/2108.04014/) (2021)
- [Efficient Logit-based Knowledge Distillation of Deep Spiking Neural Networks for Full-Range Timestep Deployment](https://mlsanity.com/paper/2501.15925/) (2025)
- [Pixel Distillation: A New Knowledge Distillation Scheme for Low-Resolution Image Recognition](https://mlsanity.com/paper/2112.09532/) (2021)
- [In-Distribution Consistency Regularization Improves the Generalization of Quantization-Aware Training](https://mlsanity.com/paper/2402.13497/) (2024)
- [A Real-time Low-cost Artificial Intelligence System for Autonomous Spraying in Palm Plantations](https://mlsanity.com/paper/2103.04132/) (2021)
- [Quantized Neural Networks via {-1, +1} Encoding Decomposition and Acceleration](https://mlsanity.com/paper/2106.09886/) (2021)
- [MEC-Quant: Maximum Entropy Coding for Extremely Low Bit Quantization-Aware Training](https://mlsanity.com/paper/2509.15514/) (2025)
- [Q2: Quantization-Aware Gradient Balancing and Attention Alignment for Low-Bit Quantization](https://mlsanity.com/paper/2511.05898/) (2025)

## Abstract (arXiv)

In this paper, we propose a simple and general framework for training very tiny CNNs for object detection. Due to limited representation ability, it is challenging to train very tiny networks for complicated tasks like detection. To the best of our knowledge, our method, called Quantization Mimic, is the first one focusing on very tiny networks. We utilize two types of acceleration methods: mimic and quantization. Mimic improves the performance of a student network by transfering knowledge from a teacher network. Quantization converts a full-precision network to a quantized one without large degradation of performance. If the teacher network is quantized, the search scope of the student network will be smaller. Using this feature of the quantization, we propose Quantization Mimic. It first quantizes the large network, then mimic a quantized small network. The quantization operation can help student network to better match the feature maps from teacher network. To evaluate our approach, we carry out experiments on various popular CNNs including VGG and Resnet, as well as different detection frameworks including Faster R-CNN and R-FCN. Experiments on Pascal VOC and WIDER FACE verify that our Quantization Mimic algorithm can be applied on various settings and outperforms state-of-the-art model acceleration methods given limited computing resouces.