# Neural Network Distiller: A Python Package For DNN Compression Research

arXiv: https://arxiv.org/abs/1910.12232 · HTML: https://mlsanity.com/paper/1910.12232/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Oct 2019.

## Built on by (most cited first, 12 of 12)

- [Auto-Split: A General Framework of Collaborative Edge-Cloud AI](https://mlsanity.com/paper/2108.13041/) (2021)
- [Dual-side Sparse Tensor Core](https://mlsanity.com/paper/2105.09564/) (2021)
- [Shrinking Bigfoot: Reducing wav2vec 2.0 footprint](https://mlsanity.com/paper/2103.15760/) (2021)
- [NxMTransformer: Semi-Structured Sparsification for Natural Language Understanding via ADMM](https://mlsanity.com/paper/2110.15766/) (2021)
- [JaxPruner: A concise library for sparsity research](https://mlsanity.com/paper/2304.14082/) (2023)
- [Large-scale Knowledge Distillation with Elastic Heterogeneous Computing Resources](https://mlsanity.com/paper/2207.06667/) (2022)
- [Robustness in Compressed Neural Networks for Object Detection](https://mlsanity.com/paper/2102.05509/) (2021)
- [Distilled Neural Networks for Efficient Learning to Rank](https://mlsanity.com/paper/2202.10728/) (2022)
- [Benchmarking Quantized Neural Networks on FPGAs with FINN](https://mlsanity.com/paper/2102.01341/) (2021)
- [Finding Deviated Behaviors of the Compressed DNN Models for Image Classifications](https://mlsanity.com/paper/2112.02819/) (2021)
- [Evaluation Metrics for DNNs Compression](https://mlsanity.com/paper/2305.10616/) (2023)
- [Structured Network Pruning by Measuring Filter-wise Interactions](https://mlsanity.com/paper/2307.00758/) (2023)

## Abstract (arXiv)

This paper presents the philosophy, design and feature-set of Neural Network Distiller, an open-source Python package for DNN compression research. Distiller is a library of DNN compression algorithms implementations, with tools, tutorials and sample applications for various learning tasks. Its target users are both engineers and researchers, and the rich content is complemented by a design-for-extensibility to facilitate new research. Distiller is open-source and is available on Github at https://github.com/NervanaSystems/distiller.