# Knowledge distillation

3,513 papers in 12 research directions. Data through 14 Oct 2026.
HTML: https://mlsanity.com/distill/

## Directions (largest first)

- [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/index.md): 900 papers, 3 new this week
- [Knowledge distillation for LLMs](https://mlsanity.com/distill/knowledge-distillation-for-llms/index.md): 623 papers, 6 new this week
- [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/index.md): 451 papers, 0 new this week
- [On-policy and self-distillation for LLMs](https://mlsanity.com/distill/on-policy-and-self-distillation-for-llms/index.md): 446 papers, 22 new this week
- [Multimodal distillation](https://mlsanity.com/distill/multimodal-distillation/index.md): 308 papers, 2 new this week
- [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/index.md): 197 papers, 1 new this week
- [Federated, robust and private distillation](https://mlsanity.com/distill/federated-robust-and-private-distillation/index.md): 191 papers, 0 new this week
- [Distillation for medical imaging](https://mlsanity.com/distill/distillation-for-medical-imaging/index.md): 191 papers, 0 new this week
- [Graph neural network distillation](https://mlsanity.com/distill/graph-neural-network-distillation/index.md): 98 papers, 0 new this week
- [Distillation for continual learning](https://mlsanity.com/distill/distillation-for-continual-learning/index.md): 51 papers, 0 new this week
- [Distillation for recommender systems](https://mlsanity.com/distill/distillation-for-recommender-systems/index.md): 34 papers, 0 new this week
- [Distillation for spiking neural networks](https://mlsanity.com/distill/distillation-for-spiking-neural-networks/index.md): 23 papers, 0 new this week