mlsanityData to 7 Oct 2026

Slimmable Neural Networks

arXiv 1812.08928 ↗First posted 21 Dec 2018cs.CV cs.AIcited by 301

Not on the map. Why?

Abstract (arXiv)

We present a simple and general method to train a single neural network executable at different widths (number of channels in a layer), permitting instant and adaptive accuracy-efficiency trade-offs at runtime. Instead of training individual networks with different width configurations, we train a shared network with switchable batch normalization. At runtime, the network can adjust its width on the fly according to on-device benchmarks and resource constraints, rather than downloading and offloading different models. Our trained networks, named slimmable neural networks, achieve similar (and in many cases better) ImageNet classification accuracy than individually trained models of MobileNet v1, MobileNet v2, ShuffleNet and ResNet-50 at different widths respectively. We also demonstrate better performance of slimmable models compared with individual ones across a wide range of applications including COCO bounding-box object detection, instance segmentation and person keypoint detection without tuning hyper-parameters. Lastly we visualize and discuss the learned features of slimmable networks. Code and models are available at: https://github.com/JiahuiYu/slimmable_networks

Key facts. Cited by 301 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 76; 2022: 61; 2023: 58; 2024: 35; 2025: 38; 2026: 33.

301 papers.

Sort:

By area:

  1. 2022Matryoshka Representation Learningcited by 268
  2. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  3. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
  4. 2021AutoFormer: Searching Transformers for Visual Recognitioncited by 186
  5. 2021Group Fisher Pruning for Practical Network Compressioncited by 130
  6. 2022Rethinking Vision Transformers for MobileNet Size and Speedcited by 121
  7. 2021Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approachcited by 105
  8. 2021Dynamic Slimmable Networkcited by 83
  9. 2023MatFormer: Nested Transformer for Elastic Inferencecited by 79
  10. 2021Convolutional Neural Network Pruning with Structural Redundancy Reductioncited by 73
  11. 2024Scaling Diffusion Transformers to 16 Billion Parameterscited by 65
  12. 2021BossNAS: Exploring Hybrid CNN-transformers with Block-wisely Self-supervised Neural Architecture Searchcited by 63
38 more
  1. 2021Lost in Pruning: The Effects of Pruning Neural Networks beyond Test Accuracycited by 62
  2. 2021Pufferfish: Communication-efficient Models At No Extra Costcited by 61
  3. 2021A Survey on Green Deep Learningcited by 52
  4. 2022Revisiting Random Channel Pruning for Neural Network Compressioncited by 50
  5. 2021Self-Damaging Contrastive Learningcited by 48
  6. 2023EVC: Towards Real-Time Neural Image Compression with Mask Decaycited by 46
  7. 2024ElasticTok: Adaptive Tokenization for Image and Videocited by 44
  8. 2023MobileDiffusion: Instant Text-to-Image Generation on Mobile Devicescited by 42
  9. 2024Flextron: Many-in-One Flexible Large Language Modelcited by 42
  10. 2021HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformerscited by 39
  11. 2021Slimmable Compressive Autoencoders for Practical Neural Image Compressioncited by 38
  12. 2021Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AIcited by 38
  13. 2021AlphaNet: Improved Training of Supernets with Alpha-Divergencecited by 36
  14. 2021Dynamic Network Quantization for Efficient Video Inferencecited by 36
  15. 2022Q-ViT: Fully Differentiable Quantization for Vision Transformercited by 36
  16. 2022Automated Progressive Learning for Efficient Training of Vision Transformerscited by 36
  17. 2021Revisiting Dynamic Convolution via Matrix Decompositioncited by 35
  18. 2022Overview of Deep Learning-based CSI Feedback in Massive MIMO Systemscited by 35
  19. 2021Single Image Depth Prediction with Wavelet Decompositioncited by 34
  20. 2022Efficient Split-Mix Federated Learning for On-Demand and In-Situ Customizationcited by 34
  21. 2022Fine-tuning Image Transformers using Learnable Memorycited by 34
  22. 2022Expediting Large-Scale Vision Transformer for Dense Prediction without Fine-tuningcited by 33
  23. 2021Recursive-NeRF: An Efficient and Dynamically Growing NeRFcited by 32
  24. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
  25. 2021FedHM: Efficient Federated Learning for Heterogeneous Models via Low-rank Factorizationcited by 31
  26. 2025Vision-Language Models for Edge Networks: A Comprehensive Surveycited by 29
  27. 2021ProgFed: Effective, Communication, and Computation Efficient Federated Learning by Progressive Trainingcited by 27
  28. 2023CoTFormer: A Chain-of-Thought Driven Architecture with Budget-Adaptive Computation Cost at Inferencecited by 27
  29. 2021CompOFA: Compound Once-For-All Networks for Faster Multi-Platform Deploymentcited by 26
  30. 2021MicroNet: Improving Image Recognition with Extremely Low FLOPscited by 26
  31. 2023Stitchable Neural Networkscited by 26
  32. 2021Learning Deep Multimodal Feature Representation with Asymmetric Multi-layer Fusioncited by 25
  33. 2021Sharpness-aware Quantization for Deep Neural Networkscited by 25
  34. 2021DPICT: Deep Progressive Image Compression Using Trit-Planescited by 25
  35. 2023Cuttlefish: Low-Rank Model Training without All the Tuningcited by 25
  36. 2024AdaRevD: Adaptive Patch Exiting Reversible Decoder Pushes the Limit of Image Deblurringcited by 25
  37. 2024Mixture of Nested Experts: Adaptive Processing of Visual Tokenscited by 25
  38. 2021Towards Compact Single Image Super-Resolution via Contrastive Self-distillationcited by 24

50 most cited of 301 All 301 (JSON)