mlsanityData to 7 Oct 2026

Network of Experts for Large-Scale Image Categorization

arXiv 1604.06119 ↗First posted 20 Apr 2016cs.CVcited by 54

Not on the map. Why?

Cited by 29 in Mixture-of-experts, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a tree-structured network architecture for large scale image classification. The trunk of the network contains convolutional layers optimized over all classes. At a given depth, the trunk splits into separate branches, each dedicated to discriminate a different subset of classes. Each branch acts as an expert classifying a set of categories that are difficult to tell apart, while the trunk provides common knowledge to all experts in the form of shared features. The training of our "network of experts" is completely end-to-end: the partition of categories into disjoint subsets is learned simultaneously with the parameters of the network trunk and the experts are trained jointly by minimizing a single learning objective over all classes. The proposed structure can be built from any existing convolutional neural network (CNN). We demonstrate its generality by adapting 4 popular CNNs for image categorization into the form of networks of experts. Our experiments on CIFAR100 and ImageNet show that in every case our method yields a substantial improvement in accuracy over the base CNN, and gives the best result achieved so far on CIFAR100. Finally, the improvement in accuracy comes at little additional cost: compared to the base network, the training time is only moderately increased and the number of parameters is comparable or in some cases even lower.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Apr 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2017: 1; 2018: 0; 2019: 0; 2020: 0; 2021: 9; 2022: 13; 2023: 11; 2024: 9; 2025: 5; 2026: 6.

54 papers.

Sort:

By area:

  1. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  2. 2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
  3. 2022M3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-designcited by 97
  4. 2022Deep Hierarchical Semantic Segmentationcited by 84
  5. 2021Generalizable Person Re-identification with Relevance-aware Mixture of Expertscited by 77
  6. 2017Hard Mixtures of Experts for Large Scale Weakly Supervised Visioncited by 67
  7. 2023Sparse MoE as the New Dropout: Scaling Dense and Self-Slimmable Transformerscited by 59
  8. 2022Meta-DMoE: Adapting to Domain Shift by Meta-Distillation from Mixture-of-Expertscited by 50
  9. 2021SpeechMoE: Scaling to Large Acoustic Models with Dynamic Routing Mixture of Expertscited by 48
  10. 2022The Neural Race Reduction: Dynamics of Abstraction in Gated Networkscited by 37
  11. 2023Patch-level Routing in Mixture-of-Experts is Provably Sample-efficient for Convolutional Neural Networkscited by 35
  12. 2023Robust Mixture-of-Expert Training for Convolutional Neural Networkscited by 30
38 more
  1. 2023Logic-induced Diagnostic Reasoning for Semi-supervised Semantic Segmentationcited by 29
  2. 2022Adaptive Mixture of Experts Learning for Generalizable Face Anti-Spoofingcited by 28
  3. 2023Enhancing NeRF akin to Enhancing LLMs: Generalizable NeRF Transformer with Mixture-of-View-Expertscited by 27
  4. 2021SpeechMoE2: Mixture-of-Experts Model with Improved Routingcited by 26
  5. 2024Routers in Vision Mixture of Experts: An Empirical Studycited by 22
  6. 2024Psychometry: An Omnifit Model for Image Reconstruction from Human Brain Activitycited by 22
  7. 2022HydraFusion: Context-Aware Selective Sensor Fusion for Robust and Efficient Autonomous Vehicle Perceptioncited by 16
  8. 2022Neural Networks are Decision Treescited by 16
  9. 2023Learning Topology-Specific Experts for Molecular Property Predictioncited by 15
  10. 2022Hierarchical classification at multiple operating pointscited by 14
  11. 2022Spatial Mixture-of-Expertscited by 14
  12. 2022Sparsely-gated Mixture-of-Expert Layers for CNN Interpretabilitycited by 13
  13. 2023ProTeCt: Prompt Tuning for Taxonomic Open Set Classificationcited by 12
  14. 2023Domain Generalization Using Large Pretrained Models with Mixture-of-Adapterscited by 11
  15. 2024Semi-Supervised Video Desnowing Network via Temporal Decoupling Experts and Distribution-Driven Contrastive Regularizationcited by 11
  16. 2022Fine-grain Inference on Out-of-Distribution Data with Hierarchical Classificationcited by 10
  17. 2023Revisiting Single-gated Mixtures of Expertscited by 10
  18. 2025HAMoBE: Hierarchical and Adaptive Mixture of Biometric Experts for Video-based Person ReIDcited by 10
  19. 2022A Survey of Neural Treescited by 7
  20. 2024Multimodal Clinical Trial Outcome Prediction with Large Language Modelscited by 7
  21. 2024LLMBind: A Unified Modality-Task Integration Frameworkcited by 6
  22. 2024A Gated Residual Kolmogorov-Arnold Networks for Mixtures of Expertscited by 6
  23. 2025Advancing MoE Efficiency: A Collaboration-Constrained Routing (C2R) Strategy for Better Expert Parallelism Designcited by 6
  24. 2025Extracting Uncertainty Estimates from Mixtures of Experts for Semantic Segmentationcited by 6
  25. 2021MMF: Multi-Task Multi-Structure Fusion for Hierarchical Image Classificationcited by 5
  26. 20223M: Multi-loss, Multi-path and Multi-level Neural Networks for speech recognitioncited by 5
  27. 2024Towards Adversarial Robustness of Model-Level Mixture-of-Experts Architectures for Semantic Segmentationcited by 5
  28. 2025A Survey on Dynamic Neural Networks: from Computer Vision to Multi-modal Sensor Fusioncited by 5
  29. 2026On the Holistic Approach for Detecting Human Image Forgerycited by 5
  30. 2023Taxonomic Class Incremental Learningcited by 3
  31. 2024Unraveling Complex Data Diversity in Underwater Acoustic Target Recognition through Convolution-based Mixture of Expertscited by 3
  32. 2021TAL: Two-stream Adaptive Learning for Generalizable Person Re-identificationcited by 2
  33. 2024big.LITTLE Vision Transformer for Efficient Visual Recognitioncited by 2
  34. 2021SONG: Self-Organizing Neural Graphscited by 1
  35. 2021A Hierarchical Assessment of Adversarial Severitycited by 1
  36. 2026Systematic Exploration of 4-Expert Heterogeneous Mixture-of-Experts via Automated Pipeline Searchcited by 1
  37. 2023LEURN: Learning Explainable Univariate Rules with Neural Networkscited by 0
  38. 2025SHMoAReg: Spark Deformable Image Registration via Spatial Heterogeneous Mixture of Experts and Attention Headscited by 0

50 most cited of 54 All 54 (JSON)