mlsanityData to 14 Oct 2026

DAMEX: Dataset-aware Mixture-of-Experts for visual understanding of mixture-of-datasets

arXiv 2311.04894 ↗First posted 8 Nov 2023cs.CV cs.AI cs.LGcited by 23

On the map: Mixture-of-experts → Vision mixture-of-experts

Abstract (arXiv)

Construction of a universal detector poses a crucial question: How can we most effectively train a model on a large mixture of datasets? The answer lies in learning dataset-specific features and ensembling their knowledge but do all this in a single model. Previous methods achieve this by having separate detection heads on a common backbone but that results in a significant increase in parameters. In this work, we present Mixture-of-Experts as a solution, highlighting that MoEs are much more than a scalability tool. We propose Dataset-Aware Mixture-of-Experts, DAMEX where we train the experts to become an `expert' of a dataset by learning to route each dataset tokens to its mapped expert. Experiments on Universal Object-Detection Benchmark show that we outperform the existing state-of-the-art by average +10.2 AP score and improve over our non-MoE baseline by average +2.0 AP score. We also observe consistent gains while mixing datasets with (1) limited availability, (2) disparate domains and (3) divergent label sets. Further, we qualitatively show that DAMEX is robust against expert representation collapse.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 38 of its 47 references that are in the map. First posted on arXiv 8 Nov 2023.

Builds on

38 of 47 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Towards Understanding Mixture of Experts in Deep Learning4 Aug
  2. 2021Scaling Vision with Sparse Mixture of Experts10 Jun

Other references · 36

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  3. 2023Segment Anythingcited by 8,234
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  5. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  6. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  7. 2019Do ImageNet Classifiers Generalize to ImageNet?cited by 1,515
  8. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  9. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
  10. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,071
  11. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  12. 2021Florence: A New Foundation Model for Computer Visioncited by 804
24 more
  1. 2017DOTA: A Large-scale Dataset for Object Detection in Aerial Imagescited by 623
  2. 2022DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETRcited by 573
  3. 2022DN-DETR: Accelerate DETR Training by Introducing Query DeNoisingcited by 488
  4. 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 421
  5. 2015WIDER FACE: A Face Detection Benchmarkcited by 298
  6. 2020VarifocalNet: An IoU-aware Dense Object Detectorcited by 231
  7. 2018Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptationcited by 218
  8. 2022Omnivore: A Single Model for Many Visual Modalitiescited by 215
  9. 2020Scaled-YOLOv4: Scaling Cross Stage Partial Networkcited by 211
  10. 2022Tutel: Adaptive Mixture-of-Experts at Scalecited by 201
  11. 2018Why do deep convolutional networks generalize so poorly to small image transformations?cited by 198
  12. 2021Dynamic Head: Unifying Object Detection Heads with Attentionscited by 187
  13. 2022On the Representation Collapse of Sparse Mixture of Expertscited by 157
  14. 2021Simple multi-dataset detectioncited by 127
  15. 2019Hierarchical Deep Stereo Matching on High-resolution Imagescited by 126
  16. 2021MSeg: A Composite Dataset for Multi-domain Semantic Segmentationcited by 122
  17. 2020Generalized Focal Loss V2: Learning Reliable Localization Quality Estimation for Dense Object Detectioncited by 104
  18. 2019Towards Universal Object Detection by Domain Attentioncited by 89
  19. 2020Object Detection with a Unified Label Space from Multiple Datasetscited by 65
  20. 2021Towards General Purpose Vision Systemscited by 46
  21. 2017Deep Lesion Graphs in the Wild: Relationship Learning and Organization of Significant Radiology Image Findings in a Diverse Large-scale Lesion Databasecited by 42
  22. 2021M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretrainingcited by 31
  23. 2016Multiview RGB-D Dataset for Object Instance Detectioncited by 26
  24. 2020Universal-RCNN: Universal Object Detector via Transferable Graph R-CNNcited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 7; 2026: 11.

23 papers.

Sort:

By area:

  1. 2024SM3Det: A Unified Model for Multi-Modal Remote Sensing Object Detectioncited by 22
  2. 2024XTrack: Multimodal Training Boosts RGB-X Video Object Trackerscited by 21
  3. 2024PLeaS -- Merging Models with Permutations and Least Squarescited by 16
  4. 2024Routing Experts: Learning to Route Dynamic Experts in Multi-modal Large Language Modelscited by 14
  5. 2026Sparse Models, Sparse Safety: Unsafe Routes in Mixture-of-Experts LLMscited by 9
  6. 2025Fast-Slow-Thinking: Complex Task Solving with Large Language Modelscited by 7
  7. 2025Long-Tailed Distribution-Aware Router For Mixture-of-Experts in Large Vision-Language Modelcited by 7
  8. 2025GEMINUS: Dual-aware Global and Scene-Adaptive Mixture-of-Experts for End-to-End Autonomous Drivingcited by 5
  9. 2025MAPEX: Modality-Aware Pruning of Experts for Remote Sensing Foundation Modelscited by 3
  10. 2026HI-MoE: Hierarchical Instance-Conditioned Mixture-of-Experts for Object Detectioncited by 3
  11. 2026UniRoute: Unified Routing Mixture-of-Experts for Modality-Adaptive Remote Sensing Change Detectioncited by 2
  12. 2024Multi-Stage Multi-Modal Pre-Training for Automatic Speech Recognitioncited by 1
11 more
  1. 2025COME: Dual Structure-Semantic Learning with Collaborative MoE for Universal Lesion Detection Across Heterogeneous Ultrasound Datasetscited by 1
  2. 2025Tokenize Once, Recommend Anywhere: Unified Item Tokenization for Multi-domain LLM-based Recommendationcited by 1
  3. 2025Expert Switching for Robust AAV Landing: A Dual-Detector Framework in Simulationcited by 0
  4. 2026Understanding Task Aggregation for Generalizable Ultrasound Foundation Modelscited by 0
  5. 2026Bidirectional Learning of Facial Action Units and Expressions via Structured Semantic Mapping across Heterogeneous Datasetscited by 0
  6. 2026Multi-Domain Learning with Global Expert Mappingcited by 0
  7. 2026HMR-Net: Hierarchical Modular Routing for Cross-Domain Object Detection in Aerial Imagescited by 0
  8. 2026Teacher-Guided Routing for Sparse Vision Mixture-of-Expertscited by 0
  9. 2026SARIF: Segment Anything for Robust Image Forensicscited by 0
  10. 2026FaceMoE: Mixture of Experts for Low-Resolution Face Recognitioncited by 0
  11. 2026RouteRec: Behavior-Guided Sparse Routing for Sequential Recommendationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Domain-Specialized Object Detection via Model-Level Mixtures of Experts
  2. 2024Mixture of Experts in Image Classification: What's the Sweet Spot?
  3. 2024Multilinear Mixture of Experts: Scalable Expert Specialization through Factorization
  4. 2025Efficient Data Driven Mixture-of-Expert Extraction from Trained Networks
  5. 2026Mixture-of-Experts Models in Vision: Routing, Optimization, and Generalization
  6. 2023MoCaE: Mixture of Calibrated Experts Significantly Improves Object Detection
  7. 2020Biased Mixtures Of Experts: Enabling Computer Vision Inference Under Data Transfer Limitations
  8. 2025MixtureKit: A General Framework for Composing, Training, and Visualizing Mixture-of-Experts Models