mlsanityData to 7 Oct 2026

HMAR: Hierarchical Modality-Aware Expert and Dynamic Routing Medical Image Retrieval Architecture

arXiv 2603.16679 ↗First posted 17 Mar 2026cs.CVcited by 0

On the map: Mixture-of-experts → Medical and biomedical MoE

Abstract (arXiv)

Medical image retrieval (MIR) is a critical component of computer-aided diagnosis, yet existing systems suffer from three persistent limitations: uniform feature encoding that fails to account for the varying clinical importance of anatomical structures, ambiguous similarity metrics based on coarse classification labels, and an exclusive focus on global image similarity that cannot meet the clinical demand for fine-grained region-specific retrieval. We propose HMAR (Hierarchical Modality-Aware Expert and Dynamic Routing), an adaptive retrieval framework built on a Mixture-of-Experts (MoE) architecture. HMAR employs a dual-expert mechanism: Expert0 extracts global features for holistic similarity matching, while Expert1 learns position-invariant local representations for precise lesion-region retrieval. A two-stage contrastive learning strategy eliminates the need for expensive bounding-box annotations, and a sliding-window matching algorithm enables dense local comparison at inference time. Hash codes are generated via Kolmogorov-Arnold Network (KAN) layers for efficient Hamming-distance search. Experiments on the RadioImageNet-CT dataset (16 clinical patterns, 29,903 images) show that HMAR achieves mean Average Precision (mAP) of 0.711 and 0.724 for 64-bit and 128-bit hash codes, improving over the state-of-the-art ACIR method by 0.7% and 1.1%, respectively.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical and biomedical MoE, one of 12 directions in Mixture-of-experts. Builds on 10 of its 14 references that are in the map. First posted on arXiv 17 Mar 2026.

Builds on

10 of 14 references.

References · 10

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  5. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  6. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  7. 2024KAN: Kolmogorov-Arnold Networkscited by 634
  8. 2017HashNet: Deep Learning to Hash by Continuationcited by 121
  9. 2025Revisiting Medical Image Retrieval via Knowledge Consolidationcited by 3
  10. 2020A survey on deep hashing for image retrievalcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Dual-Adaptive SAM3: Hierarchical Routing over Low-Rank Expert Layers for Parameter-Efficient Medical Image Segmentation
  2. 2026A Vision-language Framework for Comparative Reasoning in Radiology
  3. 2025RAD: Towards Trustworthy Retrieval-Augmented Multi-modal Clinical Diagnosis
  4. 2025Enhancing Medical Cross-Modal Hashing Retrieval using Dropout-Voting Mixture-of-Experts Fusion
  5. 2025REN: Anatomically-Informed Mixture-of-Experts for Interstitial Lung Disease Diagnosis
  6. 2025Adapting Foundation Vision-Language Models to Medical Diagnosis via Query-Driven Expert Bridging
  7. 2026Generalist-Specialist Mixture-of-Experts for Rare Pathology Detection in Multimodal Imaging
  8. 2026Sparse Spectral LoRA: Routed Experts for Medical VLMs