mlsanityData to 14 Oct 2026

A Retrospective Systematic Study on Hierarchical Sparse Query Transformer-assisted Ultrasound Screening for Early Hepatocellular Carcinoma

arXiv 2502.03772 ↗First posted 6 Feb 2025cs.CV cs.AIcited by 1

On the map: Mixture-of-experts → Medical and biomedical MoE

Abstract (arXiv)

Hepatocellular carcinoma (HCC), ranking as the third leading cause of cancer-related mortality worldwide, demands urgent improvements in early detection to enhance patient survival. While ultrasound remains the preferred screening modality due to its cost-effectiveness and real-time capabilities, its sensitivity (59%-78%) heavily relies on radiologists' expertise, leading to inconsistent diagnostic outcomes and operational inefficiencies. Recent advancements in AI technology offer promising solutions to bridge this gap. This study introduces the Hierarchical Sparse Query Transformer (HSQformer), a novel hybrid architecture that synergizes CNNs' local feature extraction with Vision Transformers' global contextual awareness through latent space representation and sparse learning. By dynamically activating task-specific experts via a Mixture-of-Experts (MoE) framework, HSQformer achieves hierarchical feature integration without structural redundancy. Evaluated across three clinical scenarios: single-center, multi-center, and high-risk patient cohorts, HSQformer outperforms state-of-the-art models (e.g., 95.38% AUC in multi-center testing) and matches senior radiologists' diagnostic accuracy while significantly surpassing junior counterparts. These results highlight the potential of AI-assisted tools to standardize HCC screening, reduce dependency on human expertise, and improve early diagnosis rates. The full code is available at https://github.com/Asunatan/HSQformer.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Medical and biomedical MoE, one of 12 directions in Mixture-of-experts. Builds on 43 of its 51 references that are in the map. First posted on arXiv 6 Feb 2025.

Builds on

43 of 51 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2016Densely Connected Convolutional Networkscited by 6,720
  7. 2017Squeeze-and-Excitation Networkscited by 4,789
  8. 2022A ConvNet for the 2020scited by 3,579
  9. 2018YOLOv3: An Incremental Improvementcited by 2,659
  10. 2018CBAM: Convolutional Block Attention Modulecited by 2,477
  11. 2017Deformable Convolutional Networkscited by 1,488
  12. 2015Multi-Scale Context Aggregation by Dilated Convolutionscited by 1,133
31 more
  1. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  2. 2021CvT: Introducing Convolutions to Vision Transformerscited by 947
  3. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  4. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 751
  5. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 670
  6. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 642
  7. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 527
  8. 2021MetaFormer Is Actually What You Need for Visioncited by 524
  9. 2021RepVGG: Making VGG-style ConvNets Great Againcited by 522
  10. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 482
  11. 2021Medical Transformer: Gated Axial-Attention for Medical Image Segmentationcited by 320
  12. 2023Hiera: A Hierarchical Vision Transformer without the Bells-and-Whistlescited by 290
  13. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 288
  14. 2021Escaping the Big Data Paradigm with Compact Transformerscited by 264
  15. 2024A Survey on Mixture of Experts in Large Language Modelscited by 237
  16. 2022Focal Modulation Networkscited by 186
  17. 2019ACNet: Strengthening the Kernel Skeletons for Powerful CNN via Asymmetric Convolution Blockscited by 178
  18. 2022DaViT: Dual Attention Vision Transformerscited by 175
  19. 2021VOLO: Vision Outlooker for Visual Recognitioncited by 165
  20. 2022More ConvNets in the 2020s: Scaling up Kernels Beyond 51x51 using Sparsitycited by 142
  21. 2023RepViT: Revisiting Mobile CNN From ViT Perspectivecited by 123
  22. 2024LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-trainingcited by 123
  23. 2021MPViT: Multi-Path Vision Transformer for Dense Predictioncited by 122
  24. 2021Involution: Inverting the Inherence of Convolution for Visual Recognitioncited by 99
  25. 2021Coordination Among Neural Modules Through a Shared Global Workspacecited by 85
  26. 2021P2T: Pyramid Pooling Transformer for Scene Understandingcited by 84
  27. 2023Slide-Transformer: Hierarchical Vision Transformer with Local Self-Attentioncited by 38
  28. 2024PeLK: Parameter-efficient Large Kernel ConvNets with Peripheral Convolutioncited by 28
  29. 2021Learned Queries for Efficient Local Attentioncited by 27
  30. 2024LLaMA-MoE v2: Exploring Sparsity of LLaMA from Perspective of Mixture-of-Experts with Post-Trainingcited by 26
  31. 2024Exploring the Synergies of Hybrid CNNs and ViTs Architectures for Computer Vision: A surveycited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understandingcited by 6

Similar, not cited

Close in wording, no citation either way

  1. 2025EchoVLM: Dynamic Mixture-of-Experts Vision-Language Model for Universal Ultrasound Intelligence
  2. 2026Hepato-LLaVA: An Expert MLLM with Sparse Topo-Pack Attention for Hepatocellular Pathology Analysis on Whole Slide Images
  3. 2026Look-Closer-Then-Diagnose: Confidence-Aware Ultrasound VQA via Active Zooming
  4. 2026BioFact-MoE: Biologically Factorized Mixture of Experts for Vision-Language Prognostic Modeling in Hepatocellular Carcinoma
  5. 2024MambaEviScrib: Mamba and Evidence-Guided Consistency Enhance CNN Robustness for Scribble-Based Weakly Supervised Ultrasound Image Segmentation
  6. 2026UltrasoundAgents: Hierarchical Multi-Agent Evidence-Chain Reasoning for Breast Ultrasound Diagnosis
  7. 2025SPARS: Self-Play Adversarial Reinforcement Learning for Segmentation of Liver Tumours
  8. 2026HMAR: Hierarchical Modality-Aware Expert and Dynamic Routing Medical Image Retrieval Architecture