mlsanityData to 7 Oct 2026

Learning Hierarchical Attention for Weakly-supervised Chest X-Ray Abnormality Localization and Diagnosis

arXiv 2112.12349 ↗First posted 23 Dec 2021cs.CVcited by 24

Not on the map. Why?

Abstract (arXiv)

We consider the problem of abnormality localization for clinical applications. While deep learning has driven much recent progress in medical imaging, many clinical challenges are not fully addressed, limiting its broader usage. While recent methods report high diagnostic accuracies, physicians have concerns trusting these algorithm results for diagnostic decision-making purposes because of a general lack of algorithm decision reasoning and interpretability. One potential way to address this problem is to further train these models to localize abnormalities in addition to just classifying them. However, doing this accurately will require a large amount of disease localization annotations by clinical experts, a task that is prohibitively expensive to accomplish for most applications. In this work, we take a step towards addressing these issues by means of a new attention-driven weakly supervised algorithm comprising a hierarchical attention mining framework that unifies activation- and gradient-based visual attention in a holistic manner. Our key algorithmic innovations include the design of explicit ordinal attention constraints, enabling principled model training in a weakly-supervised fashion, while also facilitating the generation of visual-attention-driven model explanations by means of localization cues. On two large-scale chest X-ray datasets (NIH ChestX-ray14 and CheXpert), we demonstrate significant localization performance improvements over the current state of the art while also achieving competitive classification performance. Our code is available on https://github.com/oyxhust/HAM.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 26 of its 44 references that are in the map. First posted on arXiv 23 Dec 2021.

Builds on

26 of 44 references.

References · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2017Squeeze-and-Excitation Networkscited by 4,763
  4. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  5. 2017Non-local Neural Networkscited by 2,290
  6. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  7. 2016Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfercited by 1,059
  8. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  9. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 936
  10. 2017Deep Learning for Medical Image Analysiscited by 399
  11. 2019GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyondcited by 331
  12. 2017Dilated Residual Networkscited by 293
14 more
  1. 2017Understanding Convolution for Semantic Segmentationcited by 168
  2. 2018Tell Me Where to Look: Guided Attention Inference Networkcited by 168
  3. 2018Attention Branch Network: Learning of Attention Mechanism for Visual Explanationcited by 108
  4. 2018Comparison of Deep Learning Approaches for Multi-Label Chest X-Ray Classificationcited by 57
  5. 2020Interpreting Chest X-rays via CNNs that Exploit Hierarchical Disease Dependencies and Uncertainty Labelscited by 56
  6. 2017Thoracic Disease Identification and Localization with Limited Supervisioncited by 55
  7. 2018Diagnose like a Radiologist: Attention Guided Convolutional Neural Network for Thorax Disease Classificationcited by 36
  8. 2018Learning to recognize Abnormalities in Chest X-Rays with Location-Aware Dense Networkscited by 33
  9. 2018Weakly Supervised Medical Diagnosis and Localization from Multiple Resolutionscited by 22
  10. 2020Multi-label Thoracic Disease Image Classification with Cross-Attention Networkscited by 22
  11. 2018Weakly Supervised Deep Learning for Thoracic Disease Classification and Localization on Chest X-rayscited by 21
  12. 2018Sharpen Focus: Learning with Attention Separability and Consistencycited by 18
  13. 2015ProNet: Learning to Propose Object-specific Boxes for Cascaded Neural Networkscited by 8
  14. 2019Weakly Supervised Universal Fracture Detection in Pelvic X-rayscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 10; 2024: 3; 2025: 4; 2026: 3.

24 papers.

Sort:

By area:

  1. 2023CLIP in Medical Imaging: A Surveycited by 40
  2. 2022Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosiscited by 38
  3. 2022Improving deep neural network generalization and robustness to background bias via layer-wise relevance propagation optimizationcited by 25
  4. 2023MProtoNet: A Case-Based Interpretable Model for Brain Tumor Classification with 3D Multi-parametric Magnetic Resonance Imagingcited by 11
  5. 2023I-AI: A Controllable & Interpretable AI System for Decoding Radiologists' Intense Focus for Accurate CXR Diagnosescited by 10
  6. 2023Chest X-ray Image Classification: A Causal Perspectivecited by 8
  7. 2025WeakMedSAM: Weakly-Supervised Medical Image Segmentation via SAM with Sub-Class Exploration and Prompt Affinity Miningcited by 8
  8. 2024Gaze-directed Vision GNN for Mitigating Shortcut Learning in Medical Imagecited by 7
  9. 2022ThoraX-PriorNet: A Novel Attention-Based Architecture Using Anatomical Prior Probability Maps for Thoracic Disease Classificationcited by 5
  10. 2023Expert Uncertainty and Severity Aware Chest X-Ray Classification by Multi-Relationship Graph Learningcited by 4
  11. 2024Cross- and Intra-image Prototypical Learning for Multi-label Disease Diagnosis and Interpretationcited by 4
  12. 2023Deep Reinforcement Learning Framework for Thoracic Diseases Classification via Prior Knowledge Guidancecited by 3
12 more
  1. 2023DisAsymNet: Disentanglement of Asymmetrical Abnormality on Bilateral Mammograms using Self-adversarial Learningcited by 3
  2. 2023Learning to Generalize towards Unseen Domains via a Content-Aware Style Invariant Model for Disease Detection from Chest X-rayscited by 2
  3. 2023Instrumental Variable Learning for Chest X-ray Classificationcited by 2
  4. 2024Faster ISNet for Background Bias Mitigation on Deep Neural Networkscited by 1
  5. 2025CLARiTy: A Vision Transformer for Multi-Label Classification and Weakly-Supervised Localization of Chest X-ray Pathologiescited by 1
  6. 2026Can Agents Distinguish Visually Hard-to-Separate Diseases in a Zero-Shot Setting? A Pilot Studycited by 1
  7. 2022Rethinking Saliency Map: An Context-aware Perturbation Method to Explain EEG-based Deep Learning Modelcited by 0
  8. 2023Applications of Sequential Learning for Medical Image Classificationcited by 0
  9. 2025Artificially Generated Visual Scanpath Improves Multi-label Thoracic Disease Classification in Chest X-Ray Imagescited by 0
  10. 2025A Hierarchical Slice Attention Network for Appendicitis Classification in 3D CT Scanscited by 0
  11. 2026Inhibited Self-Attention: Sharpening Focus in Vision Transformerscited by 0
  12. 2026Exponential Pixelating Integral transform with dual fractal features for enhanced chest X-ray abnormality detectioncited by 0