# Hybrid Explanation-Guided Learning for Transformer-Based Chest X-Ray Diagnosis

arXiv: https://arxiv.org/abs/2510.12704 · HTML: https://mlsanity.com/paper/2510.12704/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 19 references that are in the map.
- First posted on arXiv 14 Oct 2025.

## Builds on (most cited first, 16 of 16)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseases](https://mlsanity.com/paper/1705.02315/) (2017)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [RadGraph: Extracting Clinical Entities and Relations from Radiology Reports](https://mlsanity.com/paper/2106.14463/) (2021)
- [Tell Me Where to Look: Guided Attention Inference Network](https://mlsanity.com/paper/1802.10171/) (2018)
- [Reading Race: AI Recognises Patient's Racial Identity In Medical Images](https://mlsanity.com/paper/2107.10356/) (2021)
- [Interpretations are useful: penalizing explanations to align neural networks with prior knowledge](https://mlsanity.com/paper/1909.13584/) (2019)
- [Making deep neural networks right for the right scientific reasons by interacting with their explanations](https://mlsanity.com/paper/2001.05371/) (2020)
- [Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Images](https://mlsanity.com/paper/2302.14042/) (2023)
- [Learn Your Reference Model for Real Good Alignment](https://mlsanity.com/paper/2404.09656/) (2024)
- [ChestX-Det10: Chest X-ray Dataset on Detection of Thoracic Abnormalities](https://mlsanity.com/paper/2006.10550/) (2020)
- [Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosis](https://mlsanity.com/paper/2204.02976/) (2022)
- [Going Beyond XAI: A Systematic Survey for Explanation-Guided Learning](https://mlsanity.com/paper/2212.03954/) (2022)
- [Gaze-directed Vision GNN for Mitigating Shortcut Learning in Medical Image](https://mlsanity.com/paper/2406.14050/) (2024)
- [DWARF: Disease-weighted network for attention map refinement](https://mlsanity.com/paper/2406.17032/) (2024)

## Built on by (most cited first, 1 of 1)

- [HarMoE: Multi-Source Chest Radiograph Pretraining with Dataset-Disentangled Experts](https://mlsanity.com/paper/2608.02252/) (2026)

## Abstract (arXiv)

Transformer-based deep learning models have demonstrated exceptional performance in medical imaging by leveraging attention mechanisms for feature representation and interpretability. However, these models are prone to learning spurious correlations, leading to biases and limited generalization. While human-AI attention alignment can mitigate these issues, it often depends on costly manual supervision. In this work, we propose a Hybrid Explanation-Guided Learning (H-EGL) framework that combines self-supervised and human-guided constraints to enhance attention alignment and improve generalization. The self-supervised component of H-EGL leverages class-distinctive attention without relying on restrictive priors, promoting robustness and flexibility. We validate our approach on chest X-ray classification using the Vision Transformer (ViT), where H-EGL outperforms two state-of-the-art Explanation-Guided Learning (EGL) methods, demonstrating superior classification accuracy and generalization capability. Additionally, it produces attention maps that are better aligned with human expertise.