mlsanityData to 14 Oct 2026

Creation and Validation of a Chest X-Ray Dataset with Eye-tracking and Report Dictation for AI Development

arXiv 2009.07386 ↗First posted 15 Sep 2020cs.CVcited by 56

Not on the map. Why?

Abstract (arXiv)

We developed a rich dataset of Chest X-Ray (CXR) images to assist investigators in artificial intelligence. The data were collected using an eye tracking system while a radiologist reviewed and reported on 1,083 CXR images. The dataset contains the following aligned data: CXR image, transcribed radiology report text, radiologist's dictation audio and eye gaze coordinates data. We hope this dataset can contribute to various areas of research particularly towards explainable and multimodal deep learning / machine learning methods. Furthermore, investigators in disease classification and localization, automated radiology report generation, and human-machine interaction can benefit from these data. We report deep learning experiments that utilize the attention maps produced by eye gaze dataset to show the potential utility of this data.

Key facts. Cited by 56 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Sep 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 15; 2023: 11; 2024: 14; 2025: 5; 2026: 7.

56 papers.

Sort:

By area:

  1. 2023CLIP in Medical Imaging: A Surveycited by 40
  2. 2022Follow My Eye: Using Gaze to Supervise Computer-Aided Diagnosiscited by 38
  3. 2021REFLACX, a dataset of reports and eye-tracking data for localization of abnormalities in chest x-rayscited by 34
  4. 2022Aligning Eyes between Humans and Deep Neural Network through Interactive Attention Alignmentcited by 27
  5. 2023GazeGNN: A Gaze-Guided Graph Neural Network for Chest X-ray Classificationcited by 25
  6. 2021Human Attention in Fine-grained Classificationcited by 21
  7. 2022Eye-gaze-guided Vision Transformer for Rectifying Shortcut Learningcited by 20
  8. 2022RadioTransformer: A Cascaded Global-Focal Transformer for Visual Attention-guided Disease Classificationcited by 19
  9. 2022CheXRelNet: An Anatomy-Aware Model for Tracking Longitudinal Relationships between Chest X-Rayscited by 19
  10. 2024Improving Medical Multi-modal Contrastive Learning with Expert Annotationscited by 16
  11. 2024Eye-gaze Guided Multi-modal Alignment for Medical Representation Learningcited by 16
  12. 2022Rectify ViT Shortcut Learning by Visual Saliencycited by 14
38 more
  1. 2022Probabilistic Integration of Object Level Annotations in Chest X-ray Classificationcited by 10
  2. 2023I-AI: A Controllable & Interpretable AI System for Decoding Radiologists' Intense Focus for Accurate CXR Diagnosescited by 10
  3. 2024Trends, Applications, and Challenges in Human Attention Modellingcited by 9
  4. 2021Comparing radiologists' gaze and saliency maps generated by interpretability methods for chest x-rayscited by 7
  5. 2023Mining Gaze for Contrastive Learning toward Computer-Assisted Diagnosiscited by 7
  6. 2024Gaze-directed Vision GNN for Mitigating Shortcut Learning in Medical Imagecited by 7
  7. 2021Leveraging Human Selective Attention for Medical Image Analysis with Limited Training Datacited by 5
  8. 2024Gaze-Guided Graph Neural Network for Action Anticipation Conditioned on Intentioncited by 5
  9. 2022Gaze-Guided Class Activation Mapping: Leveraging Human Attention for Network Attention in Chest X-rays Classificationcited by 4
  10. 2022Multi-task UNet: Jointly Boosting Saliency Prediction and Disease Classification on Chest X-ray Imagescited by 4
  11. 2023Do humans and Convolutional Neural Networks attend to similar areas during scene classification: Effects of task and image typecited by 4
  12. 2023MENTOR: Human Perception-Guided Pretraining for Increased Generalizationcited by 4
  13. 2023GlanceSeg: Real-time microaneurysm lesion segmentation with gaze-map-guided foundation model for early detection of diabetic retinopathycited by 4
  14. 2024RadGazeGen: Radiomics and Gaze-guided Medical Image Generation using Diffusion Modelscited by 4
  15. 2024Gaze-Assisted Medical Image Segmentationcited by 4
  16. 2022Improving Disease Classification Performance and Explainability of Deep Learning Models in Radiology with Heatmap Generatorscited by 3
  17. 2022Localization supervision of chest x-ray classifiers using label-specific eye-tracking annotationcited by 3
  18. 2024A Transformer-Based Model for the Prediction of Human Gaze Behavior on Videoscited by 3
  19. 2023Integrating Eye-Gaze Data into CXR DL Approaches: A Preliminary studycited by 2
  20. 2024When Eye-Tracking Meets Machine Learning: A Systematic Review on Applications in Medical Image Analysiscited by 2
  21. 2026Thinking with Gaze: Sequential Eye-Tracking as Visual Reasoning Supervision for Medical VLMscited by 2
  22. 2026CheXthought: A global multimodal dataset of clinical chain-of-thought reasoning and visual attention for chest X-ray interpretationcited by 2
  23. 2026Vision-language models for chest radiography do not always need the imagecited by 2
  24. 2022Improving X-ray Diagnostics through Eye-Tracking and XRcited by 1
  25. 2022Where and What: Driver Attention-based Object Detectioncited by 1
  26. 2022Using Multi-modal Data for Improving Generalizability and Explainability of Disease Classification in Radiologycited by 1
  27. 2023Discrimination of Radiologists Utilizing Eye-Tracking Technology and Machine Learning: A Case Studycited by 1
  28. 2024Vision-language models for decoding provider attention during neonatal resuscitationcited by 1
  29. 2024Tumor Location-weighted MRI-Report Contrastive Learning: A Framework for Improving the Explainability of Pediatric Brain Tumor Diagnosiscited by 1
  30. 2024GazeSearch: Radiology Findings Search Benchmarkcited by 1
  31. 2025Eyes on the Image: Gaze Supervised Multimodal Learning for Chest X-ray Diagnosis and Report Generationcited by 1
  32. 2025MedPAO: A Protocol-Driven Agent for Structuring Medical Reportscited by 1
  33. 2026Following the Diagnostic Trace: Visual Cognition-guided Cooperative Network for Chest X-Ray Diagnosiscited by 1
  34. 2022ResNet Structure Simplification with the Convolutional Kernel Redundancy Measurecited by 0
  35. 2023Eye tracking guided deep multiple instance learning with dual cross-attention for fundus disease detectioncited by 0
  36. 2023Multi-Scale Feature Fusion using Parallel-Attention Block for COVID-19 Chest X-ray Diagnosiscited by 0
  37. 2024Supporting Mitosis Detection AI Training with Inter-Observer Eye-Gaze Consistenciescited by 0
  38. 2024Decoding Radiologists' Intentions: A Novel System for Accurate Region Identification in Chest X-ray Image Analysiscited by 0

50 most cited of 56 All 56 (JSON)