mlsanityData to 7 Oct 2026

Generalized Radiograph Representation Learning via Cross-supervision between Images and Free-text Radiology Reports

arXiv 2111.03452 ↗First posted 4 Nov 2021eess.IV cs.CV cs.LGcited by 80

Not on the map. Why?

Cited by 20 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Pre-training lays the foundation for recent successes in radiograph analysis supported by deep learning. It learns transferable image representations by conducting large-scale fully-supervised or self-supervised learning on a source domain. However, supervised pre-training requires a complex and labor intensive two-stage human-assisted annotation process while self-supervised learning cannot compete with the supervised paradigm. To tackle these issues, we propose a cross-supervised methodology named REviewing FreE-text Reports for Supervision (REFERS), which acquires free supervision signals from original radiology reports accompanying the radiographs. The proposed approach employs a vision transformer and is designed to learn joint representations from multiple views within every patient study. REFERS outperforms its transfer learning and self-supervised learning counterparts on 4 well-known X-ray datasets under extremely limited supervision. Moreover, REFERS even surpasses methods based on a source domain of radiographs with human-assisted structured labels. Thus REFERS has the potential to replace canonical pre-training methodologies.

Key facts. Cited by 80 papers in the map (data through 7 Oct 2026). Builds on 24 of its 45 references that are in the map. First posted on arXiv 4 Nov 2021.

Builds on

24 of 45 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  7. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2016Densely Connected Convolutional Networkscited by 6,691
  10. 2016Layer Normalizationcited by 5,579
  11. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  12. 2014Going Deeper with Convolutionscited by 4,231
12 more
  1. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  2. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,410
  3. 2014How transferable are features in deep neural networks?cited by 1,216
  4. 2017Mixed Precision Trainingcited by 1,135
  5. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  6. 2014cuDNN: Efficient Primitives for Deep Learningcited by 240
  7. 2020VinDr-CXR: An open dataset of chest X-rays with radiologist's annotationscited by 236
  8. 2019Scaling and Benchmarking Self-Supervised Visual Representation Learningcited by 201
  9. 2018TieNet: Text-Image Embedding Network for Common Thorax Disease Classification and Reporting in Chest X-rayscited by 139
  10. 2020COVID-19 Image Data Collection: Prospective Predictions Are the Futurecited by 100
  11. 2020Comparing to Learn: Surpassing ImageNet Pretraining on Radiographs By Comparing Image Representationscited by 69
  12. 2021Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learningcited by 68

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 8; 2023: 24; 2024: 23; 2025: 16; 2026: 9.

80 papers.

Sort:

By area:

  1. 2022Multi-Granularity Cross-modal Alignment for Generalized Medical Visual Representation Learningcited by 159
  2. 2023On the Challenges and Perspectives of Foundation Models for Medical Image Analysiscited by 135
  3. 2023Knowledge-enhanced Visual-Language Pre-training on Chest Radiology Imagescited by 128
  4. 2023Improved Distribution Matching for Dataset Condensationcited by 124
  5. 2023CXR-CLIP: Toward Large Scale Chest X-ray Language-Image Pre-trainingcited by 81
  6. 2023Advancing Radiograph Representation Learning with Masked Record Modelingcited by 75
  7. 2022A Survey on Graph Neural Networks and Graph Transformers in Computer Vision: A Task-Oriented Perspectivecited by 56
  8. 2023Multimodal LLMs for health grounded in individual-specific datacited by 48
  9. 2023A Transformer-based representation-learning model with unified processing of multimodal input for clinical diagnosticscited by 42
  10. 2023Overcoming Data Scarcity in Biomedical Imaging with a Foundational Multi-Task Modelcited by 40
  11. 2023CLIP in Medical Imaging: A Surveycited by 40
  12. 2023Enhancing Representation in Radiography-Reports Foundation Model: A Granular Alignment Algorithm Using Masked Contrastive Learningcited by 34
38 more
  1. 2023Medical Vision Language Pretraining: A surveycited by 24
  2. 2023ScribbleVC: Scribble-supervised Medical Image Segmentation with Vision-Class Embeddingcited by 23
  3. 2022Neighborhood Collective Estimation for Noisy Label Identification and Correctioncited by 22
  4. 2023Multi-task Paired Masking with Alignment Modeling for Medical Vision-Language Pre-trainingcited by 22
  5. 2024Advancing human-centric AI for robust X-ray analysis through holistic self-supervised learningcited by 18
  6. 2024Medical Multimodal Foundation Models in Clinical Diagnosis and Treatment: Applications, Challenges, and Future Directionscited by 17
  7. 2022Centrality and Consistency: Two-Stage Clean Samples Identification for Learning with Instance-Dependent Noisy Labelscited by 16
  8. 2022A survey on attention mechanisms for medical applications: are we moving towards better algorithms?cited by 15
  9. 2023Label-Efficient Deep Learning in Medical Image Analysis: Challenges and Future Directionscited by 14
  10. 2023A scoping review on multimodal deep learning in biomedical images and textscited by 13
  11. 2024Unlocking the Power of Spatial and Temporal Information in Medical Multimodal Pre-trainingcited by 13
  12. 2022Anatomy-aware Self-supervised Learning for Anomaly Detection in Chest Radiographscited by 12
  13. 2023Is attention all you need in medical image analysis? A reviewcited by 12
  14. 2025MedUnifier: Unifying Vision-and-Language Pre-training on Medical Data with Vision Generation Task using Discrete Visual Representationscited by 11
  15. 2023Deep Learning in Breast Cancer Imaging: A Decade of Progress and Future Directionscited by 10
  16. 2024PairAug: What Can Augmented Image-Text Pairs Do for Radiology?cited by 10
  17. 2026Efficient Chest X-ray Representation Learning via Semantic-Partitioned Contrastive Learningcited by 10
  18. 2023Gene-induced Multimodal Pre-training for Image-omic Classificationcited by 9
  19. 2024Adaptive Betweenness Clustering for Semi-Supervised Domain Adaptationcited by 9
  20. 2022RankDNN: Learning to Rank for Few-shot Learningcited by 7
  21. 2024MLIP: Medical Language-Image Pre-training with Masked Local Representation Learningcited by 7
  22. 2024Pre-training on High Definition X-ray Images: An Experimental Studycited by 7
  23. 2022UNet-2022: Exploring Dynamics in Non-isomorphic Architecturecited by 6
  24. 2023Masked Contrastive Reconstruction for Cross-modal Medical Image-Report Retrievalcited by 6
  25. 2024BenchX: A Unified Benchmark Framework for Medical Vision-Language Pretraining on Chest X-Rayscited by 6
  26. 2024Enhancing the vision-language foundation model with key semantic knowledge-emphasized report refinementcited by 5
  27. 2025Efficient Medical Vision-Language Alignment Through Adapting Masked Vision Modelscited by 5
  28. 2026A unified multi-task framework enables interpretable chest radiograph analysiscited by 5
  29. 2023A Survey on Image-text Multimodal Modelscited by 4
  30. 2024Enhancing medical vision-language contrastive learning via inter-matching relation modellingcited by 4
  31. 2024EVOKE: Elevating Chest X-ray Report Generation via Multi-View Contrastive Learning and Patient-Specific Knowledgecited by 4
  32. 2025RadZero: Similarity-Based Cross-Attention for Explainable Vision-Language Alignment in Chest X-ray with Zero-Shot Multi-Task Capabilitycited by 4
  33. 2023Protein Representation Learning via Knowledge Enhanced Primary Structure Modelingcited by 3
  34. 2023UniDCP: Unifying Multiple Medical Vision-language Tasks via Dynamic Cross-modal Learnable Promptscited by 3
  35. 2024Enhancing Representation in Medical Vision-Language Foundation Models via Multi-Scale Information Extraction Techniquescited by 3
  36. 2024MMCLIP: Cross-modal Attention Masked Modelling for Medical Language-Image Pre-Trainingcited by 3
  37. 2024A Large Model for Non-invasive and Personalized Management of Breast Cancer from Multiparametric MRIcited by 3
  38. 2025Self-Supervised Anatomical Consistency Learning for Vision-Grounded Medical Report Generationcited by 3

50 most cited of 80 All 80 (JSON)