RadJEPA: Radiology Encoder for Chest X-Rays via Joint Embedding Predictive Architecture
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
Abstract (arXiv)
Vision-language pretraining has driven progress in medical image representation learning, but it depends on paired image-text data and can inherit reporting bias from clinical narratives. We study whether language-free predictive pretraining can produce an image encoder that transfers effectively to radiology report generation. RadJEPA is a chest-X-ray adaptation of I-JEPA, pretrained on approximately 840K unlabeled radiographs using latent context-to-target prediction. Our primary contribution is an extensive empirical evaluation of this language-free encoder for report generation: the frozen image encoder is coupled to a trainable two-layer projector and language decoder, and is also substituted into four established vision-language backbones. Across MIMIC-CXR and IU-Xray, RadJEPA matches or exceeds the evaluated image-only and image-text baselines on lexical, entity-relation, and clinical-label metrics. Controlled MIMIC-only comparisons provide evidence that the predictive objective contributes beyond domain-specific pretraining, while broader comparisons also reflect differences in pretraining data, model capacity, and input resolution. Complementary classification and segmentation experiments assess transfer beyond report generation.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 34 of its 66 references that are in the map. First posted on arXiv 22 Jan 2026.
Builds on
34 of 66 references.
Closest ancestors · 1
Same direction, newest first
Other references · 33
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
- 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
- 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
- 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
- 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
- 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 972
21 more
- 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
- 2021LiT: Zero-Shot Transfer with Locked-image text Tuningcited by 577
- 2020VirTex: Learning Visual Representations from Textual Annotationscited by 339
- 2020CheXbert: Combining Automatic Labelers and Expert Annotations for Accurate Radiology Report Labeling Using BERTcited by 337
- 2021RadGraph: Extracting Clinical Entities and Relations from Radiology Reportscited by 319
- 2019PadChest: A large chest x-ray image dataset with multi-label annotated reportscited by 309
- 2018nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentationcited by 291
- 2020VinDr-CXR: An open dataset of chest X-rays with radiologist's annotationscited by 239
- 2023Learning to Exploit Temporal Structure for Biomedical Vision-Language Processingcited by 211
- 2022MaskCLIP: Masked Self-Distillation Advances Contrastive Language-Image Pretrainingcited by 187
- 2019Models Genesis: Generic Autodidactic Models for 3D Medical Image Analysiscited by 159
- 2024GREEN: Generative Radiology Report Evaluation and Error Notationcited by 128
- 2021Deep Learning for Chest X-ray Analysis: A Surveycited by 84
- 2023Advancing Radiograph Representation Learning with Masked Record Modelingcited by 75
- 2024Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Maskingcited by 65
- 2021VinDr-RibCXR: A Benchmark Dataset for Automatic Segmentation and Labeling of Individual Ribs on Chest X-rayscited by 23
- 2025CheXWorld: Exploring Image World Modeling for Radiograph Representation Learningcited by 19
- 2023Rethinking Radiology Report Generation via Causal Inspired Counterfactual Augmentationcited by 3
- 2025X-WIN: Building Chest Radiograph World Model via Predictive Sensingcited by 2
- 2024Sumotosima: A Framework and Dataset for Classifying and Summarizing Otoscopic Imagescited by 1
- 2025Early Exit and Multi Stage Knowledge Distillation in VLMs for Video Summarizationcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025Exploring the Capabilities of Large Language Model Encoders for Image-Text Retrieval in Chest X-rays
- 2026SentZero: An Enhanced Sentence-Centric Vision-Language Pretraining for Multi-Task Zero-Shot Chest X-Ray Analysis
- 2024Gla-AI4BioMed at RRG24: Visual Instruction-tuned Adaptation for Radiology Report Generation
- 2026RadPRISM: Schema-stratified radiology-report supervision for concept-disentangled image representations and visual grounding
- 2023ELIXR: Towards a general purpose X-ray artificial intelligence system through alignment of large language models and radiology vision encoders
- 2024Grounded Knowledge-Enhanced Medical Vision-Language Pre-training for Chest X-Ray
- 2025A Reality Check of Vision-Language Pre-training in Radiology: Have We Progressed Using Text?
- 2023MAIRA-1: A specialised large multimodal model for radiology report generation