mlsanityData to 7 Oct 2026

Beyond Representation Learning: A Systematic Study of Joint-Embedding Predictive Generation for 3D Brain MRI

arXiv 2608.28787 ↗First posted 28 Aug 2026cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Joint-embedding predictive architectures (JEPAs) have primarily been developed for self-supervised representation learning. Denoising JEPA (D-JEPA) recently demonstrated strong generative capabilities on natural images, yet the applicability to 3D medical imaging remains unexplored. Building on the D-JEPA framework, we present Med-D-JEPA, a systematic adaptation and evaluation of joint-embedding predictive generation for 3D brain MRI. Med-D-JEPA operates on continuous latent tokens produced by a 3D KL-regularized adversarial variational autoencoder, and combines masked context prediction, representation-level alignment, per-token diffusion, and iterative next-set-of-token sampling. We evaluate unconditional and class-conditional generation quality on BraTS2019 and OASIS-1 datasets; downstream classification utility; and preliminary whole-tumor segmentation on BraTS2020. Across different generation settings, Med-D-JEPA achieves superior or competitive performance compared to several strong baselines on fidelity and diversity metrics. Compared to training with real samples, Med-D-JEPA-based synthetic pretraining improves classification AUC from 0.63 to 0.85 on BraTS2019 and from 0.78 to 0.87 on OASIS-1. In the segmentation study, pretraining on Med-D-JEPA samples improves Dice from 0.74 to 0.80 and reduces HD95 from 13.40 to 9.56 mm. These findings establish joint-embedding predictive generation as a promising direction for 3D medical image synthesis and encourage further research in this direction.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 42 of its 60 references that are in the map. First posted on arXiv 28 Aug 2026.

Builds on

42 of 60 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 2016Layer Normalizationcited by 5,579
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  8. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  9. 2017Improved Training of Wasserstein GANscited by 2,223
  10. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  11. 2017A Survey on Deep Learning in Medical Image Analysiscited by 1,182
  12. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
30 more
  1. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  2. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  3. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  4. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  5. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  6. 2018Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challengecited by 508
  7. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  8. 2022MONAI: An open-source framework for deep learning in healthcarecited by 438
  9. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  10. 2019Med3D: Transfer Learning for 3D Medical Image Analysiscited by 282
  11. 2018Generative Adversarial Network in Medical Imaging: A Reviewcited by 257
  12. 2017Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognitioncited by 178
  13. 2024MAISI: Medical AI for Synthetic Imagingcited by 86
  14. 2023Conditional Diffusion Models for Semantic 3D Brain MRI Synthesiscited by 85
  15. 2023Generative AI for Medical Imaging: extending the MONAI Frameworkcited by 68
  16. 2019Generation of 3D Brain MRI Using Auto-Encoding Generative Adversarial Networkscited by 65
  17. 2023GenerateCT: Text-Conditional Generation of 3D Chest CT Volumescited by 64
  18. 2024VISTA3D: A Unified Segmentation Foundation Model For 3D Medical Imagingcited by 62
  19. 2025VL-JEPA: Joint Embedding Predictive Architecture for Vision-languagecited by 61
  20. 2020Hierarchical Amortized Training for Memory-efficient High Resolution 3D GANcited by 60
  21. 2017Variational Approaches for Auto-Encoding Generative Adversarial Networkscited by 52
  22. 20243D MedDiffusion: A 3D Medical Latent Diffusion Model for Controllable and High-quality Medical Image Generationcited by 37
  23. 2020Pseudo-healthy synthesis with pathology disentanglement and adversarial learningcited by 19
  24. 2022Morphology-preserving Autoregressive 3D Generative Modelling of the Braincited by 15
  25. 2023Metadata-Conditioned Generative Models to Synthesize Anatomically-Plausible 3D Brain MRIscited by 12
  26. 2024Denoising with a Joint-Embedding Predictive Architecturecited by 12
  27. 2026Is Your Driving World Model an All-Around Player?cited by 5
  28. 2024High-Resolution Image Synthesis via Next-Token Predictioncited by 3
  29. 2026Masked Generative Transformer Is What You Need for Image Editingcited by 3
  30. 2026Optimizing 3D Diffusion Models for Medical Imaging via Multi-Scale Reward Learningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026I Act Therefore I Am: When Is JEPA's Action-Conditioning Enough to Learn Causal Mechanisms?cited by 0