mlsanityData to 7 Oct 2026

Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture

arXiv 2301.08243 ↗First posted 19 Jan 2023cs.CV cs.AI cs.LGcited by 982

On the map: World models → Latent predictive world models

Abstract (arXiv)

This paper demonstrates an approach for learning highly semantic image representations without relying on hand-crafted data-augmentations. We introduce the Image-based Joint-Embedding Predictive Architecture (I-JEPA), a non-generative approach for self-supervised learning from images. The idea behind I-JEPA is simple: from a single context block, predict the representations of various target blocks in the same image. A core design choice to guide I-JEPA towards producing semantic representations is the masking strategy; specifically, it is crucial to (a) sample target blocks with sufficiently large scale (semantic), and to (b) use a sufficiently informative (spatially distributed) context block. Empirically, when combined with Vision Transformers, we find I-JEPA to be highly scalable. For instance, we train a ViT-Huge/14 on ImageNet using 16 A100 GPUs in under 72 hours to achieve strong downstream performance across a wide range of tasks, from linear classification to object counting and depth prediction.

Key facts. Cited by 982 papers in the map (data through 7 Oct 2026). Placed in Latent predictive world models, one of 11 directions in World models. Builds on 56 of its 82 references that are in the map. First posted on arXiv 19 Jan 2023.

Builds on

56 of 82 references.

References · 56

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  9. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  10. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  11. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  12. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
38 more
  1. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  2. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  4. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  5. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  6. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  7. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  8. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  9. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
  10. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  11. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  12. 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,445
  13. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  14. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  15. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  16. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  17. 2016Colorful Image Colorizationcited by 1,113
  18. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  19. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
  20. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  21. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
  22. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  23. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  24. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  25. 2017Large Batch Training of Convolutional Networkscited by 529
  26. 2019Self-labelling via simultaneous clustering and representation learningcited by 525
  27. 2019A Large-scale Study of Representation Learning with the Visual Task Adaptation Benchmarkcited by 419
  28. 2019On Mutual Information Maximization for Representation Learningcited by 315
  29. 2022Masked Siamese Networks for Label-Efficient Learningcited by 301
  30. 2022Context Autoencoder for Self-Supervised Representation Learningcited by 297
  31. 2021Understanding self-supervised Learning Dynamics without Contrastive Pairscited by 254
  32. 2016Learning Representations for Automatic Colorizationcited by 251
  33. 2020Representation Learning via Invariant Causal Mechanismscited by 197
  34. 2017Colorization as a Proxy Task for Visual Understandingcited by 175
  35. 2017Learning Discrete Representations via Information Maximizing Self-Augmented Trainingcited by 149
  36. 2022VICRegL: Self-Supervised Learning of Local Visual Featurescited by 118
  37. 2021Are Large-scale Datasets Necessary for Self-Supervised Pre-training?cited by 117
  38. 2021Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samplescited by 114

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 48; 2024: 165; 2025: 229; 2026: 540.

982 papers.

Sort:

By area:

  1. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  2. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  3. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  4. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  5. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  6. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 490
  7. 2026World Action Models are Zero-shot Policiescited by 411
  8. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  9. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 357
  10. 2025Diffusion Transformers with Representation Autoencoderscited by 275
  11. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  12. 2023A Cookbook of Self-Supervised Learningcited by 248
38 more
  1. 2026LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixelscited by 237
  2. 2025LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristicscited by 227
  3. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 212
  4. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  5. 2023CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoderscited by 167
  6. 2024Exploring scalable medical image encoders beyond text supervisioncited by 129
  7. 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Modelcited by 125
  8. 2024Scalable Pre-training of Large Autoregressive Image Modelscited by 115
  9. 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
  10. 2024Multimodal Autoregressive Pre-training of Large Vision Encoderscited by 108
  11. 2025Sonata: Self-Supervised Learning of Reliable Point Representationscited by 108
  12. 2026V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learningcited by 104
  13. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 99
  14. 2025Masked Autoencoders Are Effective Tokenizers for Diffusion Modelscited by 93
  15. 2024Beyond A*: Better Planning with Transformers via Search Dynamics Bootstrappingcited by 89
  16. 2024Large Concept Models: Language Modeling in a Sentence Representation Spacecited by 86
  17. 2024World Models for Autonomous Driving: An Initial Surveycited by 79
  18. 2024DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignmentcited by 79
  19. 2024MMEarth: Exploring Multi-Modal Pretext Tasks For Geospatial Representation Learningcited by 76
  20. 2025What matters for Representation Alignment: Global Information or Spatial Structure?cited by 76
  21. 2024AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalitiescited by 73
  22. 2024Learning and Leveraging World Models in Visual Representation Learningcited by 71
  23. 2023The effectiveness of MAE pre-pretraining for billion-scale pretrainingcited by 68
  24. 2024Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Maskingcited by 64
  25. 2024OmniSat: Self-Supervised Modality Fusion for Earth Observationcited by 63
  26. 2025Scaling Language-Free Visual Representation Learningcited by 62
  27. 2023Large AI Models in Health Informatics: Applications, Challenges, and the Futurecited by 61
  28. 2025VL-JEPA: Joint Embedding Predictive Architecture for Vision-languagecited by 61
  29. 2024TripletCLIP: Improving Compositional Reasoning of CLIP via Synthetic Vision-Language Negativescited by 57
  30. 2023MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Featurescited by 53
  31. 2023A-JEPA: Joint-Embedding Predictive Architecture Can Listencited by 50
  32. 2025A Survey: Learning Embodied Intelligence from Physical Simulators and World Modelscited by 50
  33. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 50
  34. 2024EEG2Rep: Enhancing Self-supervised EEG Representation Through Informative Masked Inputscited by 48
  35. 2024SD-DiT: Unleashing the Power of Self-supervised Discrimination in Diffusion Transformercited by 47
  36. 2024Lexicon3D: Probing Visual Foundation Models for Complex 3D Scene Understandingcited by 46
  37. 2026World Action Models: The Next Frontier in Embodied AIcited by 46
  38. 2023Predicting Gradient is Better: Exploring Self-Supervised Learning for SAR ATR with a Joint-Embedding Predictive Architecturecited by 43

50 most cited of 982 All 982 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Joint Embedding Predictive Architectures Focus on Slow Features
  2. 2026A Lightweight Library for Energy-Based Joint-Embedding Predictive Architectures
  3. 2026Mask-Aware Execution for Efficient JEPA Training
  4. 2025Improving Joint Embedding Predictive Architecture with Diffusion Noise
  5. 2026From Pixels, Without Pre-training: Joint Generative and Self-Supervised Representation Learning in One Model
  6. 2026Causal-JEPA: Learning World Models through Object-Level Latent Masking
  7. 2026Successive Capacity Growth: Task-Complexity-Driven Width and Depth Expansion for Vision Transformer Encoders in JEPA World Models
  8. 2025USP: Unified Self-Supervised Pretraining for Image Generation and Understanding