mlsanityData to 7 Oct 2026

Introduction to Latent Variable Energy-Based Models: A Path Towards Autonomous Machine Intelligence

arXiv 2306.02572 ↗First posted 5 Jun 2023cs.LG cond-mat.dis-nn stat.MLcited by 52

Not on the map. Why?

Abstract (arXiv)

Current automated systems have crucial limitations that need to be addressed before artificial intelligence can reach human-like levels and bring new technological revolutions. Among others, our societies still lack Level 5 self-driving cars, domestic robots, and virtual assistants that learn reliable world models, reason, and plan complex action sequences. In these notes, we summarize the main ideas behind the architecture of autonomous intelligence of the future proposed by Yann LeCun. In particular, we introduce energy-based and latent variable models and combine their advantages in the building block of LeCun's proposal, that is, in the hierarchical joint embedding predictive architecture (H-JEPA).

Key facts. Cited by 52 papers in the map (data through 7 Oct 2026). Builds on 27 of its 73 references that are in the map. First posted on arXiv 5 Jun 2023.

Builds on

27 of 73 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  9. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  10. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,424
  11. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  12. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
15 more
  1. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 723
  2. 2020Image Segmentation Using Deep Learning: A Surveycited by 659
  3. 2017Wasserstein GANcited by 635
  4. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  5. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  6. 2018fastMRI: An Open Dataset and Benchmarks for Accelerated MRIcited by 464
  7. 2020A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promisescited by 230
  8. 2013OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networkscited by 224
  9. 2016Energy-based Generative Adversarial Networkcited by 119
  10. 2019A Survey of Deep Learning Applications to Autonomous Vehicle Controlcited by 101
  11. 2018The History Began from AlexNet: A Comprehensive Survey on Deep Learning Approachescited by 74
  12. 2020An Introduction to Electrocatalyst Design using Machine Learning for Renewable Energy Storagecited by 45
  13. 2018Learning to Predict the Cosmological Structure Formationcited by 38
  14. 2022Ranking Info Noise Contrastive Estimation: Boosting Contrastive Learning via Ranked Positivescited by 37
  15. 2022Ab-initio quantum chemistry with neural-network wavefunctionscited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 15; 2025: 14; 2026: 22.

52 papers.

Sort:

By area:

  1. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 175
  2. 2024The pitfalls of next-token predictioncited by 156
  3. 2025Vid2World: Crafting Video Diffusion Models to Interactive World Modelscited by 67
  4. 2024Agent Planning with World Knowledge Modelcited by 66
  5. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 49
  6. 2024Benchmarking Agentic Workflow Generationcited by 36
  7. 2025Exploring the Evolution of Physics Cognition in Video Generation: A Surveycited by 31
  8. 2025Energy-Based Transformers are Scalable Learners and Thinkerscited by 30
  9. 2025Roll the dice & look before you leap: Going beyond the creative limits of next-token predictioncited by 28
  10. 2024Imitating Language via Scalable Inverse Reinforcement Learningcited by 21
  11. 2024IoT-LLM: a framework for enhancing Large Language Model reasoning from real-world sensor datacited by 18
  12. 2024SymbolicAI: A framework for logic-based approaches combining generative models and solverscited by 15
38 more
  1. 2024GenAINet: Enabling Wireless Collective Intelligence via Knowledge Transfer and Reasoningcited by 13
  2. 2024M3-JEPA: Multimodal Alignment via Multi-gate MoE based on the Joint-Embedding Predictive Architecturecited by 12
  3. 2025seq-JEPA: Autoregressive Predictive Learning of Invariant-Equivariant World Modelscited by 7
  4. 2025WPT: World-to-Policy Transfer via Online World Model Distillationcited by 4
  5. 2026Understanding Self-Supervised Learning via Latent Distribution Matchingcited by 4
  6. 2026Nano World Models: A Minimalist Implementation of Future Video Predictioncited by 4
  7. 2026Temporally Centered SIGReg Improves LeWorldModel Representations for Robot Policy Learningcited by 4
  8. 2025Neural Langevin Machine: a local asymmetric learning rule can be creativecited by 3
  9. 2026A Definition and Roadmap for World Modelscited by 3
  10. 2025Hyper-SET: Designing Transformers via Hyperspherical Energy Minimizationcited by 2
  11. 2025A Hybrid Co-Finetuning Approach for Visual Bug Detection in Video Gamescited by 2
  12. 202510 Open Challenges Steering the Future of Vision-Language-Action Modelscited by 2
  13. 2026PanoWorld: Geometry-Consistent Panoramic Video World Modelingcited by 2
  14. 2025Enhancing Gradient-based Discrete Sampling via Parallel Temperingcited by 1
  15. 2026Remapping and navigation of an embedding space via error minimization: a fundamental organizational principle of cognition in natural and artificial systemscited by 1
  16. 2026PEPR: Privileged Event-based Predictive Regularization for Domain Generalizationcited by 1
  17. 2026GT-SVJ: Generative-Transformer-Based Self-Supervised Video Judge For Efficient Video Reward Modelingcited by 1
  18. 2026Future Dynamic 3D Reconstruction: Toward 3D World Modeling with Disentangled Ego-Motioncited by 1
  19. 2023Model of models -- Part 1cited by 0
  20. 2024Operational Latent Spacescited by 0
  21. 2024JEMA: A Joint Embedding Framework for Scalable Co-Learning with Multimodal Alignmentcited by 0
  22. 2024Recurrent Joint Embedding Predictive Architecture with Recurrent Forward Propagation Learningcited by 0
  23. 2024Machine learning the Ising transition: A comparison between discriminative and generative approachescited by 0
  24. 2024Future Research Avenues for Artificial Intelligence in Digital Gaming: An Exploratory Reportcited by 0
  25. 2025Deep Learning-Assisted Detection of Sarcopenia in Cross-Sectional Computed Tomography Imagingcited by 0
  26. 2025Computational Concept of the Psyche (in Russian)cited by 0
  27. 2025Learning the Neighborhood: Contrast-Free Multimodal Self-Supervised Molecular Graph Pretrainingcited by 0
  28. 2026Computational Concept of the Psychecited by 0
  29. 2026A unifying view of contrastive learning, importance sampling, and bridge sampling for energy-based modelscited by 0
  30. 2026Towards Human Motion World Models via Executable Behaviour Representationscited by 0
  31. 2026Efficient Rationale-based Retrieval: On-policy Distillation from Generative Rerankers based on JEPAcited by 0
  32. 2026Three Necessary Principles for Self-Supervised Visual Representation Learningcited by 0
  33. 2026Physiological World Models for Human State Transitionscited by 0
  34. 2026AlignJEPA: Predictive Vision-Language Alignment for Remote Sensing Foundation Modelscited by 0
  35. 2026When Graph-JEPA Learns the Wrong Thing: Diagnosing and Repairing Category-Conditional Collapsecited by 0
  36. 2026TetrisCNN for interpretable detection of phases of matter from experimental quantum simulator datacited by 0
  37. 2026Agent-Editing World Model: Rethinking World Modeling for LLM Agentscited by 0
  38. 2026Predictive Self-Supervised Learning Provably Identifies Stochastic Signals under Nuisancecited by 0

50 most cited of 52 All 52 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026D-JEPA: A Decision-Aligned Latent World Model
  2. 2025From Classical Probabilistic Latent Variable Models to Modern Generative AI: A Unified Perspective
  3. 2026HanoiWorld : A Joint Embedding Predictive Architecture BasedWorld Model for Autonomous Vehicle Controller
  4. 2026Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challenges
  5. 2024Leveraging Hybrid Intelligence Towards Sustainable and Energy-Efficient Machine Learning
  6. 2026AD-E2E-JEPA: A Joint-Embedding Predictive Architecture For End-to-End Autonomous Driving
  7. 2026World Models: A Comprehensive Survey of Architectures, Methodologies, Reasoning Paradigms, and Applications
  8. 2026RoboJEPA: Scaling Robotic Latent World Models