mlsanityData to 14 Oct 2026

Discrete JEPA: Learning Discrete Token Representations without Reconstruction

arXiv 2506.14373 ↗First posted 17 Jun 2025cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

The cornerstone of cognitive intelligence lies in extracting hidden patterns from observations and leveraging these principles to systematically predict future outcomes. However, current image tokenization methods demonstrate significant limitations in tasks requiring symbolic abstraction and logical reasoning capabilities essential for systematic inference. To address this challenge, we propose Discrete-JEPA, extending the latent predictive coding framework with semantic tokenization and novel complementary objectives to create robust tokenization for symbolic reasoning tasks. Discrete-JEPA dramatically outperforms baselines on visual symbolic prediction tasks, while striking visual evidence reveals the spontaneous emergence of deliberate systematic patterns within the learned semantic token space. Though an initial model, our approach promises a significant impact for advancing Symbolic world modeling and planning capabilities in artificial intelligence systems.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 37 of its 46 references that are in the map. First posted on arXiv 17 Jun 2025.

Builds on

37 of 46 references.

References · 37

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  7. 2017Neural Discrete Representation Learningcited by 4,712
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  10. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  11. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  12. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
25 more
  1. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  2. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  3. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
  4. 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 903
  5. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  6. 2022Autoregressive Image Generation using Residual Quantizationcited by 722
  7. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  8. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 440
  9. 2022MAGVIT: Masked Generative Video Transformercited by 346
  10. 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 302
  11. 2024WorldCoder, a Model-Based LLM Agent: Building World Models by Writing Code and Interacting with the Environmentcited by 107
  12. 2025Masked Autoencoders Are Effective Tokenizers for Diffusion Modelscited by 96
  13. 2021Neural Production Systems: Learning Rule-Governed Visual Dynamicscited by 75
  14. 2022SQ-VAE: Variational Bayes on Discrete Representation with Self-annealed Stochastic Quantizationcited by 68
  15. 2023MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Featurescited by 57
  16. 2023A-JEPA: Joint-Embedding Predictive Architecture Can Listencited by 53
  17. 2025Democratizing Text-to-Image Masked Generative Models with Compact Text-Aware One-Dimensional Tokenscited by 50
  18. 2024LARP: Tokenizing Videos with a Learned Autoregressive Generative Priorcited by 46
  19. 2024Towards Semantic Equivalence of Tokenization in Multimodal LLMcited by 45
  20. 2022Temporally Consistent Transformers for Video Generationcited by 44
  21. 2022Joint Embedding Predictive Architectures Focus on Slow Featurescited by 30
  22. 2024Slot State Space Modelscited by 17
  23. 2023Neurosymbolic Grounding for Compositional World Modelscited by 14
  24. 2024Denoising with a Joint-Embedding Predictive Architecturecited by 14
  25. 2025Dreamweaver: Learning Compositional World Models from Pixelscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Probing the Latent World: Emergent Discrete Symbols and Physical Structure in Latent Representationscited by 0
  2. 2026Transcribe, Then Reason: Two-Pass Decomposition for Multimodal Reviewcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Why Struggle with Continuous Latents? Interpretable Discrete Latent Reasoning via Rendered Compression
  2. 2026Symbolic Grounding Reveals Representational Bottlenecks in Abstract Visual Reasoning
  3. 2026Communication-Inspired Tokenization for Structured Image Representations
  4. 2025I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?
  5. 2026Semantic-Aware Prefix Learning for Token-Efficient Image Generation
  6. 2026Token-Disentangled Latent Test-Time Scaling for Vision-Language Reasoning
  7. 2022Emergence of Machine Language: Towards Symbolic Intelligence with Neural Networks
  8. 2026Lngram: N-gram Conditional Memory in Latent Space