mlsanityData to 7 Oct 2026

3D-JEPA: A Joint Embedding Predictive Architecture for 3D Self-Supervised Representation Learning

arXiv 2409.15803 ↗First posted 24 Sep 2024cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

Invariance-based and generative methods have shown a conspicuous performance for 3D self-supervised representation learning (SSRL). However, the former relies on hand-crafted data augmentations that introduce bias not universally applicable to all downstream tasks, and the latter indiscriminately reconstructs masked regions, resulting in irrelevant details being saved in the representation space. To solve the problem above, we introduce 3D-JEPA, a novel non-generative 3D SSRL framework. Specifically, we propose a multi-block sampling strategy that produces a sufficiently informative context block and several representative target blocks. We present the context-aware decoder to enhance the reconstruction of the target blocks. Concretely, the context information is fed to the decoder continuously, facilitating the encoder in learning semantic modeling rather than memorizing the context information related to target blocks. Overall, 3D-JEPA predicts the representation of target blocks from a context block using the encoder and context-aware decoder architecture. Various downstream tasks on different datasets demonstrate 3D-JEPA's effectiveness and efficiency, achieving higher accuracy with fewer pretraining epochs, e.g., 88.65% accuracy on PB_T50_RS with 150 pretraining epochs.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 33 of its 39 references that are in the map. First posted on arXiv 24 Sep 2024.

Builds on

33 of 39 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  4. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  7. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  8. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  9. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  10. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  11. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  12. 2022BEVFusion: Multi-Task Multi-Sensor Fusion with Unified Bird's-Eye View Representationcited by 844
21 more
  1. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  2. 2019Self-labelling via simultaneous clustering and representation learningcited by 525
  3. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 502
  4. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
  5. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 456
  6. 2022PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategiescited by 432
  7. 2022Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Frameworkcited by 376
  8. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 263
  9. 2022CrossPoint: Self-Supervised Cross-Modal Contrastive Learning for 3D Point Cloud Understandingcited by 198
  10. 2021Spatio-temporal Self-Supervised Representation Learning for 3D Point Cloudscited by 180
  11. 2022Autoencoders as Cross-Modal Teachers: Can Pretrained 2D Image Transformers Help 3D Representation Learning?cited by 121
  12. 2022Occupancy-MAE: Self-supervised Pre-training Large-scale LiDAR Point Clouds with Masked Occupancy Autoencoderscited by 57
  13. 2022The Hidden Uniform Cluster Prior in Self-Supervised Learningcited by 57
  14. 2023MC-JEPA: A Joint-Embedding Predictive Architecture for Self-Supervised Learning of Motion and Content Featurescited by 53
  15. 2023Point2Vec for Self-Supervised Representation Learning on Point Cloudscited by 28
  16. 2022Distillation with Contrast is All You Need for Self-Supervised Point Cloud Representation Learningcited by 19
  17. 20223D-OAE: Occlusion Auto-Encoders for Self-Supervised Learning on Point Cloudscited by 18
  18. 2023UniScene: Multi-Camera Unified Pre-training via 3D Scene Reconstruction for Autonomous Drivingcited by 15
  19. 2023LiDARFormer: A Unified Transformer-based Multi-task Network for LiDAR Perceptioncited by 11
  20. 2023ViPFormer: Efficient Vision-and-Pointcloud Transformer for Unsupervised Pointcloud Understandingcited by 8
  21. 2024Unsupervised Pre-Training for 3D Leaf Instance Segmentationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 11.

14 papers.

Sort:

By area:

  1. 2025CrossJEPA: Cross-Modal Joint-Embedding Predictive Architecture for Efficient 3D Representation Learning from 2D Imagescited by 4
  2. 2026AeroJEPA: Learning Semantic Latent Representations for Scalable 3D Aerodynamic Field Modelingcited by 2
  3. 2025PointGAC: Geometric-Aware Codebook for Masked Point Cloud Modelingcited by 0
  4. 2025Label-Efficient Grasp Joint Prediction with Point-JEPAcited by 0
  5. 2026TERRA: Task-Embedded Reasoning and Representation Architecture for Cross-Domain Applicationscited by 0
  6. 2026Synesthesia via Direct Latent Augmentation:Bypassing the Decode-Encode Loop for Cross-Modal Distillationcited by 0
  7. 2026Neural Voxel Dynamics: Learning Volumetric Feature Advection for 3D Physics in V-JEPA Latent Spacecited by 0
  8. 2026SJEPA: Learning Elegant Latent Dynamics with Hybrid Symbolic-Neural Predictorscited by 0
  9. 2026SR-JEPA: Learning Predictive Latent State in 3D Scenescited by 0
  10. 2026Human-JEPA: A Human-Centric Vision Model that Perceives and Anticipatescited by 0
  11. 2026Polis: 3D Self-Supervision at City Scalecited by 0
  12. 2026GeoLAM: Learning Geometry-Grounded Latent Actions from Unlabeled Human Videoscited by 0
2 more
  1. 2026Mask-Aware Execution for Efficient JEPA Trainingcited by 0
  2. 2026I Act Therefore I Am: When Is JEPA's Action-Conditioning Enough to Learn Causal Mechanisms?cited by 0