# Self-Supervised Representation Learning with Joint Embedding Predictive Architecture for Automotive LiDAR Object Detection

arXiv: https://arxiv.org/abs/2501.04969 · HTML: https://mlsanity.com/paper/2501.04969/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 43 references that are in the map.
- First posted on arXiv 9 Jan 2025.

## Builds on (most cited first, 28 of 28)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [PointRCNN: 3D Object Proposal Generation and Detection from Point Cloud](https://mlsanity.com/paper/1812.04244/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [One Million Scenes for Autonomous Driving: ONCE Dataset](https://mlsanity.com/paper/2106.11037/) (2021)
- [Self-Supervised Pretraining of 3D Features on any Point-Cloud](https://mlsanity.com/paper/2101.02691/) (2021)
- [Scaling and Benchmarking Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/1905.01235/) (2019)
- [Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds](https://mlsanity.com/paper/2109.00179/) (2021)
- [Visual Point Cloud Forecasting enables Scalable Autonomous Driving](https://mlsanity.com/paper/2312.17655/) (2023)
- [ALSO: Automotive Lidar Self-supervision by Occupancy estimation](https://mlsanity.com/paper/2212.05867/) (2022)
- [Occupancy-MAE: Self-supervised Pre-training Large-scale LiDAR Point Clouds with Masked Occupancy Autoencoders](https://mlsanity.com/paper/2206.09900/) (2022)
- [Understanding Collapse in Non-Contrastive Siamese Representation Learning](https://mlsanity.com/paper/2209.15007/) (2022)
- [UnO: Unsupervised Occupancy Fields for Perception and Forecasting](https://mlsanity.com/paper/2406.08691/) (2024)
- [MV-JAR: Masked Voxel Jigsaw and Reconstruction for LiDAR-Based Self-Supervised Pre-Training](https://mlsanity.com/paper/2303.13510/) (2023)
- [Spatiotemporal Self-supervised Learning for Point Clouds in the Wild](https://mlsanity.com/paper/2303.16235/) (2023)
- [AD-PT: Autonomous Driving Pre-Training with Large-scale Point Cloud Dataset](https://mlsanity.com/paper/2306.00612/) (2023)
- [T-MAE: Temporal Masked Autoencoders for Point Cloud Representation Learning](https://mlsanity.com/paper/2312.10217/) (2023)
- [Equivariant Spatio-Temporal Self-Supervision for LiDAR Object Detection](https://mlsanity.com/paper/2404.11737/) (2024)

## Built on by (most cited first, 13 of 13)

- [A Survey of World Models for Autonomous Driving](https://mlsanity.com/paper/2501.11260/) (2025)
- [Drive-JEPA: Video JEPA Meets Multimodal Trajectory Distillation for End-to-End Driving](https://mlsanity.com/paper/2601.22032/) (2026)
- [Zero-Shot Cross-City Generalization in End-to-End Autonomous Driving: Self-Supervised versus Supervised Representations](https://mlsanity.com/paper/2603.11417/) (2026)
- [Dueling World Models: Advantage-Style Action Channels for Common-Mode Distractor Rejection](https://mlsanity.com/paper/2608.06706/) (2026)
- [Cross4D-JEPA: Dense Cross-modal Correspondence Distillation for 4D Point Cloud Representation Learning](https://mlsanity.com/paper/2607.00514/) (2026)
- [GhostPoint: Self-Supervised Representation Learning by Hallucinating Occluded LiDAR Structure](https://mlsanity.com/paper/2608.14428/) (2026)
- [HanoiWorld : A Joint Embedding Predictive Architecture BasedWorld Model for Autonomous Vehicle Controller](https://mlsanity.com/paper/2601.01577/) (2026)
- [SR-JEPA: Learning Predictive Latent State in 3D Scenes](https://mlsanity.com/paper/2608.05774/) (2026)
- [Human-JEPA: A Human-Centric Vision Model that Perceives and Anticipates](https://mlsanity.com/paper/2608.21160/) (2026)
- [Representation Learning in Diffusion and Flow-based Model: An Application Aspect](https://mlsanity.com/paper/2608.24068/) (2026)
- [Hydra: A Navigation World Action Model with Discrete Latent Planning and Continuous Flow-Matching Execution](https://mlsanity.com/paper/2608.28995/) (2026)
- [Mask-Aware Execution for Efficient JEPA Training](https://mlsanity.com/paper/2609.22674/) (2026)
- [AD-E2E-JEPA: A Joint-Embedding Predictive Architecture For End-to-End Autonomous Driving](https://mlsanity.com/paper/2609.34085/) (2026)

## Abstract (arXiv)

Recently, self-supervised representation learning relying on vast amounts of unlabeled data has been explored as a pre-training method for autonomous driving. However, directly applying popular contrastive or generative methods to this problem is insufficient and may even lead to negative transfer. In this paper, we present AD-L-JEPA, a novel self-supervised pre-training framework with a joint embedding predictive architecture (JEPA) for automotive LiDAR object detection. Unlike existing methods, AD-L-JEPA is neither generative nor contrastive. Instead of explicitly generating masked regions, our method predicts Bird's-Eye-View embeddings to capture the diverse nature of driving scenes. Furthermore, our approach eliminates the need to manually form contrastive pairs by employing explicit variance regularization to avoid representation collapse. Experimental results demonstrate consistent improvements on the LiDAR 3D object detection downstream task across the KITTI3D, Waymo, and ONCE datasets, while reducing GPU hours by 1.9x-2.7x and GPU memory by 2.8x-4x compared with the state-of-the-art method Occupancy-MAE. Notably, on the largest ONCE dataset, pre-training on 100K frames yields a 1.61 mAP gain, better than all other methods pre-trained on either 100K or 500K frames, and pre-training on 500K frames yields a 2.98 mAP gain, better than all other methods pre-trained on either 500K or 1M frames. AD-L-JEPA constitutes the first JEPA-based pre-training method for autonomous driving. It offers better quality, faster, and more GPU-memory-efficient self-supervised representation learning. The source code of AD-L-JEPA is ready to be released.