mlsanityData to 7 Oct 2026

LargeAD: Large-Scale Cross-Sensor Data Pretraining for Autonomous Driving

arXiv 2501.04005 ↗First posted 7 Jan 2025cs.CV cs.LG cs.ROcited by 11

Not on the map. Why?

Abstract (arXiv)

Recent advancements in vision foundation models (VFMs) have revolutionized visual perception in 2D, yet their potential for 3D scene understanding, particularly in autonomous driving applications, remains underexplored. In this paper, we introduce LargeAD, a versatile and scalable framework designed for large-scale 3D pretraining across diverse real-world driving datasets. Our framework leverages VFMs to extract semantically rich superpixels from 2D images, which are aligned with LiDAR point clouds to generate high-quality contrastive samples. This alignment facilitates cross-modal representation learning, enhancing the semantic consistency between 2D and 3D data. We introduce several key innovations: (i) VFM-driven superpixel generation for detailed semantic representation, (ii) a VFM-assisted contrastive learning strategy to align multimodal features, (iii) superpoint temporal consistency to maintain stable representations across time, and (iv) multi-source data pretraining to generalize across various LiDAR configurations. Our approach achieves substantial gains over state-of-the-art methods in linear probing and fine-tuning for LiDAR-based segmentation and object detection. Extensive experiments on 11 large-scale multi-sensor datasets highlight our superior performance, demonstrating adaptability, efficiency, and robustness in real-world autonomous driving scenarios.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 111 of its 145 references that are in the map. First posted on arXiv 7 Jan 2025.

Builds on

111 of 145 references.

References · 111

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  6. 2023Segment Anythingcited by 8,158
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  10. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  11. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  12. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
38 more
  1. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  2. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  3. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  4. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  5. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  6. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  7. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  8. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,379
  9. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,202
  10. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  11. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  12. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  13. 2016Colorful Image Colorizationcited by 1,113
  14. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  15. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  16. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 584
  17. 2022OpenScene: 3D Scene Understanding with Open Vocabulariescited by 556
  18. 2023Segment Everything Everywhere All at Oncecited by 546
  19. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 502
  20. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  21. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 422
  22. 2020Understanding the Behaviour of Contrastive Losscited by 397
  23. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 388
  24. 2022Scaling Language-Image Pre-training via Maskingcited by 317
  25. 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
  26. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 274
  27. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  28. 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 197
  29. 2021Unsupervised Semantic Segmentation by Contrasting Object Mask Proposalscited by 190
  30. 2023CLIP2Scene: Towards Label-efficient 3D Scene Understanding by CLIPcited by 178
  31. 2021RPVNet: A Deep and Efficient Range-Point-Voxel Fusion Network for LiDAR Point Cloud Segmentationcited by 173
  32. 2020SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentationcited by 170
  33. 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 165
  34. 2021(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Networkcited by 164
  35. 2021Efficient Visual Pretraining with Contrastive Detectioncited by 150
  36. 2019xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentationcited by 129
  37. 2023Robo3D: Towards Robust and Reliable 3D Perception against Corruptionscited by 128
  38. 2022Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Datacited by 126

50 most cited of 111 All 111 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 5.

11 papers.

  1. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  2. 2025Beyond One Shot, Beyond One Perspective: Cross-View and Long-Horizon Distillation for Better LiDAR Representationscited by 7
  3. 2025U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequencescited by 5
  4. 2025Enhanced Spatiotemporal Consistency for Image-to-LiDAR Data Pretrainingcited by 4
  5. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  6. 2026AdaSFormer: Adaptive Serialized Transformers for Monocular Semantic Scene Completion from Indoor Environmentscited by 3
  7. 2026TOLiD: Bridging the Architecture Gap in Vision Foundation Model to LiDAR Pretraining via Token Lifting for Distillationcited by 1
  8. 2025Learning to Navigate Socially Through Proactive Risk Perceptioncited by 0
  9. 2026LiDAR-Anchored Collaborative Distillation for Robust 2D Representationscited by 0
  10. 2026PanDA: Unsupervised Domain Adaptation for Multimodal 3D Panoptic Segmentation in Autonomous Drivingcited by 0
  11. 2026OmniLiDAR: A Unified Diffusion Framework for Multi-Domain 3D LiDAR Generationcited by 0