mlsanityData to 7 Oct 2026

LiMoE: Mixture of LiDAR Representation Learners from Automotive Scenes

arXiv 2501.04004 ↗First posted 7 Jan 2025cs.CV cs.LG cs.ROcited by 19

On the map: Mixture-of-experts → Vision mixture-of-experts

Abstract (arXiv)

LiDAR data pretraining offers a promising approach to leveraging large-scale, readily available datasets for enhanced data utilization. However, existing methods predominantly focus on sparse voxel representation, overlooking the complementary attributes provided by other LiDAR representations. In this work, we propose LiMoE, a framework that integrates the Mixture of Experts (MoE) paradigm into LiDAR data representation learning to synergistically combine multiple representations, such as range images, sparse voxels, and raw points. Our approach consists of three stages: i) Image-to-LiDAR Pretraining, which transfers prior knowledge from images to point clouds across different representations; ii) Contrastive Mixture Learning (CML), which uses MoE to adaptively activate relevant attributes from each representation and distills these mixed features into a unified 3D network; iii) Semantic Mixture Supervision (SMS), which combines semantic logits from multiple representations to boost downstream segmentation performance. Extensive experiments across eleven large-scale LiDAR datasets demonstrate our effectiveness and superiority. The code has been made publicly accessible.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 72 of its 102 references that are in the map. First posted on arXiv 7 Jan 2025.

Builds on

72 of 102 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022A Review of Sparse Expert Models in Deep Learning4 Sep
  2. 2022Residual Mixture of Experts20 Apr
  3. 2021Scaling Vision with Sparse Mixture of Experts10 Jun

Other references · 69

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2023Segment Anythingcited by 8,158
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  9. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  10. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  11. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  12. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
38 more
  1. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  2. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  3. 2019RandLA-Net: Efficient Semantic Segmentation of Large-Scale Point Cloudscited by 584
  4. 2023Segment Everything Everywhere All at Oncecited by 546
  5. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 502
  6. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 485
  7. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 422
  8. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 388
  9. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 298
  10. 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
  11. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 274
  12. 2020PointASNL: Robust Point Clouds Processing using Nonlocal Neural Networks with Adaptive Samplingcited by 238
  13. 2020SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Drivingcited by 226
  14. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  15. 2022Tutel: Adaptive Mixture-of-Experts at Scalecited by 199
  16. 2023A Simple Framework for Open-Vocabulary Segmentation and Detectioncited by 197
  17. 2021RPVNet: A Deep and Efficient Range-Point-Voxel Fusion Network for LiDAR Point Cloud Segmentationcited by 173
  18. 2021Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Trackingcited by 165
  19. 2023Robo3D: Towards Robust and Reliable 3D Perception against Corruptionscited by 128
  20. 2022Image-to-Lidar Self-Supervised Distillation for Autonomous Driving Datacited by 126
  21. 2022Vision-Centric BEV Perception: A Surveycited by 126
  22. 2023Rethinking Range View Representation for LiDAR Segmentationcited by 126
  23. 2020SemanticPOSS: A Point Cloud Dataset with Large Quantity of Dynamic Instancescited by 123
  24. 2023Segment Any Point Cloud Sequences by Distilling Vision Foundation Modelscited by 123
  25. 2024LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-trainingcited by 122
  26. 2020RELLIS-3D Dataset: Data, Benchmarks and Analysiscited by 118
  27. 2021Panoptic-PolarNet: Proposal-free LiDAR Point Cloud Panoptic Segmentationcited by 107
  28. 2022LaserMix for Semi-Supervised LiDAR Semantic Segmentationcited by 92
  29. 2020AMVNet: Assertion-based Multi-View Fusion Network for LiDAR Semantic Segmentationcited by 87
  30. 2020LiDAR-based Panoptic Segmentation via Dynamic Shifting Networkcited by 84
  31. 2021Transfer Learning from Synthetic to Real LiDAR Point Cloud for Semantic Segmentationcited by 76
  32. 2022TorchSparse: Efficient Point Cloud Inference Enginecited by 72
  33. 2022Scribble-Supervised LiDAR Semantic Segmentationcited by 69
  34. 2022ALSO: Automotive Lidar Self-supervision by Occupancy estimationcited by 68
  35. 2024Convolution Meets LoRA: Parameter Efficient Finetuning for Segment Anything Modelcited by 67
  36. 2023UniSeg: A Unified Multi-Modal LiDAR Segmentation Network and the OpenPCSeg Codebasecited by 57
  37. 2022Unsupervised Point Cloud Representation Learning with Deep Neural Networks: A Surveycited by 54
  38. 2023Using a Waffle Iron for Automotive Point Cloud Semantic Segmentationcited by 53

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 10; 2026: 9.

19 papers.

Sort:

By area:

  1. 2025Perspective-Invariant 3D Object Detectioncited by 16
  2. 2025Beyond One Shot, Beyond One Perspective: Cross-View and Long-Horizon Distillation for Better LiDAR Representationscited by 7
  3. 2025U4D: Uncertainty-Aware 4D World Modeling from LiDAR Sequencescited by 5
  4. 2025Enhanced Spatiotemporal Consistency for Image-to-LiDAR Data Pretrainingcited by 4
  5. 2025Robust Experts: the Effect of Adversarial Training on CNNs with Sparse Mixture-of-Experts Layerscited by 4
  6. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  7. 2025CleverDistiller: Simple and Spatially Consistent Cross-modal Distillationcited by 3
  8. 2026AdaSFormer: Adaptive Serialized Transformers for Monocular Semantic Scene Completion from Indoor Environmentscited by 3
  9. 2025Zero-Shot 3D Visual Grounding from Vision-Language Modelscited by 2
  10. 2026The Rise of Sparse Mixture-of-Experts: A Survey from Algorithmic Foundations to Decentralized Architectures and Vertical Domain Applicationscited by 2
  11. 2025EvidenceMoE: A Physics-Guided Mixture-of-Experts with Evidential Critics for Advancing Fluorescence Light Detection and Ranging in Scattering Mediacited by 1
  12. 2026Design and Behavior of Sparse Mixture-of-Experts Layers in CNN-based Semantic Segmentationcited by 1
7 more
  1. 2026Unified Prediction and Planning via Conflict-Aware Disjoint Parameter Trainingcited by 1
  2. 2025DoReMi: Bridging 3D Domains via Topology-Aware Domain-Representation Mixture of Expertscited by 0
  3. 2026LiDAR-Anchored Collaborative Distillation for Robust 2D Representationscited by 0
  4. 2026OmniLiDAR: A Unified Diffusion Framework for Multi-Domain 3D LiDAR Generationcited by 0
  5. 2026One Model to Translate Them All: Universal Any-to-Any Translation for Heterogeneous Collaborative Perceptioncited by 0
  6. 2026OmniPlan: An Adaptive Framework for Timely and Near-Optimal Network Planning Optimizationcited by 0
  7. 2026STAR: A Spatial-Topology Aware Routing Framework for Generalizable 3D Scene Understandingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Point-MoE: Large-Scale Multi-Dataset Training with Mixture-of-Experts for 3D Semantic Segmentation
  2. 2025NUC-Net: Non-uniform Cylindrical Partition Network for Efficient LiDAR Semantic Segmentation
  3. 2026Geometry Without Coordinates: LiDAR Diffusion as a 3D Feature Bridge
  4. 2026Bridging the Modality Gap in Roadside LiDAR: A Training-Free Vision-Language Model Framework for Vehicle Classification
  5. 2024DiffSSC: Semantic LiDAR Scan Completion using Denoising Diffusion Probabilistic Models
  6. 2025LOSC: LiDAR Open-voc Segmentation Consolidator
  7. 2026CloudDiffusion: Diffusion-Based Scene Completion in the Point Cloud Domain
  8. 2026SemLT3D: Semantic-Guided Expert Distillation for Camera-only Long-Tailed 3D Object Detection