mlsanityData to 14 Oct 2026

Vanilla ViT for Automotive Point Cloud Semantic Segmentation

arXiv 2605.31177 ↗First posted 29 May 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Plain Transformers have become the de-facto architecture for processing text, audio, image, and video, offering a unified backbone for multimodal learning. However, state-of-the-art architectures for point cloud semantic segmentation remain dominated by U-Nets architectures where convolutions are interleaved with local or windowed attentions. In this work, we show how to effectively leverage vanilla, non-hierarchical ViTs for segmentation of large-scale automotive lidar scenes. We bridge the performance gap thanks to a carefully designed tokenizer, a lightweight decoder segmentation head, and tailored data augmentations. Our approach, VaViT for Vanilla ViT, matches or exceeds the performance of state-of-the-art methods while maintaining the simplicity of ViT architecture. We provide extensive evaluations on nuScenes, SemanticKITTI, and Waymo Open Dataset to validate the efficiency of our method. Code and models are available at https://github.com/valeoai/VaViT.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 54 of its 63 references that are in the map. First posted on arXiv 29 May 2026.

Builds on

54 of 63 references.

References · 54

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  6. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  7. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  8. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  9. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  10. 2021ViViT: A Video Vision Transformercited by 1,620
  11. 2020Point Transformercited by 1,470
  12. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
38 more
  1. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  2. 2019KPConv: Flexible and Deformable Convolution for Point Cloudscited by 1,205
  3. 2021Scaling Vision Transformerscited by 999
  4. 2016Deep Networks with Stochastic Depthcited by 979
  5. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 694
  6. 2021AST: Audio Spectrogram Transformercited by 643
  7. 2023Point Transformer V3: Simpler, Faster, Strongercited by 589
  8. 2022PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategiescited by 436
  9. 2020Searching Efficient 3D Architectures with Sparse Point-Voxel Convolutioncited by 423
  10. 2022Multimodal Learning with Transformers: A Surveycited by 400
  11. 2020Cylindrical and Asymmetrical 3D Convolution Networks for LiDAR Segmentationcited by 389
  12. 2017Large-scale Point Cloud Semantic Segmentation with Superpoint Graphscited by 385
  13. 2022Rethinking Network Design and Local Geometry in Point Cloud: A Simple Residual MLP Frameworkcited by 379
  14. 2020PolarNet: An Improved Grid Representation for Online LiDAR Point Clouds Semantic Segmentationcited by 275
  15. 2024Rotary Position Embedding for Vision Transformercited by 236
  16. 2020SalsaNext: Fast, Uncertainty-aware Semantic Segmentation of LiDAR Point Clouds for Autonomous Drivingcited by 226
  17. 2020LMSCNet: Lightweight Multiscale 3D Semantic Completioncited by 188
  18. 2021RPVNet: A Deep and Efficient Range-Point-Voxel Fusion Network for LiDAR Point Cloud Segmentationcited by 173
  19. 2020SqueezeSegV3: Spatially-Adaptive Convolution for Efficient Point-Cloud Segmentationcited by 170
  20. 2022Stratified Transformer for 3D Point Cloud Segmentationcited by 167
  21. 2021(AF)2-S3Net: Attentive Feature Fusion with Adaptive Feature Selection for Sparse Semantic Segmentation Networkcited by 164
  22. 20222DPASS: 2D Priors Assisted Semantic Segmentation on LiDAR Point Cloudscited by 158
  23. 2023Rethinking Range View Representation for LiDAR Segmentationcited by 128
  24. 2023Spherical Transformer for LiDAR-based 3D Recognitioncited by 125
  25. 2022Point-to-Voxel Knowledge Distillation for LiDAR Semantic Segmentationcited by 119
  26. 2021Fast Point Transformercited by 104
  27. 2022LaserMix for Semi-Supervised LiDAR Semantic Segmentationcited by 93
  28. 2020AMVNet: Assertion-based Multi-View Fusion Network for LiDAR Semantic Segmentationcited by 87
  29. 2023RangeViT: Towards Vision Transformers for 3D Semantic Segmentation in Autonomous Drivingcited by 76
  30. 2022PolarMix: A General Data Augmentation Technique for LiDAR Point Cloudscited by 74
  31. 2023FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformercited by 73
  32. 2021PointMixer: MLP-Mixer for Point Cloud Understandingcited by 71
  33. 2023Efficient 3D Semantic Segmentation with Superpoint Transformercited by 57
  34. 2023Using a Waffle Iron for Automotive Point Cloud Semantic Segmentationcited by 53
  35. 2020KPRNet: Improving projection-based LiDAR semantic segmentationcited by 47
  36. 2023Three Pillars improving Vision Foundation Model Distillation for Lidarcited by 44
  37. 2020TORNADO-Net: mulTiview tOtal vaRiatioN semAntic segmentation with Diamond inceptiOn modulecited by 36
  38. 2022GFNet: Geometric Flow Network for 3D Point Cloud Semantic Segmentationcited by 36

50 most cited of 54 All 54 (JSON)

Built on by

Not cited yet.