mlsanityData to 14 Oct 2026

T-MAE: Temporal Masked Autoencoders for Point Cloud Representation Learning

arXiv 2312.10217 ↗First posted 15 Dec 2023cs.CVcited by 11

Not on the map. Why?

Abstract (arXiv)

The scarcity of annotated data in LiDAR point cloud understanding hinders effective representation learning. Consequently, scholars have been actively investigating efficacious self-supervised pre-training paradigms. Nevertheless, temporal information, which is inherent in the LiDAR point cloud sequence, is consistently disregarded. To better utilize this property, we propose an effective pre-training strategy, namely Temporal Masked Auto-Encoders (T-MAE), which takes as input temporally adjacent frames and learns temporal dependency. A SiamWCA backbone, containing a Siamese encoder and a windowed cross-attention (WCA) module, is established for the two-frame input. Considering that the movement of an ego-vehicle alters the view of the same instance, temporal modeling also serves as a robust and natural data augmentation, enhancing the comprehension of target objects. SiamWCA is a powerful architecture but heavily relies on annotated data. Our T-MAE pre-training strategy alleviates its demand for annotated data. Comprehensive experiments demonstrate that T-MAE achieves the best performance on both Waymo and ONCE datasets among competitive self-supervised approaches. Codes will be released at https://github.com/codename1995/T-MAE

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 60 of its 73 references that are in the map. First posted on arXiv 15 Dec 2023.

Builds on

60 of 73 references.

References · 60

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020End-to-End Object Detection with Transformerscited by 7,414
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  8. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  10. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  11. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  12. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
38 more
  1. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,616
  2. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  3. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  4. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 891
  5. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 748
  6. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 653
  7. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 576
  8. 2016Fully-Convolutional Siamese Networks for Object Trackingcited by 568
  9. 2020PointContrast: Unsupervised Pre-training for 3D Point Cloud Understandingcited by 504
  10. 2022Masked Autoencoders for Point Cloud Self-supervised Learningcited by 459
  11. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 429
  12. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 264
  13. 2021One Million Scenes for Autonomous Driving: ONCE Datasetcited by 218
  14. 2021Self-Supervised Pretraining of 3D Features on any Point-Cloudcited by 217
  15. 2020Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contextscited by 210
  16. 2021Embracing Single Stride 3D Object Detector with Sparse Transformercited by 193
  17. 2021Spatio-temporal Self-Supervised Representation Learning for 3D Point Cloudscited by 180
  18. 2022Masked Discrimination for Self-Supervised Learning on Point Cloudscited by 155
  19. 2022Learning 3D Representations from 2D Pre-trained Models via Image-to-Point Masked Autoencoderscited by 144
  20. 2022Not All Points Are Equal: Learning Highly Efficient Point-based Detectors for 3D LiDAR Point Cloudscited by 131
  21. 2021SuMa++: Efficient LiDAR-based Semantic SLAMcited by 100
  22. 2022SWFormer: Sparse Window Transformer for 3D Object Detection in Point Cloudscited by 92
  23. 2023Siamese Masked Autoencoderscited by 76
  24. 20213D-MAN: 3D Multi-frame Attention Network for Object Detectioncited by 73
  25. 2022ALSO: Automotive Lidar Self-supervision by Occupancy estimationcited by 68
  26. 2022Crafting Better Contrastive Views for Siamese Representation Learningcited by 67
  27. 2018A Twofold Siamese Network for Real-Time Object Trackingcited by 62
  28. 2016Siamese Instance Search for Trackingcited by 60
  29. 2022Occupancy-MAE: Self-supervised Pre-training Large-scale LiDAR Point Clouds with Masked Occupancy Autoencoderscited by 58
  30. 2022GD-MAE: Generative Decoder for MAE Pre-training on LiDAR Point Cloudscited by 57
  31. 2022Unsupervised Point Cloud Representation Learning with Deep Neural Networks: A Surveycited by 54
  32. 2022Graph R-CNN: Towards Accurate 3D Object Detection with Semantic-Decorated Local Graphcited by 42
  33. 2023Masked Motion Predictors are Strong 3D Action Representation Learnerscited by 41
  34. 2023MGMAE: Motion Guided Masking for Video Masked Autoencodingcited by 41
  35. 2021CT-ICP: Real-time Elastic LiDAR Odometry with Loop Closurecited by 38
  36. 2023GeoMAE: Masked Geometric Target Prediction for Self-supervised Point Cloud Pre-Trainingcited by 38
  37. 2021RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detectioncited by 37
  38. 2022Masked Autoencoders in 3D Point Cloud Representation Learningcited by 30

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 7; 2026: 2.

11 papers.

  1. 20244D Contrastive Superflows are Dense 3D Representation Learnerscited by 33
  2. 2025Self-Supervised Representation Learning with Joint Embedding Predictive Architecture for Automotive LiDAR Object Detectioncited by 13
  3. 2025Masked Point-Entity Contrast for Open-Vocabulary 3D Scene Understandingcited by 10
  4. 2025TurboTrain: Towards Efficient and Balanced Multi-Task Learning for Multi-Agent Perception and Predictioncited by 9
  5. 2025Advances in 4D Representation: Geometry, Motion, and Interactioncited by 4
  6. 2025Forging Spatial Intelligence: A Roadmap of Multi-Modal Data Pre-Training for Autonomous Systemscited by 4
  7. 2024TREND: Unsupervised 3D Representation Learning via Temporal Forecasting for LiDAR Perceptioncited by 2
  8. 2025Temporal Overlapping Prediction: A Self-supervised Pre-training Method for LiDAR Moving Object Segmentationcited by 0
  9. 2025LiDAR-BIND-T: Improved and Temporally Consistent Sensor Modality Translation and Fusion for Robotic Applicationscited by 0
  10. 2026Hierarchical Mesh Transformers with Topology-Guided Pretraining for Morphometric Analysis of Brain Structurescited by 0
  11. 2026FILTR: Extracting Topological Features from Pretrained 3D Modelscited by 0