MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequences
Not on the map. Why?
Abstract (arXiv)
Understanding dynamic 3D environment is crucial for robotic agents and many other applications. We propose a novel neural network architecture called MeteorNet for learning representations for dynamic 3D point cloud sequences. Different from previous work that adopts a grid-based representation and applies 3D or 4D convolutions, our network directly processes point clouds. We propose two ways to construct spatiotemporal neighborhoods for each point in the point cloud sequence. Information from these neighborhoods is aggregated to learn features per point. We benchmark our network on a variety of 3D recognition tasks including action recognition, semantic segmentation and scene flow estimation. MeteorNet shows stronger performance than previous grid-based methods while achieving state-of-the-art performance on Synthia. MeteorNet also outperforms previous baseline methods that are able to process at most two consecutive point clouds. To the best of our knowledge, this is the first work on deep learning for dynamic raw point cloud sequences.
Key facts. Cited by 117 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Oct 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 22; 2022: 25; 2023: 32; 2024: 13; 2025: 12; 2026: 13.
117 papers.
Sort:
By area:
- 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
- 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
- 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
- 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 144
- 20214DComplete: Non-Rigid Motion Estimation Beyond the Observable Surfacecited by 123
- 2021Neural Scene Flow Priorcited by 91
- 2022PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequencescited by 68
- 2021Weakly Supervised Learning of Rigid 3D Scene Flowcited by 60
- 2021CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimationcited by 56
- 2021Learning Semantic Segmentation of Large-Scale Point Clouds with Random Samplingcited by 51
- 2021Scalable Scene Flow from Point Clouds in the Real Worldcited by 49
- 2022Dynamic 3D Scene Analysis by Point Cloud Accumulationcited by 47
38 more
- 2021MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronizationcited by 46
- 2021StereOBJ-1M: Large-scale Stereo Image Dataset for 6D Object Pose Estimationcited by 40
- 2023Fast Neural Scene Flowcited by 39
- 2023Geometric Clifford Algebra Networkscited by 37
- 2021HCRF-Flow: Scene Flow from Point Clouds with Continuous High-order CRFs and Position-aware Flow Embeddingcited by 36
- 2022Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutionscited by 35
- 2022Point Primitive Transformer for Long-Term 4D Point Cloud Video Understandingcited by 35
- 2023Building Rearticulable Models for Arbitrary 3D Objects from 4D Point Cloudscited by 34
- 2024ICP-Flow: LiDAR Scene Flow Estimation with ICPcited by 34
- 2021Learning Parallel Dense Correspondence from Spatio-Temporal Descriptors for Efficient and Robust 4D Reconstructioncited by 31
- 2022What Matters for 3D Scene Flow Networkcited by 31
- 2021FESTA: Flow Estimation via Spatial-Temporal Attention for Scene Point Cloudscited by 30
- 2022Complete-to-Partial 4D Distillation for Self-Supervised Point Cloud Sequence Representation Learningcited by 30
- 2022DiffCloud: Real-to-Sim from Point Clouds with Differentiable Simulation and Rendering of Deformable Objectscited by 29
- 2023RPEFlow: Multimodal Fusion of RGB-PointCloud-Event for Joint Optical Flow and Scene Flow Estimationcited by 26
- 2021Efficient 3D Deep LiDAR Odometrycited by 25
- 2022No Pain, Big Gain: Classify Dynamic Point Cloud Sequences with Static Models by Fitting Feature-level Space-time Surfacescited by 25
- 2022Domain generalization of 3D semantic segmentation in autonomous drivingcited by 25
- 2022OGC: Unsupervised 3D Object Segmentation from Rigid Dynamics of Point Cloudscited by 24
- 2023Self-Supervised 3D Scene Flow Estimation Guided by Superpointscited by 24
- 2023Advancements in Point Cloud Data Augmentation for Deep Learning: A Surveycited by 24
- 2021Spatial-Temporal Transformer for 3D Point Cloud Sequencescited by 22
- 2023PointCMP: Contrastive Mask Prediction for Self-supervised Learning on Point Cloud Videoscited by 22
- 20223D Object Detection with a Self-supervised Lidar Scene Flow Backbonecited by 19
- 2023Banana: Banach Fixed-Point Network for Pointcloud Segmentation with Inter-Part Equivariancecited by 19
- 2023Masked Spatio-Temporal Structure Prediction for Self-supervised Learning on Point Cloud Videoscited by 19
- 2022Revealing Occlusions with 4D Neural Fieldscited by 17
- 2023Re-Evaluating LiDAR Scene Flow for Autonomous Drivingcited by 17
- 2026Align then Adapt: Rethinking Parameter-Efficient Transfer Learning in 4D Perceptioncited by 17
- 2023Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videoscited by 16
- 2023DELFlow: Dense Efficient Learning of Scene Flow for Large-Scale Point Cloudscited by 14
- 2022RMS-FlowNet: Efficient and Robust Multi-Scale Scene Flow Estimation for Large-Scale Point Cloudscited by 13
- 2023SVQNet: Sparse Voxel-Adjacent Query Network for 4D Spatio-Temporal LiDAR Semantic Segmentationcited by 13
- 2023Multi-Body Neural Scene Flowcited by 12
- 2023X4D-SceneFormer: Enhanced Scene Understanding on 4D Point Cloud Videos through Cross-modal Knowledge Transfercited by 12
- 2023PCPNet: An Efficient and Semantic-Enhanced Transformer Network for Point Cloud Predictioncited by 11
- 2024VG4D: Vision-Language Model Goes 4D Video Recognitioncited by 11
- 2024MAMBA4D: Efficient Long-Sequence Point Cloud Video Understanding with Disentangled Spatial-Temporal State Space Modelscited by 11
50 most cited of 117 All 117 (JSON)
- 2026Motion-Saliency Complementary Masked Modeling for Point Cloud Video Understanding31 Aug 2026
- 2026Joint-Embedding Prediction of Masked Point Tubes for Self-Supervised Learning on 4D Point Cloud Videos25 Aug 2026
- 2026RVLoss: Runoff Vote Loss for Self-Supervised LiDAR Scene Flow Estimation19 Aug 2026
- 2026ODeform: Learning Continuous 4D Motion for Shape Deformation with Neural ODEs22 Jul 2026
- 2026Tri-Efficient Transfer Learning for Point Cloud Videos23 Jun 2026
- 2026SRENet: Spectral Re-Entry Network for Point Cloud Action Recognition2 Jun 2026
- 2026Paving the Way for Point Cloud Video Representation Learning Using A PDE Model1 Jun 2026
- 2026Point Cloud Sequence Encoding for Material-conditioned Graph Network Simulators20 May 2026
- 2026STS-Mixer: Spatio-Temporal-Spectral Mixer for 4D Point Cloud Video Understanding13 Apr 2026
- 2026GATS: Gaussian Aware Temporal Scaling Transformer for Invariant 4D Spatio-Temporal Point Cloud Representation17 Mar 2026
- 2026Align then Adapt: Rethinking Parameter-Efficient Transfer Learning in 4D Perception26 Feb 2026
- 2026X-VORTEX: Spatio-Temporal Contrastive Learning for Wake Vortex Trajectory Forecasting13 Feb 2026
38 more
- 20264DPC2hat: Towards Dynamic Point Cloud Understanding with Failure-Aware Bootstrapping3 Feb 2026
- 2025Exploring Spatial-Temporal Representation via Star Graph for mmWave Radar-based Human Activity Recognition12 Dec 2025
- 2025PointNet4D: A Lightweight 4D Point Cloud Video Backbone for Online and Offline Perception in Robotic Applications1 Dec 2025
- 2025DELTAv2: Accelerating Dense 3D Tracking2 Aug 2025
- 2025Part Segmentation and Motion Estimation for Articulated Objects with Dynamic 3D Gaussians28 Jun 2025
- 2025DeSPITE: Exploring Contrastive Deep Skeleton-Pointcloud-IMU-Text Embeddings for Advanced Point Cloud Human Activity Understanding16 Jun 2025
- 2025Quality assessment of 3D human animation: Subjective and objective evaluation29 May 2025
- 2025Content-Distortion High-Order Interaction for Blind Image Quality Assessment7 Apr 2025
- 2025Uni4D: A Unified Self-Supervised Learning Framework for Point Cloud Videos7 Apr 2025
- 2025EMoTive: Event-guided Trajectory Modeling for 3D Motion Estimation14 Mar 2025
- 2025SDD-4DGS: Static-Dynamic Aware Decoupling in Gaussian Splatting for 4D Scene Reconstruction12 Mar 2025
- 2025Road Boundary Detection Using 4D mmWave Radar for Autonomous Driving3 Mar 2025
- 2025Quasi Zigzag Persistence: A Topological Framework for Analyzing Time-Varying Data22 Feb 2025
- 2024SPiKE: 3D Human Pose from Point Cloud Sequences3 Sep 2024
- 2024SiamMo: Siamese Motion-Centric 3D Object Tracking3 Aug 2024
- 2024PAPR in Motion: Seamless Point-level 3D Scene Interpolation8 Jun 2024
- 2024MAMBA4D: Efficient Long-Sequence Point Cloud Video Understanding with Disentangled Spatial-Temporal State Space Models23 May 2024
- 2024PRENet: A Plane-Fit Redundancy Encoding Point Cloud Sequence Network for Real-Time 3D Action Recognition11 May 2024
- 2024VG4D: Vision-Language Model Goes 4D Video Recognition17 Apr 2024
- 2024Let-It-Flow: Simultaneous Optimization of 3D Flow and Object Clustering12 Apr 2024
- 2024A Unified Framework for Human-centric Point Cloud Video Understanding29 Mar 2024
- 2024Self-Supervised Multi-Frame Neural Scene Flow24 Mar 2024
- 2024Fast Kernel Scene Flow9 Mar 2024
- 2024Dynamic 3D Point Cloud Sequences as 2D Videos2 Mar 2024
- 2024ICP-Flow: LiDAR Scene Flow Estimation with ICP27 Feb 2024
- 2024CrossVideo: Self-supervised Cross-modal Contrastive Learning for Point Cloud Video Understanding17 Jan 2024
- 2023Regularizing Self-supervised 3D Scene Flows with Surface Awareness and Cyclic Consistency12 Dec 2023
- 2023X4D-SceneFormer: Enhanced Scene Understanding on 4D Point Cloud Videos through Cross-modal Knowledge Transfer12 Dec 2023
- 2023Deep learning for 3D Object Detection and Tracking in Autonomous Driving: A Brief Survey10 Nov 2023
- 2023Self-Supervised 3D Scene Flow Estimation and Motion Prediction using Local Rigidity Prior17 Oct 2023
- 2023Multi-Body Neural Scene Flow16 Oct 2023
- 20234D Gaussian Splatting for Real-Time Dynamic Scene Rendering12 Oct 2023
- 2023NSM4D: Neural Scene Model Based Online 4D Point Cloud Sequence Understanding12 Oct 2023
- 2023RPEFlow: Multimodal Fusion of RGB-PointCloud-Event for Joint Optical Flow and Scene Flow Estimation26 Sep 2023
- 2023SVQNet: Sparse Voxel-Adjacent Query Network for 4D Spatio-Temporal LiDAR Semantic Segmentation25 Aug 2023
- 2023Advancements in Point Cloud Data Augmentation for Deep Learning: A Survey23 Aug 2023
- 2023Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videos18 Aug 2023
- 2023Masked Spatio-Temporal Structure Prediction for Self-supervised Learning on Point Cloud Videos18 Aug 2023
50 newest of 117 All 117 (JSON)
- 20234D Gaussian Splatting for Real-Time Dynamic Scene Renderingcited by 1,144
- 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
- 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 144
- 20214DComplete: Non-Rigid Motion Estimation Beyond the Observable Surfacecited by 123
- 2021Neural Scene Flow Priorcited by 91
- 2022PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequencescited by 68
- 2021Weakly Supervised Learning of Rigid 3D Scene Flowcited by 60
- 2021CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimationcited by 56
- 2021Learning Semantic Segmentation of Large-Scale Point Clouds with Random Samplingcited by 51
- 2021Scalable Scene Flow from Point Clouds in the Real Worldcited by 49
- 2022Dynamic 3D Scene Analysis by Point Cloud Accumulationcited by 47
- 2021MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronizationcited by 46
12 most cited of 112 All 112 (JSON)
- 2026Motion-Saliency Complementary Masked Modeling for Point Cloud Video Understanding31 Aug 2026
- 2026Joint-Embedding Prediction of Masked Point Tubes for Self-Supervised Learning on 4D Point Cloud Videos25 Aug 2026
- 2026RVLoss: Runoff Vote Loss for Self-Supervised LiDAR Scene Flow Estimation19 Aug 2026
- 2026ODeform: Learning Continuous 4D Motion for Shape Deformation with Neural ODEs22 Jul 2026
- 2026Tri-Efficient Transfer Learning for Point Cloud Videos23 Jun 2026
- 2026SRENet: Spectral Re-Entry Network for Point Cloud Action Recognition2 Jun 2026
- 2026Paving the Way for Point Cloud Video Representation Learning Using A PDE Model1 Jun 2026
- 2026Point Cloud Sequence Encoding for Material-conditioned Graph Network Simulators20 May 2026
- 2026STS-Mixer: Spatio-Temporal-Spectral Mixer for 4D Point Cloud Video Understanding13 Apr 2026
- 2026GATS: Gaussian Aware Temporal Scaling Transformer for Invariant 4D Spatio-Temporal Point Cloud Representation17 Mar 2026
- 2026Align then Adapt: Rethinking Parameter-Efficient Transfer Learning in 4D Perception26 Feb 2026
- 2026X-VORTEX: Spatio-Temporal Contrastive Learning for Wake Vortex Trajectory Forecasting13 Feb 2026
12 newest of 112 All 112 (JSON)