mlsanityData to 14 Oct 2026

Graph Neural Network and Spatiotemporal Transformer Attention for 3D Video Object Detection from Point Clouds

arXiv 2207.12659 ↗First posted 26 Jul 2022cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

Previous works for LiDAR-based 3D object detection mainly focus on the single-frame paradigm. In this paper, we propose to detect 3D objects by exploiting temporal information in multiple frames, i.e., the point cloud videos. We empirically categorize the temporal information into short-term and long-term patterns. To encode the short-term data, we present a Grid Message Passing Network (GMPNet), which considers each grid (i.e., the grouped points) as a node and constructs a k-NN graph with the neighbor grids. To update features for a grid, GMPNet iteratively collects information from its neighbors, thus mining the motion cues in grids from nearby frames. To further aggregate the long-term frames, we propose an Attentive Spatiotemporal Transformer GRU (AST-GRU), which contains a Spatial Transformer Attention (STA) module and a Temporal Transformer Attention (TTA) module. STA and TTA enhance the vanilla GRU to focus on small objects and better align the moving objects. Our overall framework supports both online and offline video object detection in point clouds. We implement our algorithm based on prevalent anchor-based and anchor-free detectors. The evaluation results on the challenging nuScenes benchmark show the superior performance of our method, achieving the 1st on the leaderboard without any bells and whistles, by the time the paper is submitted.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 84 of its 100 references that are in the map. First posted on arXiv 26 Jul 2022.

Builds on

84 of 100 references.

References · 84

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  5. 2020End-to-End Object Detection with Transformerscited by 7,414
  6. 2017Inductive Representation Learning on Large Graphscited by 6,320
  7. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  8. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  9. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  10. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  11. 2017Neural Message Passing for Quantum Chemistrycited by 3,487
  12. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
38 more
  1. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  2. 2017Non-local Neural Networkscited by 2,293
  3. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  4. 2016Convolutional Neural Networks on Graphs with Fast Localized Spectral Filteringcited by 2,089
  5. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  6. 2017Modeling Relational Data with Graph Convolutional Networkscited by 1,694
  7. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,436
  8. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  9. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
  10. 2020Center-based 3D Object Detection and Trackingcited by 1,144
  11. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  12. 2013Spectral Networks and Locally Connected Networks on Graphscited by 1,102
  13. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 941
  14. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 891
  15. 2015Gated Graph Sequence Neural Networkscited by 796
  16. 2015Convolutional Networks on Graphs for Learning Molecular Fingerprintscited by 754
  17. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 724
  18. 2018Deformable ConvNets v2: More Deformable, Better Resultscited by 670
  19. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  20. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  21. 2015Cyclical Learning Rates for Training Neural Networkscited by 629
  22. 2016Interaction Networks for Learning about Objects, Relations and Physicscited by 509
  23. 2019Graph U-Netscited by 492
  24. 2018Pseudo-LiDAR from Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Drivingcited by 457
  25. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  26. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  27. 2015Unsupervised Learning of Video Representations using LSTMscited by 435
  28. 2019PIXOR: Real-time 3D Object Detection from Point Cloudscited by 358
  29. 2019Class-balanced Grouping and Sampling for Point Cloud 3D Object Detectioncited by 350
  30. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 326
  31. 2016Learning Convolutional Neural Networks for Graphscited by 314
  32. 2019STD: Sparse-to-Dense 3D Object Detector for Point Cloudcited by 282
  33. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  34. 2015Deep Convolutional Networks on Graph-Structured Datacited by 246
  35. 2016Molecular Graph Convolutions: Moving Beyond Fingerprintscited by 238
  36. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  37. 2019Disentangling Monocular 3D Object Detectioncited by 217
  38. 2020Multi-Task Multi-Sensor Fusion for 3D Object Detectioncited by 203

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 6; 2023: 4; 2024: 3; 2025: 4.

17 papers.

Sort:

By area:

  1. 2024IS-Fusion: Instance-Scene Collaborative Fusion for Multimodal 3D Object Detectioncited by 57
  2. 2022SSDA3D: Semi-supervised Domain Adaptation for 3D Object Detection from Point Cloudcited by 37
  3. 2023Weakly Supervised Monocular 3D Object Detection using Multi-View Projection and Direction Consistencycited by 18
  4. 2022Point Cloud Attacks in Graph Spectral Domain: When 3D Geometry Meets Graph Signal Processingcited by 16
  5. 2022MGTANet: Encoding Sequential LiDAR Points Using Long Short-Term Motion-Guided Temporal Attention for 3D Object Detectioncited by 15
  6. 2024OLiDM: Object-aware LiDAR Diffusion Models for Autonomous Drivingcited by 13
  7. 2022LWSIS: LiDAR-guided Weakly Supervised Instance Segmentation for Autonomous Drivingcited by 9
  8. 2024Enhancing Traffic Safety with Parallel Dense Video Captioning for End-to-End Event Analysiscited by 9
  9. 2023Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detectioncited by 8
  10. 2022Boosting Monocular 3D Object Detection with Object-Centric Auxiliary Depth Supervisioncited by 7
  11. 2023Fuzzy-NMS: Improving 3D Object Detection with Fuzzy Classification in NMScited by 3
  12. 2023Hierarchical Graph Pattern Understanding for Zero-Shot VOScited by 2
5 more
  1. 2025M^3Detection: Multi-Frame Multi-Level Feature Fusion for Multi-Modal 3D Object Detection with Camera and 4D Imaging Radarcited by 1
  2. 2022Leveraging Multi-stream Information Fusion for Trajectory Prediction in Low-illumination Scenarios: A Multi-channel Graph Convolutional Approachcited by 0
  3. 2025Dual-Domain Homogeneous Fusion with Cross-Modal Mamba and Progressive Decoder for 3D Object Detectioncited by 0
  4. 2025Multimodal Spatio-temporal Graph Learning for Alignment-free RGBT Video Object Detectioncited by 0
  5. 2025PointSlice: Accurate and Efficient Slice-Based Representation for 3D Object Detection from Point Cloudscited by 0