mlsanityData to 7 Oct 2026

TransPillars: Coarse-to-Fine Aggregation for Multi-Frame 3D Object Detection

arXiv 2208.03141 ↗First posted 4 Aug 2022cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

3D object detection using point clouds has attracted increasing attention due to its wide applications in autonomous driving and robotics. However, most existing studies focus on single point cloud frames without harnessing the temporal information in point cloud sequences. In this paper, we design TransPillars, a novel transformer-based feature aggregation technique that exploits temporal features of consecutive point cloud frames for multi-frame 3D object detection. TransPillars aggregates spatial-temporal point cloud features from two perspectives. First, it fuses voxel-level features directly from multi-frame feature maps instead of pooled instance features to preserve instance details with contextual information that are essential to accurate object localization. Second, it introduces a hierarchical coarse-to-fine strategy to fuse multi-scale features progressively to effectively capture the motion of moving objects and guide the aggregation of fine features. Besides, a variant of deformable transformer is introduced to improve the effectiveness of cross-frame feature matching. Extensive experiments show that our proposed TransPillars achieves state-of-art performance as compared to existing multi-frame detection approaches. Code will be released.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 43 of its 48 references that are in the map. First posted on arXiv 4 Aug 2022.

Builds on

43 of 48 references.

References · 43

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2020End-to-End Object Detection with Transformerscited by 7,384
  6. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  7. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  8. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  9. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  10. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  11. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  12. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,707
31 more
  1. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  2. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  3. 2020Center-based 3D Object Detection and Trackingcited by 1,143
  4. 2018PointRCNN: 3D Object Proposal Generation and Detection from Point Cloudcited by 939
  5. 2019PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detectioncited by 890
  6. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  7. 2019Deep Hough Voting for 3D Object Detection in Point Cloudscited by 648
  8. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  9. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  10. 2021CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windowscited by 526
  11. 20203DSSD: Point-based 3D Single Stage Object Detectorcited by 440
  12. 2019From Points to Parts: 3D Object Detection from Point Cloud with Part-aware and Part-aggregation Networkcited by 366
  13. 2017Relation Networks for Object Detectioncited by 305
  14. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 286
  15. 2019STD: Sparse-to-Dense 3D Object Detector for Point Cloudcited by 282
  16. 2020Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Netcited by 228
  17. 2021Offboard 3D Object Detection from Point Cloud Sequencescited by 144
  18. 2020Memory Enhanced Global-Local Aggregation for Video Object Detectioncited by 128
  19. 2021Generative Adversarial Transformerscited by 120
  20. 2021Shuffle Transformer: Rethinking Spatial Shuffle for Vision Transformercited by 115
  21. 2019Sequence Level Semantics Aggregation for Video Object Detectioncited by 87
  22. 20213D-MAN: 3D Multi-frame Attention Network for Object Detectioncited by 73
  23. 2020LiDAR-based Online 3D Video Object Detection with Graph-based Message Passing and Spatiotemporal Transformer Attentioncited by 65
  24. 2021PTTR: Relational 3D Point Cloud Object Tracking with Transformercited by 65
  25. 2022Accelerating DETR Convergence via Semantic-Aligned Matchingcited by 58
  26. 2020Temporal-Channel Transformer for 3D Lidar-Based Video Object Detection in Autonomous Drivingcited by 55
  27. 2021MSG-Transformer: Exchanging Local Spatial Information by Manipulating Messenger Tokenscited by 46
  28. 2020An LSTM Approach to Temporal 3D Object Detection in LiDAR Point Cloudscited by 45
  29. 2021Meta-DETR: Image-Level Few-Shot Object Detection with Inter-Class Correlation Exploitationcited by 44
  30. 2020Object-aware Feature Aggregation for Video Object Detectioncited by 3
  31. 2020Leveraging Temporal Information for 3D Detection and Domain Adaptationcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 1; 2025: 1.

6 papers.

  1. 2023Online Map Vectorization for Autonomous Driving: A Rasterization Perspectivecited by 54
  2. 2022Towards Efficient Use of Multi-Scale Features in Transformer-Based Object Detectorscited by 10
  3. 2023Spatial-Temporal Graph Enhanced DETR Towards Multi-Frame 3D Object Detectioncited by 8
  4. 2025Towards Scalable Spatial Intelligence via 2D-to-3D Data Liftingcited by 8
  5. 2024VADet: Multi-frame LiDAR 3D Object Detection using Variable Aggregationcited by 2
  6. 2023SUIT: Learning Significance-guided Information for 3D Temporal Detectioncited by 1