# D-Align: Dual Query Co-attention Network for 3D Object Detection Based on Multi-frame Point Cloud Sequence

arXiv: https://arxiv.org/abs/2210.00087 · HTML: https://mlsanity.com/paper/2210.00087/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 29 references that are in the map.
- First posted on arXiv 30 Sep 2022.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection](https://mlsanity.com/paper/1711.06396/) (2017)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [3DSSD: Point-based 3D Single Stage Object Detector](https://mlsanity.com/paper/2002.10187/) (2020)
- [Class-balanced Grouping and Sampling for Point Cloud 3D Object Detection](https://mlsanity.com/paper/1908.09492/) (2019)
- [Unifying Voxel-based Representation with Transformer for 3D Object Detection](https://mlsanity.com/paper/2206.00630/) (2022)
- [Delving Deeper into Convolutional Networks for Learning Video Representations](https://mlsanity.com/paper/1511.06432/) (2015)
- [TANet: Robust 3D Object Detection from Point Clouds with Triple Attention](https://mlsanity.com/paper/1912.05163/) (2019)
- [AFDetV2: Rethinking the Necessity of the Second Stage for Object Detection from Point Clouds](https://mlsanity.com/paper/2112.09205/) (2021)
- [Object as Hotspots: An Anchor-Free 3D Object Detection Approach via Firing of Hotspots](https://mlsanity.com/paper/1912.12791/) (2019)
- [SSN: Shape Signature Networks for Multi-class Object Detection from Point Clouds](https://mlsanity.com/paper/2004.02774/) (2020)
- [3D-MAN: 3D Multi-frame Attention Network for Object Detection](https://mlsanity.com/paper/2103.16054/) (2021)
- [LiDAR-based Online 3D Video Object Detection with Graph-based Message Passing and Spatiotemporal Transformer Attention](https://mlsanity.com/paper/2004.01389/) (2020)
- [AFDet: Anchor Free One Stage 3D Object Detection](https://mlsanity.com/paper/2006.12671/) (2020)
- [Temporal-Channel Transformer for 3D Lidar-Based Video Object Detection in Autonomous Driving](https://mlsanity.com/paper/2011.13628/) (2020)
- [An LSTM Approach to Temporal 3D Object Detection in LiDAR Point Clouds](https://mlsanity.com/paper/2007.12392/) (2020)
- [Boosting 3D Object Detection by Simulating Multimodality on Point Clouds](https://mlsanity.com/paper/2206.14971/) (2022)
- [Robust Deep Multi-modal Learning Based on Gated Information Fusion Network](https://mlsanity.com/paper/1807.06233/) (2018)
- [It's All Around You: Range-Guided Cylindrical Network for 3D Object Detection](https://mlsanity.com/paper/2012.03121/) (2020)
- [SA-Det3D: Self-Attention Based Context-Aware 3D Object Detection](https://mlsanity.com/paper/2101.02672/) (2021)

## Built on by (most cited first, 1 of 1)

- [Grid-Centric Traffic Scenario Perception for Autonomous Driving: A Comprehensive Review](https://mlsanity.com/paper/2303.01212/) (2023)

## Abstract (arXiv)

LiDAR sensors are widely used for 3D object detection in various mobile robotics applications. LiDAR sensors continuously generate point cloud data in real-time. Conventional 3D object detectors detect objects using a set of points acquired over a fixed duration. However, recent studies have shown that the performance of object detection can be further enhanced by utilizing spatio-temporal information obtained from point cloud sequences. In this paper, we propose a new 3D object detector, named D-Align, which can effectively produce strong bird's-eye-view (BEV) features by aligning and aggregating the features obtained from a sequence of point sets. The proposed method includes a novel dual-query co-attention network that uses two types of queries, including target query set (T-QS) and support query set (S-QS), to update the features of target and support frames, respectively. D-Align aligns S-QS to T-QS based on the temporal context features extracted from the adjacent feature maps and then aggregates S-QS with T-QS using a gated attention mechanism. The dual queries are updated through multiple attention layers to progressively enhance the target frame features used to produce the detection results. Our experiments on the nuScenes dataset show that the proposed D-Align method greatly improved the performance of a single frame-based baseline method and significantly outperformed the latest 3D object detectors.