# Pixel-Aligned Recurrent Queries for Multi-View 3D Object Detection

arXiv: https://arxiv.org/abs/2310.01401 · HTML: https://mlsanity.com/paper/2310.01401/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 51 references that are in the map.
- First posted on arXiv 2 Oct 2023.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes](https://mlsanity.com/paper/1702.04405/) (2017)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [BEVFormer: Learning Bird's-Eye-View Representation from Multi-Camera Images via Spatiotemporal Transformers](https://mlsanity.com/paper/2203.17270/) (2022)
- [On the Continuity of Rotation Representations in Neural Networks](https://mlsanity.com/paper/1812.07035/) (2018)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitization](https://mlsanity.com/paper/1905.05172/) (2019)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [PETR: Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2203.05625/) (2022)
- [ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data](https://mlsanity.com/paper/2111.08897/) (2021)
- [PETRv2: A Unified Framework for 3D Perception from Multi-Camera Images](https://mlsanity.com/paper/2206.01256/) (2022)
- [BEVDet4D: Exploit Temporal Cues in Multi-camera 3D Object Detection](https://mlsanity.com/paper/2203.17054/) (2022)
- [NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Video](https://mlsanity.com/paper/2104.00681/) (2021)
- [Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistency](https://mlsanity.com/paper/1704.06254/) (2017)
- [ImVoxelNet: Image to Voxels Projection for Monocular and Multi-View General-Purpose 3D Object Detection](https://mlsanity.com/paper/2106.01178/) (2021)
- [Scan2CAD: Learning CAD Model Alignment in RGB-D Scans](https://mlsanity.com/paper/1811.11187/) (2018)
- [Kinematic 3D Object Detection in Monocular Video](https://mlsanity.com/paper/2007.09548/) (2020)
- [SimpleRecon: 3D Reconstruction Without 3D Convolutions](https://mlsanity.com/paper/2208.14743/) (2022)
- [Joint Monocular 3D Vehicle Detection and Tracking](https://mlsanity.com/paper/1811.10742/) (2018)
- [FroDO: From Detections to 3D Objects](https://mlsanity.com/paper/2005.05125/) (2020)
- [NeRF-Det: Learning Geometry-Aware Volumetric Representation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2307.14620/) (2023)

## Built on by (most cited first, 14 of 14)

- [EFM3D: A Benchmark for Measuring Progress Towards 3D Egocentric Foundation Models](https://mlsanity.com/paper/2406.10224/) (2024)
- [DaReNeRF: Direction-aware Representation for Dynamic Scenes](https://mlsanity.com/paper/2403.02265/) (2024)
- [Boxer: Robust Lifting of Open-World 2D Bounding Boxes to 3D](https://mlsanity.com/paper/2604.05212/) (2026)
- [GO-N3RDet: Geometry Optimized NeRF-enhanced 3D Object Detector](https://mlsanity.com/paper/2503.15211/) (2025)
- [NeRF-Det++: Incorporating Semantic Cues and Perspective-aware Depth Supervision for Indoor Multi-View 3D Detection](https://mlsanity.com/paper/2402.14464/) (2024)
- [Redundant Queries in DETR-Based 3D Detection Methods: Unnecessary and Prunable](https://mlsanity.com/paper/2412.02054/) (2024)
- [NeRF-DetS: Enhanced Adaptive Spatial-wise Sampling and View-wise Fusion Strategies for NeRF-based Indoor Multi-view 3D Object Detection](https://mlsanity.com/paper/2404.13921/) (2024)
- [Boosting Multi-View Indoor 3D Object Detection via Adaptive 3D Volume Construction](https://mlsanity.com/paper/2507.18331/) (2025)
- [ODTFormer: Efficient Obstacle Detection and Tracking with Stereo Cameras Based on Transformer](https://mlsanity.com/paper/2403.14626/) (2024)
- [VisDiff: SDF-Guided Polygon Generation for Visibility Reconstruction and Recognition](https://mlsanity.com/paper/2410.05530/) (2024)
- [DaRePlane: Direction-aware Representations for Dynamic Scene Reconstruction](https://mlsanity.com/paper/2410.14169/) (2024)
- [White Aggregation and Restoration for Few-shot 3D Point Cloud Semantic Segmentation](https://mlsanity.com/paper/2509.13907/) (2025)
- [Sparse Multiview Open-Vocabulary 3D Detection](https://mlsanity.com/paper/2509.15924/) (2025)
- [ShelfChange3D: Object-Level 3D Change Detection for Retail Shelf Monitoring](https://mlsanity.com/paper/2610.01283/) (2026)

## Abstract (arXiv)

We present PARQ - a multi-view 3D object detector with transformer and pixel-aligned recurrent queries. Unlike previous works that use learnable features or only encode 3D point positions as queries in the decoder, PARQ leverages appearance-enhanced queries initialized from reference points in 3D space and updates their 3D location with recurrent cross-attention operations. Incorporating pixel-aligned features and cross attention enables the model to encode the necessary 3D-to-2D correspondences and capture global contextual information of the input images. PARQ outperforms prior best methods on the ScanNet and ARKitScenes datasets, learns and detects faster, is more robust to distribution shifts in reference points, can leverage additional input views without retraining, and can adapt inference compute by changing the number of recurrent iterations.