# A Unified 3D Object Perception Framework for Real-Time Outside-In Multi-Camera Systems

arXiv: https://arxiv.org/abs/2601.10819 · HTML: https://mlsanity.com/paper/2601.10819/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 30 references that are in the map.
- First posted on arXiv 15 Jan 2026.

## Builds on (most cited first, 24 of 24)

- [Efficient Geometry-aware 3D Generative Adversarial Networks](https://mlsanity.com/paper/2112.07945/) (2021)
- [Center-based 3D Object Detection and Tracking](https://mlsanity.com/paper/2006.11275/) (2020)
- [BEVDet: High-performance Multi-camera 3D Object Detection in Bird-Eye-View](https://mlsanity.com/paper/2112.11790/) (2021)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [DETR3D: 3D Object Detection from Multi-view Images via 3D-to-2D Queries](https://mlsanity.com/paper/2110.06922/) (2021)
- [HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking](https://mlsanity.com/paper/2009.07736/) (2020)
- [PETR: Position Embedding Transformation for Multi-View 3D Object Detection](https://mlsanity.com/paper/2203.05625/) (2022)
- [BEVFusion: A Simple and Robust LiDAR-Camera Fusion Framework](https://mlsanity.com/paper/2205.13790/) (2022)
- [Exploring Object-Centric Temporal Modeling for Efficient Multi-View 3D Object Detection](https://mlsanity.com/paper/2303.11926/) (2023)
- [BEVFormer v2: Adapting Modern Image Backbones to Bird's-Eye-View Recognition via Perspective Supervision](https://mlsanity.com/paper/2211.10439/) (2022)
- [Joint Discriminative and Generative Learning for Person Re-identification](https://mlsanity.com/paper/1904.07223/) (2019)
- [Multiple Object Tracking: A Literature Review](https://mlsanity.com/paper/1409.7618/) (2014)
- [Sparse4D: Multi-view 3D Object Detection with Sparse Spatial-Temporal Fusion](https://mlsanity.com/paper/2211.10581/) (2022)
- [DriveGAN: Towards a Controllable High-Quality Neural Simulation](https://mlsanity.com/paper/2104.15060/) (2021)
- [CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification](https://mlsanity.com/paper/1903.09254/) (2019)
- [Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control](https://mlsanity.com/paper/2503.14492/) (2025)
- [Sparse4D v2: Recurrent Temporal Fusion with Sparse Model](https://mlsanity.com/paper/2305.14018/) (2023)
- [Multi-modal Sensor Fusion for Auto Driving Perception: A Survey](https://mlsanity.com/paper/2202.02703/) (2022)
- [UniTR: A Unified and Efficient Multi-Modal Transformer for Bird's-Eye-View Representation](https://mlsanity.com/paper/2308.07732/) (2023)
- [Sparse4D v3: Advancing End-to-End 3D Detection and Tracking](https://mlsanity.com/paper/2311.11722/) (2023)
- [EarlyBird: Early-Fusion for Multi-View Tracking in the Bird's Eye View](https://mlsanity.com/paper/2310.13350/) (2023)
- [The 9th AI City Challenge](https://mlsanity.com/paper/2508.13564/) (2025)
- [MCBLT: Multi-Camera Multi-Object 3D Tracking in Long Videos](https://mlsanity.com/paper/2412.00692/) (2024)
- [Online 3D Multi-Camera Perception through Robust 2D Tracking and Depth-based Late Aggregation](https://mlsanity.com/paper/2509.09946/) (2025)

## Built on by (most cited first, 2 of 2)

- [Online Multi-Camera 3D Tracking via ID Prediction over Recurrent Sparse Queries](https://mlsanity.com/paper/2609.18363/) (2026)
- [Physical AI Smart Spaces: A Large-Scale Benchmark for Multi-Camera 3D Perception in Smart Spaces](https://mlsanity.com/paper/2610.02580/) (2026)

## Abstract (arXiv)

Accurate 3D object perception and multi-target multi-camera (MTMC) tracking are fundamental for the digital transformation of industrial infrastructure. However, transitioning "inside-out" autonomous driving models to "outside-in" static camera networks presents significant challenges due to heterogeneous camera placements and extreme occlusion. In this paper, we present an adapted Sparse4D framework specifically optimized for large-scale infrastructure environments. Our system leverages absolute world-coordinate geometric priors and introduces an occlusion-aware ReID embedding module to maintain identity stability across distributed sensor networks. To bridge the Sim2Real domain gap without manual labeling, we employ a generative data augmentation strategy using the NVIDIA COSMOS framework, creating diverse environmental styles that enhance the model's appearance-invariance. Evaluated on the AI City Challenge 2025 benchmark, our camera-only framework achieves a state-of-the-art HOTA of 45.22. Furthermore, we address real-time deployment constraints by developing an optimized TensorRT plugin for Multi-Scale Deformable Aggregation (MSDA). Our hardware-accelerated implementation achieves a 2.15× speedup on modern GPU architectures, enabling a single Blackwell-class GPU to support over 64 concurrent camera streams.