# Multistream Network for LiDAR and Camera-based 3D Object Detection in Outdoor Scenes

arXiv: https://arxiv.org/abs/2507.19304 · HTML: https://mlsanity.com/paper/2507.19304/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 51 references that are in the map.
- First posted on arXiv 25 Jul 2025.

## Builds on (most cited first, 30 of 31)

- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [PointPillars: Fast Encoders for Object Detection from Point Clouds](https://mlsanity.com/paper/1812.05784/) (2018)
- [PV-RCNN: Point-Voxel Feature Set Abstraction for 3D Object Detection](https://mlsanity.com/paper/1912.13192/) (2019)
- [Multi-View 3D Object Detection Network for Autonomous Driving](https://mlsanity.com/paper/1611.07759/) (2016)
- [TransFusion: Robust LiDAR-Camera Fusion for 3D Object Detection with Transformers](https://mlsanity.com/paper/2203.11496/) (2022)
- [Voxel R-CNN: Towards High Performance Voxel-based 3D Object Detection](https://mlsanity.com/paper/2012.15712/) (2020)
- [Multi-Modal Fusion Transformer for End-to-End Autonomous Driving](https://mlsanity.com/paper/2104.09224/) (2021)
- [Joint 3D Proposal Generation and Object Detection from View Aggregation](https://mlsanity.com/paper/1712.02294/) (2017)
- [Voxel Transformer for 3D Object Detection](https://mlsanity.com/paper/2109.02497/) (2021)
- [Multi-Task Multi-Sensor Fusion for 3D Object Detection](https://mlsanity.com/paper/2012.12397/) (2020)
- [CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detection](https://mlsanity.com/paper/2009.00784/) (2020)
- [Offboard 3D Object Detection from Point Cloud Sequences](https://mlsanity.com/paper/2103.05073/) (2021)
- [Focal Sparse Convolutional Networks for 3D Object Detection](https://mlsanity.com/paper/2204.12463/) (2022)
- [SE-SSD: Self-Ensembling Single-Stage Object Detector From Point Cloud](https://mlsanity.com/paper/2104.09804/) (2021)
- [Improving 3D Object Detection with Channel-wise Transformer](https://mlsanity.com/paper/2108.10723/) (2021)
- [Sparse Fuse Dense: Towards High Quality 3D Detection with Depth Completion](https://mlsanity.com/paper/2203.09780/) (2022)
- [Virtual Sparse Convolution for Multimodal 3D Object Detection](https://mlsanity.com/paper/2303.02314/) (2023)
- [Pyramid R-CNN: Towards Better Performance and Adaptability for 3D Object Detection](https://mlsanity.com/paper/2109.02499/) (2021)
- [M3DeTR: Multi-representation, Multi-scale, Mutual-relation 3D Object Detection with Transformers](https://mlsanity.com/paper/2104.11896/) (2021)
- [Behind the Curtain: Learning Occluded Shapes for 3D Object Detection](https://mlsanity.com/paper/2112.02205/) (2021)
- [Transformation-Equivariant 3D Object Detection for Autonomous Driving](https://mlsanity.com/paper/2211.11962/) (2022)
- [VPFNet: Improving 3D Object Detection with Virtual Point based LiDAR and Stereo Data Fusion](https://mlsanity.com/paper/2111.14382/) (2021)
- [Graph R-CNN: Towards Accurate 3D Object Detection with Semantic-Decorated Local Graph](https://mlsanity.com/paper/2208.03624/) (2022)
- [OcTr: Octree-based Transformer for 3D Object Detection](https://mlsanity.com/paper/2303.12621/) (2023)
- [Object-Centric Stereo Matching for 3D Object Detection](https://mlsanity.com/paper/1909.07566/) (2019)
- [PVT-SSD: Single-Stage 3D Object Detector with Point-Voxel Transformer](https://mlsanity.com/paper/2305.06621/) (2023)
- [Dense Voxel Fusion for 3D Object Detection](https://mlsanity.com/paper/2203.00871/) (2022)
- [LiDAR-CS Dataset: LiDAR Point Cloud Dataset with Cross-Sensors for 3D Object Detection](https://mlsanity.com/paper/2301.12515/) (2023)
- [3D Dual-Fusion: Dual-Domain Dual-Query Camera-LiDAR Fusion for 3D Object Detection](https://mlsanity.com/paper/2211.13529/) (2022)
- [MLF-DET: Multi-Level Fusion for Cross-Modal 3D Object Detection](https://mlsanity.com/paper/2307.09155/) (2023)

## Built on by (most cited first, 2 of 2)

- [BAWSeg: A UAV Multispectral Benchmark for Barley Weed Segmentation](https://mlsanity.com/paper/2603.01932/) (2026)
- [Fusion4CA: Boosting 3D Object Detection via Comprehensive Image Exploitation](https://mlsanity.com/paper/2603.05305/) (2026)

## Abstract (arXiv)

Fusion of LiDAR and RGB data has the potential to enhance outdoor 3D object detection accuracy. To address real-world challenges in outdoor 3D object detection, fusion of LiDAR and RGB input has started gaining traction. However, effective integration of these modalities for precise object detection task still remains a largely open problem. To address that, we propose a MultiStream Detection (MuStD) network, that meticulously extracts task-relevant information from both data modalities. The network follows a three-stream structure. Its LiDAR-PillarNet stream extracts sparse 2D pillar features from the LiDAR input while the LiDAR-Height Compression stream computes Bird's-Eye View features. An additional 3D Multimodal stream combines RGB and LiDAR features using UV mapping and polar coordinate indexing. Eventually, the features containing comprehensive spatial, textural and geometric information are carefully fused and fed to a detection head for 3D object detection. Our extensive evaluation on the challenging KITTI Object Detection Benchmark using public testing server at https://www.cvlibs.net/datasets/kitti/eval_object_detail.php?&result=d162ec699d6992040e34314d19ab7f5c217075e0 establishes the efficacy of our method by achieving new state-of-the-art or highly competitive results in different categories while remaining among the most efficient methods. Our code will be released through MuStD GitHub repository at https://github.com/IbrahimUWA/MuStD.git