mlsanityData to 7 Oct 2026

PF3Det: A Prompted Foundation Feature Assisted Visual LiDAR 3D Detector

arXiv 2504.03563 ↗First posted 4 Apr 2025cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

3D object detection is crucial for autonomous driving, leveraging both LiDAR point clouds for precise depth information and camera images for rich semantic information. Therefore, the multi-modal methods that combine both modalities offer more robust detection results. However, efficiently fusing LiDAR points and images remains challenging due to the domain gaps. In addition, the performance of many models is limited by the amount of high quality labeled data, which is expensive to create. The recent advances in foundation models, which use large-scale pre-training on different modalities, enable better multi-modal fusion. Combining the prompt engineering techniques for efficient training, we propose the Prompted Foundational 3D Detector (PF3Det), which integrates foundation model encoders and soft prompts to enhance LiDAR-camera feature fusion. PF3Det achieves the state-of-the-art results under limited training data, improving NDS by 1.19% and mAP by 2.42% on the nuScenes dataset, demonstrating its efficiency in 3D detection.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 58 of its 66 references that are in the map. First posted on arXiv 4 Apr 2025.

Builds on

58 of 66 references.

References · 58

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  7. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  8. 2023Segment Anythingcited by 8,158
  9. 2017Mask R-CNNcited by 7,259
  10. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  11. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
  12. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
38 more
  1. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  2. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  3. 2022Visual Prompt Tuningcited by 1,674
  4. 2017VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detectioncited by 1,323
  5. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
  6. 2016Multi-View 3D Object Detection Network for Autonomous Drivingcited by 721
  7. 2022A Survey on In-context Learningcited by 717
  8. 2021GPT Understands, Toocited by 715
  9. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
  10. 2017Frustum PointNets for 3D Object Detection from RGB-D Datacited by 640
  11. 2021PointCLIP: Point Cloud Understanding by CLIPcited by 461
  12. 2019PointPainting: Sequential Fusion for 3D Object Detectioncited by 449
  13. 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 334
  14. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 326
  15. 2017Joint 3D Proposal Generation and Object Detection from View Aggregationcited by 325
  16. 2023AudioGPT: Understanding and Generating Speech, Music, Sound, and Talking Headcited by 264
  17. 2020Deep Continuous Fusion for Multi-Sensor 3D Object Detectioncited by 263
  18. 2022ULIP: Learning a Unified Representation of Language, Images, and Point Clouds for 3D Understandingcited by 262
  19. 2023Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessmentcited by 227
  20. 2022DeepFusion: Lidar-Camera Deep Fusion for Multi-Modal 3D Object Detectioncited by 224
  21. 2020Multi-Task Multi-Sensor Fusion for 3D Object Detectioncited by 203
  22. 2021Probabilistic and Geometric Depth: Detecting Objects in Perspectivecited by 190
  23. 20203D-CVF: Generating Joint Camera and LiDAR Features Using Cross-View Spatial Feature Fusion for 3D Object Detectioncited by 176
  24. 2017PointFusion: Deep Sensor Fusion for 3D Bounding Box Estimationcited by 170
  25. 2021Multimodal Virtual Point 3D Detectioncited by 166
  26. 2019MVX-Net: Multimodal VoxelNet for 3D Object Detectioncited by 162
  27. 2020CLOCs: Camera-LiDAR Object Candidates Fusion for 3D Object Detectioncited by 147
  28. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  29. 20223D Object Detection for Autonomous Driving: A Comprehensive Surveycited by 132
  30. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  31. 2022MSMDFusion: Fusing LiDAR and Camera at Multiple Scales with Multi-Depth Seeds for 3D Object Detectioncited by 62
  32. 2022CLIP-GEN: Language-Free Training of a Text-to-Image Generator with CLIPcited by 60
  33. 2024Multi-Modal Data-Efficient 3D Scene Understanding for Autonomous Drivingcited by 53
  34. 2022LESS: Label-Efficient Semantic Segmentation for LiDAR Point Cloudscited by 41
  35. 2022Robust Structured Declarative Classifiers for 3D Point Clouds: Defending Adversarial Attacks with Implicit Gradientscited by 17
  36. 2018A General Pipeline for 3D Detection of Vehiclescited by 15
  37. 2022FusionRCNN: LiDAR-Camera Fusion for Two-stage 3D Object Detectioncited by 14
  38. 2024Depth-Wise Convolutions in Vision Transformers for Efficient Training on Small Datasetscited by 14

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Towards Open-Vocabulary Multimodal 3D Object Detection with Attributescited by 2
  2. 2026Privacy-Concealing Cooperative Perception for BEV Scene Segmentationcited by 2
  3. 2026Distilling 3D Spatial Reasoning into a Lightweight Vision-Language Model with CoTcited by 0
  4. 2026RAF: Reliability-Aware Fusion of Camera, LiDAR, and 4D RADAR for Robust 3D Object Detection in Adverse Weathercited by 0