mlsanityData to 7 Oct 2026

FP3: A 3D Foundation Policy for Robotic Manipulation

arXiv 2503.08950 ↗First posted 11 Mar 2025cs.RO cs.AIcited by 31

Not on the map. Why?

Abstract (arXiv)

Following its success in natural language processing and computer vision, foundation models that are pre-trained on large-scale multi-task datasets have also shown great potential in robotics. However, most existing robot foundation models rely solely on 2D image observations, ignoring 3D geometric information, which is essential for robots to perceive and reason about the 3D world. In this paper, we introduce FP3, a first large-scale 3D foundation policy model for robotic manipulation. FP3 builds on a scalable diffusion transformer architecture and is pre-trained on 60k trajectories with point cloud observations. With the model design and diverse pre-training data, FP3 can be efficiently fine-tuned for downstream tasks while exhibiting strong generalization capabilities. Experiments on real robots demonstrate that with only 80 demonstrations, FP3 is able to learn a new task with over 90% success rates in novel environments with unseen objects, significantly surpassing existing robot foundation models.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 76 of its 87 references that are in the map. First posted on arXiv 11 Mar 2025.

Builds on

76 of 87 references.

References · 76

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  8. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  9. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  10. 2022Scalable Diffusion Models with Transformerscited by 5,931
  11. 2022Flow Matching for Generative Modelingcited by 5,426
  12. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
38 more
  1. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
  5. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  6. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  7. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  8. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  9. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  10. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  11. 2022Video Diffusion Modelscited by 2,280
  12. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  13. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  14. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  15. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  16. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
  17. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
  18. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  19. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
  20. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  21. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  22. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  23. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  24. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
  25. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
  26. 2022PointNeXt: Revisiting PointNet++ with Improved Training and Scaling Strategiescited by 432
  27. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
  28. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
  29. 2024Multilingual E5 Text Embeddings: A Technical Reportcited by 357
  30. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  31. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
  32. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 231
  33. 2023RVT: Robotic View Transformer for 3D Object Manipulationcited by 224
  34. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 216
  35. 2023Any-point Trajectory Modeling for Policy Learningcited by 207
  36. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 177
  37. 2023Look Before You Leap: Unveiling the Power of GPT-4V in Robotic Vision-Language Planningcited by 161
  38. 2023Motion Planning Diffusion: Learning and Planning of Robot Motions with Diffusion Modelscited by 155

50 most cited of 76 All 76 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 16; 2026: 15.

31 papers.

Sort:

By area:

  1. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  2. 2025BridgeVLA: Input-Output Alignment for Efficient 3D Manipulation Learning with Vision-Language Modelscited by 77
  3. 2025Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challengescited by 72
  4. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 46
  5. 2025CoMo: Learning Continuous Latent Motion from Internet Videos for Scalable Robot Learningcited by 37
  6. 2025DeepVerse: 4D Autoregressive Video Generation as a World Modelcited by 36
  7. 2025GeoAware-VLA: Implicit Geometry Aware Vision-Language-Action Modelcited by 24
  8. 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipescited by 22
  9. 2025QDepth-VLA: Quantized Depth Prediction as Auxiliary Supervision for Vision-Language-Action Modelscited by 19
  10. 2025StereoVLA: Enhancing Vision-Language-Action Models with Stereo Visioncited by 16
  11. 2026ActiveVLA: Injecting Active Perception into Vision-Language-Action Models for Precise 3D Robotic Manipulationcited by 14
  12. 2025GP3: A 3D Geometry-Aware Policy with Multi-View Images for Robotic Manipulationcited by 11
19 more
  1. 2026SpatialVAM:Spatial-Aware Multi-View Video Diffusion as a Data-Efficient Robot Policycited by 8
  2. 2025Survey of Vision-Language-Action Models for Embodied Manipulationcited by 7
  3. 2026AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Modelscited by 7
  4. 2026PEAfowl: Perception-Enhanced Multi-View Vision-Language-Action for Bimanual Manipulationcited by 6
  5. 2026ConsisVLA-4D: Advancing Spatiotemporal Consistency in Efficient 3D-Perception and 4D-Reasoning for Robotic Manipulationcited by 6
  6. 2026PointACT: Vision-Language-Action Models with Multi-Scale Point-Action Interactioncited by 6
  7. 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challengescited by 4
  8. 2025Tenma: Robust Cross-Embodiment Robot Manipulation with Diffusion Transformercited by 3
  9. 2025UMIGen: A Unified Framework for Egocentric Point Cloud Generation and Cross-Embodiment Robotic Imitation Learningcited by 3
  10. 2025Avi: Action from Volumetric Inferencecited by 2
  11. 2026R3D: Revisiting 3D Policy Learningcited by 1
  12. 2026StereoPolicy: Improving Robotic Manipulation Policies via Stereo Perceptioncited by 1
  13. 20263DThinkVLA: Endowing Vision-Language-Action Models with Latent 3D Priors via 3D-Thinking-Guided Co-trainingcited by 1
  14. 2026See like a Robot: Robot-Centric Pointmaps for VLA Modelscited by 1
  15. 2026BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulationcited by 1
  16. 2026KAN We Flow? Advancing Robotic Manipulation with 3D Flow Matching via KAN & RWKVcited by 0
  17. 2026Tele-Catch: Adaptive Teleoperation for Dexterous Dynamic 3D Object Catchingcited by 0
  18. 20263DWay: Generalizing Robot Manipulation via 3D Consistent Waypointscited by 0
  19. 2026Attention-DP3: Spatially Object-aware 3D Diffusion Policy via Geometry-aligned Attentional Conditioningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 20254D Visual Pre-training for Robot Learning
  2. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representations
  3. 2026PAIWorld: A 3D-Consistent World Foundation Model for Robotic Manipulation
  4. 2025SPEAR-1: Scaling Beyond Robot Demonstrations via 3D Understanding
  5. 20253D Flow Diffusion Policy: Visuomotor Policy Learning via Generating Flow in 3D Space
  6. 2024GenDP: 3D Semantic Fields for Category-Level Generalizable Diffusion Policy
  7. 2025AnchorDP3: 3D Affordance Guided Sparse Diffusion Policy for Robotic Manipulation
  8. 2025ISS Policy : Scalable Diffusion Policy with Implicit Scene Supervision