mlsanityData to 14 Oct 2026

GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives

arXiv 2609.39601 ↗First posted 30 Sep 2026cs.CV cs.AI cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Precise grounding matters. It specifies which object is the target and where that object is, even in clutter and for tiny objects, and it has to be fast enough for closed-loop control. Yet vision-language-action (VLA) and world-action models (WAMs) take perception from general-purpose vision-language and video-generation backbones, which still fail in these settings. We introduce GroundingPI, a 4B grounding foundation model that generates points and boxes as quantized coordinates in a shared vocabulary. Training combines multimodal and spatial pretraining, supervised fine-tuning, and reinforcement learning with GRPO, using supervision from public datasets and dedicated data engines. Against 44 baselines across 34 grounding benchmarks spanning 11 perceptual capabilities, GroundingPI establishes a new state of the art, averaging 73.68%, above the larger GPT-6 Astra (71.54%). As a downstream visual backbone, GroundingPI improves performance on robotic manipulation and autonomous driving. On RoboTwin 2.0, it outperforms every mainstream backbone we evaluate in all four out-of-distribution settings, by up to 24.8% relative to the strongest backbone. On RoboCasa-GR1, GroundingPI trained with 50% of the demonstrations outperforms those baselines trained with 75%. On nuScenes, used as the visual backbone, GroundingPI attains an average open-loop L2 error of 0.296 m. We systematically analyze GroundingPI's pretraining in scale and data composition. Downstream autonomous driving and robotic manipulation improve as the pretraining is scaled. Analyzing the data recipe across these 11 perceptual capabilities shows dense grounding's substantial benefits for both, and OCR's potential as a catalyst for perceptual learning. These results support grounding as a perceptual foundation, and dedicated perceptual pretraining as a promising direction for foundation models of physical intelligence.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 52 of its 96 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

52 of 96 references.

References · 52

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2022Flow Matching for Generative Modelingcited by 5,702
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  7. 2025Qwen3-VL Technical Reportcited by 2,793
  8. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  9. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  10. 2017Decoupled Weight Decay Regularizationcited by 1,831
  11. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  12. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,319
38 more
  1. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  2. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  3. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
  4. 2024PaliGemma: A versatile 3B VLM for transfercited by 696
  5. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  6. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  7. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 518
  8. 2026World Action Models are Zero-shot Policiescited by 465
  9. 2025Seed1.5-VL Technical Reportcited by 373
  10. 2021Pix2seq: A Language Modeling Framework for Object Detectioncited by 349
  11. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 300
  12. 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 251
  13. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 232
  14. 2023PixelLM: Pixel Reasoning with Large Multimodal Modelcited by 222
  15. 2025World Simulation with Video Foundation Models for Physical AIcited by 200
  16. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automationcited by 190
  17. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
  18. 2025Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Bettercited by 146
  19. 2025PaddleOCR 3.0 Technical Reportcited by 143
  20. 2026Kimi K3: Open Frontier Intelligencecited by 138
  21. 2025Perception-R1: Pioneering Perception Policy with Reinforcement Learningcited by 120
  22. 2025MiMo-VL Technical Reportcited by 115
  23. 2023LLaVA-Grounding: Grounded Visual Chat with Large Multimodal Modelscited by 113
  24. 2026StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developingcited by 112
  25. 2024DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perceptioncited by 82
  26. 2018Vision Meets Drones: A Challengecited by 81
  27. 2024Towards Synergistic, Generalized, and Efficient Dual-System for Robotic Manipulationcited by 73
  28. 2025ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform Datacited by 60
  29. 2023M6Doc: A Large-Scale Multi-Format, Multi-Type, Multi-Layout, Multi-Language, Multi-Annotation Category Dataset for Modern Document Layout Analysiscited by 46
  30. 2026RynnBrain: Open Embodied Foundation Modelscited by 35
  31. 2026OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretrainingcited by 22
  32. 2026AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understandingcited by 12
  33. 2026UniT: Toward a Unified Physical Language for Human-to-Humanoid Policy Learning and World Modelingcited by 10
  34. 2026SG-VLA: Learning Spatially-Grounded Vision-Language-Action Models for Mobile Manipulationcited by 9
  35. 2026RynnBrain 1.1: Towards More Capable and Generalizable Embodied Foundation Modelcited by 8
  36. 2026Vision as Unified Multimodal Generationcited by 7
  37. 2026LongAct: Harnessing Intrinsic Activation Patterns for Long-Context Reinforcement Learningcited by 3
  38. 2026SUGAR: A Scalable Human-Video-Driven Generalizable Humanoid Loco-Manipulation Learning Frameworkcited by 3

50 most cited of 52 All 52 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025RoboGround: Robotic Manipulation with Grounded Vision-Language Priors
  2. 2026Spa-Bench: A Real-Robot Benchmark for Spatial Grounding in Vision-Language-Action Models
  3. 2026GROW2: Grounding Which and Where for Robot Tool Use
  4. 2026Grounded Action Model: 3D Grounding as a Foundation for Robotics
  5. 2025Clutter-Robust Vision-Language-Action Models through Object-Centric and Geometry Grounding
  6. 2026RoboIRGBench: Benchmarking Implicit Referential Grounding in Vision-Language-Action Models
  7. 2026GroundingVLN: Reasoning and Acting with Grounding for Vision-Language Navigation
  8. 2025InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policy