mlsanityData to 14 Oct 2026

RoboGround: Robotic Manipulation with Grounded Vision-Language Priors

arXiv 2504.21530 ↗First posted 30 Apr 2025cs.RO cs.CVcited by 51

Not on the map. Why?

Cited by 30 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advancements in robotic manipulation have highlighted the potential of intermediate representations for improving policy generalization. In this work, we explore grounding masks as an effective intermediate representation, balancing two key advantages: (1) effective spatial guidance that specifies target objects and placement areas while also conveying information about object shape and size, and (2) broad generalization potential driven by large-scale vision-language models pretrained on diverse grounding datasets. We introduce RoboGround, a grounding-aware robotic manipulation system that leverages grounding masks as an intermediate representation to guide policy networks in object manipulation tasks. To further explore and enhance generalization, we propose an automated pipeline for generating large-scale, simulated data with a diverse set of objects and instructions. Extensive experiments show the value of our dataset and the effectiveness of grounding masks as intermediate guidance, significantly enhancing the generalization abilities of robot policies.

Key facts. Cited by 51 papers in the map (data through 14 Oct 2026). Builds on 47 of its 50 references that are in the map. First posted on arXiv 30 Apr 2025.

Builds on

47 of 50 references.

References · 47

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Segment Anythingcited by 8,234
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  6. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  7. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  8. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  9. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  10. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  11. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  12. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
35 more
  1. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  2. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  3. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
  4. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
  5. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  6. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  7. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 622
  8. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
  9. 2023Ferret: Refer and Ground Anything Anywhere at Any Granularitycited by 509
  10. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 455
  11. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 410
  12. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
  13. 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 364
  14. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 327
  15. 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 269
  16. 2023Any-point Trajectory Modeling for Policy Learningcited by 215
  17. 2024RT-H: Action Hierarchies Using Languagecited by 210
  18. 2022AnyGrasp: Robust and Efficient Grasp Perception in Spatial and Temporal Domainscited by 208
  19. 2024Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulationcited by 161
  20. 2023Open-World Object Manipulation using Pre-trained Vision-Language Modelscited by 156
  21. 2023RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketchescited by 133
  22. 2024Flow as the Cross-Domain Manipulation Interfacecited by 132
  23. 2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Promptingcited by 129
  24. 2023GenSim: Generating Robotic Simulation Tasks via Large Language Modelscited by 125
  25. 2024Groma: Localized Visual Tokenization for Grounding Multimodal Large Language Modelscited by 119
  26. 2024Ferret-v2: An Improved Baseline for Referring and Grounding with Large Language Modelscited by 97
  27. 2022FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objectscited by 91
  28. 2023RoboTAP: Tracking Arbitrary Points for Few-Shot Visual Imitationcited by 90
  29. 2024The All-Seeing Project V2: Towards General Relation Comprehension of the Open Worldcited by 90
  30. 2024General Flow as Foundation Affordance for Scalable Robot Learningcited by 85
  31. 2023Imitating Task and Motion Planning with Visuomotor Transformerscited by 66
  32. 2022ToolFlowNet: Robotic Manipulation with Tools via Predicting Tool Flow from Point Cloudscited by 54
  33. 2024Closed-Loop Visuomotor Control with Generative Expectation for Robotic Manipulationcited by 53
  34. 2024RT-Sketch: Goal-Conditioned Imitation Learning from Hand-Drawn Sketchescited by 41
  35. 2023KITE: Keypoint-Conditioned Policies for Semantic Manipulationcited by 39

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 13; 2026: 38.

51 papers.

Sort:

By area:

  1. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 84
  2. 2025InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policycited by 84
  3. 2025InstructVLA: Vision-Language-Action Instruction Tuning from Understanding to Manipulationcited by 72
  4. 2026InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulationcited by 50
  5. 2025Pure Vision Language Action (VLA) Models: A Comprehensive Surveycited by 49
  6. 2025ShapeLLM-Omni: A Native Multimodal LLM for 3D Generation and Understandingcited by 38
  7. 2026VP-VLA: Visual Prompting as an Interface for Vision-Language-Action Modelscited by 17
  8. 2026FutureVLA: Joint Visuomotor Prediction for Vision-Language-Action Modelcited by 16
  9. 2026OAT: Ordered Action Tokenizationcited by 15
  10. 2025Point What You Mean: Visually Grounded Instruction Policycited by 10
  11. 2025Clutter-Robust Vision-Language-Action Models through Object-Centric and Geometry Groundingcited by 10
  12. 2026ST4VLA: Spatially Guided Training for Vision-Language-Action Modelscited by 9
38 more
  1. 2026HiVLA: A Visual-Grounded-Centric Hierarchical Embodied Manipulation Systemcited by 8
  2. 2026Gaze2Act: Gaze-Conditioned Vision-Language-Action Policies for Interactive Robot Manipulationcited by 8
  3. 2025Bring My Cup! Personalizing Vision-Language-Action Models with Visual Attentive Promptingcited by 6
  4. 2025IA-VLA: Input Augmentation for Vision-Language-Action models in settings with semantically complex taskscited by 4
  5. 2026Afford-VLA: Action-Aligned Visual Planning via Internalized Affordancecited by 4
  6. 2025Executable Analytic Concepts as the Missing Link Between VLM Insight and Precise Manipulationcited by 3
  7. 2026MultihopSpatial: Multi-hop Compositional Spatial Reasoning Benchmark for Vision-Language Modelcited by 3
  8. 2025Beyond Success: Refining Elegant Robot Manipulation from Mixed-Quality Data via Just-in-Time Interventioncited by 2
  9. 2026From Instruction to Event: Sound-Triggered Mobile Manipulationcited by 2
  10. 2026AnySlot: Goal-Conditioned Vision-Language-Action Policies for Zero-Shot Slot-Level Placementcited by 2
  11. 2026From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulationcited by 2
  12. 2026From Failure to Feedback: Group Revision Unlocks Hard Cases in Object-Level Groundingcited by 2
  13. 2026Direct Action-Head Injection of A Grounded 3D Point Unlocks Spatial and Task Generalizationcited by 2
  14. 2026Hijacking Robots with a Piece of Paper: A Systematic Study of Physical Prompt Injection in VLM-Controlled Robotscited by 2
  15. 2026InstructMove: A Text-Indispensable Benchmark for Instruction-Following Manipulationcited by 2
  16. 2026AR2-4FV: Anchored Referring and Re-identification for Long-Term Grounding in Fixed-View Videoscited by 1
  17. 2026Decoupling Semantics and Geometric Grounding: Spatial Visual Prompts for Language-Conditioned Imitation Learningcited by 1
  18. 2026SAM3D-Guided Object-Centric Representation Alignment for Vision-Language-Action Modelscited by 1
  19. 2026Cross-Embodiment Transfer via Behavior-Aligned Representationscited by 1
  20. 2025CDE: Concept-Driven Exploration for Reinforcement Learningcited by 0
  21. 2025Prompt2Craft: Generating Functional Craft Assemblies with LLMscited by 0
  22. 2026System Design for Maintaining Internal State Consistency in Long-Horizon Robotic Tabletop Gamescited by 0
  23. 2026PRIMED: Adaptive Modality Suppression for Referring Audio-Visual Segmentation via Biased Competitioncited by 0
  24. 2026Premover: Fast Vision-Language-Action Control by Acting Before Instructions Are Completecited by 0
  25. 2026Towards Open-World Referring Expression Comprehension: A Benchmark with Training-free Multi-task Consistency Checkercited by 0
  26. 2026Bounding Boxes as Goals: Language-Conditioned Grasping via Neuro-Symbolic Planningcited by 0
  27. 2026FeVOS: Foresight Expression Video Object Segmentationcited by 0
  28. 2026What Language Does and What the Evidence Supports: A Functional Role Taxonomy and Evidence Audit of Language Grounding in Embodied Agentscited by 0
  29. 2026SpaceVLA: Spatially Grounded VLA for Robotic Manipulation with User-Authored Grasp and Place Anchorscited by 0
  30. 2026Act with Intent: Distilling Behavior Intent for Vision-Language-Action Modelscited by 0
  31. 2026GRAFT: Grounded and Efficient Online Reinforcement Adaptation for Fine-Grained Robot Manipulationcited by 0
  32. 2026DeicticVLA: Unifying Instruction Modes Based on Language and Deictic Gestures in a Single VLAcited by 0
  33. 2026What Matters, When? Diagnosing and Improving Conditional Visual Grounding in Visuomotor Imitation Policiescited by 0
  34. 20263DWay: Generalizing Robot Manipulation via 3D Consistent Waypointscited by 0
  35. 2026Attention-DP3: Spatially Object-aware 3D Diffusion Policy via Geometry-aligned Attentional Conditioningcited by 0
  36. 2026MaskHarness-WAM: Instance-Grounded Harnessing for Long-Horizon Robot Manipulationcited by 0
  37. 2026Beyond Token Importance: Preserving Spatial Scaffolds for Efficient Vision-Language-Action Inferencecited by 0
  38. 2026Correcting WHERE, Preserving HOW: Compositional Generalization for Vision-Language-Action Models via Referential Guidancecited by 0

50 most cited of 51 All 51 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026GroundingPI: A Grounding Foundation Model towards Physical Intelligence with Visual Primitives
  2. 2026RoboIRGBench: Benchmarking Implicit Referential Grounding in Vision-Language-Action Models
  3. 2026Spa-Bench: A Real-Robot Benchmark for Spatial Grounding in Vision-Language-Action Models
  4. 2023Transferring Foundation Models for Generalizable Robotic Manipulation
  5. 2026RoboInter: A Holistic Intermediate Representation Suite Towards Robotic Manipulation
  6. 2026GROW2: Grounding Which and Where for Robot Tool Use
  7. 2026Grounded Action Model: 3D Grounding as a Foundation for Robotics
  8. 2025PEEK: Guiding and Minimal Image Representations for Zero-Shot Generalization of Robot Manipulation Policies