# INTENTION: Inferring Tendencies of Humanoid Robot Motion Through Interactive Intuition and Grounded VLM

arXiv: https://arxiv.org/abs/2508.04931 · HTML: https://mlsanity.com/paper/2508.04931/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 36 references that are in the map.
- First posted on arXiv 6 Aug 2025.

## Builds on (most cited first, 26 of 26)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Open X-Embodiment: Robotic Learning Datasets and RT-X Models](https://mlsanity.com/paper/2310.08864/) (2023)
- [VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models](https://mlsanity.com/paper/2307.05973/) (2023)
- [SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities](https://mlsanity.com/paper/2401.12168/) (2024)
- [Language to Rewards for Robotic Skill Synthesis](https://mlsanity.com/paper/2306.08647/) (2023)
- [RT-H: Action Hierarchies Using Language](https://mlsanity.com/paper/2403.01823/) (2024)
- [Distilled Feature Fields Enable Few-Shot Language-Guided Manipulation](https://mlsanity.com/paper/2308.07931/) (2023)
- [Language Embedded Radiance Fields for Zero-Shot Task-Oriented Grasping](https://mlsanity.com/paper/2309.07970/) (2023)
- [Bootstrap Your Own Skills: Learning to Solve New Tasks with Large Language Model Guidance](https://mlsanity.com/paper/2310.10021/) (2023)
- [Real-World Robot Applications of Foundation Models: A Review](https://mlsanity.com/paper/2402.05741/) (2024)
- [Prompt a Robot to Walk with Large Language Models](https://mlsanity.com/paper/2309.09969/) (2023)
- [Distilling and Retrieving Generalizable Knowledge for Robot Manipulation via Language Corrections](https://mlsanity.com/paper/2311.10678/) (2023)
- [SayTap: Language to Quadrupedal Locomotion](https://mlsanity.com/paper/2306.07580/) (2023)
- [A Unified MPC Framework for Whole-Body Dynamic Locomotion and Manipulation](https://mlsanity.com/paper/2103.00946/) (2021)
- [Optimization-Based Control for Dynamic Legged Robots](https://mlsanity.com/paper/2211.11644/) (2022)
- [RoboNurse-VLA: Robotic Scrub Nurse System based on Vision-Language-Action Model](https://mlsanity.com/paper/2409.19590/) (2024)
- [LoHoRavens: A Long-Horizon Language-Conditioned Benchmark for Robotic Tabletop Manipulation](https://mlsanity.com/paper/2310.12020/) (2023)
- [Gesture-Informed Robot Assistance via Foundation Models](https://mlsanity.com/paper/2309.02721/) (2023)
- [SRL-VIC: A Variable Stiffness-Based Safe Reinforcement Learning for Contact-Rich Robotic Tasks](https://mlsanity.com/paper/2406.13744/) (2024)
- [Autonomous Behavior Planning For Humanoid Loco-manipulation Through Grounded Language Model](https://mlsanity.com/paper/2408.08282/) (2024)
- [HYPERmotion: Learning Hybrid Behavior Planning for Autonomous Loco-manipulation](https://mlsanity.com/paper/2406.14655/) (2024)
- [Bresa: Bio-inspired Reflexive Safe Reinforcement Learning for Contact-Rich Robotic Tasks](https://mlsanity.com/paper/2503.21989/) (2025)
- [Grounding Language Models in Autonomous Loco-manipulation Tasks](https://mlsanity.com/paper/2409.01326/) (2024)

## Built on by (most cited first, 1 of 1)

- [Vision-Language-Policy Model for Dynamic Robot Task Planning](https://mlsanity.com/paper/2512.19178/) (2025)

## Abstract (arXiv)

Traditional control and planning for robotic manipulation heavily rely on precise physical models and predefined action sequences. While effective in structured environments, such approaches often fail in real-world scenarios due to modeling inaccuracies and struggle to generalize to novel tasks. In contrast, humans intuitively interact with their surroundings, demonstrating remarkable adaptability, making efficient decisions through implicit physical understanding. In this work, we propose INTENTION, a novel framework enabling robots with learned interactive intuition and autonomous manipulation in diverse scenarios, by integrating Vision-Language Models (VLMs) based scene reasoning with interaction-driven memory. We introduce Memory Graph to record scenes from previous task interactions which embodies human-like understanding and decision-making about different tasks in real world. Meanwhile, we design an Intuitive Perceptor that extracts physical relations and affordances from visual scenes. Together, these components empower robots to infer appropriate interaction behaviors in new scenes without relying on repetitive instructions. Videos: https://robo-intention.github.io