# RoboAfford++: A Generative AI-Enhanced Dataset for Multimodal Affordance Learning in Robotic Manipulation and Navigation

arXiv: https://arxiv.org/abs/2511.12436 · HTML: https://mlsanity.com/paper/2511.12436/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Nov 2025.

## Built on by (most cited first, 17 of 17)

- [PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulation](https://mlsanity.com/paper/2601.07060/) (2026)
- [HY-Embodied-0.5: Embodied Foundation Models for Real-World Agents](https://mlsanity.com/paper/2604.07430/) (2026)
- [GEM: Generative Supervision Helps Embodied Intelligence](https://mlsanity.com/paper/2605.28548/) (2026)
- [Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Models](https://mlsanity.com/paper/2606.11324/) (2026)
- [Data Pyramid for Embodied Manipulation: A Survey](https://mlsanity.com/paper/2607.24744/) (2026)
- [RoboStressBench: Benchmarking VLM Robustness to Physical Visual Stress in Embodied Scenes](https://mlsanity.com/paper/2606.00828/) (2026)
- [SEF-MAP: Subspace-Decomposed Expert Fusion for Robust Multimodal HD Map Prediction](https://mlsanity.com/paper/2602.21589/) (2026)
- [Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text](https://mlsanity.com/paper/2607.21072/) (2026)
- [Weather-Conditioned Branch Routing for Robust LiDAR-Radar 3D Object Detection](https://mlsanity.com/paper/2604.05405/) (2026)
- [EPIC-Bench: A Perception-Centric Benchmark for Fine-Grained Embodied Visual Grounding in Vision-Language Models](https://mlsanity.com/paper/2605.17070/) (2026)
- [BEACON: Language-Conditioned Navigation Affordance Prediction under Occlusion](https://mlsanity.com/paper/2603.09961/) (2026)
- [SpatialPoint: Spatial-aware Point Prediction for Embodied Localization](https://mlsanity.com/paper/2603.26690/) (2026)
- [SceneParser: Hierarchical Scene Parsing for Visual Semantics Understanding](https://mlsanity.com/paper/2605.14923/) (2026)
- [AFUN: Towards an Affordance Foundation Model for Functionality Understanding](https://mlsanity.com/paper/2606.02551/) (2026)
- [EgoAfford: Task-Oriented Affordance Grounding via Egocentric Referring Segmentation](https://mlsanity.com/paper/2608.04533/) (2026)
- [Capek 0.5: An Execution-Centric Vision-Language Model for Embodied Intelligence](https://mlsanity.com/paper/2608.06756/) (2026)
- [PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models](https://mlsanity.com/paper/2609.14973/) (2026)

## Abstract (arXiv)

Robotic manipulation and navigation are fundamental capabilities of embodied intelligence, enabling effective robot interactions with the physical world. Achieving these capabilities requires a cohesive understanding of the environment, including object recognition to localize target objects, object affordances to identify potential interaction areas and spatial affordances to discern optimal areas for both object placement and robot movement. While Vision-Language Models (VLMs) excel at high-level task planning and scene understanding, they often struggle to infer actionable positions for physical interaction, such as functional grasping points and permissible placement regions. This limitation stems from the lack of fine-grained annotations for object and spatial affordances in their training datasets. To tackle this challenge, we introduce RoboAfford++, a generative AI-enhanced dataset for multimodal affordance learning for both robotic manipulation and navigation. Our dataset comprises 869,987 images paired with 2.0 million question answering (QA) annotations, covering three critical tasks: object affordance recognition to identify target objects based on attributes and spatial relationships, object affordance prediction to pinpoint functional parts for manipulation, and spatial affordance localization to identify free space for object placement and robot navigation. Complementing this dataset, we propose RoboAfford-Eval, a comprehensive benchmark for assessing affordance-aware prediction in real-world scenarios, featuring 338 meticulously annotated samples across the same three tasks. Extensive experimental results reveal the deficiencies of existing VLMs in affordance learning, while fine-tuning on the RoboAfford++ dataset significantly enhances their ability to reason about object and spatial affordances, validating the dataset's effectiveness.