mlsanityData to 7 Oct 2026

A3VLM: Actionable Articulation-Aware Vision Language Model

arXiv 2406.07549 ↗First posted 11 Jun 2024cs.ROcited by 33

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Vision Language Models (VLMs) have received significant attention in recent years in the robotics community. VLMs are shown to be able to perform complex visual reasoning and scene understanding tasks, which makes them regarded as a potential universal solution for general robotics problems such as manipulation and navigation. However, previous VLMs for robotics such as RT-1, RT-2, and ManipLLM have focused on directly learning robot-centric actions. Such approaches require collecting a significant amount of robot interaction data, which is extremely costly in the real world. Thus, we propose A3VLM, an object-centric, actionable, articulation-aware vision language model. A3VLM focuses on the articulation structure and action affordances of objects. Its representation is robot-agnostic and can be translated into robot actions using simple action primitives. Extensive experiments in both simulation benchmarks and real-world settings demonstrate the effectiveness and stability of A3VLM. We release our code and other materials at https://github.com/changhaonan/A3VLM.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 36 of its 39 references that are in the map. First posted on arXiv 11 Jun 2024.

Builds on

36 of 39 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024ManipVQA: Injecting Robotic Affordance and Physically Grounded Information into Multi-Modal Large Language Models17 Mar
  2. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interaction27 Feb
  3. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation24 Dec
  4. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul

Other references · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023Segment Anythingcited by 8,158
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  9. 2022Code as Policies: Language Model Programs for Embodied Controlcited by 1,172
  10. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
  11. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  12. 2021Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modelingcited by 647
20 more
  1. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  2. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
  3. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 326
  4. 2023SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Modelscited by 283
  5. 2023Scalable 3D Captioning with Pretrained Modelscited by 201
  6. 2021Where2Act: From Pixels to Actions for Articulated 3D Objectscited by 179
  7. 2023RoboAgent: Generalization and Efficiency in Robot Manipulation via Semantic Augmentations and Action Chunkingcited by 167
  8. 2019Category-Level Articulated Object Pose Estimationcited by 158
  9. 2022GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Partscited by 135
  10. 2024SPHINX-X: Scaling Data and Parameters for a Family of Multi-modal Large Language Modelscited by 130
  11. 2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Promptingcited by 126
  12. 2019Shape2Motion: Joint Analysis of Motion Parts and Attributes from 3D Shapescited by 117
  13. 2023Instruct2Act: Mapping Multi-modality Instructions to Robotic Actions with Large Language Modelcited by 115
  14. 2022FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objectscited by 91
  15. 2021UMPNet: Universal Manipulation Policy Network for Articulated Objectscited by 64
  16. 2023FlowBot++: Learning Generalized Articulated Objects Manipulation via Articulation Projectioncited by 34
  17. 2016High precision grasp pose detection in dense cluttercited by 33
  18. 2023Context-Aware Entity Grounding with Open-Vocabulary 3D Scene Graphscited by 33
  19. 20233D Implicit Transporter for Temporally Consistent Keypoint Discoverycited by 20
  20. 2023LGMCTS: Language-Guided Monte-Carlo Tree Search for Executable Semantic Object Rearrangementcited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 8; 2025: 8; 2026: 17.

33 papers.

Sort:

By area:

  1. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  2. 2024ShowUI: One Vision-Language-Action Model for GUI Visual Agentcited by 226
  3. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
  4. 2024Visual Prompting in Multimodal Large Language Models: A Surveycited by 52
  5. 2025SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulationcited by 46
  6. 2025Embodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulationcited by 36
  7. 2026ACoT-VLA: Action Chain-of-Thought for Vision-Language-Action Modelscited by 30
  8. 2024CoA-VLA: Improving Vision-Language-Action Models via Visual-Textual Chain-of-Affordancecited by 29
  9. 2026PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulationcited by 20
  10. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  11. 2024GLOVER: Generalizable Open-Vocabulary Affordance Reasoning for Task-Oriented Graspingcited by 15
  12. 2024Understanding Depth and Height Perception in Large Visual-Language Modelscited by 8
21 more
  1. 2024UniAff: A Unified Representation of Affordances for Tool Usage and Articulation with Vision-Language Modelscited by 6
  2. 2025CrayonRobo: Object-Centric Prompt-Driven Vision-Language-Action Model for Robotic Manipulationcited by 6
  3. 2025Focusing on What Matters: Object-Agent-centric Tokenization for Vision Language Action modelscited by 6
  4. 2025Physically Ground Commonsense Knowledge for Articulated Object Manipulation with Analytic Conceptscited by 5
  5. 2024Enhanced Vision-Language Models for Diverse Sensor Understanding: Cost-Efficient Optimization and Benchmarkingcited by 3
  6. 2025Neural Network Reprogrammability: A Unified Theme on Model Reprogramming, Prompt Tuning, and Prompt Instructioncited by 3
  7. 2026Embodied3DBench: Benchmarking Low-Level Embodied Spatial Intelligence of Vision Language Modelscited by 3
  8. 2026ArtPro: Self-Supervised Articulated Object Reconstruction with Adaptive Integration of Mobility Proposalscited by 1
  9. 2026ArtMesh: Part-Aware Articulated Mesh Fields with Motion-Consistent Dynamicscited by 1
  10. 2026Action with Visual Primitivescited by 1
  11. 2026Analytic Concept-Centric Memory for Agentic Embodied Manipulationcited by 1
  12. 2026UnfoldArt: Zero-Shot Recovery of Full Articulated 3D Objects from Text or Imagecited by 1
  13. 2026Flow-Enabled Generalization to Human Demonstrations in Few-Shot Imitation Learningcited by 0
  14. 2026Learning Structured Robot Policies from Vision-Language Models via Synthetic Neuro-Symbolic Supervisioncited by 0
  15. 2026StateVLM: A State-Aware Vision-Language Model for Robotic Affordance Reasoningcited by 0
  16. 2026GSAM: A Generalizable and Safe Robotic Framework for Articulated Object Manipulationcited by 0
  17. 2026From Perception to Assistance: Open-Vocabulary Shared Autonomy for Robotic Manipulationcited by 0
  18. 2026ArtLang: Structured Language-to-Kinematics Grounding for Articulated 3D Actuationcited by 0
  19. 20263DWay: Generalizing Robot Manipulation via 3D Consistent Waypointscited by 0
  20. 2026UniPart: Towards Zero-shot Language-Grounded 3D Part Segmentation for Embodied Interactioncited by 0
  21. 2026Bridging Thought and Action: Taming Long-Horizon Instability in Open-Source LLM Agents with a MetaTool-Enhanced ROS Frameworkcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applications
  2. 2024What Matters in Building Vision-Language-Action Models for Generalist Robots
  3. 2025Large VLM-based Vision-Language-Action Models for Robotic Manipulation: A Survey
  4. 2025Evo-0: Vision-Language-Action Model with Implicit Spatial Understanding
  5. 2025Vision Language Action Models in Robotic Manipulation: A Systematic Review
  6. 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges
  7. 20253D-Grounded Vision-Language Framework for Robotic Task Planning: Automated Prompt Synthesis and Supervised Reasoning
  8. 2025VLA-0: Building State-of-the-Art VLAs with Zero Modification