PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulation
On the map: Vision-language-action models → Reasoning and chain-of-thought in VLA
Abstract (arXiv)
Recent advancements in vision-language-action (VLA) models have shown promise in robotic manipulation, yet they continue to struggle with long-horizon, multi-step tasks. Existing methods lack internal reasoning mechanisms that can identify task-relevant interaction cues or track progress within a subtask, leading to critical execution errors such as repeated actions, missed steps, and premature termination. To address these challenges, we introduce PALM, a VLA framework that structures policy learning around interaction-centric affordance reasoning and subtask progress cues. PALM distills complementary affordance representations that capture object relevance, contact geometry, spatial placements, and motion dynamics, and serve as task-relevant anchors for visuomotor control. To further stabilize long-horizon execution, PALM predicts continuous within-subtask progress, enabling seamless subtask transitions. Across extensive simulation and real-world experiments, PALM consistently outperforms baselines, achieving a 91.8% success rate on LIBERO-LONG, a 12.5% improvement in average length on CALVIN ABC->D, and a 2x improvement over real-world baselines across three long-horizon generalization settings.
Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Reasoning and chain-of-thought in VLA, one of 13 directions in Vision-language-action models. Builds on 145 of its 162 references that are in the map. First posted on arXiv 11 Jan 2026.
Builds on
145 of 162 references.
Closest ancestors · 3
Same direction, newest first
- 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models27 Mar
- 2024CoA-VLA: Improving Vision-Language-Action Models via Visual-Textual Chain-of-Affordance29 Dec
- 2024Robotic Control via Embodied Chain-of-Thought Reasoning11 Jul
Other references · 142
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023Segment Anythingcited by 8,158
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 2,993
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
38 more
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
- 2021Perceiver: General Perception with Iterative Attentioncited by 1,089
- 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
- 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,051
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 849
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
- 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilitiescited by 679
- 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 678
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
- 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
- 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
- 2023CoTracker: It is Better to Track Togethercited by 516
- 2023Learning Universal Policies via Text-Guided Video Generationcited by 506
- 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
- 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 377
- 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
- 2023Vision-Language Foundation Models as Effective Robot Imitatorscited by 344
- 20243D-VLA: A 3D Vision-Language-Action Generative World Modelcited by 320
- 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
- 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 298
- 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 290
- 2025π*0.6: a VLA That Learns From Experiencecited by 280
- 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 279
- 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
- 2023Zero-Shot Robotic Manipulation with Pretrained Image-Editing Diffusion Modelscited by 264
- 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
- 2022Masked Visual Pre-training for Motor Controlcited by 246
- 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Roboticscited by 242
- 2020The EPIC-KITCHENS Dataset: Collection, Challenges and Baselinescited by 221
- 2023Affordances from Human Videos as a Versatile Representation for Roboticscited by 221
- 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 218
- 2023Where are we in the search for an Artificial Visual Cortex for Embodied Intelligence?cited by 215
- 2020Keep Doing What Worked: Behavioral Modelling Priors for Offline Reinforcement Learningcited by 213
50 most cited of 142 All 142 (JSON)
Built on by
Citing papers per year: 2026: 20.
20 papers.
Sort:
By area:
- 2026World Action Models: A Surveycited by 10
- 2026EgoForge: Goal-Directed Egocentric World Simulatorcited by 9
- 2026Towards Long-horizon Embodied Agents with Tool-Aligned Vision-Language-Action Modelscited by 8
- 2026Gaze2Act: Gaze-Conditioned Vision-Language-Action Policies for Interactive Robot Manipulationcited by 8
- 2026AffordVLA: Injecting Affordance Representations into Vision-Language-Action Models via Implicit Feature Alignmentcited by 5
- 2026What's Missing in Screen-to-Action? Towards a UI-in-the-Loop Paradigm for Multimodal GUI Reasoningcited by 4
- 2026FusionAgent: A Multimodal Agent with Dynamic Model Selection for Human Recognitioncited by 3
- 2026ShardMemo: Scope-Before-Routing for Agentic Memory Retrievalcited by 1
- 2026Frequency-Modulated Visual Restoration for Matryoshka Large Multimodal Modelscited by 1
- 2026Self-Evaluation Is Already There: Eliciting Latent Judge Calibration in Base LLMs with Minimal Datacited by 1
- 2026GIVE: Grounding Human Gestures in Vision-Language-Action Modelscited by 1
- 2026SparkVLA: Stop-Aware Hierarchical VLA with Adaptive Action Chunking for Long-Horizon Manipulationcited by 1
8 more
- 2026Completion at the Boundary (CaB): Deployable Switching with Completion-Aware Control under Limited Calibrationcited by 0
- 2026CogniRoute: Learning to Route Social Evidence in Omni-Modal Modelscited by 0
- 2026LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignmentscited by 0
- 2026Mind-VLA: Instruction-Aware Spatial Representation Alignment for Vision-Language-Action Modelscited by 0
- 2026Explicit State Elicitation Is Not Enough: A Controlled Audit of Memory-Policy Classificationcited by 0
- 2026Task-Specified Active Metrological Inspection with Measurement-Steered VLA Manipulation and Deterministic Evidence Gatingcited by 0
- 2026H-VLA: Hierarchical Vision-Language-Action Model with Key-Action Reasoning and Motion Planning in a Unified Action Spacecited by 0
- 2026LIBERO-MAX: Do Robot Policies Adapt When the World Changes?cited by 0
- 2026LIBERO-MAX: Do Robot Policies Adapt When the World Changes?29 Sep 2026
- 2026H-VLA: Hierarchical Vision-Language-Action Model with Key-Action Reasoning and Motion Planning in a Unified Action Space19 Sep 2026
- 2026Task-Specified Active Metrological Inspection with Measurement-Steered VLA Manipulation and Deterministic Evidence Gating13 Sep 2026
- 2026Explicit State Elicitation Is Not Enough: A Controlled Audit of Memory-Policy Classification18 Aug 2026
- 2026SparkVLA: Stop-Aware Hierarchical VLA with Adaptive Action Chunking for Long-Horizon Manipulation17 Aug 2026
- 2026Mind-VLA: Instruction-Aware Spatial Representation Alignment for Vision-Language-Action Models5 Aug 2026
- 2026LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignments27 Jul 2026
- 2026CogniRoute: Learning to Route Social Evidence in Omni-Modal Models18 Jun 2026
- 2026World Action Models: A Survey18 Jun 2026
- 2026GIVE: Grounding Human Gestures in Vision-Language-Action Models11 Jun 2026
- 2026Self-Evaluation Is Already There: Eliciting Latent Judge Calibration in Base LLMs with Minimal Data3 Jun 2026
- 2026Completion at the Boundary (CaB): Deployable Switching with Completion-Aware Control under Limited Calibration29 May 2026
8 more
- 2026Gaze2Act: Gaze-Conditioned Vision-Language-Action Policies for Interactive Robot Manipulation28 May 2026
- 2026AffordVLA: Injecting Affordance Representations into Vision-Language-Action Models via Implicit Feature Alignment17 May 2026
- 2026Towards Long-horizon Embodied Agents with Tool-Aligned Vision-Language-Action Models13 May 2026
- 2026What's Missing in Screen-to-Action? Towards a UI-in-the-Loop Paradigm for Multimodal GUI Reasoning8 Apr 2026
- 2026FusionAgent: A Multimodal Agent with Dynamic Model Selection for Human Recognition27 Mar 2026
- 2026EgoForge: Goal-Directed Egocentric World Simulator20 Mar 2026
- 2026Frequency-Modulated Visual Restoration for Matryoshka Large Multimodal Models11 Mar 2026
- 2026ShardMemo: Scope-Before-Routing for Agentic Memory Retrieval29 Jan 2026
- 2026Towards Long-horizon Embodied Agents with Tool-Aligned Vision-Language-Action Modelscited by 8
- 2026Gaze2Act: Gaze-Conditioned Vision-Language-Action Policies for Interactive Robot Manipulationcited by 8
- 2026AffordVLA: Injecting Affordance Representations into Vision-Language-Action Models via Implicit Feature Alignmentcited by 5
- 2026GIVE: Grounding Human Gestures in Vision-Language-Action Modelscited by 1
- 2026SparkVLA: Stop-Aware Hierarchical VLA with Adaptive Action Chunking for Long-Horizon Manipulationcited by 1
- 2026Mind-VLA: Instruction-Aware Spatial Representation Alignment for Vision-Language-Action Modelscited by 0
- 2026SparkVLA: Stop-Aware Hierarchical VLA with Adaptive Action Chunking for Long-Horizon Manipulation17 Aug 2026
- 2026Mind-VLA: Instruction-Aware Spatial Representation Alignment for Vision-Language-Action Models5 Aug 2026
- 2026GIVE: Grounding Human Gestures in Vision-Language-Action Models11 Jun 2026
- 2026Gaze2Act: Gaze-Conditioned Vision-Language-Action Policies for Interactive Robot Manipulation28 May 2026
- 2026AffordVLA: Injecting Affordance Representations into Vision-Language-Action Models via Implicit Feature Alignment17 May 2026
- 2026Towards Long-horizon Embodied Agents with Tool-Aligned Vision-Language-Action Models13 May 2026
- 2026World Action Models: A Surveycited by 10
- 2026EgoForge: Goal-Directed Egocentric World Simulatorcited by 9
- 2026LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignmentscited by 0
- 2026LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignments27 Jul 2026
- 2026World Action Models: A Survey18 Jun 2026
- 2026EgoForge: Goal-Directed Egocentric World Simulator20 Mar 2026
- 2026Self-Evaluation Is Already There: Eliciting Latent Judge Calibration in Base LLMs with Minimal Datacited by 1
- 2026Completion at the Boundary (CaB): Deployable Switching with Completion-Aware Control under Limited Calibrationcited by 0
- 2026CogniRoute: Learning to Route Social Evidence in Omni-Modal Modelscited by 0
- 2026Explicit State Elicitation Is Not Enough: A Controlled Audit of Memory-Policy Classificationcited by 0
- 2026Task-Specified Active Metrological Inspection with Measurement-Steered VLA Manipulation and Deterministic Evidence Gatingcited by 0
- 2026LIBERO-MAX: Do Robot Policies Adapt When the World Changes?cited by 0
- 2026LIBERO-MAX: Do Robot Policies Adapt When the World Changes?29 Sep 2026
- 2026Task-Specified Active Metrological Inspection with Measurement-Steered VLA Manipulation and Deterministic Evidence Gating13 Sep 2026
- 2026Explicit State Elicitation Is Not Enough: A Controlled Audit of Memory-Policy Classification18 Aug 2026
- 2026CogniRoute: Learning to Route Social Evidence in Omni-Modal Models18 Jun 2026
- 2026Self-Evaluation Is Already There: Eliciting Latent Judge Calibration in Base LLMs with Minimal Data3 Jun 2026
- 2026Completion at the Boundary (CaB): Deployable Switching with Completion-Aware Control under Limited Calibration29 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding
- 2025Affordance Field Intervention: Enabling VLAs to Escape Memory Traps in Robotic Manipulation
- 2026AffordTrajDP: Dynamic Affordance-Guided Visuomotor Policy Learning for Robotic Manipulation
- 2024Affordance-Guided Reinforcement Learning via Visual Prompting
- 2025A0: An Affordance-Aware Hierarchical Model for General Robotic Manipulation
- 2026Goal2Skill: Long-Horizon Manipulation with Adaptive Planning and Reflection
- 2024Affordance-based Robot Manipulation with Flow Matching
- 2026AffordanceGrasp-R1:Leveraging Reasoning-Based Affordance Segmentation with Reinforcement Learning for Robotic Grasping