# HARP-VLA: Human-Robot Aligned Representation Learning for Vision-Language-Action Model arXiv: https://arxiv.org/abs/2605.31234 · HTML: https://mlsanity.com/paper/2605.31234/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Placed in Human-video and latent-action pretraining, one of 13 directions in Vision-language-action models. - Builds on 34 of its 37 references that are in the map. - First posted on arXiv 29 May 2026. ## Placed in - Vision-language-action models / [Human-video and latent-action pretraining](https://mlsanity.com/vla/human-video-and-latent-action-pretraining/) ## Builds on (most cited first, 30 of 34) - [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023) - [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023) - [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023) - [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023) - [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024) - [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022) - [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025) - [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024) - [Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success](https://mlsanity.com/paper/2502.19645/) (2025) - [RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation](https://mlsanity.com/paper/2410.07864/) (2024) - [RLBench: The Robot Learning Benchmark & Learning Environment](https://mlsanity.com/paper/1909.12271/) (2019) - [BridgeData V2: A Dataset for Robot Learning at Scale](https://mlsanity.com/paper/2308.12952/) (2023) - [CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks](https://mlsanity.com/paper/2112.03227/) (2021) - [UniVLA: Learning to Act Anywhere with Task-centric Latent Actions](https://mlsanity.com/paper/2505.06111/) (2025) - [Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models](https://mlsanity.com/paper/2402.07865/) (2024) - [Latent Action Pretraining from Videos](https://mlsanity.com/paper/2410.11758/) (2024) - [TAPIR: Tracking Any Point with per-frame Initialization and temporal Refinement](https://mlsanity.com/paper/2306.08637/) (2023) - [MimicPlay: Long-Horizon Imitation Learning by Watching Human Play](https://mlsanity.com/paper/2302.12422/) (2023) - [RH20T: A Comprehensive Robotic Dataset for Learning Diverse Skills in One-Shot](https://mlsanity.com/paper/2307.00595/) (2023) - [EgoMimic: Scaling Imitation Learning via Egocentric Video](https://mlsanity.com/paper/2410.24221/) (2024) - [RT-Trajectory: Robotic Task Generalization via Hindsight Trajectory Sketches](https://mlsanity.com/paper/2311.01977/) (2023) - [Flow as the Cross-Domain Manipulation Interface](https://mlsanity.com/paper/2407.15208/) (2024) - [WiLoR: End-to-end 3D Hand Localization and Reconstruction in-the-wild](https://mlsanity.com/paper/2409.12259/) (2024) - [Learning by Watching: Physical Imitation of Manipulation Skills from Human Videos](https://mlsanity.com/paper/2101.07241/) (2021) - [IGOR: Image-GOal Representations are the Atomic Control Units for Foundation Models in Embodied AI](https://mlsanity.com/paper/2411.00785/) (2024) - [DexUMI: Using Human Hand as the Universal Manipulation Interface for Dexterous Manipulation](https://mlsanity.com/paper/2505.21864/) (2025) - [RoVi-Aug: Robot and Viewpoint Augmentation for Cross-Embodiment Robot Learning](https://mlsanity.com/paper/2409.03403/) (2024) - [UniSkill: Imitating Human Videos via Cross-Embodiment Skill Representations](https://mlsanity.com/paper/2505.08787/) (2025) - [Human2Robot: Learning Robot Actions from Paired Human-Robot Videos](https://mlsanity.com/paper/2502.16587/) (2025) - [MimicDreamer: Aligning Human and Robot Demonstrations for Scalable VLA Training](https://mlsanity.com/paper/2509.22199/) (2025) ## Built on by (most cited first, 5 of 5) - [Action QFormer: Structured Representation Shaping under Action Supervision in Vision-Language-Action Models](https://mlsanity.com/paper/2607.14635/) (2026) - [GALA: Geometry-Aware Latent Action Modeling for Vision-Language-Action Model Pretraining across Embodiments](https://mlsanity.com/paper/2609.21948/) (2026) - [Frame-Level Temporal Alignment for Human-to-Robot Visual Adaptation](https://mlsanity.com/paper/2610.04372/) (2026) - [PAIR: Bridging Perception and Action in Vision-Language-Action Models](https://mlsanity.com/paper/2610.09016/) (2026) - [SPECTRA-VLA: Base-Fine Hyperspectral Token Transfer at Frozen Vision-Language-Action Interfaces](https://mlsanity.com/paper/2610.19651/) (2026) ## Abstract (arXiv) Learning generalizable vision-language-action (VLA) models from large-scale human videos is promising but challenging due to cross-embodiment discrepancies in both visual observations and executable actions. While latent action models reduce the action execution gap by learning action abstractions, they still rely on visual features. Thus, misaligned human and robot visual representations can lead to inconsistencies in policy inputs and induce domain-dependent latent actions, hindering effective co-training with human videos.To address this, we propose HARP, a human-robot aligned representation learning framework for more effective VLA pretraining from human videos. Specifically, HARP uses paired human-robot demonstrations as cross-embodiment bridges and abundant unpaired human and robot videos as a scalable dynamics supervision data source. It trains a robot-adapted visual encoder and a latent action model with manipulation-centric auxiliary cues and a source-relative pair-discriminative alignment loss, which adapts robot representations toward human semantics while preserving pair-level discrimination. The learned aligned vision encoder and latent action model provide a unified vision and action representation for VLA-style policy learning, where human and robot videos provide vision-language-to-latent-action supervision and a lightweight robot action head grounds latent actions into executable commands. Experiments on feature visualization, simulation, and real-world manipulation show improved human-robot alignment and downstream policy performance, achieving 4.481 average length on CALVIN ABCrightarrowD and a 7.1 percentage-point improvement in real-world success rate over the strongest evaluated baseline. Code, videos, and demos are available at https://puzhenyuan.github.io/HARP-VLA-website/.