# Improving Cross-embodiment Transfer in Latent Action Models with Action-Similarity Supervision arXiv: https://arxiv.org/abs/2609.19846 · HTML: https://mlsanity.com/paper/2609.19846/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Builds on 31 of its 33 references that are in the map. - First posted on arXiv 17 Sep 2026. ## Builds on (most cited first, 30 of 31) - [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024) - [Supervised Contrastive Learning](https://mlsanity.com/paper/2004.11362/) (2020) - [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024) - [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025) - [GR00T N1: An Open Foundation Model for Generalist Humanoid Robots](https://mlsanity.com/paper/2503.14734/) (2025) - [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019) - [DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset](https://mlsanity.com/paper/2403.12945/) (2024) - [Genie: Generative Interactive Environments](https://mlsanity.com/paper/2402.15391/) (2024) - [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020) - [SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Robotics](https://mlsanity.com/paper/2506.01844/) (2025) - [RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation](https://mlsanity.com/paper/2506.18088/) (2025) - [UniVLA: Learning to Act Anywhere with Task-centric Latent Actions](https://mlsanity.com/paper/2505.06111/) (2025) - [AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems](https://mlsanity.com/paper/2503.06669/) (2025) - [MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrations](https://mlsanity.com/paper/2310.17596/) (2023) - [Latent Action Pretraining from Videos](https://mlsanity.com/paper/2410.11758/) (2024) - [X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Model](https://mlsanity.com/paper/2510.10274/) (2025) - [Soft-DTW: a Differentiable Loss Function for Time-Series](https://mlsanity.com/paper/1703.01541/) (2017) - [A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation](https://mlsanity.com/paper/2507.05331/) (2025) - [Learning to Act without Actions](https://mlsanity.com/paper/2312.10812/) (2023) - [AdaWorld: Learning Adaptable World Models with Latent Actions](https://mlsanity.com/paper/2503.18938/) (2025) - [IGOR: Image-GOal Representations are the Atomic Control Units for Foundation Models in Embodied AI](https://mlsanity.com/paper/2411.00785/) (2024) - [villa-X: Enhancing Latent Action Modeling in Vision-Language-Action Models](https://mlsanity.com/paper/2507.23682/) (2025) - [Latent Action Learning Requires Supervision in the Presence of Distractors](https://mlsanity.com/paper/2502.00379/) (2025) - [Emergence of Human to Robot Transfer in Vision-Language-Action Models](https://mlsanity.com/paper/2512.22414/) (2025) - [CLAM: Continuous Latent Action Models for Robot Learning from Unlabeled Demonstrations](https://mlsanity.com/paper/2505.04999/) (2025) - [What Do Latent Action Models Actually Learn?](https://mlsanity.com/paper/2506.15691/) (2025) - [ConLA: Contrastive Latent Action Learning from Human Videos for Robotic Manipulation](https://mlsanity.com/paper/2602.00557/) (2026) - [MVP-LAM: Learning Action-Centric Latent Action via Cross-Viewpoint Reconstruction](https://mlsanity.com/paper/2602.03668/) (2026) - [What Makes Video World Model Latents Action-Relevant: Prediction over Reconstruction](https://mlsanity.com/paper/2606.07687/) (2026) - [Learning to Act Robustly with View-Invariant Latent Actions](https://mlsanity.com/paper/2601.02994/) (2026) ## Built on by (most cited first, 1 of 1) - [OneLA: Learning Many Worlds from Latent Actions](https://mlsanity.com/paper/2610.20160/) (2026) ## Abstract (arXiv) As generalist robot policies gain vision and language from web-scale pretraining, demonstrations remain costly to collect and tied to the robot that recorded them. Latent action models (LAMs) address both by learning latent actions from action-free videos that can be shared across embodiments, however, in practice, LAMs are sensitive to background visual noise, and the same motion from two different robots may be encoded with different latents. One solution to the background visual noise is to add an auxiliary loss predicting the robot action from the latent action, further associating the latent action space to the embodiment specific robot action space. We study a different use of the same labels, through action-similarity supervision. The similarity between any two latent actions is trained to match the similarity of the two ground-truth robot action sequences. The ground-truth actions are never predicted by the LAM, so the latent action does not need to encode embodiment specifics. We evaluate cross-embodiment transfer on RoboTwin 2.0 in a controlled setup, two bimanual robots demonstrate disjoint task sets, a policy is trained on all the demonstrations, and each robot is evaluated closed-loop on the tasks only the other demonstrated. With the policy architecture and its hyperparameters, the dataset, and the evaluation protocol fixed, predicting latent actions instead of ground-truth actions more than doubles cross-embodiment success. Given the same ground-truth actions, similarity supervision transfers better than an auxiliary loss that predicts the ground-truth action during the LAM training. Computing the similarities on end-effector motion rather than joint-space motion, and letting the loss compare latent actions across the two robots, gives the best approach of the study.