mlsanityData to 7 Oct 2026

Learning Additively Compositional Latent Actions for Embodied AI

arXiv 2604.03340 ↗First posted 3 Apr 2026cs.CV cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Latent action learning infers pseudo-action labels from visual transitions, providing an approach to leverage internet-scale video for embodied AI. However, most methods learn latent actions without structural priors that encode the additive, compositional structure of physical motion. As a result, latents often entangle irrelevant scene details or information about future observations with true state changes and miscalibrate motion magnitude. We introduce Additively Compositional Latent Action Model (AC-LAM), which enforces scene-wise additive composition structure over short horizons on the latent action space. These AC constraints encourage simple algebraic structure in the latent action space~(identity, inverse, cycle consistency) and suppress information that does not compose additively. Empirically, AC-LAM learns more structured, motion-specific, and displacement-calibrated latent actions and provides stronger supervision for downstream policy learning, outperforming state-of-the-art LAMs across simulated and real-world tabletop tasks.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 49 of its 63 references that are in the map. First posted on arXiv 3 Apr 2026.

Builds on

49 of 63 references.

References · 49

  1. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,120
  2. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  3. 2017Neural Discrete Representation Learningcited by 4,650
  4. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  5. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  6. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  7. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  9. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  10. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  11. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  12. 2024Genie: Generative Interactive Environmentscited by 706
37 more
  1. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  2. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  3. 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
  4. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  5. 2022HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interactioncited by 321
  6. 2024Latent Action Pretraining from Videoscited by 302
  7. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 248
  8. 2020The EPIC-KITCHENS Dataset: Collection, Challenges and Baselinescited by 221
  9. 2022Interactive Language: Talking to Robots in Real Timecited by 217
  10. 2023Structured World Models from Human Videoscited by 144
  11. 2023FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulationcited by 132
  12. 2023Learning to Act without Actionscited by 116
  13. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
  14. 2022From Play to Policy: Conditional Behavior Generation from Uncurated Robot Datacited by 103
  15. 2022Grounding Language with Visual Affordances over Unstructured Datacited by 101
  16. 2022Robot Learning on the Job: Human-in-the-Loop Autonomy and Learning During Deploymentcited by 101
  17. 2022Latent Plans for Task-Agnostic Offline Reinforcement Learningcited by 81
  18. 2024FMB: a Functional Manipulation Benchmark for Generalizable Robotic Learningcited by 79
  19. 2022Learning and Retrieval from Prior Data for Skill-based Imitation Learningcited by 77
  20. 2024IGOR: Image-GOal Representations are the Atomic Control Units for Foundation Models in Embodied AIcited by 68
  21. 2023HYDRA: Hybrid Robot Actions for Imitation Learningcited by 54
  22. 2023Linear Spaces of Meanings: Compositional Structures in Vision-Language Modelscited by 48
  23. 2025VideoWorld: Exploring Knowledge Learning from Unlabeled Videoscited by 42
  24. 2026Learning Latent Action World Models In The Wildcited by 42
  25. 2025Latent Action Learning Requires Supervision in the Presence of Distractorscited by 40
  26. 2025CLAM: Continuous Latent Action Models for Robot Learning from Unlabeled Demonstrationscited by 39
  27. 2025CoMo: Learning Continuous Latent Motion from Internet Videos for Scalable Robot Learningcited by 37
  28. 2022Egocentric Prediction of Action Target in 3Dcited by 31
  29. 2025What Do Latent Action Models Actually Learn?cited by 30
  30. 2024HO-Cap: A Capture System and Dataset for 3D Reconstruction and Pose Tracking of Hand-Object Interactioncited by 25
  31. 2025Modeling Fine-Grained Hand-Object Dynamics for Egocentric Video Representation Learningcited by 20
  32. 2025Co-Evolving Latent Action World Modelscited by 12
  33. 2025LAOF: Robust Latent Action Learning with Optical Flow Constraintscited by 12
  34. 2026MVP-LAM: Learning Action-Centric Latent Action via Cross-Viewpoint Reconstructioncited by 11
  35. 2025Not Only Text: Exploring Compositionality of Visual Representations in Vision-Language Modelscited by 8
  36. 2025LatBot: Distilling Universal Latent Actions for Vision-Language-Action Modelscited by 7
  37. 2025How Do VLAs Effectively Inherit from VLMs?cited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026ALAM: Algebraically Consistent Latent Action Model for Vision-Language-Action Modelscited by 3
  2. 2026Causally Debiased Latent Action Model for Embodied Action Conditioned World Modelscited by 3
  3. 2026DLAM: Distributional Latent Actions with Temporal Constraintscited by 1
  4. 2026Latent Actions from Factorized Transition Effects under Agent Ambiguitycited by 0
  5. 2026Dueling World Models: Advantage-Style Action Channels for Common-Mode Distractor Rejectioncited by 0
  6. 2026Algebraic Consistency Alone Does Not Certify Temporal Structure in Latent Action Modelscited by 0