LM-X: Explainable Vision--Language--Action Modeling via Progress, Event, and Uncertainty Prediction
Not on the map. Why?
Abstract (arXiv)
Large-scale vision--language--action (VLA) policies have advanced generalist robot control, yet most remain stimulus-to-action black boxes: actions are exposed, but their explanatory state is not. They provide no native account of three explanatory signals: task progress, the next semantic transition, or local command reliability. Prior work shows that progress and event structure aid long-horizon control and that uncertainty supports monitoring; however, such capabilities are typically added or extracted only after action pretraining. The field therefore lacks a VLA foundation model whose explanatory state is jointly pretrained with control. Drawing on biological sensorimotor organization, in which outcome-sensitive, event-segmented, and probabilistic predictions structure behavior, we introduce LM-X. LM-X learns three directly supervised online signals: return-to-go (RTG) estimates visible progress and state quality; event-to-go (ETG) predicts the action sequence to the next semantic event; and heteroscedastic action-flow variance reports local command reliability. RTG conditions ETG and both condition action generation; uncertainty is estimated inside the action expert, making explanation part of control rather than a post-hoc description. We pretrain LM-X on more than 20,000 hours of heterogeneous real-robot trajectories, including over 1,000 hours of failed rollouts. LM-X achieves 74.1% success on 50 randomized-hard RoboTwin2.0 tasks and 73.5% on seven real-robot tasks, compared with 55.4% and 50.7% for GR00T N1.7. Its signals track progress and regression, anticipate event-scale motion, detect high-error actions, and provide advance failure warning. These results establish LM-X as an explainable VLA foundation model that couples transparent predictive state with stronger generalist control. Github: https://github.com/loongOpen/LoongWu-LM-X-VLA
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 43 of its 62 references that are in the map. First posted on arXiv 26 Aug 2026.
Builds on
43 of 62 references.
References · 43
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
- 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
31 more
- 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,546
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
- 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
- 2025Mean Flows for One-step Generative Modelingcited by 561
- 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 518
- 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 451
- 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 405
- 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 365
- 2025π*0.6: a VLA That Learns From Experiencecited by 303
- 2024Flow Matching Guide and Codecited by 301
- 2024RT-H: Action Hierarchies Using Languagecited by 210
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
- 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 138
- 2026MEM: Multi-Scale Embodied Memory for Vision Language Action Modelscited by 61
- 2025ReinboT: Amplifying Robot Visual-Language Manipulation with Reinforcement Learningcited by 47
- 2023BayesDiff: Estimating Pixel-wise Uncertainty in Diffusion via Bayesian Inferencecited by 35
- 2025Generative Uncertainty in Diffusion Modelscited by 23
- 2024Shedding Light on Large Generative Networks: Estimating Epistemic Uncertainty in Diffusion Modelscited by 22
- 2026EventVLA: Event-Driven Visual Evidence Memory for Long-Horizon Vision-Language-Action Policiescited by 22
- 2024Diffusion Model Guided Sampling with Pixel-Wise Aleatoric Uncertainty Estimationcited by 20
- 2026PRTS: A Primitive Reasoning and Tasking System via Contrastive Representationscited by 9
- 2026Shifting Uncertainty to Critical Moments: Towards Reliable Uncertainty Quantification for VLA Modelcited by 7
- 2026Why Does Action Chunking Improve Behavioral Cloning Performance in Robotic Control?cited by 7
- 2025EveryDayVLA: A Vision-Language-Action Model for Affordable Robotic Manipulationcited by 5
- 2026SCALE: Self-uncertainty Conditioned Adaptive Looking and Execution for Vision-Language-Action Modelscited by 4
- 2026Flow Matching with Uncertainty Quantification and Guidancecited by 4
- 2026Uncertainty Quantification for Flow-Based Vision-Language-Action Modelscited by 4
- 2026STEAM: Self-Supervised Temporal Ensemble Advantage Modeling for Real-World Robot Learningcited by 3
- 2026Improving Vision-Language-Action Model Fine-Tuning with Structured Stage and Keyframe Supervisioncited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Beyond Task Success: Behavioral and Representational Diagnostics for WAM and VLA
- 2026Rethinking Visual-Language-Action Model Scaling: Alignment, Mixture, and Regularization
- 2026Not All Features Are Created Equal: A Mechanistic Study of Vision-Language-Action Models
- 2025An Anatomy of Vision-Language-Action Models: From Modules to Milestones and Challenges
- 2025LoHoVLA: A Unified Vision-Language-Action Model for Long-Horizon Embodied Tasks
- 2026LaWAM: Latent World Action Models for Efficient Dynamics-Aware Robot Policies
- 2025Mechanistic interpretability for steering vision-language-action models
- 2026G0.5: One Autoregressive Stream for Robot Reasoning and Action