mlsanityData to 14 Oct 2026

PhysBrain 1.0 Technical Report

arXiv 2605.15298 ↗First posted 14 May 2026cs.RO cs.AI cs.CLcited by 2

On the map: Vision-language-action models → Human-video and latent-action pretraining

Abstract (arXiv)

Vision-language-action models have advanced rapidly, but robot trajectories alone provide limited coverage for learning broad physical understanding. PhysBrain 1.0 studies a complementary route: converting large-scale human egocentric video into structured physical commonsense supervision before robot adaptation. Our data engine extracts scene elements, spatial dynamics, action execution, and depth-aware relations, then turns them into question-answer supervision for training PhysBrain VLMs. The resulting physical priors are further transferred to VLA policies through a capability-preserving and language-sensitive adaptation design. Across multimodal QA benchmarks and embodied control benchmarks, including ERQA, PhysBench, SimplerEnv-WidowX, LIBERO, and RoboCasa, PhysBrain 1.0 achieves SOTA results and shows especially strong out-of-domain performance on SimplerEnv. These results suggest that scaling physical commonsense from human interaction video can provide an effective bridge from multimodal understanding to robot action.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Human-video and latent-action pretraining, one of 13 directions in Vision-language-action models. Builds on 34 of its 40 references that are in the map. First posted on arXiv 14 May 2026.

Builds on

34 of 40 references.

References · 34

  1. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  2. 2025Qwen3-VL Technical Reportcited by 2,793
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  4. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  5. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  6. 2019Towards VQA Models That Can Readcited by 1,718
  7. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  8. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  9. 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
  10. 2024Octo: An Open-Source Generalist Robot Policycited by 1,484
  11. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  12. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
22 more
  1. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 883
  2. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
  3. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
  4. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 519
  5. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
  6. 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 451
  7. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 422
  8. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
  9. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 304
  10. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 303
  11. 2020The EPIC-KITCHENS Dataset: Collection, Challenges and Baselinescited by 225
  12. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 211
  13. 2025PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understandingcited by 160
  14. 2026StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developingcited by 112
  15. 2025F1: A Vision-Language-Action Model Bridging Understanding and Generation to Actionscited by 86
  16. 2025VideoVLA: Video Generators Can Be Generalizable Robot Manipulatorscited by 64
  17. 2025Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgettingcited by 58
  18. 2025ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledgecited by 49
  19. 2026LangForce: Bayesian Decomposition of Vision Language Action Models via Latent Action Queriescited by 27
  20. 2026TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformerscited by 14
  21. 2025Euclid's Gift: Enhancing Spatial Perception and Reasoning in Vision-Language Models via Geometric Surrogate Taskscited by 13
  22. 2026Green-VLA: Staged Vision-Language-Action Model for Generalist Robotscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026PhysBrain 1.5: From Vision-Language Models to Physical Foundation Modelscited by 0
  2. 2026HABILIS Brain 0: Geometry-Change Supervision for Vision-Language-Action and Residual Flow Recoverycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025PhysBrain: Human Egocentric Data as a Bridge from Vision Language Models to Physical Intelligence
  2. 2026PhysMind: From Video to Executable Worlds for Training-Free Physical Reasoning
  3. 2026PhysVLA: Towards Physically-Grounded VLA for Embodied Robotic Manipulation
  4. 2026CometVLA: Co-Training on an Embodied Data Pyramid towards Physical Understanding
  5. 2026From Human Videos to Robot Manipulation: A Survey on Scalable Vision-Language-Action Learning with Human-Centric Data
  6. 2025Visual Embodied Brain: Let Multimodal Large Language Models See, Think, and Control in Spaces
  7. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videos
  8. 2025PhysVidBench: Language-Grounded Evaluation of Physical Commonsense in Text-to-Video Models