mlsanityData to 14 Oct 2026

PhysBrain: Human Egocentric Data as a Bridge from Vision Language Models to Physical Intelligence

arXiv 2512.16793 ↗First posted 18 Dec 2025cs.ROcited by 11

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

Robotic generalization relies on physical intelligence: the ability to reason about state changes, contact-rich interactions, and long-horizon planning under egocentric perception and action. Vision Language Models (VLMs) are essential to Vision-Language-Action (VLA) systems, but the reliance on third-person training data creates a viewpoint gap for humanoid robots. Collecting massive robot-centric data is an ideal but impractical solution due to cost and diversity constraints. Conversely, human egocentric videos offer a highly scalable data source with rich interaction context, yet the embodiment mismatch prevents the direct application. To bridge this gap, we propose an Egocentric2Embodiment Translation Pipeline that transforms raw human egocentric videos into multi-level, schema-driven embodiment supervision with enforced evidence grounding and temporal consistency, enabling the construction of the Egocentric2Embodiment dataset (E2E-3M) at scale. An egocentric-aware embodied brain, termed PhysBrain, is obtained by training on the E2E-3M dataset. PhysBrain exhibits substantially improved egocentric understanding, particularly for planning. It provides an egocentric-aware initialization that enables more sample-efficient VLA fine-tuning and higher success rates, demonstrating effective transfer from human egocentric supervision to downstream robot control.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 43 of its 56 references that are in the map. First posted on arXiv 18 Dec 2025.

Builds on

43 of 56 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Advancing Egocentric Video Question Answering with Multimodal Large Language Models6 Apr
  2. 2024What Matters in Building Vision-Language-Action Models for Generalist Robots18 Dec
  3. 2024EgoPlan-Bench2: A Benchmark for Multimodal Large Language Model Planning in Real-World Scenarios5 Dec
  4. 2024Robotic Control via Embodied Chain-of-Thought Reasoning11 Jul
  5. 2023EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planning11 Dec
  6. 2023EgoThink: Evaluating First-Person Perspective Thinking Capability of Vision-Language Models27 Nov
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul

Other references · 36

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2022Scalable Diffusion Models with Transformerscited by 6,054
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2024GPT-4o System Cardcited by 4,588
  7. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  8. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  9. 2025Qwen3-VL Technical Reportcited by 2,793
  10. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  11. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  12. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
24 more
  1. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  2. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  3. 2019ZeRO: Memory Optimizations Toward Training Trillion Parameter Modelscited by 1,301
  4. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  5. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  6. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
  7. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
  8. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
  9. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 304
  10. 2022Rectified Flow: A Marginal Preserving Approach to Optimal Transportcited by 283
  11. 2022Egocentric Video-Language Pretrainingcited by 232
  12. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 211
  13. 2023EgoVLPv2: Egocentric Video-Language Pre-training with Fusion in the Backbonecited by 130
  14. 2025OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoningcited by 107
  15. 2025Being-H0: Vision-Language-Action Pretraining from Large-Scale Human Videoscited by 102
  16. 2025Scalable Vision-Language-Action Model Pretraining for Robotic Manipulation with Real-Life Human Activity Videoscited by 70
  17. 2025ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledgecited by 49
  18. 2025TGRPO :Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimizationcited by 49
  19. 2024Emma-X: An Embodied Multimodal Action Model with Grounded Chain of Thought and Look-ahead Spatial Reasoningcited by 47
  20. 2025RynnVLA-001: Using Human Demonstrations to Improve Robot Manipulationcited by 25
  21. 2025Video2Act: A Dual-System Video Diffusion Policy with Robotic Spatio-Motional Modelingcited by 13
  22. 2025DualVLA: Building a Generalizable Embodied Agent via Partial Decoupling of Reasoning and Actioncited by 12
  23. 2025EgoM2P: Egocentric Multimodal Multitask Pretrainingcited by 9
  24. 2025Bridging Perspectives: A Survey on Cross-view Collaborative Intelligence with Egocentric-Exocentric Visioncited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 11.

11 papers.

  1. 2026LangForce: Bayesian Decomposition of Vision Language Action Models via Latent Action Queriescited by 27
  2. 2026TwinBrainVLA: Unleashing the Potential of Generalist VLMs for Embodied Tasks via Asymmetric Mixture-of-Transformerscited by 14
  3. 2026Egocentric World Model for Photorealistic Hand-Object Interaction Synthesiscited by 12
  4. 2026IntentVLA: Short-Horizon Intent Modeling for Aliased Robot Manipulationcited by 7
  5. 2026FrameSkip: Learning from Fewer but More Informative Frames in VLA Trainingcited by 6
  6. 2026Robot Learning from Human Videos: A Surveycited by 5
  7. 2026In-Context VLA: Endowing Vision-Language-Action Models with Language via In-Context Post-Training and Agentic Tool Usecited by 2
  8. 2026Watching Movies Like a Human: Egocentric Emotion Understanding for Embodied Companionscited by 0
  9. 2026SIEVE: Structure-Aware Data Selection for Imitation Learning with VLA Modelscited by 0
  10. 2026Decoupling Intention from Trajectory: A Representational Deduction Framework for World Action Modelscited by 0
  11. 2026Think Like a World Model, Act Like a VLA: Distilling World-Model Representations into Compact Robot Policiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Ego2Robot: Scalable Robot Data Synthesis from Egocentric Human Data
  2. 2026Vision-Language Models for Egocentric Video: From Hand-Object Interaction to Embodied AI
  3. 2026PhysBrain 1.5: From Vision-Language Models to Physical Foundation Models
  4. 2026PhysBrain 1.0 Technical Report
  5. 2026ACE-Ego-0: Unifying Egocentric Human and Robotic Data for VLA Pretraining
  6. 2026IronMind: Scaling Humanoid Dexterous Manipulation via Camera-Space Ego-Centric Pretraining ↗
  7. 2025Developing Vision-Language-Action Model from Egocentric Videos
  8. 2026EgoLAP: Learning from Egocentric Human Data through Language-Action Reasoning