mlsanityData to 7 Oct 2026

LEGS: Fine-Tuning Teleop-Free VLAs for Humanoid Loco-manipulation in an Embodied Gaussian Splatting World

arXiv 2606.01458 ↗First posted 31 May 2026cs.ROcited by 0

On the map: Vision-language-action models → Tactile, force-aware and humanoid VLA

Abstract (arXiv)

Training vision-language-action (VLA) policies for humanoid loco-manipulation is constrained by the high cost and complexity of collecting human teleoperation demonstrations. VLA policies fine-tuned in simulators have, until now, failed to transfer effectively in humanoid loco-manipulation tasks. We present LEGS (Loco-manipulation via Embodied Gaussian Splatting), a hybrid simulator that composites a mesh foreground (robot, objects, props) over a photorealistic 3D Gaussian Splatting (3DGS) background reconstructed from a handheld scene capture. LEGS uses a procedural motion-primitive generator to synthesize labeled demonstrations at scale without human teleoperation, and a deterministic two-stage color calibration to align the rendered 3DGS image to the robot's deployment camera. On a Unitree G1 humanoid robot, across three pick-and-place tasks of increasing whole-body difficulty and three VLA backbones (psi_0, pi_0.5, GR00T N1.6), a policy trained purely on LEGS data matches or exceeds one trained on human teleoperation demos on every experiment. It also outperforms a mesh-only simulation baseline that ablates the effect of the 3DGS background, showing that photorealistic rendering is a key enabler for synthetic data transfer. Humanoid motion is recorded independently of scene appearance in LEGS, allowing the same auto-generated demonstrations to be re-rendered under new backgrounds and object meshes--covering a new scene at more than 15x lower cost than teleoperation--to augment training data for robustness to scene variations. Under combined object-and-scene appearance shift, the policy trained on re-rendered LEGS-AUG data maintains task success while the baseline trained on teleoperation data fails entirely. Our project page is located at https://legsvla.github.io/.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Tactile, force-aware and humanoid VLA, one of 13 directions in Vision-language-action models. Builds on 35 of its 37 references that are in the map. First posted on arXiv 31 May 2026.

Builds on

35 of 37 references.

References · 35

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  2. 2024SAM 2: Segment Anything in Images and Videoscited by 2,784
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  4. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  5. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,401
  6. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,167
  7. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 373
  8. 2025SAM 3D: 3Dfy Anything in Imagescited by 230
  9. 2023Foundation Models in Robotics: Applications, Challenges, and the Futurecited by 212
  10. 2024OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learningcited by 167
  11. 2024HumanPlus: Humanoid Shadowing and Imitation from Humanscited by 164
  12. 2025TWIST: Teleoperated Whole-Body Imitation Systemcited by 86
23 more
  1. 2025SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Controlcited by 83
  2. 2025HOMIE: Humanoid Loco-Manipulation with Isomorphic Exoskeleton Cockpitcited by 65
  3. 2022Approximate Convex Decomposition for 3D Meshes with Collision-Aware Concavity and Tree Searchcited by 61
  4. 2024Mobile-TeleVision: Predictive Motion Priors for Humanoid Whole-Body Controlcited by 41
  5. 2024Splat-MOVER: Multi-Stage, Open-Vocabulary Robotic Manipulation via Editable Gaussian Splattingcited by 37
  6. 2025WholeBodyVLA: Towards Unified Latent VLA for Whole-Body Loco-Manipulation Controlcited by 32
  7. 2024Opt2Skill: Imitating Dynamically-feasible Whole-Body Trajectories for Versatile Humanoid Loco-Manipulationcited by 28
  8. 2026Psi0: An Open Foundation Model Towards Universal Humanoid Loco-Manipulationcited by 25
  9. 2025GaussGym: An open-source real-to-sim framework for learning locomotion from pixelscited by 22
  10. 2025VIRAL: Visual Sim-to-Real at Scale for Humanoid Loco-Manipulationcited by 21
  11. 2025Viser: Imperative, Web-based 3D Visualization in Pythoncited by 19
  12. 2026EgoHumanoid: Unlocking In-the-Wild Loco-Manipulation with Robot-Free Egocentric Demonstrationcited by 18
  13. 2025GSWorld: Closed-Loop Photo-Realistic Simulation Suite for Robotic Manipulationcited by 16
  14. 2025GRaD-Nav++: Vision-Language Model Enabled Visual Drone Navigation with Gaussian Radiance Fields and Differentiable Dynamicscited by 14
  15. 2024SOUS VIDE: Cooking Visual Drone Navigation Policies in a Gaussian Splatting Vacuumcited by 13
  16. 2025Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfercited by 13
  17. 2026Humanoid Manipulation Interface: Humanoid Whole-Body Manipulation from Robot-Free Demonstrationscited by 11
  18. 2025DemoHLM: From One Demonstration to Generalizable Humanoid Loco-Manipulationcited by 10
  19. 2024ARCH: Hierarchical Hybrid Learning for Long-Horizon Contact-Rich Robotic Assemblycited by 7
  20. 2025SINGER: An Onboard Generalist Vision-Language Navigation Policy for Dronescited by 6
  21. 2025GRaD-Nav: Efficiently Learning Visual Drone Navigation with Gaussian Radiance Fields and Differentiable Dynamicscited by 5
  22. 2026π, But Make It Fly: Physics-Guided Transfer of VLA Models to Aerial Manipulationcited by 5
  23. 2026GS-Playground: A High-Throughput Photorealistic Simulator for Vision-Informed Robot Learningcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026SIMPLE: Simulation-Based Policy Learning and Evaluation for Humanoid Loco-manipulation
  2. 2026OpenHLM: An Empirical Recipe for Whole-Body Humanoid Loco-Manipulation
  3. 2026FetchMan: Learning Visual Humanoid Loco-Manipulation Policies from Simulated Experiences ↗
  4. 2026HumanoidMimicGen: Data Generation for Loco-Manipulation via Whole-Body Planning
  5. 2026MotionWAM: Towards Foundation World Action Models for Real-Time Humanoid Loco-Manipulation
  6. 2023Deep Imitation Learning for Humanoid Loco-manipulation through Human Teleoperation
  7. 2026Humanoid Loco-Manipulation With Discrete VLA Model
  8. 2025TWIST2: Scalable, Portable, and Holistic Humanoid Data Collection System