mlsanityData to 14 Oct 2026

Assistron: Bayesian Shared Autonomy with Off-the-shelf Vision-Language-Action Models

arXiv 2606.23147 ↗First posted 22 Jun 2026cs.ROcited by 0

On the map: Vision-language-action models → Generalist VLA models and benchmarks

Abstract (arXiv)

We propose Assistron, a shared autonomy model that leverages Vision-Language-Action (VLA) models to assist the user in daily activities. Our approach is grounded in two core principles: (1)~minimizing human cognitive and physical effort by leveraging VLA-driven autonomy for macro-movements, and (2)~prioritizing human intervention specifically at critical failure points. Driven by the user's verbal language commands, Assistron utilizes the VLA to autonomously execute macro-reaching trajectories, saving users' effort. In contact-rich interactions where VLAs tend to fail, Assistron employs a phase-aware interaction detection mechanism and solicits the user to intervene, in turn adjusting the VLA's action generation via flow matching guidance. Critically, our formulation eliminates the need for VLA fine-tuning, protecting its broad behavioral priors from catastrophic forgetting and ensuring the model does not become a narrow specialist. We validate our approach on a comprehensive multi-task scene recovery benchmark encompassing diverse daily manipulation skills. Empirical results demonstrate that Assistron significantly improves task success rates over pure autonomous baselines while significantly reducing human cognitive and physical workload compared to traditional teleoperation, offering a scalable, smooth, and effortless paradigm for assistive manipulation. The code is available in https://github.com/mousecpn/Assistron.git.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Generalist VLA models and benchmarks, one of 13 directions in Vision-language-action models. Builds on 33 of its 49 references that are in the map. First posted on arXiv 22 Jun 2026.

Builds on

33 of 49 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025π0.5: a Vision-Language-Action Model with Open-World Generalization22 Apr

Other references · 32

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  4. 2025π*0.6: a VLA That Learns From Experiencecited by 303
  5. 2025Hi Robot: Open-Ended Instruction Following with Hierarchical Vision-Language-Action Modelscited by 221
  6. 2025Real-Time Execution of Action Chunking Flow Policiescited by 216
  7. 2025MemoryVLA: Perceptual-Cognitive Memory in Vision-Language-Action Models for Robotic Manipulationcited by 192
  8. 2024Yell At Your Robot: Improving On-the-Fly from Language Correctionscited by 124
  9. 2021ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learningcited by 108
  10. 2023Training-free Linear Image Inverses via Flowscited by 72
  11. 2025DemoGen: Synthetic Demonstration Generation for Data-Efficient Visuomotor Policy Learningcited by 69
  12. 2025RoboArena: Distributed Real-World Evaluation of Generalist Robot Policiescited by 64
20 more
  1. 2025On the Guidance of Flow Matchingcited by 45
  2. 2025Running VLAs at Real-time Speedcited by 43
  3. 2017Shared Autonomy via Hindsight Optimization for Teleoperation and Teamingcited by 40
  4. 2024Inference-Time Policy Steering through Human Interactionscited by 38
  5. 2023To the Noise and Back: Diffusion for Shared Autonomycited by 22
  6. 2022PATO: Policy Assisted TeleOperation for Scalable Robot Data Collectioncited by 19
  7. 2025Learning a Thousand Tasks in a Daycited by 11
  8. 2024Human-Agent Joint Learning for Efficient Robot Manipulation Skill Acquisitioncited by 10
  9. 2025End-to-End Dexterous Arm-Hand VLA Policies via Shared Autonomy: VR Teleoperation Augmented by Autonomous Hand VLA Policy for Efficient Data Collectioncited by 9
  10. 2024Robot Trajectron: Trajectory Prediction-based Shared Control for Robot Manipulationcited by 7
  11. 2026AR-VLA: True Autoregressive Action Expert for Vision-Language-Action Modelscited by 6
  12. 2026Towards Human-Like Manipulation through RL-Augmented Teleoperation and Mixture-of-Dexterous-Experts VLAcited by 5
  13. 2021On Optimizing Interventions in Shared Autonomycited by 3
  14. 2024Shared Autonomy with IDA: Interventional Diffusion Assistancecited by 3
  15. 2026DiSCo: Diffusion Sequence Copilots for Shared Autonomycited by 3
  16. 2015Assistive Planning in Complex, Dynamic Environments: a Probabilistic Approachcited by 1
  17. 2025Casper: Inferring Diverse Intents for Assistive Teleoperation with Vision Language Modelscited by 1
  18. 2025Robot Trajectron V2: A Probabilistic Shared Control Framework for Navigationcited by 1
  19. 2026AURA: Multimodal Shared Autonomy for Real-World Urban Navigationcited by 1
  20. 2026Adaptor: Advancing Assistive Teleoperation with Few-Shot Learning and Cross-Operator Generalizationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning to Assist: Collaborative VLAs for Implicit Human-Robot Collaboration ↗
  2. 2026AC^2-VLA: Action-Context-Aware Adaptive Computation in Vision-Language-Action Models for Efficient Robotic Manipulation
  3. 2026VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Model
  4. 2026From Knowing to Doing Precisely: A General Self-Correction and Termination Framework for VLA models
  5. 2026VLA-REPLICA: A Low-Cost, Reproducible Benchmark for Real-World Evaluation of Vision-Language-Action Models
  6. 2026SEES: A Self-Evolving Embodied System via Failure-Guided VLA Policy Adaptation
  7. 2025Vision Language Action Models in Robotic Manipulation: A Systematic Review
  8. 2026TAO-DA: Towards Autonomous Operation--A Dual-Arm Vision-Language-Action Model for Coordinated Manipulation