# Towards Deploying VLA without Fine-Tuning: Plug-and-Play Inference-Time VLA Policy Steering via Embodied Evolutionary Diffusion

arXiv: https://arxiv.org/abs/2511.14178 · HTML: https://mlsanity.com/paper/2511.14178/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models.
- Builds on 26 of its 33 references that are in the map.
- First posted on arXiv 18 Nov 2025.

## Placed in

- Vision-language-action models / [Reinforcement learning and post-training](https://mlsanity.com/vla/reinforcement-learning-and-post-training/)

## Builds on (most cited first, 26 of 26)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [Inner Monologue: Embodied Reasoning through Planning with Language Models](https://mlsanity.com/paper/2207.05608/) (2022)
- [Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success](https://mlsanity.com/paper/2502.19645/) (2025)
- [RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation](https://mlsanity.com/paper/2410.07864/) (2024)
- [Eureka: Human-Level Reward Design via Coding Large Language Models](https://mlsanity.com/paper/2310.12931/) (2023)
- [RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation](https://mlsanity.com/paper/2506.18088/) (2025)
- [Robotic Control via Embodied Chain-of-Thought Reasoning](https://mlsanity.com/paper/2407.08693/) (2024)
- [ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation](https://mlsanity.com/paper/2409.01652/) (2024)
- [ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI](https://mlsanity.com/paper/2410.00425/) (2024)
- [Text2Motion: From Natural Language Instructions to Feasible Plans](https://mlsanity.com/paper/2303.12153/) (2023)
- [Foundation Models in Robotics: Applications, Challenges, and the Future](https://mlsanity.com/paper/2312.07843/) (2023)
- [Steering Your Diffusion Policy with Latent Space Reinforcement Learning](https://mlsanity.com/paper/2506.15799/) (2025)
- [GRAPE: Generalizing Robot Policy via Preference Alignment](https://mlsanity.com/paper/2411.19309/) (2024)
- [Steering Your Generalists: Improving Robotic Foundation Models via Value Guidance](https://mlsanity.com/paper/2410.13816/) (2024)
- [DINOv2 Meets Text: A Unified Framework for Image- and Pixel-Level Vision-Language Alignment](https://mlsanity.com/paper/2412.16334/) (2024)
- [RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Models](https://mlsanity.com/paper/2506.17811/) (2025)
- [Hume: Introducing System-2 Thinking in Visual-Language-Action Model](https://mlsanity.com/paper/2505.21432/) (2025)
- [From Foresight to Forethought: VLM-In-the-Loop Policy Steering via Latent Alignment](https://mlsanity.com/paper/2502.01828/) (2025)
- [Inference-Time Policy Steering through Human Interactions](https://mlsanity.com/paper/2411.16627/) (2024)
- [RoVer: Robot Reward Model as Test-Time Verifier for Vision-Language-Action Model](https://mlsanity.com/paper/2510.10975/) (2025)
- [Diffusion Models are Evolutionary Algorithms](https://mlsanity.com/paper/2410.02543/) (2024)
- [ManiDP: Manipulability-Aware Diffusion Policy for Posture-Dependent Bimanual Manipulation](https://mlsanity.com/paper/2510.23016/) (2025)
- [Extremum Flow Matching for Offline Goal Conditioned Reinforcement Learning](https://mlsanity.com/paper/2505.19717/) (2025)
- [Human-Humanoid Robots Cross-Embodiment Behavior-Skill Transfer Using Decomposed Adversarial Learning from Demonstration](https://mlsanity.com/paper/2412.15166/) (2024)

## Built on by (most cited first, 8 of 8)

- [VLS: Steering Pretrained Robot Policies via Vision-Language Models](https://mlsanity.com/paper/2602.03973/) (2026)
- [Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Training](https://mlsanity.com/paper/2604.23121/) (2026)
- [Look Before You Leap: Distilling Tree Search into Action Evaluation for Frozen VLA Models](https://mlsanity.com/paper/2607.03751/) (2026)
- [Self-Improving VLA Policies: Selected Diffusion Noise for Spurious-Robust Action Smoothing](https://mlsanity.com/paper/2606.14084/) (2026)
- [Pose6DAug: Physically Plausible Multi-view Object Swapping for Robot Data Augmentation](https://mlsanity.com/paper/2606.20118/) (2026)
- [Trust Your Instincts: Confidence-Driven Test-Time RL for Vision-Language-Action Models](https://mlsanity.com/paper/2606.29892/) (2026)
- [A Causality-aware Infer-diagnose-refine Framework for Test-time Modality Adaptation in VLA Models](https://mlsanity.com/paper/2607.25516/) (2026)
- [RL2-VLA: Adaptive RL Latent Compositional Steering with Test-Time Scaling for Vision-Language-Action Models](https://mlsanity.com/paper/2607.26991/) (2026)

## Abstract (arXiv)

Vision-Language-Action (VLA) models have demonstrated significant potential in real-world robotic manipulation. However, pre-trained VLA policies still suffer from substantial performance degradation during downstream deployment. Although fine-tuning can mitigate this issue, its reliance on costly demonstration collection and intensive computation makes it impractical in real-world settings. In this work, we introduce VLA-Pilot, a plug-and-play inference-time policy steering method for zero-shot deployment of pre-trained VLA without any additional fine-tuning or data collection. We evaluate VLA-Pilot on six real-world downstream manipulation tasks across two distinct robotic embodiments, encompassing both in-distribution and out-of-distribution scenarios. Experimental results demonstrate that VLA-Pilot substantially boosts the success rates of off-the-shelf pre-trained VLA policies, enabling robust zero-shot generalization to diverse tasks and embodiments. Experimental videos and code are available at: https://rip4kobe.github.io/vla-pilot/.