mlsanityData to 14 Oct 2026

On-the-Fly VLA Adaptation via Test-Time Reinforcement Learning

arXiv 2601.06748 ↗First posted 11 Jan 2026cs.ROcited by 14

On the map: Vision-language-action models → Reinforcement learning and post-training

Abstract (arXiv)

Vision-Language-Action models have recently emerged as a powerful paradigm for general-purpose robot learning, enabling agents to map visual observations and natural-language instructions into executable robotic actions. Though popular, they are primarily trained via supervised fine-tuning or training-time reinforcement learning, requiring explicit fine-tuning phases, human interventions, or controlled data collection. Consequently, existing methods remain unsuitable for challenging simulated- or physical-world deployments, where robots must respond autonomously and flexibly to evolving environments. To address this limitation, we introduce a Test-Time Reinforcement Learning for VLAs (TT-VLA), a framework that enables on-the-fly policy adaptation during inference. TT-VLA formulates a dense reward mechanism that leverages step-by-step task-progress signals to refine action policies during test time while preserving the SFT/RL-trained priors, making it an effective supplement to current VLA models. Empirical results show that our approach enhances overall adaptability, stability, and task success in dynamic, previously unseen scenarios under simulated and real-world settings. We believe TT-VLA offers a principled step toward self-improving, deployment-ready VLAs.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models. Builds on 49 of its 73 references that are in the map. First posted on arXiv 11 Jan 2026.

Builds on

49 of 73 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025EVOLVE-VLA: Test-Time Training from Environment Feedback for Vision-Language-Action Models16 Dec
  2. 2025VLA-RFT: Vision-Language-Action Reinforcement Fine-tuning with Verified Rewards in World Simulators1 Oct
  3. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learning19 Sep
  4. 2025TGRPO :Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimization10 Jun
  5. 2025What Can RL Bring to VLA Generalization? An Empirical Study26 May
  6. 2025RFTF: Reinforcement Fine-tuning for Embodied Agents with Temporal Feedback26 May
  7. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning24 May
  8. 2025Interactive Post-Training for Vision-Language-Action Models22 May
  9. 2025ReinboT: Amplifying Robot Visual-Language Manipulation with Reinforcement Learning12 May
  10. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy8 Feb
  11. 2024GRAPE: Generalizing Robot Policy via Preference Alignment28 Nov
  12. 2024Octo: An Open-Source Generalist Robot Policy20 May

Other references · 37

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  7. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  8. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  9. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  10. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 665
  11. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 516
  12. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 407
25 more
  1. 2023An Embodied Generalist Agent in 3D Worldcited by 365
  2. 2024TraceVLA: Visual Trace Prompting Enhances Spatial-Temporal Awareness for Generalist Robotic Policiescited by 304
  3. 2020On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselinescited by 255
  4. 2024Reinforcement learningcited by 247
  5. 2024ReFT: Reasoning with Reinforced Fine-Tuningcited by 239
  6. 2024ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AIcited by 229
  7. 2019Skew-Fit: State-Covering Self-Supervised Reinforcement Learningcited by 174
  8. 2022Finetune like you pretrain: Improved finetuning of zero-shot vision modelscited by 150
  9. 2025VLA-Adapter: An Effective Paradigm for Tiny-Scale Vision-Language-Action Modelcited by 145
  10. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Datacited by 138
  11. 2025NORA: A Small Open-Sourced Generalist Vision Language Action Model for Embodied Taskscited by 127
  12. 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoningcited by 108
  13. 2024RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulationcited by 103
  14. 2024C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersioncited by 83
  15. 2024Policy Agnostic RL: Offline RL and Online RL Fine-Tuning of Any Class and Backbonecited by 79
  16. 2024Exploring the Adversarial Vulnerabilities of Vision-Language-Action Models in Roboticscited by 73
  17. 2025RoboMonkey: Scaling Test-Time Sampling and Verification for Vision-Language-Action Modelscited by 61
  18. 2021Online Adaptation to Label Distribution Shiftcited by 44
  19. 2025Don't Blind Your VLA: Aligning Visual Representations for OOD Generalizationcited by 35
  20. 2025DynaPrompt: Dynamic Test-Time Prompt Tuningcited by 31
  21. 2025dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thoughtcited by 30
  22. 2025Robot-R1: Reinforcement Learning for Enhanced Embodied Reasoning in Roboticscited by 21
  23. 2023CML-MOTS: Collaborative Multi-task Learning for Multi-Object Tracking and Segmentationcited by 9
  24. 2025Re-Imagining Multimodal Instruction Tuning: A Representation Viewcited by 9
  25. 2025When Test-Time Adaptation Meets Self-Supervised Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

By area:

  1. 2026RoboTTT: Context Scaling for Robot Policiescited by 19
  2. 2026Why AI systems don't learn and what to do about it: Lessons on autonomous learning from cognitive sciencecited by 11
  3. 2026WAM-TTT: Steering World-Action Models by Watching Human Play at Test Timecited by 8
  4. 2026Test-Time Training for Visual Foresight Vision-Language-Action Modelscited by 4
  5. 2026VANE: Reliable Test-Time Training for Vision-Language-Action Models via Future Visual Representation Predictioncited by 2
  6. 2026DreamAvoid: Critical-Phase Test-Time Dreaming to Avoid Failures in VLA Policiescited by 1
  7. 2026CounterAlign: Counterfactual Supervision for Vision-Language-Action Modelscited by 1
  8. 2026Adaptive Control in Autonomous Driving via Real-Time Recurrent RLcited by 0
  9. 2026One-to-Two Acting: A Novel Framework for Single-arm Agent Action Expansion to Dual Armscited by 0
  10. 2026ROAD-VLA: Robust Online Adaptation via Self-Distillation for Vision-Language-Action Modelscited by 0
  11. 2026Adapting Vision Foundation Models with Cascaded Semanticscited by 0
  12. 2026Unsupervised Post-Training of Foundation Models: A Surveycited by 0
2 more
  1. 2026Demonstration-Free Success-Probability Reward Learning for Generalist Robot Policiescited by 0
  2. 2026Taming VLAs under Robot Execution Errors: Self-Compensation and Stress Testingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Deploying VLA without Fine-Tuning: Plug-and-Play Inference-Time VLA Policy Steering via Embodied Evolutionary Diffusion
  2. 2026VLRLA: Deployment-Time Adaptation of Vision-Language-Action Models through In-Context Reinforcement Learning
  3. 2025Reflection-Based Task Adaptation for Self-Improving VLA
  4. 2026Agentic-VLA: Efficient Online Adaptation for Vision-Language-Action Models
  5. 2026Trust Your Instincts: Confidence-Driven Test-Time RL for Vision-Language-Action Models
  6. 2026RA-VLA: Retrieval-Augmented VLA for Test-Time Adaptation
  7. 2025π*0.6: a VLA That Learns From Experience
  8. 2026Self-Adaptive VLA for Robust Robot Deployment