mlsanityData to 14 Oct 2026

ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Models

arXiv 2506.16211 ↗First posted 19 Jun 2025cs.ROcited by 43

On the map: Vision-language-action models → Reinforcement learning and post-training

Abstract (arXiv)

Learning real-world robotic manipulation is challenging, particularly when limited demonstrations are available. Existing methods for few-shot manipulation often rely on simulation-augmented data or pre-built modules like grasping and pose estimation, which struggle with sim-to-real gaps and lack extensibility. While large-scale imitation pre-training shows promise, adapting these general-purpose policies to specific tasks in data-scarce settings remains unexplored. To achieve this, we propose ControlVLA, a novel framework that bridges pre-trained VLA models with object-centric representations via a ControlNet-style architecture for efficient fine-tuning. Specifically, to introduce object-centric conditions without overwriting prior knowledge, ControlVLA zero-initializes a set of projection layers, allowing them to gradually adapt the pre-trained manipulation policies. In real-world experiments across 6 diverse tasks, including pouring cubes and folding clothes, our method achieves a 76.7% success rate while requiring only 10-20 demonstrations -- a significant improvement over traditional approaches that require more than 100 demonstrations to achieve comparable success. Additional experiments highlight ControlVLA's extensibility to long-horizon tasks and robustness to unseen objects and backgrounds.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning and post-training, one of 13 directions in Vision-language-action models. Builds on 58 of its 62 references that are in the map. First posted on arXiv 19 Jun 2025.

Builds on

58 of 62 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Octo: An Open-Source Generalist Robot Policy20 May

Other references · 57

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  6. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  7. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  8. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  9. 2024SAM 2: Segment Anything in Images and Videoscited by 2,863
  10. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,838
  11. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  12. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
38 more
  1. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  2. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  3. 2020ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAMcited by 878
  4. 2020Object-Centric Learning with Slot Attentioncited by 876
  5. 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
  6. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  7. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 694
  8. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 691
  9. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 614
  10. 2023Learning Interactive Real-World Simulatorscited by 458
  11. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
  12. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 395
  13. 2024Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robotscited by 394
  14. 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
  15. 2024Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperationcited by 391
  16. 2023An Embodied Generalist Agent in 3D Worldcited by 365
  17. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 327
  18. 2018Deep Object Pose Estimation for Semantic Robotic Grasping of Household Objectscited by 241
  19. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 238
  20. 2024MotionLCM: Real-time Controllable Motion Generation via Latent Consistency Modelcited by 168
  21. 2024ControlNet++: Improving Conditional Controls with Efficient Consistency Feedbackcited by 154
  22. 2024Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulationcited by 144
  23. 2023DisCo: Disentangled Control for Realistic Human Dance Generationcited by 142
  24. 2022VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priorscited by 136
  25. 2013Stabilitycited by 120
  26. 2024DexMimicGen: Automated Data Generation for Bimanual Dexterous Manipulation via Imitation Learningcited by 116
  27. 20226-DoF Pose Estimation of Household Objects for Robotic Manipulation: An Accessible Dataset and Benchmarkcited by 86
  28. 2024RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)cited by 69
  29. 2025ManipTrans: Efficient Dexterous Bimanual Manipulation Transfer via Residual Learningcited by 66
  30. 2023Learning Generalizable Manipulation Policies with Object-Centric 3D Representationscited by 61
  31. 2017Deep Object-Centric Representations for Generalizable Robot Learningcited by 50
  32. 2023An Investigation into Pre-Training Object-Centric Representations for Reinforcement Learningcited by 49
  33. 2018Deep Object-Centric Policies for Autonomous Drivingcited by 41
  34. 2024SPOT: SE(3) Pose Trajectory Diffusion for Object-Centric Manipulationcited by 38
  35. 2019Object-Centric Task and Motion Planning in Dynamic Environmentscited by 35
  36. 2024G3Flow: Generative 3D Semantic Flow for Pose-aware and Generalizable Object Manipulationcited by 35
  37. 2024DenseMatcher: Learning 3D Semantic Correspondence for Category-Level Manipulation from a Single Democited by 34
  38. 2024GenDP: 3D Semantic Fields for Category-Level Generalizable Diffusion Policycited by 25

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 17; 2026: 26.

43 papers.

Sort:

By area:

  1. 2025GWM: Towards Scalable Gaussian World Models for Robotic Manipulationcited by 39
  2. 2025A Survey on Efficient Vision-Language-Action Modelscited by 34
  3. 2025dVLA: Diffusion Vision-Language-Action Model with Multimodal Chain-of-Thoughtcited by 30
  4. 2025ActiveUMI: Robotic Manipulation with Active Perception from Robot-Free Human Demonstrationscited by 19
  5. 2025Can World Simulators Reason? Gen-ViRe: A Generative Visual Reasoning Benchmarkcited by 19
  6. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  7. 2026MaskWAM: Unifying Mask Prompting and Prediction for World-Action Modelscited by 16
  8. 2025Mechanistic Finetuning of Vision-Language-Action Models via Few-Shot Demonstrationscited by 15
  9. 2026TaF-VLA: Tactile-Force Alignment in Vision-Language-Action Models for Force-aware Manipulationcited by 13
  10. 2025PEEK: Guiding and Minimal Image Representations for Zero-Shot Generalization of Robot Manipulation Policiescited by 12
  11. 2025SlotVLA: Towards Modeling of Object-Relation Representations in Robotic Manipulationcited by 9
  12. 2025Mind to Hand: Purposeful Robotic Control via Embodied Reasoningcited by 8
31 more
  1. 2026OFlow: Injecting Object-Aware Temporal Flow Matching for Robust Robotic Manipulationcited by 8
  2. 2026Gaze2Act: Gaze-Conditioned Vision-Language-Action Policies for Interactive Robot Manipulationcited by 8
  3. 2025Survey of Vision-Language-Action Models for Embodied Manipulationcited by 7
  4. 2025RoDyn: Taming Interactive Robot-Dynamic 2.5D World Model for Robotic Manipulationcited by 7
  5. 2026X-Tokenizer: A Multimodal Action Tokenizer for Vision-Language-Action Pretrainingcited by 7
  6. 2026Breaking Lock-In: Preserving Steerability under Low-Data VLA Post-Trainingcited by 6
  7. 2025VLA^2: Empowering Vision-Language-Action Models with an Agentic Framework for Unseen Concept Manipulationcited by 4
  8. 2025PosA-VLA: Enhancing Action Generation via Pose-Conditioned Anchor Attentioncited by 4
  9. 2026Embodied3DBench: Benchmarking Low-Level Embodied Spatial Intelligence of Vision Language Modelscited by 4
  10. 2025GraSP-VLA: Graph-based Symbolic Action Representation for Long-Horizon Planning with VLA Policiescited by 3
  11. 2026ST-VLA: Enabling 4D-Aware Spatiotemporal Understanding for General Robot Manipulationcited by 3
  12. 2026TMRL: Diffusion Timestep-Modulated Pretraining Enables Exploration for Efficient Policy Finetuningcited by 3
  13. 2026Towards Predictive, Aligned, and Scalable Robot Learningcited by 3
  14. 2026StructSAM: Structure- and Spectrum-Preserving Token Merging for Segment Anything Modelscited by 2
  15. 2026ELAN4D: Embodiment-Centric 4D Supervision for Vision-Language-Action Models via Plug-and-Play Adaptationcited by 2
  16. 2026FOCA: Future-Oriented Conditioning for Data-Efficient Vision-Language-Action Adaptationcited by 2
  17. 2026SkillMemo: Expert-guided Skill Memory Framework for Compositional Embodied Manipulationcited by 2
  18. 2025FT-NCFM: An Influence-Aware Data Distillation Framework for Efficient VLA Modelscited by 1
  19. 2026UniManip: General-Purpose Zero-Shot Robotic Manipulation with Agentic Operational Graphcited by 1
  20. 2026PVI: Plug-in Visual Injection for Vision-Language-Action Modelscited by 1
  21. 2026CoEnv: Driving Embodied Multi-Agent Collaboration via Compositional Environmentcited by 1
  22. 2025GLUE: Global-Local Unified Encoding for Imitation Learning via Key-Patch Trackingcited by 0
  23. 2026UniDexTok: A Unified Dexterous Hand Tokenizer from Real Datacited by 0
  24. 2026Explicit Kinematic Guidance from Analytic Concepts for Vision-Language-Action Modelscited by 0
  25. 2026Hierarchical Skill Retrieval for Data-Efficient Adaptation of Vision-Language-Action Modelscited by 0
  26. 2026DeicticVLA: Unifying Instruction Modes Based on Language and Deictic Gestures in a Single VLAcited by 0
  27. 2026MaskHarness-WAM: Instance-Grounded Harnessing for Long-Horizon Robot Manipulationcited by 0
  28. 2026Catch Me If You Can: Real-Time Feedback Denoising for Responsive VLAscited by 0
  29. 2026Object-Centric Conditioning for Visuomotor Flow Matchingcited by 0
  30. 2026Enabling a Unified Cross-Domain Representation for Two-Finger Gripper Manipulation via Interaction-Centric Modelingcited by 0
  31. 2026Correcting WHERE, Preserving HOW: Compositional Generalization for Vision-Language-Action Models via Referential Guidancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ObjectVLA: End-to-End Open-World Object Manipulation Without Demonstration
  2. 2026PriorVLA: Prior-Preserving Adaptation for Vision-Language-Action Models
  3. 2026Simple-to-Complex Structured Demonstrations for Vision-Language-Action Learning
  4. 2025See Once, Then Act: Vision-Language-Action Model with Task Learning from One-Shot Video Demonstrations
  5. 2025Goal-VLA: Image-Generative VLMs as Object-Centric World Models Empowering Zero-shot Robot Manipulation
  6. 2025MoS-VLA: A Vision-Language-Action Model with One-Shot Skill Adaptation
  7. 2026Veo-Act: Enhancing VLA Policies with Frontier Video Models
  8. 2026SG-VLA: Learning Spatially-Grounded Vision-Language-Action Models for Mobile Manipulation