LACY: A Vision-Language Model-based Language-Action Cycle for Self-Improving Robotic Manipulation
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
also in 1 more
Abstract (arXiv)
Learning generalizable policies for robotic manipulation increasingly relies on large-scale models that map language instructions to actions (L2A). However, this one-way paradigm often produces policies that execute tasks without deeper contextual understanding, limiting their ability to generalize or explain their behavior. We argue that the complementary skill of mapping actions back to language (A2L) is essential for developing more holistic grounding. An agent capable of both acting and explaining its actions can form richer internal representations and unlock new paradigms for self-supervised learning. We introduce LACY (Language-Action Cycle), a unified framework that learns such bidirectional mappings within a single vision-language model. LACY is jointly trained on three synergistic tasks: generating parameterized actions from language (L2A), explaining observed actions in language (A2L), and verifying semantic consistency between two language descriptions (L2C). This enables a self-improving cycle that autonomously generates and filters new training data through an active augmentation strategy targeting low-confidence cases, thereby improving the model without additional human labels. Experiments on pick-and-place tasks in both simulation and the real world show that LACY improves task success rates by 56.46% on average and yields more robust language-action grounding for robotic manipulation. Project page: https://vla2026.github.io/LACY/
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 43 references that are in the map. First posted on arXiv 4 Nov 2025.
Builds on
35 of 43 references.
Closest ancestors · 3
Same direction, newest first
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success27 Feb
- 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics15 Jun
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul
Other references · 32
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
20 more
- 2018Exploration by Random Network Distillationcited by 966
- 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
- 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
- 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
- 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
- 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
- 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
- 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
- 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
- 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
- 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
- 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
- 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
- 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
- 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 125
- 2024RT-Affordance: Affordances are Versatile Intermediate Representations for Robot Manipulationcited by 40
- 2024ThinkGrasp: A Vision-Language System for Strategic Part Grasping in Cluttercited by 20
- 2022Interactive Robotic Grasping with Attribute-Guided Disambiguationcited by 19
- 2024QA-Calibration of Language Model Confidence Scorescited by 13
- 2024A Parameter-Efficient Tuning Framework for Language-guided Object Grounding and Robot Graspingcited by 6
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks
- 2023Bridging Language and Action: A Survey of Language-Conditioned Robot Manipulation
- 2023Programmatically Grounded, Compositionally Generalizable Robotic Manipulation
- 2025From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation
- 2026AC^2-VLA: Action-Context-Aware Adaptive Computation in Vision-Language-Action Models for Efficient Robotic Manipulation
- 2024A Self-Correcting Vision-Language-Action Model for Fast and Slow System Manipulation
- 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks
- 2025CodeDiffuser: Attention-Enhanced Diffusion Policy via VLM-Generated Code for Instruction Ambiguity