mlsanityData to 7 Oct 2026

LACY: A Vision-Language Model-based Language-Action Cycle for Self-Improving Robotic Manipulation

arXiv 2511.02239 ↗First posted 4 Nov 2025cs.RO cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

Learning generalizable policies for robotic manipulation increasingly relies on large-scale models that map language instructions to actions (L2A). However, this one-way paradigm often produces policies that execute tasks without deeper contextual understanding, limiting their ability to generalize or explain their behavior. We argue that the complementary skill of mapping actions back to language (A2L) is essential for developing more holistic grounding. An agent capable of both acting and explaining its actions can form richer internal representations and unlock new paradigms for self-supervised learning. We introduce LACY (Language-Action Cycle), a unified framework that learns such bidirectional mappings within a single vision-language model. LACY is jointly trained on three synergistic tasks: generating parameterized actions from language (L2A), explaining observed actions in language (A2L), and verifying semantic consistency between two language descriptions (L2C). This enables a self-improving cycle that autonomously generates and filters new training data through an active augmentation strategy targeting low-confidence cases, thereby improving the model without additional human labels. Experiments on pick-and-place tasks in both simulation and the real world show that LACY improves task success rates by 56.46% on average and yields more robust language-action grounding for robotic manipulation. Project page: https://vla2026.github.io/LACY/

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 43 references that are in the map. First posted on arXiv 4 Nov 2025.

Builds on

35 of 43 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success27 Feb
  2. 2024RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics15 Jun
  3. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control28 Jul

Other references · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,048
  7. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  8. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  9. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  10. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  11. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,490
  12. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
20 more
  1. 2018Exploration by Random Network Distillationcited by 966
  2. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 965
  3. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 745
  4. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 730
  5. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  6. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 634
  7. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  8. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 573
  9. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 545
  10. 2022BC-Z: Zero-Shot Task Generalization with Robotic Imitation Learningcited by 517
  11. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 453
  12. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  13. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 342
  14. 2023MimicGen: A Data Generation System for Scalable Robot Learning using Human Demonstrationscited by 308
  15. 2025HAMSTER: Hierarchical Action Models For Open-World Robot Manipulationcited by 125
  16. 2024RT-Affordance: Affordances are Versatile Intermediate Representations for Robot Manipulationcited by 40
  17. 2024ThinkGrasp: A Vision-Language System for Strategic Part Grasping in Cluttercited by 20
  18. 2022Interactive Robotic Grasping with Attribute-Guided Disambiguationcited by 19
  19. 2024QA-Calibration of Language Model Confidence Scorescited by 13
  20. 2024A Parameter-Efficient Tuning Framework for Language-guided Object Grounding and Robot Graspingcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks
  2. 2023Bridging Language and Action: A Survey of Language-Conditioned Robot Manipulation
  3. 2023Programmatically Grounded, Compositionally Generalizable Robotic Manipulation
  4. 2025From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation
  5. 2026AC^2-VLA: Action-Context-Aware Adaptive Computation in Vision-Language-Action Models for Efficient Robotic Manipulation
  6. 2024A Self-Correcting Vision-Language-Action Model for Fast and Slow System Manipulation
  7. 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Tasks
  8. 2025CodeDiffuser: Attention-Enhanced Diffusion Policy via VLM-Generated Code for Instruction Ambiguity