mlsanityData to 7 Oct 2026

Multi-Granularity Language-Guided Imitation Learning via Instruction Decomposition

arXiv 2609.37135 ↗First posted 29 Sep 2026cs.CVcited by 0

On the map: Offline RL and imitation learning from data → Language-conditioned policies and VLAs also fits: Robot manipulation imitation learning

Abstract (arXiv)

Using language instructions as conditions to guide robot policy learning has recently become an important research domain. However, existing language-guided policy learning methods typically use an overall task description to guide the entire demonstration trajectory. For manipulation tasks involving multiple execution stages, these methods assign the same language description to different subtasks, making it difficult to distinguish the behaviors required at different stages. In this work, we propose a multi-granularity language guidance method based on instruction decomposition. The proposed method decomposes an overall task description into more fine-grained, concrete subtask-level language instructions, thereby enhancing learning efficiency and improving performance. We evaluate the proposed method in the setting of multi-task imitation learning and validate its effectiveness.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Language-conditioned policies and VLAs, one of 13 directions in Offline RL and imitation learning from data. Builds on 23 of its 25 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

23 of 25 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024RACER: Rich Language-Guided Failure Recovery Policies for Imitation Learning23 Sep
  2. 2020Language Conditioned Imitation Learning over Unstructured Data15 May

Other references · 21

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  7. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusioncited by 2,730
  8. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,338
  9. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  10. 2021CLIPort: What and Where Pathways for Robotic Manipulationcited by 617
  11. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 564
  12. 2019Learning Latent Plans from Playcited by 310
9 more
  1. 2023Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisitioncited by 156
  2. 2024Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goalscited by 131
  3. 2024Hierarchical Diffusion Policy for Kinematics-Aware Multi-Task Robotic Manipulationcited by 92
  4. 2021Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulationcited by 85
  5. 2023PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Playcited by 48
  6. 2025RoboEval: Where Robotic Manipulation Meets Structured and Scalable Evaluationcited by 29
  7. 2024Policy Adaptation via Language Optimization: Decomposing Tasks for Few-Shot Imitationcited by 18
  8. 2024STEER: Flexible Robotic Manipulation via Dense Language Groundingcited by 13
  9. 2025Generalizable Coarse-to-Fine Robot Manipulation via Language-Aligned 3D Keypointscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022LISA: Learning Interpretable Skill Abstractions from Language
  2. 2020Language-Conditioned Imitation Learning for Robot Manipulation Tasks
  3. 2023Bridging Language and Action: A Survey of Language-Conditioned Robot Manipulation
  4. 2024Rethinking Mutual Information for Language Conditioned Skill Discovery on Imitation Learning
  5. 2026InstructMove: A Text-Indispensable Benchmark for Instruction-Following Manipulation
  6. 2026Language-Critique Imitation Learning from Suboptimal Demonstrations
  7. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Data
  8. 2020Learning to Generalize Across Long-Horizon Tasks from Human Demonstrations