mlsanityData to 7 Oct 2026

SEAL: SEmantic-Augmented Imitation Learning via Language Model

arXiv 2410.02231 ↗First posted 3 Oct 2024cs.AI cs.LG cs.SYcited by 1

On the map: Offline RL and imitation learning from data → Language-conditioned policies and VLAs

Abstract (arXiv)

Hierarchical Imitation Learning (HIL) is a promising approach for tackling long-horizon decision-making tasks. While it is a challenging task due to the lack of detailed supervisory labels for sub-goal learning, and reliance on hundreds to thousands of expert demonstrations. In this work, we introduce SEAL, a novel framework that leverages Large Language Models (LLMs)'s powerful semantic and world knowledge for both specifying sub-goal space and pre-labeling states to semantically meaningful sub-goal representations without prior knowledge of task hierarchies. SEAL employs a dual-encoder structure, combining supervised LLM-guided sub-goal learning with unsupervised Vector Quantization (VQ) for more robust sub-goal representations. Additionally, SEAL incorporates a transition-augmented low-level planner for improved adaptation to sub-goal transitions. Our experiments demonstrate that SEAL outperforms state-of-the-art HIL methods and LLM-based planning approaches, particularly in settings with small expert datasets and complex long-horizon tasks.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Language-conditioned policies and VLAs, one of 13 directions in Offline RL and imitation learning from data. Builds on 55 of its 70 references that are in the map. First posted on arXiv 3 Oct 2024.

Builds on

55 of 70 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Rethinking Mutual Information for Language Conditioned Skill Discovery on Imitation Learning27 Feb
  2. 2022What Matters in Language Conditioned Robotic Imitation Learning over Unstructured Data13 Apr
  3. 2022LISA: Learning Interpretable Skill Abstractions from Language28 Feb
  4. 2020Language-Conditioned Imitation Learning for Robot Manipulation Tasks22 Oct
  5. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigation25 Nov

Other references · 50

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2017Neural Discrete Representation Learningcited by 4,650
  5. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  7. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,302
  8. 2016Generative Adversarial Imitation Learningcited by 1,576
  9. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  10. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,029
  11. 2023VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Modelscited by 733
  12. 2022ProgPrompt: Generating Situated Robot Task Plans using Large Language Modelscited by 598
38 more
  1. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  2. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 459
  3. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  4. 2023Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agentscited by 396
  5. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 394
  6. 2023On the Planning Abilities of Large Language Models : A Critical Investigationcited by 308
  7. 2023Reward Design with Language Modelscited by 257
  8. 2018BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learningcited by 238
  9. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  10. 2023Guiding Pretraining in Reinforcement Learning with Large Language Modelscited by 185
  11. 2024LLMs Can't Plan, But Can Help Planning in LLM-Modulo Frameworkscited by 178
  12. 2019Goal-conditioned Imitation Learningcited by 159
  13. 2023Translating Natural Language to Planning Goals with Large-Language Modelscited by 132
  14. 2018Hierarchical Imitation and Reinforcement Learningcited by 88
  15. 2019Hierarchical Foresight: Self-Supervised Learning of Long-Horizon Tasks via Visual Subgoal Generationcited by 83
  16. 2020Towards Playing Full MOBA Games with Deep Reinforcement Learningcited by 78
  17. 2024Determinants of LLM-assisted Decision-Makingcited by 55
  18. 2021Hierarchical Task Learning from Language Instructions with Unified Transformers and Self-Monitoringcited by 53
  19. 2021Landmark-Guided Subgoal Generation in Hierarchical Reinforcement Learningcited by 46
  20. 2023Thought Cloning: Learning to Think while Acting by Imitating Human Thinkingcited by 30
  21. 2023Enabling Intelligent Interactions between an Agent and an LLM: A Reinforcement Learning Approachcited by 28
  22. 2018A Large-Scale Study on Regularization and Normalization in GANscited by 26
  23. 2024FuRL: Visual-Language Models as Fuzzy Rewards for Reinforcement Learningcited by 26
  24. 2020A Long Horizon Planning Framework for Manipulating Rigid Pointcloud Objectscited by 22
  25. 2022Learning Options via Compressioncited by 22
  26. 2013Hierarchical Solution of Markov Decision Processes using Macro-actionscited by 19
  27. 2021Adversarial Option-Aware Hierarchical Imitation Learningcited by 19
  28. 2021Computational Benefits of Intermediate Rewards for Goal-Reaching Policy Learningcited by 17
  29. 2024Zero-shot Imitation Policy via Search in Demonstration Datasetcited by 13
  30. 2023Cook2LTL: Translating Cooking Recipes to LTL Formulae using Large Language Modelscited by 12
  31. 2023LLM Augmented Hierarchical Agentscited by 9
  32. 2023Improving Long-Horizon Imitation Through Instruction Predictioncited by 8
  33. 2023GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimationcited by 6
  34. 2021Interactive Hierarchical Guidance using Languagecited by 5
  35. 2023Skill Disentanglement for Imitation Learning from Suboptimal Demonstrationscited by 5
  36. 2023Hierarchical reinforcement learning with natural language subgoalscited by 3
  37. 2018Human-Interactive Subgoal Supervision for Efficient Inverse Reinforcement Learningcited by 2
  38. 2021Sample Efficient Imitation Learning via Reward Function Trained in Advancecited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025FOSSIL: Harnessing Feedback on Suboptimal Samples for Data-Efficient Generalisation with Imitation Learning for Embodied Vision-and-Language Taskscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025From Code to Action: Hierarchical Learning of Diffusion-VLM Policies
  2. 2024Sub-goal Distillation: A Method to Improve Small Language Agents
  3. 2026Dual-Process Atomic Skill Learning: Decoupling Semantic Reasoning and Real-Time Control
  4. 2023Hierarchical Imitation Learning with Vector Quantized Models
  5. 2026Language-Critique Imitation Learning from Suboptimal Demonstrations
  6. 2026Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAs
  7. 2024Policy Adaptation via Language Optimization: Decomposing Tasks for Few-Shot Imitation
  8. 2024Learning with Language-Guided State Abstractions