mlsanityData to 7 Oct 2026

SCALAR: Learning and Composing Skills through LLM Guided Symbolic Planning and Deep RL Grounding

arXiv 2603.09036 ↗First posted 10 Mar 2026cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

LM-based agents excel when given high-level action APIs but struggle to ground language into low-level control. Prior work has LLMs generate skills or reward functions for RL, but these one-shot approaches lack feedback to correct specification errors. We introduce SCALAR, a bidirectional framework coupling LLM planning with RL through a learned skill library. The LLM proposes skills with preconditions and effects; RL trains policies for each skill and feeds back execution results to iteratively refine specifications, improving robustness to initial errors. Pivotal Trajectory Analysis corrects LLM priors by analyzing RL trajectories; Frontier Checkpointing optionally saves environment states at skill boundaries to improve sample efficiency. On Craftax, SCALAR achieves 88.2% diamond collection, a 1.9x improvement over the best baseline, and reaches the Gnomish Mines 9.1% of the time where prior methods fail entirely.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 42 of its 65 references that are in the map. First posted on arXiv 10 Mar 2026.

Builds on

42 of 65 references.

References · 42

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  6. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  7. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 542
  8. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  9. 2016The Option-Critic Architecturecited by 433
  10. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  11. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  12. 2016Variational Intrinsic Controlcited by 262
30 more
  1. 2017Inverse Reward Designcited by 246
  2. 2019Go-Explore: a New Approach for Hard-Exploration Problemscited by 207
  3. 2020The NetHack Learning Environmentcited by 196
  4. 2021Benchmarking the Spectrum of Agent Capabilitiescited by 193
  5. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 153
  6. 2021APS: Active Pretraining with Successor Featurescited by 109
  7. 2019Reward Tampering Problems and Solutions in Reinforcement Learning: A Causal Influence Diagram Perspectivecited by 107
  8. 2023Motif: Intrinsic Motivation from Artificial Intelligence Feedbackcited by 99
  9. 2023METRA: Scalable Unsupervised RL with Metric-Aware Abstractioncited by 80
  10. 2024Craftax: A Lightning-Fast Benchmark for Open-Ended Reinforcement Learningcited by 80
  11. 2018SDRL: Interpretable and Data-efficient Deep Reinforcement Learning Leveraging Symbolic Planningcited by 77
  12. 2024Simplifying Deep Temporal Difference Learningcited by 71
  13. 2019Wasserstein Dependency Measure for Representation Learningcited by 69
  14. 2022Lipschitz-constrained Unsupervised Skill Discoverycited by 60
  15. 2018PEORL: Integrating Symbolic Planning and Hierarchical Reinforcement Learning for Robust Decision-Makingcited by 51
  16. 2023Controllability-Aware Unsupervised Skill Discoverycited by 49
  17. 2023Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manualscited by 45
  18. 2020Avoiding Side Effects By Considering Future Taskscited by 32
  19. 2024RL-GPT: Integrating Reinforcement Learning and Code-as-policycited by 32
  20. 2024A Large Language Model-Driven Reward Design Framework via Dynamic Feedback for Reinforcement Learningcited by 29
  21. 2024Skill Set Optimization: Reinforcing Language Model Behavior via Transferable Skillscited by 24
  22. 2024MaestroMotif: Skill Design from Artificial Intelligence Feedbackcited by 24
  23. 2021Wasserstein Unsupervised Reinforcement Learningcited by 21
  24. 2024LARM: Large Auto-Regressive Model for Long-Horizon Embodied Intelligencecited by 10
  25. 2024World Models with Hints of Large Language Models for Goal Achievingcited by 6
  26. 2023Creating Multi-Level Skill Hierarchies in Reinforcement Learningcited by 5
  27. 2025Scalable Option Learning in High-Throughput Environmentscited by 5
  28. 2024Reward Guidance for Reinforcement Learning Tasks Based on Large Language Models: The LMGT Frameworkcited by 4
  29. 2025Option Discovery Using LLM-guided Semantic Hierarchical Reinforcement Learningcited by 4
  30. 2026Language Movement Primitives: Grounding Language Models in Robot Motioncited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Co-Evolving LLM Decision and Skill Bank Agents for Long-Horizon Taskscited by 13
  2. 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Librariescited by 6
  3. 2026From Cognitive Architectures to Language Agents: A Mechanism-Level Review of Lineage, Convergence, and Migration Gapscited by 1
  4. 2026Towards a Systems Foundation for Agentic Skills: Architecture, Lifecycle, and Securitycited by 0