mlsanityData to 7 Oct 2026

A Composable Specification Language for Reinforcement Learning Tasks

arXiv 2008.09293 ↗First posted 21 Aug 2020cs.LG cs.AI stat.MLcited by 68

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning is a promising approach for learning control policies for robot tasks. However, specifying complex tasks (e.g., with multiple objectives and safety constraints) can be challenging, since the user must design a reward function that encodes the entire task. Furthermore, the user often needs to manually shape the reward to ensure convergence of the learning algorithm. We propose a language for specifying complex control tasks, along with an algorithm that compiles specifications in our language into a reward function and automatically performs reward shaping. We implement our approach in a tool called SPECTRL, and show that it outperforms several state-of-the-art baselines.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 13; 2023: 9; 2024: 15; 2025: 14; 2026: 6.

68 papers.

Sort:

By area:

  1. 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 72
  2. 2021On the Expressivity of Markov Rewardcited by 71
  3. 2021Compositional Reinforcement Learning from Logical Specificationscited by 65
  4. 2021Multi-Agent Reinforcement Learning with Temporal Logic Specificationscited by 27
  5. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  6. 2021The Logical Options Frameworkcited by 24
  7. 2024DeepLTL: Learning to Efficiently Satisfy Complex LTL Specifications for Multi-Task RLcited by 20
  8. 2022Policy Optimization with Linear Temporal Logic Constraintscited by 19
  9. 2021On the (In)Tractability of Reinforcement Learning for LTL Objectivescited by 18
  10. 2024Compositional Automata Embeddings for Goal-Conditioned Reinforcement Learningcited by 18
  11. 2021In a Nutshell, the Human Asked for This: Latent Goals for Following Temporal Specificationscited by 17
  12. 2023Sample Efficient Model-free Reinforcement Learning from LTL Specifications with Optimality Guaranteescited by 17
38 more
  1. 2022GALOIS: Boosting Deep Reinforcement Learning via Generalizable Logic Synthesiscited by 16
  2. 2023Policy Synthesis and Reinforcement Learning for Discounted LTLcited by 14
  3. 2023Computably Continuous Reinforcement-Learning Objectives are PAC-learnablecited by 13
  4. 2023Compositional Policy Learning in Stochastic Control Systems with Formal Guaranteescited by 12
  5. 2022LCRL: Certified Policy Synthesis via Logically-Constrained Reinforcement Learningcited by 11
  6. 2024Reward Machines for Deep RL in Noisy and Uncertain Environmentscited by 11
  7. 2022Noisy Symbolic Abstractions for Deep RL: A case study with Reward Machinescited by 10
  8. 2024LTL-Constrained Policy Optimization with Cycle Experience Replaycited by 10
  9. 2022Specification-Guided Learning of Nash Equilibria with High Social Welfarecited by 8
  10. 2025One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learningcited by 8
  11. 2021Program Synthesis Guided Reinforcement Learning for Partially Observed Environmentscited by 7
  12. 2022Skill Machines: Temporal Logic Skill Composition in Reinforcement Learningcited by 7
  13. 2025Reinforcement Learning with Stochastic Reward Machinescited by 7
  14. 2022Model-Free Reinforcement Learning for Symbolic Automata-encoded Objectivescited by 6
  15. 2021Lifelong Reinforcement Learning with Temporal Logic Formulas and Reward Machinescited by 5
  16. 2023Reinforcement Learning with Knowledge Representation and Reasoning: A Brief Surveycited by 5
  17. 2024Adaptive Reward Design for Reinforcement Learningcited by 5
  18. 2025Ground-Compose-Reinforce: Grounding Language in Agentic Behaviours using Limited Datacited by 5
  19. 2025TGPO: Temporal Grounded Policy Optimization for Signal Temporal Logic Taskscited by 5
  20. 2022Hierarchies of Reward Machinescited by 4
  21. 2022RLang: A Declarative Language for Describing Partial World Knowledge to Reinforcement Learning Agentscited by 4
  22. 2025Reinforcement Learning with Physics-Informed Symbolic Program Priors for Zero-Shot Wireless Indoor Navigationcited by 4
  23. 2025Pushdown Reward Machines for Reinforcement Learningcited by 4
  24. 2022DistSPECTRL: Distributing Specifications in Multi-Agent Reinforcement Learning Systemscited by 3
  25. 2024Logical Specifications-guided Dynamic Task Sampling for Reinforcement Learning Agentscited by 3
  26. 2024Informativeness of Reward Functions in Reinforcement Learningcited by 3
  27. 2024Multi-Agent Reinforcement Learning with a Hierarchy of Reward Machinescited by 3
  28. 2024Inductive Generalization in Reinforcement Learning from Specificationscited by 3
  29. 2022Ablation Study of How Run Time Assurance Impacts the Training and Performance of Reinforcement Learning Agentscited by 2
  30. 2023What Kinds of Contracts Do ML APIs Need?cited by 2
  31. 2024Canopy: Property-Driven Learning for Congestion Controlcited by 2
  32. 2025HypRL: Reinforcement Learning of Control Policies for Hyperpropertiescited by 2
  33. 2026PlatoLTL: Scaling LTL-Guided Multi-Task RLcited by 2
  34. 2021Regular Decision Processes for Grid Worldscited by 1
  35. 2023Co-learning Planning and Control Policies Constrained by Differentiable Logic Specificationscited by 1
  36. 2023Counterfactuals Modulo Temporal Logicscited by 1
  37. 2024Numeric Reward Machinescited by 1
  38. 2024Generalization of Compositional Tasks with Logical Specification via Implicit Planningcited by 1

50 most cited of 68 All 68 (JSON)