mlsanityData to 7 Oct 2026

Temporal Logic Specification-Conditioned Decision Transformer for Offline Safe Reinforcement Learning

arXiv 2402.17217 ↗First posted 27 Feb 2024cs.LG cs.AIcited by 10

On the map: Offline RL and imitation learning from data → Decision transformers and sequence modeling

Abstract (arXiv)

Offline safe reinforcement learning (RL) aims to train a constraint satisfaction policy from a fixed dataset. Current state-of-the-art approaches are based on supervised learning with a conditioned policy. However, these approaches fall short in real-world applications that involve complex tasks with rich temporal and logical structures. In this paper, we propose temporal logic Specification-conditioned Decision Transformer (SDT), a novel framework that harnesses the expressive power of signal temporal logic (STL) to specify complex temporal rules that an agent should follow and the sequential modeling capability of Decision Transformer (DT). Empirical evaluations on the DSRL benchmarks demonstrate the better capacity of SDT in learning safe and high-reward policies compared with existing approaches. In addition, SDT shows good alignment with respect to different desired degrees of satisfaction of the STL specification that it is conditioned on.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Decision transformers and sequence modeling, one of 13 directions in Offline RL and imitation learning from data. Builds on 53 of its 75 references that are in the map. First posted on arXiv 27 Feb 2024.

Builds on

53 of 75 references.

Closest ancestors · 6

Same direction, newest first

  1. 2023Critic-Guided Decision Transformer for Offline Reinforcement Learning21 Dec
  2. 2023Graph Decision Transformer7 Mar
  3. 2022Q-learning Decision Transformer: Leveraging Dynamic Programming for Conditional Sequence Modelling in Offline RL8 Sep
  4. 2022You Can't Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environments31 May
  5. 2021Generalized Decision Transformer for Offline Hindsight Information Matching19 Nov
  6. 2021Decision Transformer: Reinforcement Learning via Sequence Modeling2 Jun

Other references · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
  3. 2016Concrete Problems in AI Safetycited by 1,582
  4. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  5. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,227
  6. 2017Constrained Policy Optimizationcited by 928
  7. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 833
  8. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
  9. 2021Conditional Positional Encodings for Vision Transformerscited by 402
  10. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 337
  11. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 302
  12. 2019Batch Policy Learning under Constraintscited by 242
35 more
  1. 2021Offline Reinforcement Learning with Fisher Divergence Critic Regularizationcited by 237
  2. 2022A Survey on Offline Reinforcement Learning: Taxonomy, Review, and Open Problemscited by 228
  3. 2021RvS: What is Essential for Offline RL via Supervised Learning?cited by 184
  4. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  5. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 140
  6. 2020CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guaranteecited by 128
  7. 2021KAT: A Knowledge Augmented Transformer for Vision-and-Languagecited by 125
  8. 2019Reinforcement Learning Upside Down: Don't Predict Rewards -- Just Map Them to Actionscited by 124
  9. 2021OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimationcited by 121
  10. 2016Reinforcement Learning With Temporal Logic Rewardscited by 114
  11. 2019Reward-Conditioned Policiescited by 101
  12. 2021Constraints Penalized Q-learning for Safe Offline Reinforcement Learningcited by 90
  13. 2023OmniSafe: An Infrastructure for Accelerating Safe Reinforcement Learning Researchcited by 76
  14. 2019Control Synthesis from Linear Temporal Logic Specifications using Model-Free Reinforcement Learningcited by 74
  15. 2022When does return-conditioned supervised learning work for offline reinforcement learning?cited by 73
  16. 2022COptiDICE: Offline Constrained Reinforcement Learning via Stationary Distribution Correction Estimationcited by 69
  17. 2023Constrained Decision Transformer for Offline Safe Reinforcement Learningcited by 65
  18. 2019Explicit Sparse Transformer: Concentrated Attention Through Explicit Selectioncited by 61
  19. 2023Datasets and Benchmarks for Offline Safe Reinforcement Learningcited by 58
  20. 2015Robust Online Monitoring of Signal Temporal Logiccited by 39
  21. 2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
  22. 2020Model-based Reinforcement Learning from Signal Temporal Logic Specificationscited by 18
  23. 2022Towards Safe Mechanical Ventilation Treatment Using Deep Offline Reinforcement Learningcited by 16
  24. 2023SaFormer: A Conditional Sequence Modeling Approach to Offline Safe Reinforcement Learningcited by 16
  25. 2014On-Line Monitoring for Temporal Logic Robustnesscited by 15
  26. 2023Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstrationcited by 14
  27. 2018Metrics for Signal Temporal Logic Formulaecited by 9
  28. 2021Programmatic Reward Design by Examplecited by 9
  29. 2023A Primal-Dual-Critic Algorithm for Offline Constrained Reinforcement Learningcited by 9
  30. 2022Robust Counterexample-guided Optimization for Planning from Differentiable Temporal Logiccited by 8
  31. 2021Model-Based Reinforcement Learning for Approximate Optimal Control with Temporal Logic Specificationscited by 5
  32. 2022A Near-Optimal Primal-Dual Method for Off-Policy Learning in CMDPcited by 5
  33. 2023Modularized Control Synthesis for Complex Signal Temporal Logic Specificationscited by 4
  34. 2015Robust Linear Temporal Logiccited by 2
  35. 2021A More Scalable Mixed-Integer Encoding for Metric Temporal Logiccited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 7.

10 papers.

  1. 2024OASIS: Conditional Distribution Shaping for Offline Safe Reinforcement Learningcited by 10
  2. 2025TeLoGraF: Temporal Logic Planning via Graph-encoded Flow Matchingcited by 9
  3. 2025One Subgoal at a Time: Zero-Shot Generalization to Arbitrary Linear Temporal Logic Requirements in Multi-Task Reinforcement Learningcited by 8
  4. 2026STeP: Signal Temporal Logic for Precise Specifications for Action Generation with Vision Language Modelscited by 2
  5. 2026SpecRLBench: A Benchmark for Generalization in Specification-Guided Reinforcement Learningcited by 1
  6. 2026Conditional Sequence Modeling for Safe Reinforcement Learningcited by 0
  7. 2026MoMo: Conditioned Contrastive Representation Learning for Preference-Modulated Planningcited by 0
  8. 2026Neuro-Symbolic Injection of LTLf Constraints in Autoregressive Reinforcement Learning Policiescited by 0
  9. 2026Logic-VLA: A Temporal Logic Conditioned Vision-Language-Action Modelcited by 0
  10. 2026Jaxolotl: A Unified High-Performance Benchmark Suite for LTL-Based Multi-Task RLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026GraSP-STL: A Graph-Based Framework for Zero-Shot Signal Temporal Logic Planning via Offline Goal-Conditioned Reinforcement Learning
  2. 2024Goal-Conditioned Decision Transformer for Multi-Goal Offline Reinforcement Learning
  3. 2024Predictive Coding for Decision Transformer
  4. 2023Rethinking Decision Transformer via Hierarchical Reinforcement Learning
  5. 2024Return Augmented Decision Transformer for Off-Dynamics Reinforcement Learning
  6. 2024Solving Continual Offline Reinforcement Learning with Decision Transformer
  7. 2025Should We Ever Prefer Decision Transformer for Offline Reinforcement Learning?
  8. 2025A Comparison Between Decision Transformers and Traditional Offline Reinforcement Learning Algorithms