mlsanityData to 14 Oct 2026

From Text to Trajectory: Exploring Complex Constraint Representation and Decomposition in Safe Reinforcement Learning

arXiv 2412.08920 ↗First posted 12 Dec 2024cs.CL cs.AIcited by 0

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Safe reinforcement learning (RL) requires the agent to finish a given task while obeying specific constraints. Giving constraints in natural language form has great potential for practical scenarios due to its flexible transfer capability and accessibility. Previous safe RL methods with natural language constraints typically need to design cost functions manually for each constraint, which requires domain expertise and lacks flexibility. In this paper, we harness the dual role of text in this task, using it not only to provide constraint but also as a training signal. We introduce the Trajectory-level Textual Constraints Translator (TTCT) to replace the manually designed cost function. Our empirical results demonstrate that TTCT effectively comprehends textual constraint and trajectory, and the policies trained by TTCT can achieve a lower violation rate than the standard cost function. Extra studies are conducted to demonstrate that the TTCT has zero-shot transfer capability to adapt to constraint-shift environments.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 29 of its 52 references that are in the map. First posted on arXiv 12 Dec 2024.

Builds on

29 of 52 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2020Language Models are Few-Shot Learnerscited by 32,419
  6. 2017Proximal Policy Optimization Algorithmscited by 15,215
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  8. 2017Constrained Policy Optimizationcited by 983
  9. 2020Meta-Learning in Neural Networks: A Surveycited by 950
  10. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  11. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 715
  12. 2020Keystroke Dynamics as Part of Lifeloggingcited by 334
17 more
  1. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  2. 2021ActionCLIP: A New Paradigm for Video Action Recognitioncited by 320
  3. 2020Responsive Safety in Reinforcement Learning by PID Lagrangian Methodscited by 316
  4. 2020Projection-Based Constrained Policy Optimizationcited by 203
  5. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  6. 2018Learning by Playing - Solving Sparse Reward Tasks from Scratchcited by 167
  7. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 166
  8. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 145
  9. 2023Safety-Gymnasium: A Unified Safe Reinforcement Learning Benchmarkcited by 143
  10. 2019Using Natural Language for Reward Shaping in Reinforcement Learningcited by 114
  11. 2016Model-Free Episodic Controlcited by 102
  12. 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
  13. 2022Constrained Update Projection Approach to Safe Policy Optimizationcited by 58
  14. 2019Sequence Modeling of Temporal Credit Assignment for Episodic Reinforcement Learningcited by 27
  15. 2017Beating Atari with Natural Language Guided Reinforcement Learningcited by 24
  16. 2020Safe Reinforcement Learning with Natural Language Constraintscited by 23
  17. 2024Safe Reinforcement Learning with Free-form Natural Language Constraints and Pre-Trained Language Modelscited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning Natural Language Constraints for Safe Reinforcement Learning of Language Agents
  2. 2024Safe Multi-agent Reinforcement Learning with Natural Language Constraints
  3. 2024A Survey of Constraint Formulations in Safe Reinforcement Learning
  4. 2024Towards Interpretable Reinforcement Learning with Constrained Normalizing Flow Policies
  5. 2024Safe CoR: A Dual-Expert Approach to Integrating Imitation Learning and Safe Reinforcement Learning Using Constraint Rewards
  6. 2025SafeRL-Lite: A Lightweight, Explainable, and Constrained Reinforcement Learning Library
  7. 2023Constrained Decision Transformer for Offline Safe Reinforcement Learning
  8. 2025DIAL: Distribution-Informed Adaptive Learning of Multi-Task Constraints for Safety-Critical Systems