mlsanityData to 14 Oct 2026

From LLM-Generated Specifications to Learned Quadruped Locomotion

arXiv 2609.07111 ↗First posted 7 Sep 2026cs.RO cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Quadruped robot locomotion policies are often trained using reinforcement learning, which in turn relies heavily on hand-crafted reward functions. Designing reward functions requires substantial manual engineering, and it is often unclear which local rewards will induce the desired global behavior. Shaped rewards from formal specifications in languages like Signal Temporal Logic (STL) can make rewards more interpretable, but writing STL specifications itself still requires domain expertise. We study whether large language models (LLMs) can fill this gap by generating Parametric Signal Temporal Logic (PSTL) specifications that are subsequently used for policy learning. Given a natural language locomotion objective and a constrained specification grammar, GPT-5.5 and Qwen 3.6 independently propose STL templates for command tracking, safety, and gait structure. We instantiate the parameters of the generated PSTL templates using expert trajectories and retain only specifications that are consistent with demonstrated expert behavior. The resulting specifications are then transformed into smooth, finite-history reward functions and used to train a quadruped locomotion policy with Proximal Policy Optimization (PPO) in MuJoCo XLA (MJX). We evaluate both gait-aware and gait-agnostic settings. The former specifies walking-trot, trot, and bound regimes, while the latter allows contact patterns to emerge from the task objective. We compare against hand-engineered rewards, Text2Reward-style LLM-generated reward code, and an expert-switching oracle. Gait-aware Qwen 3.6 specifications achieved 100% survival and command success across all tested speeds (0.3--2.1 m/s) and matched the target gait at high speeds, whereas Text2Reward achieved 0% for both metrics at ≥ 1.9 m/s. Videos: https://stl-locomotion.github.io/

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 46 references that are in the map. First posted on arXiv 7 Sep 2026.

Builds on

27 of 46 references.

References · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2017Deep reinforcement learning from human preferencescited by 3,912
  3. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  4. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  5. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  6. 2018Sim-to-Real: Learning Agile Locomotion For Quadruped Robotscited by 319
  7. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 298
  8. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 191
  9. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learningcited by 160
  10. 2016Reinforcement Learning With Temporal Logic Rewardscited by 117
  11. 2025MuJoCo Playgroundcited by 78
  12. 2021Minimizing Energy Consumption Leads to the Emergence of Gaits in Legged Robotscited by 76
15 more
  1. 2016Q-Learning for Robust Satisfaction of Signal Temporal Logic Specificationscited by 67
  2. 2023NL2TL: Transforming Natural Languages to Temporal Logics using Large Language Modelscited by 59
  3. 2021Fast and Efficient Locomotion via Learned Gait Transitionscited by 54
  4. 2019Dealing with Sparse Rewards in Reinforcement Learningcited by 42
  5. 2023Barkour: Benchmarking Animal-level Agility with Quadruped Robotscited by 41
  6. 2020A Smooth Robustness Measure of Signal Temporal Logic for Symbolic Controlcited by 21
  7. 2021Learning from Demonstrations using Signal Temporal Logiccited by 17
  8. 2024A Learning Framework for Diverse Legged Robot Locomotion Using Barrier-Based Style Rewardscited by 12
  9. 2023Imitating and Finetuning Model Predictive Control for Robust and Symmetric Quadrupedal Locomotioncited by 10
  10. 2024Robust-Locomotion-by-Logic: Perturbation-Resilient Bipedal Locomotion via Signal Temporal Logic Guided Model Predictive Controlcited by 8
  11. 2021Learning Quadruped Locomotion Policies using Logical Rulescited by 6
  12. 2023Walking-by-Logic: Signal Temporal Logic-Guided Model Predictive Control for Bipedal Locomotion Resilient to External Perturbationscited by 6
  13. 2024Learning to Adapt through Bio-Inspired Gait Strategies for Versatile Quadruped Locomotioncited by 6
  14. 2021FastMimic: Model-based Motion Imitation for Agile, Diverse and Generalizable Quadrupedal Locomotioncited by 3
  15. 2026Learning Gait-Aware Quadruped Locomotion with Temporal Logic Specificationscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Long-horizon Locomotion and Manipulation on a Quadrupedal Robot with Large Language Models
  2. 2026PROMO: Preference-conditioned Multi-Objective Reinforcement Learning for Quadrupedal Robots
  3. 2025Learning to Walk With Less: A Dyna-Style Approach to Quadrupedal Locomotion
  4. 2024Reinforcement Learning For Quadrupedal Locomotion: Current Advancements And Future Perspectives
  5. 2024Constrained Skill Discovery: Quadruped Locomotion with Unsupervised Reinforcement Learning
  6. 2023A Real-World Quadrupedal Locomotion Benchmark for Offline Reinforcement Learning
  7. 2020Elaborating on Learned Demonstrations with Temporal Logic Specifications
  8. 2025Learning Robotic Policy with Imagined Transition: Mitigating the Trade-off between Robustness and Optimality