mlsanityData to 14 Oct 2026

Learning Robot Policies from Sparse Success Signals via STL-Guided Stein Variational Policy Gradient

arXiv 2609.31606 ↗First posted 25 Sep 2026cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Learning robot policies for tasks with sparse success signals is challenging when completion depends on coordinated actions, precise contact outcomes, or satisfying several conditions together. Intricate physical interactions with the world further complicate these requirements. Prior work using conventional reward shaping mechanisms provides dense feedback but local progress might not translate into eventual task completion. We present Signal Temporal Logic-guided Stein Variational Policy Gradient (STL-SVPG), a population-based method that uses smooth STL robustness as a trajectory-level training objective. Differentiating this objective through the dynamics assigns credit to policy actions according to their effect on the complete task specification, rather than local progress alone. We evaluate the approach on six quadcopter and manipulator tasks that involves event-triggered responses, strictly ordered behavior, responses within specified deadlines, and physical interaction with the world. STL-SVPG achieves the highest mean success rate among the compared methods on five of six benchmarks. Simulation-trained policies trained in simulation transfer temporal and contact task behavior to the real world.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 21 of its 37 references that are in the map. First posted on arXiv 25 Sep 2026.

Builds on

21 of 37 references.

References · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  3. 2017Hindsight Experience Replaycited by 1,136
  4. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  5. 2023Eureka: Human-Level Reward Design via Coding Large Language Modelscited by 571
  6. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  7. 2016Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithmcited by 507
  8. 2020Reward Machines: Exploiting Reward Function Structure in Reinforcement Learningcited by 191
  9. 2016Reinforcement Learning With Temporal Logic Rewardscited by 117
  10. 2021LTL2Action: Generalizing LTL Instructions for Multi-Task RLcited by 74
  11. 2020Backpropagation through Signal Temporal Logic Specifications: Infusing Logical Structure into Gradient-Based Methodscited by 54
  12. 2017Stein Variational Policy Gradientcited by 44
9 more
  1. 2023Signal Temporal Logic Neural Predictive Controlcited by 13
  2. 2022Funnel-based Reward Shaping for Signal Temporal Logic Tasks in Reinforcement Learningcited by 8
  3. 2025STLCG++: A Masking Approach for Differentiable Signal Temporal Logic Specificationcited by 8
  4. 2025TGPO: Temporal Grounded Policy Optimization for Signal Temporal Logic Taskscited by 6
  5. 2020Disentangled Planning and Control in Vision Based Robotics via Reward Machinescited by 5
  6. 2019Prescribed Performance Control Guided Policy Improvement for Satisfying Signal Temporal Logic Taskscited by 4
  7. 2025Agile Flight Emerges from Multi-Agent Competitive Racingcited by 4
  8. 2026STL-SVPIO: Signal Temporal Logic guided Stein Variational Path Integral Optimizationcited by 2
  9. 2026Reinforcement Learning for Flow-Matching Policies with Density Transportcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Temporal Logic Guidance for Action-Only Diffusion Policies with World Models
  2. 2026From Pretraining to Proficiency: Real-World Subtask RL for Long-Horizon Manipulation with Minimal Human Intervention
  3. 2024Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learning
  4. 2026Demonstration-Free Success-Probability Reward Learning for Generalist Robot Policies
  5. 2025Zero-Shot Trajectory Planning for Signal Temporal Logic Tasks
  6. 2025Training-free Generation of Temporally Consistent Rewards from VLMs
  7. 2026DriftOPD: Sequence-Level Reverse-KL Distillation for One-Step VLA Policies
  8. 2026SmoothVLA: Aligning Vision-Language-Action Models with Physical Constraints via Intrinsic Smoothness Optimization