# SCoCaT: Success Conditioned Constrained Reinforcement Learning for Spacecraft Docking

arXiv: https://arxiv.org/abs/2609.06061 · HTML: https://mlsanity.com/paper/2609.06061/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot navigation and UAV control, one of 15 directions in Deep reinforcement learning methods.
- Builds on 18 of its 30 references that are in the map.
- First posted on arXiv 5 Sep 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot navigation and UAV control](https://mlsanity.com/deeprl/rl-for-robot-navigation-and-uav-control/)

## Builds on (most cited first, 18 of 18)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Responsive Safety in Reinforcement Learning by PID Lagrangian Methods](https://mlsanity.com/paper/2007.03964/) (2020)
- [End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Tasks](https://mlsanity.com/paper/1903.08792/) (2019)
- [Defining and Characterizing Reward Hacking](https://mlsanity.com/paper/2209.13085/) (2022)
- [An empirical investigation of the challenges of real-world reinforcement learning](https://mlsanity.com/paper/2003.11881/) (2020)
- [Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentation](https://mlsanity.com/paper/2202.06558/) (2022)
- [skrl: Modular and Flexible Library for Reinforcement Learning](https://mlsanity.com/paper/2202.03825/) (2022)
- [Direct Behavior Specification via Constrained Reinforcement Learning](https://mlsanity.com/paper/2112.12228/) (2021)
- [CaT: Constraints as Terminations for Legged Locomotion Reinforcement Learning](https://mlsanity.com/paper/2403.18765/) (2024)
- [Guaranteed Safe Spacecraft Docking with Control Barrier Functions](https://mlsanity.com/paper/2109.07027/) (2021)
- [Conflict-Averse Gradient Aggregation for Constrained Multi-Objective Reinforcement Learning](https://mlsanity.com/paper/2403.00282/) (2024)
- [Dynamic object goal pushing with mobile manipulators through model-free constrained reinforcement learning](https://mlsanity.com/paper/2502.01546/) (2025)
- [Non-conflicting Energy Minimization in Reinforcement Learning based Robot Control](https://mlsanity.com/paper/2509.01765/) (2025)
- [RoboRAN: A Unified Robotics Framework for Reinforcement Learning-Based Autonomous Navigation](https://mlsanity.com/paper/2505.14526/) (2025)
- [Demonstrating Reinforcement Learning and Run Time Assurance for Spacecraft Inspection Using Unmanned Aerial Vehicles](https://mlsanity.com/paper/2405.06770/) (2024)
- [Meta-Reinforcement Learning for Robust and Non-greedy Control Barrier Functions in Spacecraft Proximity Operations](https://mlsanity.com/paper/2602.07335/) (2026)

## Abstract (arXiv)

Termination-based constrained reinforcement learning is attractive for safety-critical robotic deployments: it avoids online optimization at inference, scales easily to many constraints via a single scalar per constraint, and is simpler to implement than commonly used Lagrangian methods. Instead of pricing violations through summed cost penalties, this approach makes violations structurally unprofitable by shortening the effective horizon for each violation. We identify a structural failure mode of this method class on terminal-navigation tasks: reaching a precise goal configuration while satisfying safety constraints that tighten along the final approach. When the goal sits inside the region close to where the constraints become active, the survival-weighted objective makes dwelling outside the goal region strictly preferable to entering, producing high constraint compliance with low task completion. We formalize this pathology and show that a minimal augmentation to off-the-shelf RL algorithms like PPO resolves this ``feasibility collapse''. We empirically demonstrate that adding a dense per-step success signal via an auxiliary value critic improves the task completion rate while maintaining safety-critical constraint compliance. Validation across two representative spacecraft platforms: a 6U-CubeSat spanning the mass and degree-of-freedom envelope of operational proximity operations, and a floating platform testbed for zero-shot sim-to-real transfer in our laboratory, supports the generality of these findings.