# Bellman Value Decomposition for Task Logic in Safe Optimal Control

arXiv: https://arxiv.org/abs/2602.19532 · HTML: https://mlsanity.com/paper/2602.19532/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Feb 2026.

## Built on by (most cited first, 4 of 4)

- [Value Functions for Temporal Logic: Optimal Policies and Safety Filters](https://mlsanity.com/paper/2605.01051/) (2026)
- [Towards General Language-Conditioned Latent Safety Filters](https://mlsanity.com/paper/2608.00315/) (2026)
- [Reachability-Preserving Bellman Operator for the Discounted Reach-Cost Value Function: Uniting Hamilton-Jacobi Reachability and Reinforcement Learning](https://mlsanity.com/paper/2607.07893/) (2026)
- [ResSafe: Learning Safety Filtering with Residual Reinforcement Learning for Humanoids](https://mlsanity.com/paper/2609.15988/) (2026)

## Abstract (arXiv)

Real-world tasks involve nuanced combinations of goal and safety specifications. In high dimensions, the challenge is exacerbated: formal automata become cumbersome, and the combination of sparse rewards tends to require laborious tuning. In this work, we consider the innate structure of the Bellman Value as a means to naturally organize the problem for improved automatic performance. Namely, we prove the Bellman Value for a complex task defined in temporal logic can be decomposed into a graph of Bellman Values, connected by a set of well-known Bellman equations (BEs): the Reach-Avoid BE, the Avoid BE, and a novel type, the Reach-Avoid-Loop BE. To solve the Value and optimal policy, we propose VDPPO, which embeds the decomposed Value graph into a two-layer neural net, bootstrapping the implicit dependencies. We conduct a variety of simulated and hardware experiments to test our method on complex, high-dimensional tasks involving heterogeneous teams and nonlinear dynamics. Ultimately, we find this approach greatly improves performance over existing baselines, balancing safety and liveness automatically.