# Value Iteration for Learning Concurrently Executable Robotic Control Tasks

arXiv: https://arxiv.org/abs/2504.01174 · HTML: https://mlsanity.com/paper/2504.01174/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 34 references that are in the map.
- First posted on arXiv 1 Apr 2025.

## Builds on (most cited first, 11 of 11)

- [Continuous control with deep reinforcement learning](https://mlsanity.com/paper/1509.02971/) (2015)
- [High-Dimensional Continuous Control Using Generalized Advantage Estimation](https://mlsanity.com/paper/1506.02438/) (2015)
- [A Practical Guide to Multi-Objective Reinforcement Learning and Planning](https://mlsanity.com/paper/2103.09568/) (2021)
- [A Survey of Multi-Objective Sequential Decision-Making](https://mlsanity.com/paper/1402.0590/) (2014)
- [MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policies](https://mlsanity.com/paper/1905.09808/) (2019)
- [Composable Deep Reinforcement Learning for Robotic Manipulation](https://mlsanity.com/paper/1803.06773/) (2018)
- [Value Iteration in Continuous Actions, States and Time](https://mlsanity.com/paper/2105.04682/) (2021)
- [A Set-Theoretic Approach to Multi-Task Execution and Prioritization](https://mlsanity.com/paper/2003.02968/) (2020)
- [Extended Set-based Tasks for Multi-task Execution and Prioritization](https://mlsanity.com/paper/2310.16189/) (2023)
- [Active Exploitation of Redundancies in Reconfigurable Multi-Robot Systems](https://mlsanity.com/paper/2106.12079/) (2021)
- [A Constrained-Optimization Approach to the Execution of Prioritized Stacks of Learned Multi-Robot Tasks](https://mlsanity.com/paper/2301.05346/) (2023)

## Abstract (arXiv)

Many modern robotic systems such as multi-robot systems and manipulators exhibit redundancy, a property owing to which they are capable of executing multiple tasks. This work proposes a novel method, based on the Reinforcement Learning (RL) paradigm, to train redundant robots to be able to execute multiple tasks concurrently. Our approach differs from typical multi-objective RL methods insofar as the learned tasks can be combined and executed in possibly time-varying prioritized stacks. We do so by first defining a notion of task independence between learned value functions. We then use our definition of task independence to propose a cost functional that encourages a policy, based on an approximated value function, to accomplish its control objective while minimally interfering with the execution of higher priority tasks. This allows us to train a set of control policies that can be executed simultaneously. We also introduce a version of fitted value iteration to learn to approximate our proposed cost functional efficiently. We demonstrate our approach on several scenarios and robotic systems.