# Multi-critic Learning for Whole-body End-effector Twist Tracking

arXiv: https://arxiv.org/abs/2507.08656 · HTML: https://mlsanity.com/paper/2507.08656/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jul 2025.

## Built on by (most cited first, 3 of 3)

- [Humanoid Whole-Body Badminton via an Annealed Reinforcement Learning Curriculum](https://mlsanity.com/paper/2511.11218/) (2025)
- [PUMA: Perception-driven Unified Foothold Prior for Mobility Augmented Quadruped Parkour](https://mlsanity.com/paper/2601.15995/) (2026)
- [Critic Architecture Matters: Dual vs. Unified Critics for Humanoid Loco-Manipulation](https://mlsanity.com/paper/2606.11891/) (2026)

## Abstract (arXiv)

Learning whole-body control for locomotion and arm motions in a single policy has challenges, as the two tasks have conflicting goals. For instance, efficient locomotion typically favors a horizontal base orientation, while end-effector tracking may benefit from base tilting to extend reachability. Additionally, current Reinforcement Learning (RL) approaches using a pose-based task specification lack the ability to directly control the end-effector velocity, making smoothly executing trajectories very challenging. To address these limitations, we propose an RL-based framework that allows for dynamic, velocity-aware whole-body end-effector control. Our method introduces a multi-critic actor architecture that decouples the reward signals for locomotion and manipulation, simplifying reward tuning and allowing the policy to resolve task conflicts more effectively. Furthermore, we design a twist-based end-effector task formulation that can track both discrete poses and motion trajectories. We validate our approach through a set of simulation and hardware experiments using a quadruped robot equipped with a robotic arm. The resulting controller can simultaneously walk and move its end-effector and shows emergent whole-body behaviors, where the base assists the arm in extending the workspace, despite a lack of explicit formulations. Videos and supplementary material can be found at multi-critic-locomanipulation.github.io.