# FootstepNet: an Efficient Actor-Critic Method for Fast On-line Bipedal Footstep Planning and Forecasting

arXiv: https://arxiv.org/abs/2403.12589 · HTML: https://mlsanity.com/paper/2403.12589/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 29 references that are in the map.
- First posted on arXiv 19 Mar 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 11 of 11)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.13653/) (2023)
- [DeepGait: Planning and Control of Quadrupedal Gaits using Deep Reinforcement Learning](https://mlsanity.com/paper/1909.08399/) (2019)
- [DeepQ Stepper: A framework for reactive dynamic walking on uneven terrain](https://mlsanity.com/paper/2010.14834/) (2020)
- [Bipedal Humanoid Hardware Design: A Technology Review](https://mlsanity.com/paper/2103.04675/) (2021)
- [Prioritized motion-force control of constrained fully-actuated robots: "Task Space Inverse Dynamics"](https://mlsanity.com/paper/1410.3863/) (2014)
- [Rhoban Football Club: RoboCup Humanoid Kid-Size 2023 Champion Team Paper](https://mlsanity.com/paper/2402.00612/) (2024)

## Built on by (most cited first, 4 of 4)

- [Deep Reinforcement Learning for Bipedal Locomotion: A Brief Survey](https://mlsanity.com/paper/2404.17070/) (2024)
- [Towards Safe Maneuvering of Double-Ackermann-Steering Robots with a Soft Actor-Critic Framework](https://mlsanity.com/paper/2510.10332/) (2025)
- [ManeuverNet: A Soft Actor-Critic Framework for Precise Maneuvering of Double-Ackermann-Steering Robots with Optimized Reward Functions](https://mlsanity.com/paper/2602.14726/) (2026)
- [DRL-Based Pose Control for Double-Ackermann Robots Under Actuation Uncertainties](https://mlsanity.com/paper/2606.00313/) (2026)

## Abstract (arXiv)

Designing a humanoid locomotion controller is challenging and classically split up in sub-problems. Footstep planning is one of those, where the sequence of footsteps is defined. Even in simpler environments, finding a minimal sequence, or even a feasible sequence, yields a complex optimization problem. In the literature, this problem is usually addressed by search-based algorithms (e.g. variants of A*). However, such approaches are either computationally expensive or rely on hand-crafted tuning of several parameters. In this work, at first, we propose an efficient footstep planning method to navigate in local environments with obstacles, based on state-of-the art Deep Reinforcement Learning (DRL) techniques, with very low computational requirements for on-line inference. Our approach is heuristic-free and relies on a continuous set of actions to generate feasible footsteps. In contrast, other methods necessitate the selection of a relevant discrete set of actions. Second, we propose a forecasting method, allowing to quickly estimate the number of footsteps required to reach different candidates of local targets. This approach relies on inherent computations made by the actor-critic DRL architecture. We demonstrate the validity of our approach with simulation results, and by a deployment on a kid-size humanoid robot during the RoboCup 2023 competition.