# FootQuery: Future-Touchdown-Guided Retrieval from Depth History for Perceptive Humanoid Locomotion

arXiv: https://arxiv.org/abs/2609.21447 · HTML: https://mlsanity.com/paper/2609.21447/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 28 of its 30 references that are in the map.
- First posted on arXiv 18 Sep 2026.

## Builds on (most cited first, 28 of 28)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [Legged Locomotion in Challenging Terrains using Egocentric Vision](https://mlsanity.com/paper/2211.07638/) (2022)
- [Robot Parkour Learning](https://mlsanity.com/paper/2309.05665/) (2023)
- [Humanoid Parkour Learning](https://mlsanity.com/paper/2406.10759/) (2024)
- [Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers](https://mlsanity.com/paper/2107.03996/) (2021)
- [BeamDojo: Learning Agile Humanoid Locomotion on Sparse Footholds](https://mlsanity.com/paper/2502.10363/) (2025)
- [Neural Volumetric Memory for Visual Locomotion Control](https://mlsanity.com/paper/2304.01201/) (2023)
- [Learning Vision-Based Bipedal Locomotion for Challenging Terrain](https://mlsanity.com/paper/2309.14594/) (2023)
- [Attention-Based Map Encoding for Learning Generalized Legged Locomotion](https://mlsanity.com/paper/2506.09588/) (2025)
- [Perceptive Locomotion through Nonlinear Model Predictive Control](https://mlsanity.com/paper/2208.08373/) (2022)
- [Hiking in the Wild: A Scalable Perceptive Parkour Framework for Humanoids](https://mlsanity.com/paper/2601.07718/) (2026)
- [MoRE: Mixture of Residual Experts for Humanoid Lifelike Gaits Learning on Complex Terrains](https://mlsanity.com/paper/2506.08840/) (2025)
- [Learning Motion Skills with Adaptive Assistive Curriculum Force in Humanoid Robots](https://mlsanity.com/paper/2506.23125/) (2025)
- [START: Traversing Sparse Footholds with Terrain Reconstruction](https://mlsanity.com/paper/2512.13153/) (2025)
- [Perceptive Mixed-Integer Footstep Control for Underactuated Bipedal Walking on Rough Terrain](https://mlsanity.com/paper/2501.19391/) (2025)
- [KiVi: Kinesthetic-Visuospatial Integration for Dynamic and Safe Egocentric Legged Locomotion](https://mlsanity.com/paper/2509.23650/) (2025)
- [CReF: Cross-modal and Recurrent Fusion for Depth-conditioned Humanoid Locomotion](https://mlsanity.com/paper/2603.29452/) (2026)
- [GelFusion: Enhancing Robotic Manipulation under Visual Constraints via Visuotactile Fusion](https://mlsanity.com/paper/2505.07455/) (2025)
- [Look Forward to Walk Backward: Efficient Terrain Memory for Backward Locomotion with Forward Vision](https://mlsanity.com/paper/2603.03138/) (2026)
- [Learning Locomotion on Complex Terrain for Quadrupedal Robots with Foot Position Maps and Stability Rewards](https://mlsanity.com/paper/2604.02744/) (2026)
- [SSR: Scaling Surefooted and Symmetric Humanoid Traversal to the Open World](https://mlsanity.com/paper/2605.30770/) (2026)
- [Global-Local Attention Decomposition for Terrain Encoding in Humanoid Perceptive Locomotion](https://mlsanity.com/paper/2606.00637/) (2026)
- [DELTA: Deformable Elevation-Based Local Terrain Attention Encoder for Sparse-Terrain Quadrupedal Locomotion](https://mlsanity.com/paper/2608.22033/) (2026)
- [SOLO: Stable Omni-terrain Long-Horizon Perceptive Humanoid Locomotion](https://mlsanity.com/paper/2608.26583/) (2026)
- [PGMT: Perceptive General Motion Tracking for Humanoid Robots](https://mlsanity.com/paper/2609.08511/) (2026)

## Abstract (arXiv)

Humanoid locomotion over complex terrain requires anticipating footholds that may no longer be visible at touchdown. Limited camera coverage and self-occlusion make it necessary to retrieve relevant terrain information from earlier observations. We present FootQuery, a perceptive locomotion framework that queries depth history using each foot's predicted next touchdown. The policy predicts touchdown locations and uncertainty from proprioception and uses these distributions, together with per-foot features, to query sparsely sampled historical depth frames. During training, realized contacts are projected into historical images to supervise retrieval at the regions where those contacts were visible. The retrieved per-foot features are fused with global visual memory to generate control actions. A progressive force-assistance curriculum supports early exploration, while event-consistent tread-midline shaping encourages coordinated stair contacts. Deployment requires only proprioception and onboard depth images. In simulation, the complete framework outperforms its component ablations on the most challenging tested stairs, gaps, and platforms. Real-world experiments on a Unitree G1 demonstrate continuous traversal with a single policy across outdoor stairs and indoor routes combining stair ascent and descent, platforms, and gaps. These results support organizing visual history around anticipated contacts for perceptive humanoid locomotion.