# Loco-Loco-RL: Low-Cost Terrain Mapping for Humanoid Locomotion with Reinforcement Learning

arXiv: https://arxiv.org/abs/2609.19041 · HTML: https://mlsanity.com/paper/2609.19041/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 22 references that are in the map.
- First posted on arXiv 21 Jul 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [Stabilizing Transformers for Reinforcement Learning](https://mlsanity.com/paper/1910.06764/) (2019)
- [Real-World Humanoid Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2303.03381/) (2023)
- [Humanoid Parkour Learning](https://mlsanity.com/paper/2406.10759/) (2024)
- [DreamWaQ: Learning Robust Quadrupedal Locomotion With Implicit Terrain Imagination via Deep Reinforcement Learning](https://mlsanity.com/paper/2301.10602/) (2023)
- [Adapting Rapid Motor Adaptation for Bipedal Robots](https://mlsanity.com/paper/2205.15299/) (2022)
- [Learning Humanoid Locomotion over Challenging Terrain](https://mlsanity.com/paper/2410.03654/) (2024)
- [DPL: Depth-only Perceptive Humanoid Locomotion via Realistic Depth Synthesis and Cross-Attention Terrain Reconstruction](https://mlsanity.com/paper/2510.07152/) (2025)
- [Isaac Sim-to-Real: Reinforcement Learning based Locomotion for Quadrupeds](https://mlsanity.com/paper/2607.18135/) (2026)

## Abstract (arXiv)

Informative terrain perception is important for robust reinforcement learning policies in humanoid locomotion. Still, common sensors such as depth cameras and LiDARs incur high cost, power, and processing overhead while often producing redundant, high-resolution data. This work uses a low-cost time-of-flight sensor to provide a compact 3D local terrain representation for humanoid locomotion. To efficiently use this sparse exteroceptive input, we introduce a token-compressed temporal transformer policy. Proprioceptive and terrain observations are tokenized and processed by a self-attention multi-head transformer to capture within-timestep relationships between observation terms. The attended tokens are then compressed through an MLP-based latent-space token compression module before being stored in a rolling 15-timestep history. A second cross-attention multi-head transformer extracts temporal locomotion features from this compact history for policy learning. By compressing tokens before temporal aggregation, the architecture preserves important terrain-observation structure while limiting the dimensional growth of attention over observation histories. We validate our method through sim-to-real transfer on physical hardware using a terrain-based locomotion benchmark, demonstrating robust humanoid terrain walking with low-cost local terrain sensing.