# Perception for Humanoid Robots

arXiv: https://arxiv.org/abs/2309.15616 · HTML: https://mlsanity.com/paper/2309.15616/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 112 references that are in the map.
- First posted on arXiv 27 Sep 2023.

## Builds on (most cited first, 25 of 25)

- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [FlowFusion: Dynamic Dense RGB-D SLAM Based on Optical Flow](https://mlsanity.com/paper/2003.05102/) (2020)
- [Robotic Vision for Human-Robot Interaction and Collaboration: A Survey and Systematic Review](https://mlsanity.com/paper/2307.15363/) (2023)
- [A Survey on Human-aware Robot Navigation](https://mlsanity.com/paper/2106.11650/) (2021)
- [Translating Videos to Commands for Robotic Manipulation with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1710.00290/) (2017)
- [Contact-Aided Invariant Extended Kalman Filtering for Robot State Estimation](https://mlsanity.com/paper/1904.09251/) (2019)
- [ProxEmo: Gait-based Emotion Learning and Multi-view Proxemic Fusion for Socially-Aware Robot Navigation](https://mlsanity.com/paper/2003.01062/) (2020)
- [A Mobile Robot Hand-Arm Teleoperation System by Vision and IMU](https://mlsanity.com/paper/2003.05212/) (2020)
- [A Survey on Physiological Signal Based Emotion Recognition](https://mlsanity.com/paper/2205.10466/) (2022)
- [Hybrid Contact Preintegration for Visual-Inertial-Contact State Estimation Using Factor Graphs](https://mlsanity.com/paper/1803.07531/) (2018)
- [Legged Robot State-Estimation Through Combined Forward Kinematic and Preintegrated Contact Factors](https://mlsanity.com/paper/1712.05873/) (2017)
- [State Estimation for a Humanoid Robot](https://mlsanity.com/paper/1402.5450/) (2014)
- [WOLF: A modular estimation framework for robotics based on factor graphs](https://mlsanity.com/paper/2110.12919/) (2021)
- [Memory Unscented Particle Filter for 6-DOF Tactile Localization](https://mlsanity.com/paper/1607.02757/) (2016)
- [Unsupervised Contact Learning for Humanoid Estimation and Control](https://mlsanity.com/paper/1709.07472/) (2017)
- [Tracking Human Pose During Robot-Assisted Dressing using Single-Axis Capacitive Proximity Sensing](https://mlsanity.com/paper/1709.07957/) (2017)
- [Enhanced Robot Speech Recognition Using Biomimetic Binaural Sound Source Localization](https://mlsanity.com/paper/1902.05446/) (2019)
- [Audiovisual Speaker Tracking using Nonlinear Dynamical Systems with Dynamic Stream Weights](https://mlsanity.com/paper/1903.06031/) (2019)
- [Joining Sound Event Detection and Localization Through Spatial Segregation](https://mlsanity.com/paper/1904.00055/) (2019)
- [Curved patch mapping and tracking for irregular terrain modeling: Application to bipedal robot foot placement](https://mlsanity.com/paper/2004.03405/) (2020)
- [Probabilistic Contact State Estimation for Legged Robots using Inertial Information](https://mlsanity.com/paper/2303.00538/) (2023)
- [Humanoid Momentum Estimation Using Sensed Contact Wrenches](https://mlsanity.com/paper/1507.04401/) (2015)
- [Real-Time Object Detection and Recognition on Low-Compute Humanoid Robots using Deep Learning](https://mlsanity.com/paper/2002.03735/) (2020)
- [Inertial Sensor-Based Humanoid Joint State Estimation](https://mlsanity.com/paper/1602.05134/) (2016)
- [Robust Contact State Estimation in Humanoid Walking Gaits](https://mlsanity.com/paper/2208.00278/) (2022)

## Built on by (most cited first, 5 of 5)

- [Vision-Language-Action (VLA) Models: Concepts, Progress, Applications and Challenges](https://mlsanity.com/paper/2505.04769/) (2025)
- [Humanoid Occupancy: Enabling A Generalized Multimodal Occupancy Perception System on Humanoid Robots](https://mlsanity.com/paper/2507.20217/) (2025)
- [Monocular Person Localization under Camera Ego-motion](https://mlsanity.com/paper/2503.02916/) (2025)
- [A Hybrid Autoencoder for Robust Heightmap Generation from Fused Lidar and Depth Data for Humanoid Robot Locomotion](https://mlsanity.com/paper/2602.05855/) (2026)
- [Spatiotemporal Agility: Time-Constrained Reinforcement Learning for Vision-Guided Dynamic Quadrupedal Interception](https://mlsanity.com/paper/2608.06907/) (2026)

## Abstract (arXiv)

Purpose of Review: The field of humanoid robotics, perception plays a fundamental role in enabling robots to interact seamlessly with humans and their surroundings, leading to improved safety, efficiency, and user experience. This scientific study investigates various perception modalities and techniques employed in humanoid robots, including visual, auditory, and tactile sensing by exploring recent state-of-the-art approaches for perceiving and understanding the internal state, the environment, objects, and human activities. Recent Findings: Internal state estimation makes extensive use of Bayesian filtering methods and optimization techniques based on maximum a-posteriori formulation by utilizing proprioceptive sensing. In the area of external environment understanding, with an emphasis on robustness and adaptability to dynamic, unforeseen environmental changes, the new slew of research discussed in this study have focused largely on multi-sensor fusion and machine learning in contrast to the use of hand-crafted, rule-based systems. Human robot interaction methods have established the importance of contextual information representation and memory for understanding human intentions. Summary: This review summarizes the recent developments and trends in the field of perception in humanoid robots. Three main areas of application are identified, namely, internal state estimation, external environment estimation, and human robot interaction. The applications of diverse sensor modalities in each of these areas are considered and recent significant works are discussed.