# Legged Robot State Estimation With Invariant Extended Kalman Filter Using Neural Measurement Network

arXiv: https://arxiv.org/abs/2402.00366 · HTML: https://mlsanity.com/paper/2402.00366/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 42 references that are in the map.
- First posted on arXiv 1 Feb 2024.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator](https://mlsanity.com/paper/1708.03852/) (2017)
- [On-Manifold Preintegration for Real-Time Visual-Inertial Odometry](https://mlsanity.com/paper/1512.02363/) (2015)
- [Blind Bipedal Stair Traversal via Sim-to-Real Reinforcement Learning](https://mlsanity.com/paper/2105.08328/) (2021)
- [Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion](https://mlsanity.com/paper/2202.05481/) (2022)
- [RoNIN: Robust Neural Inertial Navigation in the Wild: Benchmark, Evaluations, and New Methods](https://mlsanity.com/paper/1905.12853/) (2019)
- [Regularizing Action Policies for Smooth Control with Reinforcement Learning](https://mlsanity.com/paper/2012.06644/) (2020)
- [IONet: Learning to Cure the Curse of Drift in Inertial Odometry](https://mlsanity.com/paper/1802.02209/) (2018)
- [TLIO: Tight Learned Inertial Odometry](https://mlsanity.com/paper/2007.01867/) (2020)
- [Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotion](https://mlsanity.com/paper/2308.12517/) (2023)
- [Quaternion kinematics for the error-state Kalman filter](https://mlsanity.com/paper/1711.02508/) (2017)
- [AI-IMU Dead-Reckoning](https://mlsanity.com/paper/1904.06064/) (2019)
- [Denoising IMU Gyroscopes with Deep Learning for Open-Loop Attitude Estimation](https://mlsanity.com/paper/2002.10718/) (2020)
- [IDOL: Inertial Deep Orientation-Estimation and Localization](https://mlsanity.com/paper/2102.04024/) (2021)
- [VILENS: Visual, Inertial, Lidar, and Leg Odometry for All-Terrain Legged Robots](https://mlsanity.com/paper/2107.07243/) (2021)
- [Contact-Aided Invariant Extended Kalman Filtering for Robot State Estimation](https://mlsanity.com/paper/1904.09251/) (2019)
- [Learning Inertial Odometry for Dynamic Legged Robot State Estimation](https://mlsanity.com/paper/2111.00789/) (2021)
- [The invariant extended Kalman filter as a stable observer](https://mlsanity.com/paper/1410.1465/) (2014)
- [Legged Robot State Estimation using Invariant Kalman Filtering and Learned Contact Events](https://mlsanity.com/paper/2106.15713/) (2021)
- [Learned Inertial Odometry for Autonomous Drone Racing](https://mlsanity.com/paper/2210.15287/) (2022)
- [The Blackbird Dataset: A large-scale dataset for UAV perception in aggressive flight](https://mlsanity.com/paper/1810.01987/) (2018)
- [IMU Data Processing For Inertial Aided Navigation: A Recurrent Neural Network Based Approach](https://mlsanity.com/paper/2103.14286/) (2021)
- [Hybrid Contact Preintegration for Visual-Inertial-Contact State Estimation Using Factor Graphs](https://mlsanity.com/paper/1803.07531/) (2018)
- [Legged Robot State-Estimation Through Combined Forward Kinematic and Preintegrated Contact Factors](https://mlsanity.com/paper/1712.05873/) (2017)
- [DIDO: Deep Inertial Quadrotor Dynamical Odometry](https://mlsanity.com/paper/2203.03149/) (2022)

## Built on by (most cited first, 4 of 4)

- [MARG: MAstering Risky Gap Terrains for Legged Robots with Elevation Mapping](https://mlsanity.com/paper/2509.20036/) (2025)
- [Attention-Based Neural-Augmented Kalman Filter for Legged Robot State Estimation](https://mlsanity.com/paper/2601.18569/) (2026)
- [CoCo-InEKF: State Estimation with Learned Contact Covariances in Dynamic, Contact-Rich Scenarios](https://mlsanity.com/paper/2605.15122/) (2026)
- [TRACE: Learned Proprioceptive Odometry for Legged Robots under Unreliable Contact Conditions](https://mlsanity.com/paper/2608.05975/) (2026)

## Abstract (arXiv)

This paper introduces a novel proprioceptive state estimator for legged robots that combines model-based filters and deep neural networks. Recent studies have shown that neural networks such as multi-layer perceptron or recurrent neural networks can estimate the robot states, including contact probability and linear velocity. Inspired by this, we develop a state estimation framework that integrates a neural measurement network (NMN) with an invariant extended Kalman filter. We show that our framework improves estimation performance in various terrains. Existing studies that combine model-based filters and learning-based approaches typically use real-world data. However, our approach relies solely on simulation data, as it allows us to easily obtain extensive data. This difference leads to a gap between the learning and the inference domain, commonly referred to as a sim-to-real gap. We address this challenge by adapting existing learning techniques and regularization. To validate our proposed method, we conduct experiments using a quadruped robot on four types of terrain: textitflat, textitdebris, textitsoft, and textitslippery. We observe that our approach significantly reduces position drift compared to the existing model-based state estimator.