# Holistic Fusion: Task- and Setup-Agnostic Robot Localization and State Estimation with Factor Graphs

arXiv: https://arxiv.org/abs/2504.06479 · HTML: https://mlsanity.com/paper/2504.06479/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 94 references that are in the map.
- First posted on arXiv 8 Apr 2025.

## Builds on (most cited first, 30 of 45)

- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [Learning robust perceptive locomotion for quadrupedal robots in the wild](https://mlsanity.com/paper/2201.08117/) (2022)
- [ANYmal Parkour: Learning Agile Navigation for Quadrupedal Robots](https://mlsanity.com/paper/2306.14874/) (2023)
- [KISS-ICP: In Defense of Point-to-Point ICP -- Simple, Accurate, and Robust Registration If Done the Right Way](https://mlsanity.com/paper/2209.15397/) (2022)
- [On-Manifold Preintegration for Real-Time Visual-Inertial Odometry](https://mlsanity.com/paper/1512.02363/) (2015)
- [FAST-LIO: A Fast, Robust LiDAR-inertial Odometry Package by Tightly-Coupled Iterated Kalman Filter](https://mlsanity.com/paper/2010.08196/) (2020)
- [LO-Net: Deep Real-time Lidar Odometry](https://mlsanity.com/paper/1904.08242/) (2019)
- [TLIO: Tight Learned Inertial Odometry](https://mlsanity.com/paper/2007.01867/) (2020)
- [A General Optimization-based Framework for Global Pose Estimation with Multiple Sensors](https://mlsanity.com/paper/1901.03642/) (2019)
- [Fast Traversability Estimation for Wild Visual Navigation](https://mlsanity.com/paper/2305.08510/) (2023)
- [OKVIS2: Realtime Scalable Visual-Inertial SLAM with Loop Closure](https://mlsanity.com/paper/2202.09199/) (2022)
- [BALM: Bundle Adjustment for Lidar Mapping](https://mlsanity.com/paper/2010.08215/) (2020)
- [AI-IMU Dead-Reckoning](https://mlsanity.com/paper/1904.06064/) (2019)
- [VILENS: Visual, Inertial, Lidar, and Leg Odometry for All-Terrain Legged Robots](https://mlsanity.com/paper/2107.07243/) (2021)
- [Super Odometry: IMU-centric LiDAR-Visual-Inertial Estimator for Challenging Environments](https://mlsanity.com/paper/2104.14938/) (2021)
- [RoadRunner -- Learning Traversability Estimation for Autonomous Off-road Driving](https://mlsanity.com/paper/2402.19341/) (2024)
- [maplab: An Open Framework for Research in Visual-inertial Mapping and Localization](https://mlsanity.com/paper/1711.10250/) (2017)
- [X-ICP: Localizability-Aware LiDAR Registration for Robust Localization in Extreme Environments](https://mlsanity.com/paper/2211.16335/) (2022)
- [The Hilti SLAM Challenge Dataset](https://mlsanity.com/paper/2109.11316/) (2021)
- [GVINS: Tightly Coupled GNSS-Visual-Inertial Fusion for Smooth and Consistent State Estimation](https://mlsanity.com/paper/2103.07899/) (2021)
- [Learning Inertial Odometry for Dynamic Legged Robot State Estimation](https://mlsanity.com/paper/2111.00789/) (2021)
- [maplab 2.0 -- A Modular and Multi-Modal Mapping Framework](https://mlsanity.com/paper/2212.00654/) (2022)
- [Self-supervised Learning of LiDAR Odometry for Robotic Applications](https://mlsanity.com/paper/2011.05418/) (2020)
- [RadarSLAM: Radar based Large-Scale SLAM in All Weathers](https://mlsanity.com/paper/2005.02198/) (2020)
- [Graph-based Multi-sensor Fusion for Consistent Localization of Autonomous Construction Robots](https://mlsanity.com/paper/2203.01389/) (2022)
- [Wildcat: Online Continuous-Time 3D Lidar-Inertial SLAM](https://mlsanity.com/paper/2205.12595/) (2022)
- [Learned Inertial Odometry for Autonomous Drone Racing](https://mlsanity.com/paper/2210.15287/) (2022)
- [Learning-based Localizability Estimation for Robust LiDAR Localization](https://mlsanity.com/paper/2203.05698/) (2022)
- [Cerberus: Low-Drift Visual-Inertial-Leg Odometry For Agile Locomotion](https://mlsanity.com/paper/2209.07654/) (2022)
- [Continuous-Time State Estimation Methods in Robotics: A Survey](https://mlsanity.com/paper/2411.03951/) (2024)

## Built on by (most cited first, 1 of 1)

- [CompSLAM: Complementary Hierarchical Multi-Modal Localization and Mapping for Robot Autonomy in Underground Environments](https://mlsanity.com/paper/2505.06483/) (2025)

## Abstract (arXiv)

Seamless operation of mobile robots in challenging environments requires low-latency local motion estimation and accurate global localization. While most sensor-fusion approaches are designed for specific scenarios, this work introduces a flexible open-source solution for task- and setup-agnostic multimodal sensor fusion distinguished by its generality and usability. Holistic Fusion formulates sensor fusion as a combined estimation problem of i) the local and global robot state and ii) a (theoretically unlimited) number of dynamic variables, including automatic alignment of reference frames; this formulation fits countless real-world applications without conceptual modifications, offering a comprehensive solution beyond hard-coded/task-specific approaches. The proposed factor-graph formulation enables direct fusion of an arbitrary number of absolute, local, and landmark measurements expressed with respect to different frames by explicitly including them as states in the optimization and modeling their evolution as random walks. Moreover, local smoothness and consistency receive particular attention to prevent estimation jumps. Holistic Fusion enables low-latency and smooth online state estimation on typical robot hardware while simultaneously providing low-drift global localization at the IMU measurement rate. The efficacy of this released framework [1] is demonstrated in five real-world scenarios on three robotic platforms with distinct task requirements, highlighting the advantages of fusing multiple absolute measurement types [2]. [1] Code: https://github.com/leggedrobotics/holistic_fusion [2] Project: https://leggedrobotics.github.io/holistic_fusion