# Exosense: A Vision-Based Scene Understanding System For Exoskeletons

arXiv: https://arxiv.org/abs/2403.14320 · HTML: https://mlsanity.com/paper/2403.14320/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 37 references that are in the map.
- First posted on arXiv 21 Mar 2024.

## Builds on (most cited first, 6 of 6)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [Language-EXtended Indoor SLAM (LEXIS): A Versatile System for Real-time Visual Scene Understanding](https://mlsanity.com/paper/2309.15065/) (2023)
- [Balancing the Budget: Feature Selection and Tracking for Multi-Camera Visual-Inertial Odometry](https://mlsanity.com/paper/2109.05975/) (2021)
- [Environment-based Assistance Modulation for a Hip Exosuit via Computer Vision](https://mlsanity.com/paper/2211.15346/) (2022)
- [Multi-IMU Proprioceptive State Estimator for Humanoid Robots](https://mlsanity.com/paper/2307.14125/) (2023)

## Built on by (most cited first, 1 of 1)

- [LT-Exosense: A Vision-centric Multi-session Mapping System for Lifelong Safe Navigation of Exoskeletons](https://mlsanity.com/paper/2510.22164/) (2025)

## Abstract (arXiv)

Self-balancing exoskeletons are a key enabling technology for individuals with mobility impairments. While the current challenges focus on human-compliant hardware and control, unlocking their use for daily activities requires a scene perception system. In this work, we present Exosense, a vision-centric scene understanding system for self-balancing exoskeletons. We introduce a multi-sensor visual-inertial mapping device as well as a navigation stack for state estimation, terrain mapping and long-term operation. We tested Exosense attached to both a human leg and Wandercraft's Personal Exoskeleton in real-world indoor scenarios. This enabled us to test the system during typical periodic walking gaits, as well as future uses in multi-story environments. We demonstrate that Exosense can achieve an odometry drift of about 4 cm per meter traveled, and construct terrain maps under 1 cm average reconstruction error. It can also work in a visual localization mode in a previously mapped environment, providing a step towards long-term operation of exoskeletons.