# TESO: Online Tracking of Essential Matrix by Stochastic Optimization

arXiv: https://arxiv.org/abs/2604.19420 · HTML: https://mlsanity.com/paper/2604.19420/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 36 references that are in the map.
- First posted on arXiv 21 Apr 2026.

## Builds on (most cited first, 12 of 12)

- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [SuperGlue: Learning Feature Matching with Graph Neural Networks](https://mlsanity.com/paper/1911.11763/) (2019)
- [LoFTR: Detector-Free Local Feature Matching with Transformers](https://mlsanity.com/paper/2104.00680/) (2021)
- [The Faiss library](https://mlsanity.com/paper/2401.08281/) (2024)
- [Wide-Baseline Relative Camera Pose Estimation with Directional Learning](https://mlsanity.com/paper/2106.03336/) (2021)
- [MAN TruckScenes: A multimodal dataset for autonomous trucking in diverse conditions](https://mlsanity.com/paper/2407.07462/) (2024)
- [AI-IMU Dead-Reckoning](https://mlsanity.com/paper/1904.06064/) (2019)
- [Benchmarking Egocentric Visual-Inertial SLAM at City Scale](https://mlsanity.com/paper/2509.26639/) (2025)
- [FAR: Flexible, Accurate and Robust 6DoF Relative Camera Pose Estimation](https://mlsanity.com/paper/2403.03221/) (2024)
- [Recalibrating the KITTI Dataset Camera Setup for Improved Odometry Accuracy](https://mlsanity.com/paper/2109.03462/) (2021)

## Abstract (arXiv)

Maintaining long-term accuracy of stereo camera calibration parameters is important for autonomous systems' perception. This work proposes Online Tracking of Essential Matrix by Stochastic Optimization (TESO). The core mechanisms of TESO are: 1) a robust loss function based on kernel correlation over tentative correspondences, 2) an adaptive online stochastic optimization on the essential manifold. TESO has low CPU and memory requirements, relies on a few hyperparameters, and eliminates the need for data-driven training, enabling the usage in resource-constrained online perception systems. We evaluated the influence of TESO on geometric precision, rectification quality, and stereo depth consistency. On the large-scale MAN TruckScenes dataset, TESO tracks rotational calibration drift with 0.12 deg precision in the Y-axis (critical for stereo accuracy) while the X- and Z-axes are five times more precise. Tracking applied to sequences with simulated drift shows similar precision with respect to the reference as tracking applied to no-drift sequences, indicating the tracker is unbiased. On the KITTI dataset, TESO revealed systematic inconsistencies in extrinsic parameters across stereo pairs, confirming previous published findings. We verified that intrinsic decalibration affected these errors, as evidenced by the conflicting behavior of the rectification and depth metrics. After correcting the reference calibration, TESO improved its rotation precision around the Y-axis 20 times to 0.025 deg and its depth accuracy 50 times. Despite its lightweight design, direct optimization of the proposed TESO loss function alone achieves accuracy comparable to that of neural network-based single-frame methods.