# CTIN: Robust Contextual Transformer Network for Inertial Navigation

arXiv: https://arxiv.org/abs/2112.02143 · HTML: https://mlsanity.com/paper/2112.02143/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 51 references that are in the map.
- First posted on arXiv 3 Dec 2021.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Stand-Alone Self-Attention in Vision Models](https://mlsanity.com/paper/1906.05909/) (2019)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [Exploring Self-attention for Image Recognition](https://mlsanity.com/paper/2004.13621/) (2020)
- [Local Relation Networks for Image Recognition](https://mlsanity.com/paper/1904.11491/) (2019)
- [Contextual Transformer Networks for Visual Recognition](https://mlsanity.com/paper/2107.12292/) (2021)
- [RoNIN: Robust Neural Inertial Navigation in the Wild: Benchmark, Evaluations, and New Methods](https://mlsanity.com/paper/1905.12853/) (2019)
- [IONet: Learning to Cure the Curse of Drift in Inertial Odometry](https://mlsanity.com/paper/1802.02209/) (2018)
- [TLIO: Tight Learned Inertial Odometry](https://mlsanity.com/paper/2007.01867/) (2020)
- [Backprop KF: Learning Discriminative Deterministic State Estimators](https://mlsanity.com/paper/1605.07148/) (2016)
- [RIDI: Robust IMU Double Integration](https://mlsanity.com/paper/1712.09004/) (2017)
- [Neural Networks Are More Productive Teachers Than Human Raters: Active Mixup for Data-Efficient Knowledge Distillation from a Blackbox Model](https://mlsanity.com/paper/2003.13960/) (2020)
- [AI-IMU Dead-Reckoning](https://mlsanity.com/paper/1904.06064/) (2019)
- [Multivariate Uncertainty in Deep Learning](https://mlsanity.com/paper/1910.14215/) (2019)
- [IDOL: Inertial Deep Orientation-Estimation and Localization](https://mlsanity.com/paper/2102.04024/) (2021)
- [OxIOD: The Dataset for Deep Inertial Odometry](https://mlsanity.com/paper/1809.07491/) (2018)
- [Using Inertial Sensors for Position and Orientation Estimation](https://mlsanity.com/paper/1704.06053/) (2017)
- [Robust Data-Driven Zero-Velocity Detection for Foot-Mounted Inertial Navigation](https://mlsanity.com/paper/1910.00529/) (2019)

## Built on by (most cited first, 12 of 12)

- [Deep Learning for Inertial Positioning: A Survey](https://mlsanity.com/paper/2303.03757/) (2023)
- [Generalizable End-to-End Deep Learning Frameworks for Real-Time Attitude Estimation Using 6DoF Inertial Measurement Units](https://mlsanity.com/paper/2302.06037/) (2023)
- [LGC-Net: A Lightweight Gyroscope Calibration Network for Efficient Attitude Estimation](https://mlsanity.com/paper/2209.08816/) (2022)
- [Physical Self-Supervised Learning: IMU Sensing without Manual Labels](https://mlsanity.com/paper/2607.18361/) (2026)
- [NeurIT: Pushing the Limit of Neural Inertial Tracking for Indoor Robotic IoT](https://mlsanity.com/paper/2404.08939/) (2024)
- [IONext: Unlocking the Next Era of Inertial Odometry](https://mlsanity.com/paper/2507.17089/) (2025)
- [ConvXformer: Differentially Private Hybrid ConvNeXt-Transformer for Inertial Navigation](https://mlsanity.com/paper/2510.19352/) (2025)
- [CorVS+: Correspondence-Driven Association of Video Trajectories and Sensors for Identity-Aware Person Localization in Warehouses](https://mlsanity.com/paper/2510.26369/) (2025)
- [X-IONet: Cross-Platform Inertial Odometry Network for Pedestrian and Legged Robot](https://mlsanity.com/paper/2511.08277/) (2025)
- [FDIO: Frequency Decomposed Inertial Odometry](https://mlsanity.com/paper/2511.15645/) (2025)
- [Tracking Large-scale Shared Bikes with Inertial Motion Learning in GNSS Blocked Environments](https://mlsanity.com/paper/2605.07412/) (2026)
- [PedestrianDiffusion: Multimodal Generative Denoising and Dense State Estimation for Inertial Navigation](https://mlsanity.com/paper/2607.03349/) (2026)

## Abstract (arXiv)

Recently, data-driven inertial navigation approaches have demonstrated their capability of using well-trained neural networks to obtain accurate position estimates from inertial measurement units (IMU) measurements. In this paper, we propose a novel robust Contextual Transformer-based network for Inertial Navigation~(CTIN) to accurately predict velocity and trajectory. To this end, we first design a ResNet-based encoder enhanced by local and global multi-head self-attention to capture spatial contextual information from IMU measurements. Then we fuse these spatial representations with temporal knowledge by leveraging multi-head attention in the Transformer decoder. Finally, multi-task learning with uncertainty reduction is leveraged to improve learning efficiency and prediction accuracy of velocity and trajectory. Through extensive experiments over a wide range of inertial datasets~(e.g. RIDI, OxIOD, RoNIN, IDOL, and our own), CTIN is very robust and outperforms state-of-the-art models.