# GTT-Net: Learned Generalized Trajectory Triangulation

arXiv: https://arxiv.org/abs/2109.03408 · HTML: https://mlsanity.com/paper/2109.03408/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 46 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Builds on (most cited first, 18 of 18)

- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [OctNet: Learning Deep 3D Representations at High Resolutions](https://mlsanity.com/paper/1611.05009/) (2016)
- [Volumetric and Multi-View CNNs for Object Classification on 3D Data](https://mlsanity.com/paper/1604.03265/) (2016)
- [Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1706.01307/) (2017)
- [Panoptic Studio: A Massively Multiview System for Social Interaction Capture](https://mlsanity.com/paper/1612.03153/) (2016)
- [Deep Parametric Continuous Convolutional Neural Networks](https://mlsanity.com/paper/2101.06742/) (2021)
- [Learning Rich Features from RGB-D Images for Object Detection and Segmentation](https://mlsanity.com/paper/1407.5736/) (2014)
- [Learning Monocular 3D Human Pose Estimation from Multi-view Images](https://mlsanity.com/paper/1803.04775/) (2018)
- [C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motion](https://mlsanity.com/paper/1909.02533/) (2019)
- [BigHand2.2M Benchmark: Hand Pose Dataset and State of the Art Analysis](https://mlsanity.com/paper/1704.02612/) (2017)
- [Deep Non-Rigid Structure from Motion](https://mlsanity.com/paper/1908.00052/) (2019)
- [RGBD-Dog: Predicting Canine Pose from RGBD Sensors](https://mlsanity.com/paper/2004.07788/) (2020)
- [Deep NRSfM++: Towards Unsupervised 2D-3D Lifting in the Wild](https://mlsanity.com/paper/2001.10090/) (2020)
- [Discrete Laplace Operator Estimation for Dynamic 3D Reconstruction](https://mlsanity.com/paper/1908.11044/) (2019)
- [Self-expressive Dictionary Learning for Dynamic 3D Reconstruction](https://mlsanity.com/paper/1605.06863/) (2016)

## Built on by (most cited first, 2 of 2)

- [Deep Non-rigid Structure-from-Motion: A Sequence-to-Sequence Translation Perspective](https://mlsanity.com/paper/2204.04730/) (2022)
- [3D Trajectory Reconstruction of Moving Points Based on a Monocular Camera](https://mlsanity.com/paper/2502.19689/) (2025)

## Abstract (arXiv)

We present GTT-Net, a supervised learning framework for the reconstruction of sparse dynamic 3D geometry. We build on a graph-theoretic formulation of the generalized trajectory triangulation problem, where non-concurrent multi-view imaging geometry is known but global image sequencing is not provided. GTT-Net learns pairwise affinities modeling the spatio-temporal relationships among our input observations and leverages them to determine 3D geometry estimates. Experiments reconstructing 3D motion-capture sequences show GTT-Net outperforms the state of the art in terms of accuracy and robustness. Within the context of articulated motion reconstruction, our proposed architecture is 1) able to learn and enforce semantic 3D motion priors for shared training and test domains, while being 2) able to generalize its performance across different training and test domains. Moreover, GTT-Net provides a computationally streamlined framework for trajectory triangulation with applications to multi-instance reconstruction and event segmentation.