PixTrack: Precise 6DoF Object Pose Tracking using NeRF Templates and Feature-metric Alignment
Not on the map. Why?
Abstract (arXiv)
We present PixTrack, a vision based object pose tracking framework using novel view synthesis and deep feature-metric alignment. We follow an SfM-based relocalization paradigm where we use a Neural Radiance Field to canonically represent the tracked object. Our evaluations demonstrate that our method produces highly accurate, robust, and jitter-free 6DoF pose estimates of objects in both monocular RGB images and RGB-D images without the need of any data annotation or trajectory smoothing. Our method is also computationally efficient making it easy to have multi-object tracking with no alteration to our algorithm through simple CPU multiprocessing. Our code is available at: https://github.com/GiantAI/pixtrack
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 15 of its 22 references that are in the map. First posted on arXiv 8 Sep 2022.
Builds on
15 of 22 references.
References · 15
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
- 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,723
- 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
- 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,443
- 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,320
- 2020GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fieldscited by 785
- 2018From Coarse to Fine: Robust Hierarchical Localization at Large Scalecited by 559
- 2020INeRF: Inverting Neural Radiance Fields for Pose Estimationcited by 382
- 2017A General and Adaptive Robust Loss Functioncited by 236
- 2021Pixel-Perfect Structure-from-Motion with Featuremetric Refinementcited by 200
- 2021Back to the Feature: Learning Robust Camera Localization from Pixels to Posecited by 199
- 2022OnePose: One-Shot Object Pose Estimation without CAD Modelscited by 157
3 more
- 2020HybridPose: 6D Object Pose Estimation under Hybrid Representationscited by 107
- 2022NeRF-Pose: A First-Reconstruct-Then-Regress Approach for Weakly-supervised 6D Object Pose Estimationcited by 58
- 2022RNNPose: Recurrent 6-DoF Object Pose Refinement with Robust Correspondence Field Estimation and Pose Optimizationcited by 38
Built on by
Citing papers per year: 2024: 1.
1 paper.