# Online Adaptation for Consistent Mesh Reconstruction in the Wild

arXiv: https://arxiv.org/abs/2012.03196 · HTML: https://mlsanity.com/paper/2012.03196/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Dec 2020.

## Built on by (most cited first, 30 of 51)

- [A Comprehensive Survey on Test-Time Adaptation under Distribution Shifts](https://mlsanity.com/paper/2303.15361/) (2023)
- [BANMo: Building Animatable 3D Neural Models from Many Casual Videos](https://mlsanity.com/paper/2112.12761/) (2021)
- [NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild](https://mlsanity.com/paper/2110.07604/) (2021)
- [LASR: Learning Articulated Shape Reconstruction from a Monocular Video](https://mlsanity.com/paper/2105.02976/) (2021)
- [A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation](https://mlsanity.com/paper/2104.07645/) (2021)
- [MagicPony: Learning Articulated 3D Animals in the Wild](https://mlsanity.com/paper/2211.12497/) (2022)
- [LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discovery](https://mlsanity.com/paper/2207.03434/) (2022)
- [MFT: Long-Term Tracking of Every Pixel](https://mlsanity.com/paper/2305.12998/) (2023)
- [DOVE: Learning Deformable 3D Objects by Watching Videos](https://mlsanity.com/paper/2107.10844/) (2021)
- [Reconstructing Animatable Categories from Videos](https://mlsanity.com/paper/2305.06351/) (2023)
- [REACTO: Reconstructing Articulated Objects from a Single Video](https://mlsanity.com/paper/2404.11151/) (2024)
- [Surface-Aligned Neural Radiance Fields for Controllable 3D Human Synthesis](https://mlsanity.com/paper/2201.01683/) (2022)
- [ARAPReg: An As-Rigid-As Possible Regularization Loss for Learning Deformable Shape Generators](https://mlsanity.com/paper/2108.09432/) (2021)
- [AUV-Net: Learning Aligned UV Maps for Texture Transfer and Synthesis](https://mlsanity.com/paper/2204.03105/) (2022)
- [CASA: Category-agnostic Skeletal Animal Reconstruction](https://mlsanity.com/paper/2211.03568/) (2022)
- [Test-Time Personalization with a Transformer for Human Pose Estimation](https://mlsanity.com/paper/2107.02133/) (2021)
- [De-rendering 3D Objects in the Wild](https://mlsanity.com/paper/2201.02279/) (2022)
- [Hi-LASSIE: High-Fidelity Articulated Shape and Skeleton Discovery from Sparse Image Ensemble](https://mlsanity.com/paper/2212.11042/) (2022)
- [State of the Art in Dense Monocular Non-Rigid 3D Reconstruction](https://mlsanity.com/paper/2210.15664/) (2022)
- [Unbiased 4D: Monocular 4D Reconstruction with a Neural Deformation Model](https://mlsanity.com/paper/2206.08368/) (2022)
- [φ-SfT: Shape-from-Template with a Physics-Based Deformation Model](https://mlsanity.com/paper/2203.11938/) (2022)
- [ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collections](https://mlsanity.com/paper/2306.04619/) (2023)
- [Dynamic Gaussians Mesh: Consistent Mesh Reconstruction from Dynamic Scenes](https://mlsanity.com/paper/2404.12379/) (2024)
- [To The Point: Correspondence-driven monocular 3D category reconstruction](https://mlsanity.com/paper/2106.05662/) (2021)
- [Ponymation: Learning Articulated 3D Animal Motions from Unlabeled Online Videos](https://mlsanity.com/paper/2312.13604/) (2023)
- [Unsupervised Volumetric Animation](https://mlsanity.com/paper/2301.11326/) (2023)
- [SAOR: Single-View Articulated Object Reconstruction](https://mlsanity.com/paper/2303.13514/) (2023)
- [Fine Detailed Texture Learning for 3D Meshes with Generative Models](https://mlsanity.com/paper/2203.09362/) (2022)
- [StrobeNet: Category-Level Multiview Reconstruction of Articulated Objects](https://mlsanity.com/paper/2105.08016/) (2021)
- [Reconstructing 4D Spatial Intelligence: A Survey](https://mlsanity.com/paper/2507.21045/) (2025)

## Abstract (arXiv)

This paper presents an algorithm to reconstruct temporally consistent 3D meshes of deformable object instances from videos in the wild. Without requiring annotations of 3D mesh, 2D keypoints, or camera pose for each video frame, we pose video-based reconstruction as a self-supervised online adaptation problem applied to any incoming test video. We first learn a category-specific 3D reconstruction model from a collection of single-view images of the same category that jointly predicts the shape, texture, and camera pose of an image. Then, at inference time, we adapt the model to a test video over time using self-supervised regularization terms that exploit temporal consistency of an object instance to enforce that all reconstructed meshes share a common texture map, a base shape, as well as parts. We demonstrate that our algorithm recovers temporally consistent and reliable 3D structures from videos of non-rigid objects including those of animals captured in the wild -- an extremely challenging task rarely addressed before.