# Learning Parallel Dense Correspondence from Spatio-Temporal Descriptors for Efficient and Robust 4D Reconstruction

arXiv: https://arxiv.org/abs/2103.16341 · HTML: https://mlsanity.com/paper/2103.16341/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 52 references that are in the map.
- First posted on arXiv 30 Mar 2021.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [A Point Set Generation Network for 3D Object Reconstruction from a Single Image](https://mlsanity.com/paper/1612.00603/) (2016)
- [Convolutional Occupancy Networks](https://mlsanity.com/paper/2003.04618/) (2020)
- [Implicit Geometric Regularization for Learning Shapes](https://mlsanity.com/paper/2002.10099/) (2020)
- [Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images](https://mlsanity.com/paper/1804.01654/) (2018)
- [VIBE: Video Inference for Human Body Pose and Shape Estimation](https://mlsanity.com/paper/1912.05656/) (2019)
- [3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction](https://mlsanity.com/paper/1604.00449/) (2016)
- [Neural 3D Mesh Renderer](https://mlsanity.com/paper/1711.07566/) (2017)
- [SAL: Sign Agnostic Learning of Shapes from Raw Data](https://mlsanity.com/paper/1911.10414/) (2019)
- [OctNet: Learning Deep 3D Representations at High Resolutions](https://mlsanity.com/paper/1611.05009/) (2016)
- [DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstruction](https://mlsanity.com/paper/1905.10711/) (2019)
- [Learning 3D Human Dynamics from Video](https://mlsanity.com/paper/1812.01601/) (2018)
- [Video Based Reconstruction of 3D People Models](https://mlsanity.com/paper/1803.04758/) (2018)
- [Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputs](https://mlsanity.com/paper/1703.09438/) (2017)
- [Modulating early visual processing by language](https://mlsanity.com/paper/1707.00683/) (2017)
- [MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequences](https://mlsanity.com/paper/1910.09165/) (2019)
- [Deep Mesh Reconstruction from Single RGB Images via Topology Modification Networks](https://mlsanity.com/paper/1909.00321/) (2019)
- [Self-supervised Learning of Motion Capture](https://mlsanity.com/paper/1712.01337/) (2017)
- [Towards Accurate Markerless Human Shape and Pose Estimation over Time](https://mlsanity.com/paper/1707.07548/) (2017)
- [Hierarchical Surface Prediction for 3D Object Reconstruction](https://mlsanity.com/paper/1704.00710/) (2017)
- [High-Resolution Shape Completion Using Deep Neural Networks for Global Structure and Local Geometry Inference](https://mlsanity.com/paper/1709.07599/) (2017)
- [A Skeleton-bridged Deep Learning Approach for Generating Meshes of Complex Topologies from Single RGB Images](https://mlsanity.com/paper/1903.04704/) (2019)
- [Gaussian Process Morphable Models](https://mlsanity.com/paper/1603.07254/) (2016)
- [SkeletonNet: A Topology-Preserving Solution for Learning Mesh Reconstruction of Object Surfaces from RGB Images](https://mlsanity.com/paper/2008.05742/) (2020)
- [Sign-Agnostic Implicit Learning of Surface Self-Similarities for Shape Modeling and Reconstruction from Raw Point Clouds](https://mlsanity.com/paper/2012.07498/) (2020)

## Built on by (most cited first, 30 of 31)

- [Rotation-Invariant Transformer for Point Cloud Matching](https://mlsanity.com/paper/2303.08231/) (2023)
- [SA-ConvONet: Sign-Agnostic Optimization of Convolutional Occupancy Networks](https://mlsanity.com/paper/2105.03582/) (2021)
- [CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphism](https://mlsanity.com/paper/2203.16529/) (2022)
- [RGBD2: Generative Scene Synthesis via Incremental View Inpainting using RGBD Diffusion Models](https://mlsanity.com/paper/2212.05993/) (2022)
- [PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation](https://mlsanity.com/paper/2509.20358/) (2025)
- [GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusion](https://mlsanity.com/paper/2412.10209/) (2024)
- [Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking](https://mlsanity.com/paper/2401.06614/) (2024)
- [Recent Trends in 3D Reconstruction of General Non-Rigid Scenes](https://mlsanity.com/paper/2403.15064/) (2024)
- [Neural Shape Deformation Priors](https://mlsanity.com/paper/2210.05616/) (2022)
- [RIGA: Rotation-Invariant and Globally-Aware Descriptors for Point Cloud Registration](https://mlsanity.com/paper/2209.13252/) (2022)
- [PatchComplete: Learning Multi-Resolution Patch Priors for 3D Shape Completion on Unseen Categories](https://mlsanity.com/paper/2206.04916/) (2022)
- [FourierHandFlow: Neural 4D Hand Representation Using Fourier Query Flow](https://mlsanity.com/paper/2307.08100/) (2023)
- [DPHMs: Diffusion Parametric Head Models for Depth-based Tracking](https://mlsanity.com/paper/2312.01068/) (2023)
- [DynoSurf: Neural Deformation-based Temporally Consistent Dynamic Surface Reconstruction](https://mlsanity.com/paper/2403.11586/) (2024)
- [DNF: Unconditional 4D Generation with Dictionary-based Neural Fields](https://mlsanity.com/paper/2412.05161/) (2024)
- [Fluid Dynamics Network: Topology-Agnostic 4D Reconstruction via Fluid Dynamics Priors](https://mlsanity.com/paper/2303.09871/) (2023)
- [CanFields: Consolidating Diffeomorphic Flows for Non-Rigid 4D Interpolation from Arbitrary-Length Sequences](https://mlsanity.com/paper/2406.18582/) (2024)
- [RFNet-4D++: Joint Object Reconstruction and Flow Estimation from 4D Point Clouds with Cross-Attention Spatio-Temporal Features](https://mlsanity.com/paper/2203.16482/) (2022)
- [Fast Point Cloud to Mesh Reconstruction for Deformable Object Tracking](https://mlsanity.com/paper/2311.02749/) (2023)
- [Dynamic Neural Surfaces for Elastic 4D Shape Representation and Analysis](https://mlsanity.com/paper/2503.03132/) (2025)
- [Learning Coherent Matrixized Representation in Latent Space for Volumetric 4D Generation](https://mlsanity.com/paper/2403.13238/) (2024)
- [4DSR-GCN: 4D Video Point Cloud Upsampling using Graph Convolutional Networks](https://mlsanity.com/paper/2306.01081/) (2023)
- [NF3DM: Combining Neural Fields and Deformation Models for 3D Non-Rigid Motion Reconstruction](https://mlsanity.com/paper/2412.08511/) (2024)
- [Multiview Point Cloud Registration via Optimization in an Autoencoder Latent Space](https://mlsanity.com/paper/2504.21467/) (2025)
- [Preconditioned Deformation Grids](https://mlsanity.com/paper/2509.18097/) (2025)
- [ERNet: Efficient Non-Rigid Registration Network for Point Sequences](https://mlsanity.com/paper/2510.15800/) (2025)
- [Neu-PiG: Neural Preconditioned Grids for Fast Dynamic Surface Reconstruction on Long Sequences](https://mlsanity.com/paper/2602.22212/) (2026)
- [Learning Neural Deformation Representation for 4D Dynamic Shape Generation](https://mlsanity.com/paper/2606.01021/) (2026)
- [Temporally-Consistent Surface Reconstruction using Metrically-Consistent Atlases](https://mlsanity.com/paper/2111.06838/) (2021)
- [Point Cloud to Mesh Reconstruction: Methods, Trade-offs, and Implementation Guide](https://mlsanity.com/paper/2412.10977/) (2024)

## Abstract (arXiv)

This paper focuses on the task of 4D shape reconstruction from a sequence of point clouds. Despite the recent success achieved by extending deep implicit representations into 4D space, it is still a great challenge in two respects, i.e. how to design a flexible framework for learning robust spatio-temporal shape representations from 4D point clouds, and develop an efficient mechanism for capturing shape dynamics. In this work, we present a novel pipeline to learn a temporal evolution of the 3D human shape through spatially continuous transformation functions among cross-frame occupancy fields. The key idea is to parallelly establish the dense correspondence between predicted occupancy fields at different time steps via explicitly learning continuous displacement vector fields from robust spatio-temporal shape representations. Extensive comparisons against previous state-of-the-arts show the superior accuracy of our approach for 4D human reconstruction in the problems of 4D shape auto-encoding and completion, and a much faster network inference with about 8 times speedup demonstrates the significant efficiency of our approach. The trained models and implementation code are available at https://github.com/tangjiapeng/LPDC-Net.