mlsanityData to 7 Oct 2026

4DComplete: Non-Rigid Motion Estimation Beyond the Observable Surface

arXiv 2105.01905 ↗First posted 5 May 2021cs.CVcited by 123

Not on the map. Why?

Abstract (arXiv)

Tracking non-rigidly deforming scenes using range sensors has numerous applications including computer vision, AR/VR, and robotics. However, due to occlusions and physical limitations of range sensors, existing methods only handle the visible surface, thus causing discontinuities and incompleteness in the motion field. To this end, we introduce 4DComplete, a novel data-driven approach that estimates the non-rigid motion for the unobserved geometry. 4DComplete takes as input a partial shape and motion observation, extracts 4D time-space embedding, and jointly infers the missing geometry and motion field using a sparse fully-convolutional network. For network training, we constructed a large-scale synthetic dataset called DeformingThings4D, which consists of 1972 animation sequences spanning 31 different animals or humanoid categories with dense 4D annotation. Experiments show that 4DComplete 1) reconstructs high-resolution volumetric shape and motion field from a partial observation, 2) learns an entangled 4D feature representation that benefits both shape and motion estimation, 3) yields more accurate and natural deformation than classic non-rigid priors such as As-Rigid-As-Possible (ARAP) deformation, and 4) generalizes well to unseen objects in real-world sequences.

Key facts. Cited by 123 papers in the map (data through 7 Oct 2026). Builds on 36 of its 61 references that are in the map. First posted on arXiv 5 May 2021.

Builds on

36 of 61 references.

References · 36

  1. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  2. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  3. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  5. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  6. 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,201
  7. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  8. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  9. 2015A Large Dataset to Train Convolutional Networks for Disparity, Optical Flow, and Scene Flow Estimationcited by 878
  10. 2020Convolutional Occupancy Networkscited by 760
  11. 20173D Semantic Segmentation with Submanifold Sparse Convolutional Networkscited by 746
  12. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
24 more
  1. 2020Local Implicit Grid Representations for 3D Scenescited by 372
  2. 2020Implicit Functions in Feature Space for 3D Shape Reconstruction and Completioncited by 343
  3. 20163D Menagerie: Modeling the 3D shape and pose of animalscited by 310
  4. 2018FlowNet3D: Learning Scene Flow in 3D Point Cloudscited by 241
  5. 2017Submanifold Sparse Convolutional Networkscited by 223
  6. 2016Shape Completion using 3D-Encoder-Predictor CNNs and Shape Synthesiscited by 219
  7. 2019DeepHuman: 3D Human Reconstruction from a Single Imagecited by 182
  8. 2017Playing for Benchmarkscited by 169
  9. 2019MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequencescited by 117
  10. 2019HPLFlowNet: Hierarchical Permutohedral Lattice FlowNet for Scene Flow Estimation on Large-scale Point Cloudscited by 108
  11. 2017ScanComplete: Large-Scale Scene Completion and Semantic Segmentation for 3D Scanscited by 103
  12. 2019SG-NN: Sparse Generative Neural Networks for Self-Supervised Scene Completion of RGB-D Scanscited by 77
  13. 2019DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Datacited by 76
  14. 2016VolumeDeform: Real-time Volumetric Non-rigid Reconstructioncited by 75
  15. 2020DI-Fusion: Online Implicit 3D Reconstruction with Deep Priorscited by 70
  16. 2019FlowNet3D++: Geometric Losses For Deep Scene Flow Estimationcited by 55
  17. 2020Neural Non-Rigid Trackingcited by 49
  18. 2018Learning Rigidity in Dynamic Scenes with a Moving Camera for 3D Motion Field Estimationcited by 38
  19. 2019PointPWC-Net: A Coarse-to-Fine Network for Supervised and Self-Supervised Scene Flow Estimation on 3D Point Cloudscited by 38
  20. 2020SPSG: Self-Supervised Photometric Scene Generation from RGB-D Scanscited by 34
  21. 2019SurfelWarp: Efficient Non-Volumetric Single View Dynamic Reconstructioncited by 31
  22. 2020Robust 3D Self-portraits in Secondscited by 30
  23. 2020Learning to Optimize Non-Rigid Trackingcited by 16
  24. 2020The Whole Is Greater Than the Sum of Its Nonrigid Partscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 13; 2023: 27; 2024: 31; 2025: 26; 2026: 23.

123 papers.

Sort:

By area:

  1. 2023PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Trackingcited by 274
  2. 2023HyperDiffusion: Generating Implicit Neural Fields with Weight-Space Diffusioncited by 152
  3. 2023Infinite Photorealistic Worlds using Procedural Generationcited by 142
  4. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  5. 2021Lepard: Learning partial point cloud matching in rigid and deformable scenescited by 85
  6. 2023GeoTransformer: Fast and Robust Point Cloud Registration with Geometric Transformercited by 83
  7. 2022gDNA: Towards Generative Detailed Neural Avatarscited by 72
  8. 2023Rotation-Invariant Transformer for Point Cloud Matchingcited by 68
  9. 2023HiFi4G: High-Fidelity Human Performance Rendering via Compact Gaussian Splattingcited by 68
  10. 2023Unsupervised Learning of Robust Spectral Shape Matchingcited by 57
  11. 2022CaDeX: Learning Canonical Deformation Coordinate Space for Dynamic Surface Representation via Neural Homeomorphismcited by 56
  12. 2022Non-rigid Point Cloud Registration with Neural Deformation Pyramidcited by 41
38 more
  1. 2022Smooth Non-Rigid Shape Matching via Effective Dirichlet Energy Optimizationcited by 37
  2. 2023Dynamic Point Fieldscited by 37
  3. 2022Learning Multi-resolution Functional Maps with Spectral Attention for Robust Shape Matchingcited by 35
  4. 2022CASA: Category-agnostic Skeletal Animal Reconstructioncited by 35
  5. 2023Zero-Shot 3D Shape Correspondencecited by 33
  6. 2025AnimateAnyMesh: A Feed-Forward 4D Foundation Model for Text-Driven Universal Mesh Animationcited by 31
  7. 2021Multiway Non-rigid Point Cloud Registration via Learned Functional Map Synchronizationcited by 30
  8. 2023Spatially and Spectrally Consistent Deep Functional Mapscited by 30
  9. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
  10. 2022OcclusionFusion: Occlusion-aware Motion Estimation for Real-time Dynamic 3D Reconstructioncited by 27
  11. 2024Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Trackingcited by 27
  12. 2024Towards Open Domain Text-Driven Synthesis of Multi-Person Motionscited by 27
  13. 2025Anymate: A Dataset and Baselines for Learning 3D Object Riggingcited by 27
  14. 2022Neural Shape Deformation Priorscited by 26
  15. 2023ResFields: Residual Neural Fields for Spatiotemporal Signalscited by 26
  16. 2024Spectral Meets Spatial: Harmonising 3D Shape Matching and Interpolationcited by 26
  17. 2024Make-A-Shape: a Ten-Million-scale 3D Shape Modelcited by 25
  18. 2022Morig: Motion-aware rigging of character meshes from point cloudscited by 20
  19. 2023OmniMotionGPT: Animal Motion Generation with Limited Datacited by 18
  20. 2023Hybrid Functional Maps for Crease-Aware Non-Isometric Shape Matchingcited by 18
  21. 2024MeshUp: Multi-Target Mesh Deformation via Blended Score Distillationcited by 17
  22. 2024Partial-to-Partial Shape Matching with Geometric Consistencycited by 16
  23. 2023GarmentTracking: Category-Level Garment Pose Trackingcited by 15
  24. 2023Deep Graph-based Spatial Consistency for Robust Non-rigid Point Cloud Registrationcited by 14
  25. 2024Memory-Scalable and Simplified Functional Map Learningcited by 14
  26. 2022Implicit field supervision for robust non-rigid shape matchingcited by 13
  27. 2024Integrating Efficient Optimal Transport and Functional Maps For Unsupervised Shape Correspondence Learningcited by 12
  28. 2024SRIF: Semantic Shape Registration Empowered by Diffusion-based Image Morphing and Flow Estimationcited by 12
  29. 2024Beyond Complete Shapes: A Benchmark for Quantitative Evaluation of 3D Shape Surface Matching Algorithmscited by 11
  30. 2024Wavelet Latent Diffusion (Wala): Billion-Parameter 3D Generative Model with Compact Wavelet Encodingscited by 10
  31. 2025DiffuMatch: Category-Agnostic Spectral Diffusion Priors for Robust Non-rigid Shape Matchingcited by 10
  32. 2021Class-agnostic Reconstruction of Dynamic Objects from Videoscited by 8
  33. 2023Self-supervised Learning of Implicit Shape Representation with Dense Correspondence for Deformable Objectscited by 8
  34. 2023Revisiting Map Relations for Unsupervised Non-Rigid Shape Matchingcited by 8
  35. 2024DynoSurf: Neural Deformation-based Temporally Consistent Dynamic Surface Reconstructioncited by 8
  36. 2024Reconstruction of Manipulated Garment with Guided Deformation Priorcited by 8
  37. 2024SEED4D: A Synthetic Ego--Exo Dynamic 4D Data Generator, Driving Dataset and Benchmarkcited by 8
  38. 2024DNF: Unconditional 4D Generation with Dictionary-based Neural Fieldscited by 8

50 most cited of 123 All 123 (JSON)