mlsanityData to 7 Oct 2026

Class-agnostic Reconstruction of Dynamic Objects from Videos

arXiv 2112.02091 ↗First posted 3 Dec 2021cs.CV cs.AI cs.GRcited by 8

Not on the map. Why?

Abstract (arXiv)

We introduce REDO, a class-agnostic framework to REconstruct the Dynamic Objects from RGBD or calibrated videos. Compared to prior work, our problem setting is more realistic yet more challenging for three reasons: 1) due to occlusion or camera settings an object of interest may never be entirely visible, but we aim to reconstruct the complete shape; 2) we aim to handle different object dynamics including rigid motion, non-rigid motion, and articulation; 3) we aim to reconstruct different categories of objects with one unified framework. To address these challenges, we develop two novel modules. First, we introduce a canonical 4D implicit function which is pixel-aligned with aggregated temporal visual cues. Second, we develop a 4D transformation module which captures object dynamics to support temporal propagation and aggregation. We study the efficacy of REDO in extensive experiments on synthetic RGBD video datasets SAIL-VOS 3D and DeformingThings4D++, and on real-world video data 3DPW. We find REDO outperforms state-of-the-art dynamic reconstruction methods by a margin. In ablation studies we validate each developed component.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 53 of its 98 references that are in the map. First posted on arXiv 3 Dec 2021.

Builds on

53 of 98 references.

References · 53

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2017Mask R-CNNcited by 7,259
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  6. 2018Neural Ordinary Differential Equationscited by 3,733
  7. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  8. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  9. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  10. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  11. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  12. 2018Learning Implicit Fields for Generative Shape Modelingcited by 1,060
38 more
  1. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  2. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  3. 2017PWC-Net: CNNs for Optical Flow Using Pyramid, Warping, and Cost Volumecited by 966
  4. 2016Stacked Hourglass Networks for Human Pose Estimationcited by 913
  5. 2017End-to-end Recovery of Human Shape and Posecited by 883
  6. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  7. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  8. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  9. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  10. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  11. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  12. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  13. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  14. 2017Neural 3D Mesh Renderercited by 437
  15. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  16. 2016OctNet: Learning Deep 3D Representations at High Resolutionscited by 340
  17. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  18. 2018Learning 3D Human Dynamics from Videocited by 315
  19. 2020PolyGen: An Autoregressive Generative Model of 3D Meshescited by 267
  20. 2018FlowNet3D: Learning Scene Flow in 3D Point Cloudscited by 241
  21. 2016Volumetric and Multi-View CNNs for Object Classification on 3D Datacited by 240
  22. 2018Total Capture: A 3D Deformation Model for Tracking Faces, Hands, and Bodiescited by 239
  23. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  24. 2020Consistent Video Depth Estimationcited by 220
  25. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  26. 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
  27. 20214DComplete: Non-Rigid Motion Estimation Beyond the Observable Surfacecited by 123
  28. 2021LASR: Learning Articulated Shape Reconstruction from a Monocular Videocited by 115
  29. 2015Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Imagescited by 111
  30. 2019Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"cited by 106
  31. 2018DoubleFusion: Real-time Capture of Human Performances with Inner Body Shapes from a Single Depth Sensorcited by 95
  32. 2018LiveCap: Real-time Human Performance Capture from Monocular Videocited by 93
  33. 2019DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Datacited by 76
  34. 2016VolumeDeform: Real-time Volumetric Non-rigid Reconstructioncited by 75
  35. 2019Predicting 3D Human Dynamics from Videocited by 51
  36. 2020Neural Non-Rigid Trackingcited by 49
  37. 2021SAIL-VOS 3D: A Synthetic Dataset and Baselines for Object Detection and 3D Mesh Reconstruction from Video Datacited by 38
  38. 2019SurfelWarp: Efficient Non-Volumetric Single View Dynamic Reconstructioncited by 31

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 3; 2025: 0; 2026: 1.

8 papers.

  1. 2023Consistent4D: Consistent 360deg Dynamic Object Generation from Monocular Videocited by 124
  2. 2024GoMAvatar: Efficient Animatable Human Modeling from Monocular Video Using Gaussians-on-Meshcited by 51
  3. 2022CASA: Category-agnostic Skeletal Animal Reconstructioncited by 35
  4. 2024Recent Trends in 3D Reconstruction of General Non-Rigid Scenescited by 28
  5. 2023Total-Recon: Deformable Scene Reconstruction for Embodied View Synthesiscited by 20
  6. 2022Occupancy Planes for Single-view RGB-D Human Reconstructioncited by 4
  7. 2026Kirin: Animal Motion Generation from In-the-Wild Videocited by 1
  8. 2024ReMatching Dynamic Reconstruction Flowcited by 0