# IDEA-Net: Dynamic 3D Point Cloud Interpolation via Deep Embedding Alignment

arXiv: https://arxiv.org/abs/2203.11590 · HTML: https://mlsanity.com/paper/2203.11590/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 42 references that are in the map.
- First posted on arXiv 22 Mar 2022.

## Builds on (most cited first, 25 of 25)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [Video Enhancement with Task-Oriented Flow](https://mlsanity.com/paper/1711.09078/) (2017)
- [Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation](https://mlsanity.com/paper/1712.00080/) (2017)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [Depth-Aware Video Frame Interpolation](https://mlsanity.com/paper/1904.00830/) (2019)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [Video Frame Interpolation via Adaptive Separable Convolution](https://mlsanity.com/paper/1708.01692/) (2017)
- [Video Frame Synthesis using Deep Voxel Flow](https://mlsanity.com/paper/1702.02463/) (2017)
- [3D-CODED : 3D Correspondences by Deep Deformation](https://mlsanity.com/paper/1806.05228/) (2018)
- [Video Frame Interpolation via Adaptive Convolution](https://mlsanity.com/paper/1703.07514/) (2017)
- [Context-aware Synthesis for Video Frame Interpolation](https://mlsanity.com/paper/1803.10967/) (2018)
- [MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequences](https://mlsanity.com/paper/1910.09165/) (2019)
- [FLAVR: Flow-Agnostic Video Representations for Fast Frame Interpolation](https://mlsanity.com/paper/2012.08512/) (2020)
- [NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go](https://mlsanity.com/paper/2106.09431/) (2021)
- [PhaseNet for Video Frame Interpolation](https://mlsanity.com/paper/1804.00884/) (2018)
- [CaSPR: Learning Canonical Spatiotemporal Point Cloud Representations](https://mlsanity.com/paper/2008.02792/) (2020)
- [PointRNN: Point Recurrent Neural Network for Moving Point Cloud Processing](https://mlsanity.com/paper/1910.08287/) (2019)
- [PointINet: Point Cloud Frame Interpolation Network](https://mlsanity.com/paper/2012.10066/) (2020)
- [Intrinsic Point Cloud Interpolation via Dual Latent Space Navigation](https://mlsanity.com/paper/2004.01661/) (2020)
- [Tranquil Clouds: Neural Networks for Learning Temporally Coherent Features in Point Clouds](https://mlsanity.com/paper/1907.05279/) (2019)

## Built on by (most cited first, 9 of 9)

- [Recent Trends in 3D Reconstruction of General Non-Rigid Scenes](https://mlsanity.com/paper/2403.15064/) (2024)
- [NeuralPCI: Spatio-temporal Neural Field for 3D Point Cloud Multi-frame Non-linear Interpolation](https://mlsanity.com/paper/2303.15126/) (2023)
- [Dynamic 3D Point Cloud Sequences as 2D Videos](https://mlsanity.com/paper/2403.01129/) (2024)
- [CanFields: Consolidating Diffeomorphic Flows for Non-Rigid 4D Interpolation from Arbitrary-Length Sequences](https://mlsanity.com/paper/2406.18582/) (2024)
- [PAPR in Motion: Seamless Point-level 3D Scene Interpolation](https://mlsanity.com/paper/2406.05533/) (2024)
- [FastPCI: Motion-Structure Guided Fast Point Cloud Frame Interpolation](https://mlsanity.com/paper/2410.19573/) (2024)
- [Advances in 4D Representation: Geometry, Motion, and Interaction](https://mlsanity.com/paper/2510.19255/) (2025)
- [NeuroGauss4D-PCI: 4D Neural Fields and Gaussian Deformation Fields for Point Cloud Interpolation](https://mlsanity.com/paper/2405.14241/) (2024)
- [UniRED: Unified RGB-D Video Frame Interpolation with Event Guidance](https://mlsanity.com/paper/2606.24282/) (2026)

## Abstract (arXiv)

This paper investigates the problem of temporally interpolating dynamic 3D point clouds with large non-rigid deformation. We formulate the problem as estimation of point-wise trajectories (i.e., smooth curves) and further reason that temporal irregularity and under-sampling are two major challenges. To tackle the challenges, we propose IDEA-Net, an end-to-end deep learning framework, which disentangles the problem under the assistance of the explicitly learned temporal consistency. Specifically, we propose a temporal consistency learning module to align two consecutive point cloud frames point-wisely, based on which we can employ linear interpolation to obtain coarse trajectories/in-between frames. To compensate the high-order nonlinear components of trajectories, we apply aligned feature embeddings that encode local geometry properties to regress point-wise increments, which are combined with the coarse estimations. We demonstrate the effectiveness of our method on various point cloud sequences and observe large improvement over state-of-the-art methods both quantitatively and visually. Our framework can bring benefits to 3D motion data acquisition. The source code is publicly available at https://github.com/ZENGYIMING-EAMON/IDEA-Net.git.