# AGAR: Attention Graph-RNN for Adaptative Motion Prediction of Point Clouds of Deformable Objects

arXiv: https://arxiv.org/abs/2307.09936 · HTML: https://mlsanity.com/paper/2307.09936/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 52 references that are in the map.
- First posted on arXiv 19 Jul 2023.

## Builds on (most cited first, 21 of 21)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [A Comprehensive Survey on Graph Neural Networks](https://mlsanity.com/paper/1901.00596/) (2019)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Unsupervised Learning of Video Representations using LSTMs](https://mlsanity.com/paper/1502.04681/) (2015)
- [FlowNet3D: Learning Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/1806.01411/) (2018)
- [MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequences](https://mlsanity.com/paper/1910.09165/) (2019)
- [Just Go with the Flow: Self-Supervised Scene Flow Estimation](https://mlsanity.com/paper/1912.00497/) (2019)
- [PV-RAFT: Point-Voxel Correlation Fields for Scene Flow Estimation of Point Clouds](https://mlsanity.com/paper/2012.00987/) (2020)
- [FlowStep3D: Model Unrolling for Self-Supervised Scene Flow Estimation](https://mlsanity.com/paper/2011.10147/) (2020)
- [PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences](https://mlsanity.com/paper/2205.13713/) (2022)
- [FlowNet3D++: Geometric Losses For Deep Scene Flow Estimation](https://mlsanity.com/paper/1912.01438/) (2019)
- [Hierarchical Attention Learning of Scene Flow in 3D Point Clouds](https://mlsanity.com/paper/2010.05762/) (2020)
- [PointRNN: Point Recurrent Neural Network for Moving Point Cloud Processing](https://mlsanity.com/paper/1910.08287/) (2019)
- [FESTA: Flow Estimation via Spatial-Temporal Attention for Scene Point Clouds](https://mlsanity.com/paper/2104.00798/) (2021)
- [CloudLSTM: A Recurrent Neural Model for Spatiotemporal Point-cloud Stream Forecasting](https://mlsanity.com/paper/1907.12410/) (2019)
- [MoNet: Motion-based Point Cloud Prediction Network](https://mlsanity.com/paper/2011.10812/) (2020)
- [AGConv: Adaptive Graph Convolution on 3D Point Clouds](https://mlsanity.com/paper/2206.04665/) (2022)
- [Attention Models for Point Clouds in Deep Learning: A Survey](https://mlsanity.com/paper/2102.10788/) (2021)
- [Spatio-temporal Graph-RNN for Point Cloud Prediction](https://mlsanity.com/paper/2102.07482/) (2021)
- [Explaining Hierarchical Features in Dynamic Point Cloud Processing](https://mlsanity.com/paper/2209.15557/) (2022)

## Abstract (arXiv)

This paper focuses on motion prediction for point cloud sequences in the challenging case of deformable 3D objects, such as human body motion. First, we investigate the challenges caused by deformable shapes and complex motions present in this type of representation, with the ultimate goal of understanding the technical limitations of state-of-the-art models. From this understanding, we propose an improved architecture for point cloud prediction of deformable 3D objects. Specifically, to handle deformable shapes, we propose a graph-based approach that learns and exploits the spatial structure of point clouds to extract more representative features. Then we propose a module able to combine the learned features in an adaptative manner according to the point cloud movements. The proposed adaptative module controls the composition of local and global motions for each point, enabling the network to model complex motions in deformable 3D objects more effectively. We tested the proposed method on the following datasets: MNIST moving digits, the Mixamo human bodies motions, JPEG and CWIPC-SXR real-world dynamic bodies. Simulation results demonstrate that our method outperforms the current baseline methods given its improved ability to model complex movements as well as preserve point cloud shape. Furthermore, we demonstrate the generalizability of the proposed framework for dynamic feature learning, by testing the framework for action recognition on the MSRAction3D dataset and achieving results on-par with state-of-the-art methods