# PointRNN: Point Recurrent Neural Network for Moving Point Cloud Processing

arXiv: https://arxiv.org/abs/1910.08287 · HTML: https://mlsanity.com/paper/1910.08287/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Oct 2019.

## Built on by (most cited first, 30 of 32)

- [PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences](https://mlsanity.com/paper/2205.13713/) (2022)
- [Weakly Supervised Learning of Rigid 3D Scene Flow](https://mlsanity.com/paper/2102.08945/) (2021)
- [Cam4DOcc: Benchmark for Camera-Only 4D Occupancy Forecasting in Autonomous Driving Applications](https://mlsanity.com/paper/2311.17663/) (2023)
- [Scalable Scene Flow from Point Clouds in the Real World](https://mlsanity.com/paper/2103.01306/) (2021)
- [Self-supervised Point Cloud Prediction Using 3D Spatio-temporal Convolutional Networks](https://mlsanity.com/paper/2110.04076/) (2021)
- [Receding Moving Object Segmentation in 3D LiDAR Data Using Sparse 4D Convolutions](https://mlsanity.com/paper/2206.04129/) (2022)
- [No Pain, Big Gain: Classify Dynamic Point Cloud Sequences with Static Models by Fitting Feature-level Space-time Surfaces](https://mlsanity.com/paper/2203.11113/) (2022)
- [Spatial-Temporal Transformer for 3D Point Cloud Sequences](https://mlsanity.com/paper/2110.09783/) (2021)
- [GeoDrive: 3D Geometry-Informed Driving World Model with Precise Action Control](https://mlsanity.com/paper/2505.22421/) (2025)
- [X4D-SceneFormer: Enhanced Scene Understanding on 4D Point Cloud Videos through Cross-modal Knowledge Transfer](https://mlsanity.com/paper/2312.07378/) (2023)
- [PCPNet: An Efficient and Semantic-Enhanced Transformer Network for Point Cloud Prediction](https://mlsanity.com/paper/2304.07773/) (2023)
- [MAMBA4D: Efficient Long-Sequence Point Cloud Video Understanding with Disentangled Spatial-Temporal State Space Models](https://mlsanity.com/paper/2405.14338/) (2024)
- [Learning Scene Dynamics from Point Cloud Sequences](https://mlsanity.com/paper/2111.08755/) (2021)
- [IDEA-Net: Dynamic 3D Point Cloud Interpolation via Deep Embedding Alignment](https://mlsanity.com/paper/2203.11590/) (2022)
- [Sequential Point Clouds: A Survey](https://mlsanity.com/paper/2204.09337/) (2022)
- [Spatiotemporal Decoupling for Efficient Vision-Based Occupancy Forecasting](https://mlsanity.com/paper/2411.14169/) (2024)
- [Spatio-temporal Graph-RNN for Point Cloud Prediction](https://mlsanity.com/paper/2102.07482/) (2021)
- [NSM4D: Neural Scene Model Based Online 4D Point Cloud Sequence Understanding](https://mlsanity.com/paper/2310.08326/) (2023)
- [4DPC2hat: Towards Dynamic Point Cloud Understanding with Failure-Aware Bootstrapping](https://mlsanity.com/paper/2602.03890/) (2026)
- [Deep Learning for 3D Point Cloud Enhancement: A Survey](https://mlsanity.com/paper/2411.00857/) (2024)
- [FFPA-Net: Efficient Feature Fusion with Projection Awareness for 3D Object Detection](https://mlsanity.com/paper/2209.07419/) (2022)
- [TARS: Traffic-Aware Radar Scene Flow Estimation](https://mlsanity.com/paper/2503.10210/) (2025)
- [Uni4D: A Unified Self-Supervised Learning Framework for Point Cloud Videos](https://mlsanity.com/paper/2504.04837/) (2025)
- [DPMix: Mixture of Depth and Point Cloud Video Experts for 4D Action Segmentation](https://mlsanity.com/paper/2307.16803/) (2023)
- [3D-FCT: Simultaneous 3D Object Detection and Tracking Using Feature Correlation](https://mlsanity.com/paper/2110.02531/) (2021)
- [Flexible Networks for Learning Physical Dynamics of Deformable Objects](https://mlsanity.com/paper/2112.03728/) (2021)
- [Deep learning for 3D Object Detection and Tracking in Autonomous Driving: A Brief Survey](https://mlsanity.com/paper/2311.06043/) (2023)
- [STS-Mixer: Spatio-Temporal-Spectral Mixer for 4D Point Cloud Video Understanding](https://mlsanity.com/paper/2604.11637/) (2026)
- [AGAR: Attention Graph-RNN for Adaptative Motion Prediction of Point Clouds of Deformable Objects](https://mlsanity.com/paper/2307.09936/) (2023)
- [Translution: Unifying Self-attention and Convolution for Adaptive and Relative Modeling](https://mlsanity.com/paper/2510.10060/) (2025)

## Abstract (arXiv)

In this paper, we introduce a Point Recurrent Neural Network (PointRNN) for moving point cloud processing. At each time step, PointRNN takes point coordinates boldsymbolP ∈ ℝn × 3 and point features boldsymbolX ∈ ℝn × d as input (n and d denote the number of points and the number of feature channels, respectively). The state of PointRNN is composed of point coordinates boldsymbolP and point states boldsymbolS ∈ ℝn × d' (d' denotes the number of state channels). Similarly, the output of PointRNN is composed of boldsymbolP and new point features boldsymbolY ∈ ℝn × d'' (d'' denotes the number of new feature channels). Since point clouds are orderless, point features and states from two time steps can not be directly operated. Therefore, a point-based spatiotemporally-local correlation is adopted to aggregate point features and states according to point coordinates. We further propose two variants of PointRNN, i.e., Point Gated Recurrent Unit (PointGRU) and Point Long Short-Term Memory (PointLSTM). We apply PointRNN, PointGRU and PointLSTM to moving point cloud prediction, which aims to predict the future trajectories of points in a set given their history movements. Experimental results show that PointRNN, PointGRU and PointLSTM are able to produce correct predictions on both synthetic and real-world datasets, demonstrating their ability to model point cloud sequences. The code has been released at urlhttps://github.com/hehefan/PointRNN.