# Deep Convolution for Irregularly Sampled Temporal Point Clouds

arXiv: https://arxiv.org/abs/2105.00137 · HTML: https://mlsanity.com/paper/2105.00137/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 25 references that are in the map.
- First posted on arXiv 1 May 2021.

## Builds on (most cited first, 22 of 22)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [4D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networks](https://mlsanity.com/paper/1904.08755/) (2019)
- [KPConv: Flexible and Deformable Convolution for Point Clouds](https://mlsanity.com/paper/1904.08889/) (2019)
- [Spatio-Temporal Graph Convolutional Networks: A Deep Learning Framework for Traffic Forecasting](https://mlsanity.com/paper/1709.04875/) (2017)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [Interaction Networks for Learning about Objects, Relations and Physics](https://mlsanity.com/paper/1612.00222/) (2016)
- [Dynamic Edge-Conditioned Filters in Convolutional Neural Networks on Graphs](https://mlsanity.com/paper/1704.02901/) (2017)
- [StarCraft II: A New Challenge for Reinforcement Learning](https://mlsanity.com/paper/1708.04782/) (2017)
- [SpiderCNN: Deep Learning on Point Sets with Parameterized Convolutional Filters](https://mlsanity.com/paper/1803.11527/) (2018)
- [Transformer Hawkes Process](https://mlsanity.com/paper/2002.09291/) (2020)
- [Submanifold Sparse Convolutional Networks](https://mlsanity.com/paper/1706.01307/) (2017)
- [SPLATNet: Sparse Lattice Networks for Point Cloud Processing](https://mlsanity.com/paper/1802.08275/) (2018)
- [Set Functions for Time Series](https://mlsanity.com/paper/1909.12064/) (2019)
- [Revisiting Spatial-Temporal Similarity: A Deep Learning Framework for Traffic Prediction](https://mlsanity.com/paper/1803.01254/) (2018)
- [Interpolation-Prediction Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/1909.07782/) (2019)
- [Monte Carlo Convolution for Learning on Non-Uniformly Sampled Point Clouds](https://mlsanity.com/paper/1806.01759/) (2018)
- [Learning Temporal Point Processes via Reinforcement Learning](https://mlsanity.com/paper/1811.05016/) (2018)
- [Self-Attentive Hawkes Processes](https://mlsanity.com/paper/1907.07561/) (2019)

## Built on by (most cited first, 1 of 1)

- [Unsupervised Hebbian Learning on Point Sets in StarCraft II](https://mlsanity.com/paper/2207.12323/) (2022)

## Abstract (arXiv)

We consider the problem of modeling the dynamics of continuous spatial-temporal processes represented by irregular samples through both space and time. Such processes occur in sensor networks, citizen science, multi-robot systems, and many others. We propose a new deep model that is able to directly learn and predict over this irregularly sampled data, without voxelization, by leveraging a recent convolutional architecture for static point clouds. The model also easily incorporates the notion of multiple entities in the process. In particular, the model can flexibly answer prediction queries about arbitrary space-time points for different entities regardless of the distribution of the training or test-time data. We present experiments on real-world weather station data and battles between large armies in StarCraft II. The results demonstrate the model's flexibility in answering a variety of query types and demonstrate improved performance and efficiency compared to state-of-the-art baselines.