# Motion-Saliency Complementary Masked Modeling for Point Cloud Video Understanding

arXiv: https://arxiv.org/abs/2608.30279 · HTML: https://mlsanity.com/paper/2608.30279/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 43 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 30 of 31)

- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Training](https://mlsanity.com/paper/2203.12602/) (2022)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)
- [Masked Autoencoders for Point Cloud Self-supervised Learning](https://mlsanity.com/paper/2203.06604/) (2022)
- [HOI4D: A 4D Egocentric Dataset for Category-Level Human-Object Interaction](https://mlsanity.com/paper/2203.01577/) (2022)
- [Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training](https://mlsanity.com/paper/2205.14401/) (2022)
- [Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds](https://mlsanity.com/paper/2109.00179/) (2021)
- [MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequences](https://mlsanity.com/paper/1910.09165/) (2019)
- [PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences](https://mlsanity.com/paper/2205.13713/) (2022)
- [Point Primitive Transformer for Long-Term 4D Point Cloud Video Understanding](https://mlsanity.com/paper/2208.00281/) (2022)
- [Complete-to-Partial 4D Distillation for Self-Supervised Point Cloud Sequence Representation Learning](https://mlsanity.com/paper/2212.05330/) (2022)
- [No Pain, Big Gain: Classify Dynamic Point Cloud Sequences with Static Models by Fitting Feature-level Space-time Surfaces](https://mlsanity.com/paper/2203.11113/) (2022)
- [PointCMP: Contrastive Mask Prediction for Self-supervised Learning on Point Cloud Videos](https://mlsanity.com/paper/2305.04075/) (2023)
- [Chain-of-Thought Compression Should Not Be Blind: V-Skip for Efficient Multimodal Reasoning via Dual-Path Anchoring](https://mlsanity.com/paper/2601.13879/) (2026)
- [Masked Spatio-Temporal Structure Prediction for Self-supervised Learning on Point Cloud Videos](https://mlsanity.com/paper/2308.09245/) (2023)
- [PointCoT: A Multi-modal Benchmark for Explicit 3D Geometric Reasoning](https://mlsanity.com/paper/2602.23945/) (2026)
- [Align then Adapt: Rethinking Parameter-Efficient Transfer Learning in 4D Perception](https://mlsanity.com/paper/2602.23069/) (2026)
- [PointDico: Contrastive 3D Representation Learning Guided by Diffusion Models](https://mlsanity.com/paper/2512.08330/) (2025)
- [PointRFT: Explicit Reinforcement Fine-tuning for Point Cloud Few-shot Learning](https://mlsanity.com/paper/2603.23957/) (2026)
- [CMHANet: A Cross-Modal Hybrid Attention Network for Point Cloud Registration](https://mlsanity.com/paper/2603.12721/) (2026)
- [Mantis: Mamba-native Tuning is Efficient for 3D Point Cloud Foundation Models](https://mlsanity.com/paper/2605.03438/) (2026)
- [Diffusion Masked Pretraining for Dynamic Point Cloud](https://mlsanity.com/paper/2605.03639/) (2026)
- [Tri-Efficient Transfer Learning for Point Cloud Videos](https://mlsanity.com/paper/2606.24175/) (2026)
- [Parameter-Efficient Fine-Tuning for Spiking Point Cloud Models](https://mlsanity.com/paper/2607.29048/) (2026)
- [GaussFusion: Towards Multimodal 3D Gaussian Pretraining](https://mlsanity.com/paper/2607.05906/) (2026)

## Abstract (arXiv)

Point cloud video representation learning is crucial for 3D dynamic scene understanding. In this paper, we propose MoSaiC, a novel Motion-Saliency Complementary masked modeling framework for self-supervised point cloud video representation learning. MoSaiC couples three components: Curriculum Motion-Saliency Masking (CMSM), which guides the masking process toward motion-salient tokens under a curriculum schedule; Normal-Flow Motion (NFM) modeling, which supervises the local rigid rotation of each token in the Lie algebra so(3) as an explicit geometric motion target; and Cross-view Token Consistency Prediction (CTCP), which enforces consistency between two complementary masked views at the token level. Together, these components allow MoSaiC to effectively capture both appearance and motion dynamics. Extensive experiments on multiple downstream tasks, including action recognition, temporal action segmentation, and point-level semantic segmentation, demonstrate the effectiveness of our approach.