# Joint-Embedding Prediction of Masked Point Tubes for Self-Supervised Learning on 4D Point Cloud Videos

arXiv: https://arxiv.org/abs/2608.24093 · HTML: https://mlsanity.com/paper/2608.24093/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 34 references that are in the map.
- First posted on arXiv 25 Aug 2026.

## Builds on (most cited first, 21 of 21)

- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis](https://mlsanity.com/paper/1604.02808/) (2016)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)
- [LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels](https://mlsanity.com/paper/2603.19312/) (2026)
- [LeJEPA: Provable and Scalable Self-Supervised Learning Without the Heuristics](https://mlsanity.com/paper/2511.08544/) (2025)
- [MeteorNet: Deep Learning on Dynamic 3D Point Cloud Sequences](https://mlsanity.com/paper/1910.09165/) (2019)
- [PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences](https://mlsanity.com/paper/2205.13713/) (2022)
- [3DV: 3D Dynamic Voxel for Action Recognition in Depth Video](https://mlsanity.com/paper/2005.05501/) (2020)
- [Point Primitive Transformer for Long-Term 4D Point Cloud Video Understanding](https://mlsanity.com/paper/2208.00281/) (2022)
- [Complete-to-Partial 4D Distillation for Self-Supervised Point Cloud Sequence Representation Learning](https://mlsanity.com/paper/2212.05330/) (2022)
- [Point-JEPA: A Joint Embedding Predictive Architecture for Self-Supervised Learning on Point Cloud](https://mlsanity.com/paper/2404.16432/) (2024)
- [No Pain, Big Gain: Classify Dynamic Point Cloud Sequences with Static Models by Fitting Feature-level Space-time Surfaces](https://mlsanity.com/paper/2203.11113/) (2022)
- [PointCMP: Contrastive Mask Prediction for Self-supervised Learning on Point Cloud Videos](https://mlsanity.com/paper/2305.04075/) (2023)
- [Masked Spatio-Temporal Structure Prediction for Self-supervised Learning on Point Cloud Videos](https://mlsanity.com/paper/2308.09245/) (2023)
- [Point Contrastive Prediction with Semantic Clustering for Self-Supervised Learning on Point Cloud Videos](https://mlsanity.com/paper/2308.09247/) (2023)
- [Diffusion Masked Pretraining for Dynamic Point Cloud](https://mlsanity.com/paper/2605.03639/) (2026)
- [UST-SSM: Unified Spatio-Temporal State Space Models for Point Cloud Video Modeling](https://mlsanity.com/paper/2508.14604/) (2025)
- [STS-Mixer: Spatio-Temporal-Spectral Mixer for 4D Point Cloud Video Understanding](https://mlsanity.com/paper/2604.11637/) (2026)
- [SRENet: Spectral Re-Entry Network for Point Cloud Action Recognition](https://mlsanity.com/paper/2606.03160/) (2026)
- [Cross4D-JEPA: Dense Cross-modal Correspondence Distillation for 4D Point Cloud Representation Learning](https://mlsanity.com/paper/2607.00514/) (2026)

## Abstract (arXiv)

Self-supervised representation learning for 4D point cloud videos is challenging because annotations are costly and reconstruction-based pretraining can overemphasize low-level geometric details. We propose a JEPA-style framework that learns from unlabeled spatiotemporal point clouds through latent point-tube prediction. Instead of reconstructing raw coordinates, the model masks spatiotemporal regions and predicts their target representations from visible context representations in feature space. To stabilize latent prediction, we incorporate Sketched Isotropic Gaussian Regularization, which encourages non-collapsed embeddings without relying on explicit reconstruction targets. This formulation aims to capture both spatial structure and temporal dynamics while keeping the pretraining objective aligned with downstream semantic recognition. Experiments on action and gesture recognition benchmarks show that the learned representations improve downstream fine-tuning, limited-label learning, and cross-dataset transfer. These results suggest that JEPA-style latent prediction is a promising alternative to reconstruction-centered pretraining for 4D point cloud videos.