# Meta Episodic learning with Dynamic Task Sampling for CLIP-based Point Cloud Classification

arXiv: https://arxiv.org/abs/2404.00857 · HTML: https://mlsanity.com/paper/2404.00857/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 24 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 19 of 19)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data](https://mlsanity.com/paper/1908.04616/) (2019)
- [PointCLIP: Point Cloud Understanding by CLIP](https://mlsanity.com/paper/2112.02413/) (2021)
- [Meta-SGD: Learning to Learn Quickly for Few-Shot Learning](https://mlsanity.com/paper/1707.09835/) (2017)
- [Relation-Shape Convolutional Neural Network for Point Cloud Analysis](https://mlsanity.com/paper/1904.07601/) (2019)
- [How to train your MAML](https://mlsanity.com/paper/1810.09502/) (2018)
- [Walk in the Cloud: Learning Curves for Point Clouds Shape Analysis](https://mlsanity.com/paper/2105.01288/) (2021)
- [CLIP2Point: Transfer CLIP to Point Cloud Classification with Image-Depth Pre-training](https://mlsanity.com/paper/2210.01055/) (2022)
- [Generating Representative Samples for Few-Shot Classification](https://mlsanity.com/paper/2205.02918/) (2022)
- [Let Images Give You More:Point Cloud Cross-Modal Training for Shape Analysis](https://mlsanity.com/paper/2210.04208/) (2022)
- [Alleviating the Sample Selection Bias in Few-shot Learning by Removing Projection to the Centroid](https://mlsanity.com/paper/2210.16834/) (2022)
- [Orientation-boosted Voxel Nets for 3D Object Recognition](https://mlsanity.com/paper/1604.03351/) (2016)
- [POODLE: Improving Few-shot Learning via Penalizing Out-of-Distribution Samples](https://mlsanity.com/paper/2206.04679/) (2022)
- [Uniform Sampling over Episode Difficulty](https://mlsanity.com/paper/2108.01662/) (2021)

## Abstract (arXiv)

Point cloud classification refers to the process of assigning semantic labels or categories to individual points within a point cloud data structure. Recent works have explored the extension of pre-trained CLIP to 3D recognition. In this direction, CLIP-based point cloud models like PointCLIP, CLIP2Point have become state-of-the-art methods in the few-shot setup. Although these methods show promising performance for some classes like airplanes, desks, guitars, etc, the performance for some classes like the cup, flower pot, sink, nightstand, etc is still far from satisfactory. This is due to the fact that the adapter of CLIP-based models is trained using randomly sampled N-way K-shot data in the standard supervised learning setup. In this paper, we propose a novel meta-episodic learning framework for CLIP-based point cloud classification, addressing the challenges of limited training examples and sampling unknown classes. Additionally, we introduce dynamic task sampling within the episode based on performance memory. This sampling strategy effectively addresses the challenge of sampling unknown classes, ensuring that the model learns from a diverse range of classes and promotes the exploration of underrepresented categories. By dynamically updating the performance memory, we adaptively prioritize the sampling of classes based on their performance, enhancing the model's ability to handle challenging and real-world scenarios. Experiments show an average performance gain of 3-6% on ModelNet40 and ScanobjectNN datasets in a few-shot setup.