# Instance-aware Dynamic Prompt Tuning for Pre-trained Point Cloud Models

arXiv: https://arxiv.org/abs/2304.07221 · HTML: https://mlsanity.com/paper/2304.07221/

## Key facts

- Cited by 68 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 58 references that are in the map.
- First posted on arXiv 14 Apr 2023.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [Learning to Prompt for Vision-Language Models](https://mlsanity.com/paper/2109.01134/) (2021)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Unsupervised Representation Learning by Predicting Image Rotations](https://mlsanity.com/paper/1803.07728/) (2018)
- [Conditional Prompt Learning for Vision-Language Models](https://mlsanity.com/paper/2203.05557/) (2022)
- [Making Pre-trained Language Models Better Few-shot Learners](https://mlsanity.com/paper/2012.15723/) (2020)
- [PointCNN: Convolution On X-Transformed Points](https://mlsanity.com/paper/1801.07791/) (2018)
- [PCT: Point cloud transformer](https://mlsanity.com/paper/2012.09688/) (2020)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [GPT Understands, Too](https://mlsanity.com/paper/2103.10385/) (2021)
- [P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Tasks](https://mlsanity.com/paper/2110.07602/) (2021)
- [Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning](https://mlsanity.com/paper/2012.13255/) (2020)
- [Multimodal Few-Shot Learning with Frozen Language Models](https://mlsanity.com/paper/2106.13884/) (2021)
- [Point-BERT: Pre-training 3D Point Cloud Transformers with Masked Point Modeling](https://mlsanity.com/paper/2111.14819/) (2021)
- [Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data](https://mlsanity.com/paper/1908.04616/) (2019)
- [DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting](https://mlsanity.com/paper/2112.01518/) (2021)
- [PointContrast: Unsupervised Pre-training for 3D Point Cloud Understanding](https://mlsanity.com/paper/2007.10985/) (2020)

## Built on by (most cited first, 30 of 68)

- [MambaIR: A Simple Baseline for Image Restoration with State-Space Model](https://mlsanity.com/paper/2402.15648/) (2024)
- [PointMamba: A Simple State Space Model for Point Cloud Analysis](https://mlsanity.com/paper/2402.10739/) (2024)
- [AdaCLIP: Adapting CLIP with Hybrid Learnable Prompts for Zero-Shot Anomaly Detection](https://mlsanity.com/paper/2407.15795/) (2024)
- [BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP](https://mlsanity.com/paper/2311.16194/) (2023)
- [Mamba3D: Enhancing Local Features for 3D Point Cloud Analysis via State Space Model](https://mlsanity.com/paper/2404.14966/) (2024)
- [Dynamic Adapter Meets Prompt Tuning: Parameter-Efficient Transfer Learning for Point Cloud Analysis](https://mlsanity.com/paper/2403.01439/) (2024)
- [Parameter-Efficient Fine-Tuning for Foundation Models](https://mlsanity.com/paper/2501.13787/) (2025)
- [Parameter-Efficient Fine-Tuning in Spectral Domain for Point Cloud Learning](https://mlsanity.com/paper/2410.08114/) (2024)
- [Point-PEFT: Parameter-Efficient Fine-Tuning for 3D Pre-trained Models](https://mlsanity.com/paper/2310.03059/) (2023)
- [Towards Compact 3D Representations via Point Feature Enhancement Masked Autoencoders](https://mlsanity.com/paper/2312.10726/) (2023)
- [Domain-Agnostic Mutual Prompting for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2403.02899/) (2024)
- [Vision-Language Pre-training with Object Contrastive Learning for 3D Scene Understanding](https://mlsanity.com/paper/2305.10714/) (2023)
- [Parameter-Efficient Fine-Tuning for Pre-Trained Vision Models: A Survey and Benchmark](https://mlsanity.com/paper/2402.02242/) (2024)
- [Prompt-based Adaptation in Large-scale Vision Models: A Survey](https://mlsanity.com/paper/2510.13219/) (2025)
- [A Survey of Label-Efficient Deep Learning for 3D Point Clouds](https://mlsanity.com/paper/2305.19812/) (2023)
- [FreestyleRet: Retrieving Images from Style-Diversified Queries](https://mlsanity.com/paper/2312.02428/) (2023)
- [CLIP-Guided Generative Networks for Transferable Targeted Adversarial Attacks](https://mlsanity.com/paper/2407.10179/) (2024)
- [BoostAdapter: Improving Vision-Language Test-Time Adaptation via Regional Bootstrapping](https://mlsanity.com/paper/2410.15430/) (2024)
- [Align then Adapt: Rethinking Parameter-Efficient Transfer Learning in 4D Perception](https://mlsanity.com/paper/2602.23069/) (2026)
- [Scene Graph Generation with Role-Playing Large Language Models](https://mlsanity.com/paper/2410.15364/) (2024)
- [Positional Prompt Tuning for Efficient 3D Representation Learning](https://mlsanity.com/paper/2408.11567/) (2024)
- [PointLoRA: Low-Rank Adaptation with Token Selection for Point Cloud Learning](https://mlsanity.com/paper/2504.16023/) (2025)
- [Adapter-X: A Novel General Parameter-Efficient Fine-Tuning Framework for Vision](https://mlsanity.com/paper/2406.03051/) (2024)
- [GAPrompt: Geometry-Aware Point Cloud Prompt for 3D Vision Model](https://mlsanity.com/paper/2505.04119/) (2025)
- [LCM: Locally Constrained Compact Point Cloud Model for Masked Point Modeling](https://mlsanity.com/paper/2405.17149/) (2024)
- [MAP: Unleashing Hybrid Mamba-Transformer Vision Backbone's Potential with Masked Autoregressive Pretraining](https://mlsanity.com/paper/2410.00871/) (2024)
- [Large Point-to-Gaussian Model for Image-to-3D Generation](https://mlsanity.com/paper/2408.10935/) (2024)
- [Label-Efficient LiDAR Semantic Segmentation with 2D-3D Vision Transformer Adapters](https://mlsanity.com/paper/2503.03299/) (2025)
- [PMA: Towards Parameter-Efficient Point Cloud Understanding via Point Mamba Adapter](https://mlsanity.com/paper/2505.20941/) (2025)
- [Point-PRC: A Prompt Learning Based Regulation Framework for Generalizable Point Cloud Analysis](https://mlsanity.com/paper/2410.20406/) (2024)

## Abstract (arXiv)

Pre-trained point cloud models have found extensive applications in 3D understanding tasks like object classification and part segmentation. However, the prevailing strategy of full fine-tuning in downstream tasks leads to large per-task storage overhead for model parameters, which limits the efficiency when applying large-scale pre-trained models. Inspired by the recent success of visual prompt tuning (VPT), this paper attempts to explore prompt tuning on pre-trained point cloud models, to pursue an elegant balance between performance and parameter efficiency. We find while instance-agnostic static prompting, e.g. VPT, shows some efficacy in downstream transfer, it is vulnerable to the distribution diversity caused by various types of noises in real-world point cloud data. To conquer this limitation, we propose a novel Instance-aware Dynamic Prompt Tuning (IDPT) strategy for pre-trained point cloud models. The essence of IDPT is to develop a dynamic prompt generation module to perceive semantic prior features of each point cloud instance and generate adaptive prompt tokens to enhance the model's robustness. Notably, extensive experiments demonstrate that IDPT outperforms full fine-tuning in most tasks with a mere 7% of the trainable parameters, providing a promising solution to parameter-efficient learning for pre-trained point cloud models. Code is available at urlhttps://github.com/zyh16143998882/ICCV23-IDPT.