PLOT-TAL: Prompt Learning with Optimal Transport for Few-Shot Temporal Action Localization
Not on the map. Why?
Abstract (arXiv)
Few-shot temporal action localization (TAL) methods that adapt large models via single-prompt tuning often fail to produce precise temporal boundaries. This stems from the model learning a non-discriminative mean representation of an action from sparse data, which compromises generalization. We address this by proposing a new paradigm based on multi-prompt ensembles, where a set of diverse, learnable prompts for each action is encouraged to specialize on compositional sub-events. To enforce this specialization, we introduce PLOT-TAL, a framework that leverages Optimal Transport (OT) to find a globally optimal alignment between the prompt ensemble and the video's temporal features. Our method establishes a new state-of-the-art on the challenging few-shot benchmarks of THUMOS'14 and EPIC-Kitchens, without requiring complex meta-learning. The significant performance gains, particularly at high IoU thresholds, validate our hypothesis and demonstrate the superiority of learning distributed, compositional representations for precise temporal localization.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 29 of its 35 references that are in the map. First posted on arXiv 27 Mar 2024.
Builds on
29 of 35 references.
References · 29
- 2017Attention Is All You Needcited by 60,361
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,852
- 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
- 2015SSD: Single Shot MultiBox Detectorcited by 3,130
- 2015Learning Deep Features for Discriminative Localizationcited by 2,436
- 2013Sinkhorn Distances: Lightspeed Computation of Optimal Transportation Distancescited by 2,316
- 2021Learning to Prompt for Vision-Language Modelscited by 2,281
- 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
- 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 795
- 2019Distance-IoU Loss: Faster and Better Learning for Bounding Box Regressioncited by 459
17 more
- 2019BMN: Boundary-Matching Network for Temporal Action Proposal Generationcited by 329
- 2022ActionFormer: Localizing Moments of Actions with Transformerscited by 311
- 2021Prompting Visual-Language Models for Efficient Video Understandingcited by 310
- 2018BSN: Boundary Sensitive Network for Temporal Action Proposal Generationcited by 271
- 2021Align and Prompt: Video-and-Language Pre-training with Entity Promptscited by 149
- 2023TriDet: Temporal Action Detection with Relative Boundary Modelingcited by 142
- 2017UntrimmedNets for Weakly Supervised Action Recognition and Detectioncited by 130
- 2022TALLFormer: Temporal Action Localization with a Long-memory Transformercited by 79
- 2022Zero-Shot Temporal Action Detection via Vision-Language Promptingcited by 65
- 2022PLOT: Prompt Learning with Optimal Transport for Vision-Language Modelscited by 64
- 2021A Survey on Optimal Transport for Machine Learning: Theory and Applicationscited by 35
- 2022An Efficient Spatio-Temporal Pyramid Transformer for Action Detectioncited by 16
- 2021Few-Shot Temporal Action Localization with Query Adaptive Transformercited by 15
- 2023Temporal Action Localization with Enhanced Instant Discriminabilitycited by 13
- 2020Localizing the Common Action Among a Few Videoscited by 11
- 2020Revisiting Few-shot Activity Detection with Class Similarity Controlcited by 4
- 2022Multi-Modal Few-Shot Temporal Action Detectioncited by 3
Built on by
Citing papers per year: 2025: 2; 2026: 1.
3 papers.
- 2026CodecSight: Leveraging Video Codec Signals for Efficient Streaming VLM Inferencecited by 5
- 2025Chain-of-Evidence Multimodal Reasoning for Few-shot Temporal Action Localizationcited by 1
- 2025Reframing Dense Action Detection (RefDense): A Paradigm Shift in Problem Solving & a Novel Optimization Strategycited by 0
- 2026CodecSight: Leveraging Video Codec Signals for Efficient Streaming VLM Inference7 Apr 2026
- 2025Chain-of-Evidence Multimodal Reasoning for Few-shot Temporal Action Localization18 Apr 2025
- 2025Reframing Dense Action Detection (RefDense): A Paradigm Shift in Problem Solving & a Novel Optimization Strategy30 Jan 2025