# Weakly-Supervised Dense Action Anticipation

arXiv: https://arxiv.org/abs/2111.07593 · HTML: https://mlsanity.com/paper/2111.07593/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 50 references that are in the map.
- First posted on arXiv 15 Nov 2021.

## Builds on (most cited first, 27 of 27)

- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [MixMatch: A Holistic Approach to Semi-Supervised Learning](https://mlsanity.com/paper/1905.02249/) (2019)
- [Temporal Ensembling for Semi-Supervised Learning](https://mlsanity.com/paper/1610.02242/) (2016)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [Pseudo-Labeling and Confirmation Bias in Deep Semi-Supervised Learning](https://mlsanity.com/paper/1908.02983/) (2019)
- [Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results](https://mlsanity.com/paper/1703.01780/) (2017)
- [Label Propagation for Deep Semi-supervised Learning](https://mlsanity.com/paper/1904.04717/) (2019)
- [Weakly Supervised Learning of Instance Segmentation with Inter-pixel Relations](https://mlsanity.com/paper/1904.05044/) (2019)
- [Weakly-Supervised Semantic Segmentation via Sub-category Exploration](https://mlsanity.com/paper/2008.01183/) (2020)
- [Anti-Adversarially Manipulated Attributions for Weakly and Semi-Supervised Semantic Segmentation](https://mlsanity.com/paper/2103.08896/) (2021)
- [Rolling-Unrolling LSTMs for Action Anticipation from First-Person Video](https://mlsanity.com/paper/2005.02190/) (2020)
- [When will you do what? - Anticipating Temporal Occurrences of Activities](https://mlsanity.com/paper/1804.00892/) (2018)
- [Non-Salient Region Object Mining for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2103.14581/) (2021)
- [Weakly-Supervised Neural Text Classification](https://mlsanity.com/paper/1809.01478/) (2018)
- [Weakly Supervised Complementary Parts Models for Fine-Grained Image Classification from the Bottom Up](https://mlsanity.com/paper/1903.02827/) (2019)
- [SF-Net: Single-Frame Supervision for Temporal Action Localization](https://mlsanity.com/paper/2003.06845/) (2020)
- [Temporal Action Segmentation from Timestamp Supervision](https://mlsanity.com/paper/2103.06669/) (2021)
- [Forecasting Action through Contact Representations from First Person Video](https://mlsanity.com/paper/2102.00649/) (2021)
- [Action Recognition from Single Timestamp Supervision in Untrimmed Videos](https://mlsanity.com/paper/1904.04689/) (2019)
- [Predicting Human Activities Using Stochastic Grammar](https://mlsanity.com/paper/1708.00945/) (2017)
- [Adaptive Affinity Loss and Erroneous Pseudo-Label Refinement for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2108.01344/) (2021)
- [Forecasting future action sequences with attention: a new approach to weakly supervised action forecasting](https://mlsanity.com/paper/1912.04608/) (2019)
- [Semantic-Transferable Weakly-Supervised Endoscopic Lesions Segmentation](https://mlsanity.com/paper/1908.07669/) (2019)
- [TTPP: Temporal Transformer with Progressive Prediction for Efficient Action Anticipation](https://mlsanity.com/paper/2003.03530/) (2020)
- [Anticipation and next action forecasting in video: an end-to-end model with memory](https://mlsanity.com/paper/1901.03728/) (2019)
- [Action Anticipation for Collaborative Environments: The Impact of Contextual Information and Uncertainty-Based Prediction](https://mlsanity.com/paper/1910.00714/) (2019)

## Built on by (most cited first, 3 of 3)

- [Rethinking Learning Approaches for Long-Term Action Anticipation](https://mlsanity.com/paper/2210.11566/) (2022)
- [A Survey on Deep Learning Techniques for Action Anticipation](https://mlsanity.com/paper/2309.17257/) (2023)
- [SuPRA: Surgical Phase Recognition and Anticipation for Intra-Operative Planning](https://mlsanity.com/paper/2403.06200/) (2024)

## Abstract (arXiv)

Dense anticipation aims to forecast future actions and their durations for long horizons. Existing approaches rely on fully-labelled data, i.e. sequences labelled with all future actions and their durations. We present a (semi-) weakly supervised method using only a small number of fully-labelled sequences and predominantly sequences in which only the (one) upcoming action is labelled. To this end, we propose a framework that generates pseudo-labels for future actions and their durations and adaptively refines them through a refinement module. Given only the upcoming action label as input, these pseudo-labels guide action/duration prediction for the future. We further design an attention mechanism to predict context-aware durations. Experiments on the Breakfast and 50Salads benchmarks verify our method's effectiveness; we are competitive even when compared to fully supervised state-of-the-art models. We will make our code available at: https://github.com/zhanghaotong1/WSLVideoDenseAnticipation.