# Is there progress in activity progress prediction?

arXiv: https://arxiv.org/abs/2308.05533 · HTML: https://mlsanity.com/paper/2308.05533/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 10 Aug 2023.

## Builds on (most cited first, 23 of 23)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014)
- [EndoNet: A Deep Architecture for Recognition Tasks on Laparoscopic Videos](https://mlsanity.com/paper/1602.03012/) (2016)
- [Unsupervised learning of action classes with continuous temporal embedding](https://mlsanity.com/paper/1904.04189/) (2019)
- [An end-to-end generative framework for video segmentation and recognition](https://mlsanity.com/paper/1509.01947/) (2015)
- [LoViT: Long Video Transformer for Surgical Phase Recognition](https://mlsanity.com/paper/2305.08989/) (2023)
- [Joint Visual-Temporal Embedding for Unsupervised Learning of Actions in Untrimmed Sequences](https://mlsanity.com/paper/2001.11122/) (2020)
- [RSDNet: Learning to Predict Remaining Surgery Duration from Laparoscopic Videos Without Manual Annotations](https://mlsanity.com/paper/1802.03243/) (2018)
- [Less is More: Surgical Phase Recognition with Less Annotations through Self-Supervised Pre-training of CNN-LSTM Networks](https://mlsanity.com/paper/1805.08569/) (2018)
- [Multi-Task Recurrent Neural Network for Surgical Gesture Recognition and Progress Prediction](https://mlsanity.com/paper/2003.04772/) (2020)
- [CataNet: Predicting remaining cataract surgery duration](https://mlsanity.com/paper/2106.11048/) (2021)
- [Am I Done? Predicting Action Progress in Videos](https://mlsanity.com/paper/1705.01781/) (2017)
- [SurgMAE: Masked Autoencoders for Long Surgical Video Analysis](https://mlsanity.com/paper/2305.11451/) (2023)
- [Human Action Forecasting by Learning Task Grammars](https://mlsanity.com/paper/1709.06391/) (2017)
- [Action Completion: A Temporal Model for Moment Detection](https://mlsanity.com/paper/1805.06749/) (2018)
- [Progress Estimation and Phase Detection for Sequential Processes](https://mlsanity.com/paper/1702.08623/) (2017)
- [Progress Regression RNN for Online Spatial-Temporal Action Localization in Unconstrained Videos](https://mlsanity.com/paper/1903.00304/) (2019)
- [Weakly-Supervised Completion Moment Detection using Temporal Attention](https://mlsanity.com/paper/1910.09920/) (2019)

## Built on by (most cited first, 1 of 1)

- [Multiview Progress Prediction of Robot Activities](https://mlsanity.com/paper/2603.00151/) (2026)

## Abstract (arXiv)

Activity progress prediction aims to estimate what percentage of an activity has been completed. Currently this is done with machine learning approaches, trained and evaluated on complicated and realistic video datasets. The videos in these datasets vary drastically in length and appearance. And some of the activities have unanticipated developments, making activity progression difficult to estimate. In this work, we examine the results obtained by existing progress prediction methods on these datasets. We find that current progress prediction methods seem not to extract useful visual information for the progress prediction task. Therefore, these methods fail to exceed simple frame-counting baselines. We design a precisely controlled dataset for activity progress prediction and on this synthetic dataset we show that the considered methods can make use of the visual information, when this directly relates to the progress prediction. We conclude that the progress prediction task is ill-posed on the currently used real-world datasets. Moreover, to fairly measure activity progression we advise to consider a, simple but effective, frame-counting baseline.