# Adversarial Generative Grammars for Human Activity Prediction

arXiv: https://arxiv.org/abs/2008.04888 · HTML: https://mlsanity.com/paper/2008.04888/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Aug 2020.

## Built on by (most cited first, 19 of 19)

- [P3IV: Probabilistic Procedure Planning from Instructional Videos with Weak Supervision](https://mlsanity.com/paper/2205.02300/) (2022)
- [CaptainCook4D: A Dataset for Understanding Errors in Procedural Activities](https://mlsanity.com/paper/2312.14556/) (2023)
- [Anticipating human actions by correlating past with the future with Jaccard similarity measures](https://mlsanity.com/paper/2105.12414/) (2021)
- [Rethinking Learning Approaches for Long-Term Action Anticipation](https://mlsanity.com/paper/2210.11566/) (2022)
- [PedFormer: Pedestrian Behavior Prediction via Cross-Modal Attention Modulation and Gated Multitask Learning](https://mlsanity.com/paper/2210.07886/) (2022)
- [A Survey on Deep Learning Techniques for Action Anticipation](https://mlsanity.com/paper/2309.17257/) (2023)
- [Conditional Temporal Variational AutoEncoder for Action Video Prediction](https://mlsanity.com/paper/2108.05658/) (2021)
- [DiffAnt: Diffusion Models for Action Anticipation](https://mlsanity.com/paper/2311.15991/) (2023)
- [Human Action Anticipation: A Survey](https://mlsanity.com/paper/2410.14045/) (2024)
- [Activity Grammars for Temporal Action Segmentation](https://mlsanity.com/paper/2312.04266/) (2023)
- [Multimodal Sense-Informed Prediction of 3D Human Motions](https://mlsanity.com/paper/2405.02911/) (2024)
- [Review of Video Predictive Understanding: Early Action Recognition and Future Action Prediction](https://mlsanity.com/paper/2107.05140/) (2021)
- [Streaming egocentric action anticipation: An evaluation scheme and approach](https://mlsanity.com/paper/2306.16682/) (2023)
- [Meta-Auxiliary Learning for Adaptive Human Pose Prediction](https://mlsanity.com/paper/2304.06411/) (2023)
- [Compositional Benchmark Synthesis for Hierarchical Human Action Recognition](https://mlsanity.com/paper/2608.10765/) (2026)
- [PLAN-B: Predicting Likely Alternative Next Best Sequences for Action Prediction](https://mlsanity.com/paper/2103.15987/) (2021)
- [Finding Islands of Predictability in Action Forecasting](https://mlsanity.com/paper/2210.07354/) (2022)
- [Explainable Artificial Intelligence for Quantifying Interfering and High-Risk Behaviors in Autism Spectrum Disorder in a Real-World Classroom Environment Using Privacy-Preserving Video Analysis](https://mlsanity.com/paper/2407.21691/) (2024)
- [Improving Temporal Action Segmentation via Constraint-Aware Decoding](https://mlsanity.com/paper/2605.10149/) (2026)

## Abstract (arXiv)

In this paper we propose an adversarial generative grammar model for future prediction. The objective is to learn a model that explicitly captures temporal dependencies, providing a capability to forecast multiple, distinct future activities. Our adversarial grammar is designed so that it can learn stochastic production rules from the data distribution, jointly with its latent non-terminal representations. Being able to select multiple production rules during inference leads to different predicted outcomes, thus efficiently modeling many plausible futures. The adversarial generative grammar is evaluated on the Charades, MultiTHUMOS, Human3.6M, and 50 Salads datasets and on two activity prediction tasks: future 3D human pose prediction and future activity prediction. The proposed adversarial grammar outperforms the state-of-the-art approaches, being able to predict much more accurately and further in the future, than prior work.