# Action Anticipation with RBF Kernelized Feature Mapping RNN

arXiv: https://arxiv.org/abs/1911.07806 · HTML: https://mlsanity.com/paper/1911.07806/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2019.

## Built on by (most cited first, 20 of 20)

- [Anticipative Video Transformer](https://mlsanity.com/paper/2106.02036/) (2021)
- [Learning to Anticipate Egocentric Actions by Imagination](https://mlsanity.com/paper/2101.04924/) (2021)
- [Interaction Region Visual Transformer for Egocentric Action Anticipation](https://mlsanity.com/paper/2211.14154/) (2022)
- [Anticipating human actions by correlating past with the future with Jaccard similarity measures](https://mlsanity.com/paper/2105.12414/) (2021)
- [Can't make an Omelette without Breaking some Eggs: Plausible Action Anticipation using Large Video-Language Models](https://mlsanity.com/paper/2405.20305/) (2024)
- [Self-Regulated Learning for Egocentric Video Activity Anticipation](https://mlsanity.com/paper/2111.11631/) (2021)
- [Rethinking Learning Approaches for Long-Term Action Anticipation](https://mlsanity.com/paper/2210.11566/) (2022)
- [Predicting the Next Action by Modeling the Abstract Goal](https://mlsanity.com/paper/2209.05044/) (2022)
- [A-ACT: Action Anticipation through Cycle Transformations](https://mlsanity.com/paper/2204.00942/) (2022)
- [Interpretable Deep Feature Propagation for Early Action Recognition](https://mlsanity.com/paper/2107.05122/) (2021)
- [Review of Video Predictive Understanding: Early Action Recognition and Future Action Prediction](https://mlsanity.com/paper/2107.05140/) (2021)
- [Pedestrian Stop and Go Forecasting with Hybrid Feature Fusion](https://mlsanity.com/paper/2203.02489/) (2022)
- [Modeling long-term interactions to enhance action recognition](https://mlsanity.com/paper/2104.11520/) (2021)
- [Rich Action-semantic Consistent Knowledge for Early Action Prediction](https://mlsanity.com/paper/2201.09169/) (2022)
- [DSNet: Dynamic Skin Deformation Prediction by Recurrent Neural Network](https://mlsanity.com/paper/2201.07660/) (2021)
- [Deep learning-based approaches for human motion decoding in smart walkers for rehabilitation](https://mlsanity.com/paper/2301.05575/) (2023)
- [FactCheck: Feasibility-aware Long-term Action Anticipation with Multi-agent Collaboration](https://mlsanity.com/paper/2606.14778/) (2026)
- [Action Forecasting with Feature-wise Self-Attention](https://mlsanity.com/paper/2107.08579/) (2021)
- [Adversarial Memory Networks for Action Prediction](https://mlsanity.com/paper/2112.09875/) (2021)
- [An end-to-end multi-scale network for action prediction in videos](https://mlsanity.com/paper/2301.01216/) (2022)

## Abstract (arXiv)

We introduce a novel Recurrent Neural Network-based algorithm for future video feature generation and action anticipation called feature mapping RNN. Our novel RNN architecture builds upon three effective principles of machine learning, namely parameter sharing, Radial Basis Function kernels and adversarial training. Using only some of the earliest frames of a video, the feature mapping RNN is able to generate future features with a fraction of the parameters needed in traditional RNN. By feeding these future features into a simple multi-layer perceptron facilitated with an RBF kernel layer, we are able to accurately predict the action in the video. In our experiments, we obtain 18% improvement on JHMDB-21 dataset, 6% on UCF101-24 and 13% improvement on UT-Interaction datasets over prior state-of-the-art for action anticipation.