# Temporal Context Consistency Above All: Enhancing Long-Term Anticipation by Learning and Enforcing Temporal Constraints

arXiv: https://arxiv.org/abs/2412.19424 · HTML: https://mlsanity.com/paper/2412.19424/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 51 references that are in the map.
- First posted on arXiv 27 Dec 2024.

## Builds on (most cited first, 2 of 2)

- [DiffAnt: Diffusion Models for Action Anticipation](https://mlsanity.com/paper/2311.15991/) (2023)
- [A-ACT: Action Anticipation through Cycle Transformations](https://mlsanity.com/paper/2204.00942/) (2022)

## Abstract (arXiv)

This paper proposes a method for long-term action anticipation (LTA), the task of predicting action labels and their duration in a video given the observation of an initial untrimmed video interval. We build on an encoder-decoder architecture with parallel decoding and make two key contributions. First, we introduce a bi-directional action context regularizer module on the top of the decoder that ensures temporal context coherence in temporally adjacent segments. Second, we learn from classified segments a transition matrix that models the probability of transitioning from one action to another and the sequence is optimized globally over the full prediction interval. In addition, we use a specialized encoder for the task of action segmentation to increase the quality of the predictions in the observation interval at inference time, leading to a better understanding of the past. We validate our methods on four benchmark datasets for LTA, the EpicKitchen-55, EGTEA+, 50Salads and Breakfast demonstrating superior or comparable performance to state-of-the-art methods, including probabilistic models and also those based on Large Language Models, that assume trimmed video as input. The code will be released upon acceptance.