# Neuro-Symbolic Hierarchical Intention Anticipation in Human Behavior

arXiv: https://arxiv.org/abs/2609.17064 · HTML: https://mlsanity.com/paper/2609.17064/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 28 references that are in the map.
- First posted on arXiv 15 Sep 2026.

## Builds on (most cited first, 21 of 21)

- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning](https://mlsanity.com/paper/2506.09985/) (2025)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [Neural Probabilistic Logic Programming in DeepProbLog](https://mlsanity.com/paper/1907.08194/) (2019)
- [Anticipative Video Transformer](https://mlsanity.com/paper/2106.02036/) (2021)
- [Rolling-Unrolling LSTMs for Action Anticipation from First-Person Video](https://mlsanity.com/paper/2005.02190/) (2020)
- [HD-EPIC: A Highly-Detailed Egocentric Video Dataset](https://mlsanity.com/paper/2502.04144/) (2025)
- [Anticipative Feature Fusion Transformer for Multi-Modal Action Anticipation](https://mlsanity.com/paper/2210.12649/) (2022)
- [Interaction Region Visual Transformer for Egocentric Action Anticipation](https://mlsanity.com/paper/2211.14154/) (2022)
- [Intention-Conditioned Long-Term Human Egocentric Action Forecasting](https://mlsanity.com/paper/2207.12080/) (2022)
- [PALM: Predicting Actions through Language Models](https://mlsanity.com/paper/2311.17944/) (2023)
- [SlowFast Rolling-Unrolling LSTMs for Action Anticipation in Egocentric Videos](https://mlsanity.com/paper/2109.00829/) (2021)
- [Semantically Guided Representation Learning For Action Anticipation](https://mlsanity.com/paper/2407.02309/) (2024)
- [Intention-Guided Cognitive Reasoning for Egocentric Long-Term Action Anticipation](https://mlsanity.com/paper/2508.01742/) (2025)
- [Hierarchical Planning for Complex Tasks with Knowledge Graph-RAG and Symbolic Verification](https://mlsanity.com/paper/2504.04578/) (2025)
- [Technical Report for Ego4D Long-Term Action Anticipation Challenge 2025](https://mlsanity.com/paper/2506.02550/) (2025)
- [Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipation](https://mlsanity.com/paper/2210.05991/) (2022)
- [Vision and Intention Boost Large Language Model in Long-Term Action Anticipation](https://mlsanity.com/paper/2505.01713/) (2025)
- [Knowledge-Guided Short-Context Action Anticipation in Human-Centric Videos](https://mlsanity.com/paper/2309.05943/) (2023)
- [Action-Guided Attention for Video Action Anticipation](https://mlsanity.com/paper/2603.01743/) (2026)

## Abstract (arXiv)

Assistive autonomous systems must anticipate human goals before an observed behavior is complete. This article formulates anticipation as goal inference from a partially observed multimodal episode together with structured prediction of the remaining behavior, rather than exact motor forecasting. A compact Hierarchical Planning Decoder (HPD) is attached to a frozen neuro-symbolic recognition encoder and predicts, at four ontological levels, the next actions, the remaining activities and low-level intentions, and the episode high-level intention(HLI). The decoder is trained with soft neuro-symbolic regularization combining transition-coherence and hierarchical continuity losses, and is decoded with hard reachability masks that enforce ontological validity at inference. On a compositional four-level benchmark of 15,002 multimodal episodes built over NTU RGB+D 120 features, three headline properties are observed together. The advantage over the strongest sequential baseline grows with the anticipation horizon, from +1.7 points at step 1 to +7.3 points at step 3 (top-5). Under compositional generalization, where one parent association per multi-parent low level intention is held out, this advantage widens to +4.9 points at step 1. At the episode level, 96.8% of anticipated trajectories satisfy the joint logic constraints, above the 88.1% strongest-baseline value and the 73.9% ground-truth floor; soft logic terms alone account for a 59.8 to 71.1% relative reduction of HLI-reachability violations, and the hard masks then eliminate them entirely. Neural generation supplies predictive ranking, symbolic constraints supply onto logical validity, and their combination yields coherent hierarchical anticipation while exposing remaining challenges in compositional goal generalization and unordered set prediction.