mlsanityData to 7 Oct 2026

Neuro-Symbolic Hierarchical Intention Anticipation in Human Behavior

arXiv 2609.17064 ↗First posted 15 Sep 2026cs.AI cs.CV cs.HCcited by 0

Not on the map. Why?

Abstract (arXiv)

Assistive autonomous systems must anticipate human goals before an observed behavior is complete. This article formulates anticipation as goal inference from a partially observed multimodal episode together with structured prediction of the remaining behavior, rather than exact motor forecasting. A compact Hierarchical Planning Decoder (HPD) is attached to a frozen neuro-symbolic recognition encoder and predicts, at four ontological levels, the next actions, the remaining activities and low-level intentions, and the episode high-level intention(HLI). The decoder is trained with soft neuro-symbolic regularization combining transition-coherence and hierarchical continuity losses, and is decoded with hard reachability masks that enforce ontological validity at inference. On a compositional four-level benchmark of 15,002 multimodal episodes built over NTU RGB+D 120 features, three headline properties are observed together. The advantage over the strongest sequential baseline grows with the anticipation horizon, from +1.7 points at step 1 to +7.3 points at step 3 (top-5). Under compositional generalization, where one parent association per multi-parent low level intention is held out, this advantage widens to +4.9 points at step 1. At the episode level, 96.8% of anticipated trajectories satisfy the joint logic constraints, above the 88.1% strongest-baseline value and the 73.9% ground-truth floor; soft logic terms alone account for a 59.8 to 71.1% relative reduction of HLI-reachability violations, and the hard masks then eliminate them entirely. Neural generation supplies predictive ranking, symbolic constraints supply onto logical validity, and their combination yields coherent hierarchical anticipation while exposing remaining challenges in compositional goal generalization and unordered set prediction.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 28 references that are in the map. First posted on arXiv 15 Sep 2026.

Builds on

21 of 28 references.

References · 21

  1. 2018SlowFast Networks for Video Recognitioncited by 1,862
  2. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  3. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  4. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 738
  5. 2019Neural Probabilistic Logic Programming in DeepProbLogcited by 389
  6. 2021Anticipative Video Transformercited by 197
  7. 2020Rolling-Unrolling LSTMs for Action Anticipation from First-Person Videocited by 119
  8. 2025HD-EPIC: A Highly-Detailed Egocentric Video Datasetcited by 101
  9. 2022Anticipative Feature Fusion Transformer for Multi-Modal Action Anticipationcited by 41
  10. 2022Interaction Region Visual Transformer for Egocentric Action Anticipationcited by 36
  11. 2022Intention-Conditioned Long-Term Human Egocentric Action Forecastingcited by 35
  12. 2023PALM: Predicting Actions through Language Modelscited by 28
9 more
  1. 2021SlowFast Rolling-Unrolling LSTMs for Action Anticipation in Egocentric Videoscited by 20
  2. 2025Intention-Guided Cognitive Reasoning for Egocentric Long-Term Action Anticipationcited by 10
  3. 2024Semantically Guided Representation Learning For Action Anticipationcited by 9
  4. 2025Hierarchical Planning for Complex Tasks with Knowledge Graph-RAG and Symbolic Verificationcited by 9
  5. 2025Technical Report for Ego4D Long-Term Action Anticipation Challenge 2025cited by 9
  6. 2022Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipationcited by 6
  7. 2025Vision and Intention Boost Large Language Model in Long-Term Action Anticipationcited by 5
  8. 2023Knowledge-Guided Short-Context Action Anticipation in Human-Centric Videoscited by 4
  9. 2026Action-Guided Attention for Video Action Anticipationcited by 3

Built on by

Not cited yet.