mlsanityData to 7 Oct 2026

Zero-Shot Anticipation for Instructional Activities

arXiv 1812.02501 ↗First posted 6 Dec 2018cs.CV cs.LGcited by 43

Not on the map. Why?

Abstract (arXiv)

How can we teach a robot to predict what will happen next for an activity it has never seen before? We address this problem of zero-shot anticipation by presenting a hierarchical model that generalizes instructional knowledge from large-scale text-corpora and transfers the knowledge to the visual domain. Given a portion of an instructional video, our model predicts coherent and plausible actions multiple steps into the future, all in rich natural language. To demonstrate the anticipation capabilities of our model, we introduce the Tasty Videos dataset, a collection of 2511 recipes for zero-shot learning, recognition and anticipation.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Dec 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 13; 2023: 15; 2024: 7; 2025: 2; 2026: 1.

43 papers.

Sort:

By area:

  1. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  2. 2022Assembly101: A Large-Scale Multi-View Video Dataset for Understanding Procedural Activitiescited by 284
  3. 2022Temporal Action Segmentation: An Analysis of Modern Techniquescited by 85
  4. 2022My View is the Best View: Procedure Learning from Egocentric Videoscited by 81
  5. 2023AntGPT: Can Large Language Models Help Long-term Action Anticipation from Videos?cited by 79
  6. 2023EgoPlan-Bench: Benchmarking Multimodal Large Language Models for Human-Level Planningcited by 66
  7. 2022Future Transformer for Long-term Action Anticipationcited by 57
  8. 2023CaptainCook4D: A Dataset for Understanding Errors in Procedural Activitiescited by 47
  9. 2023Learning Procedure-aware Video Representation from Instructional Videos and Their Narrationscited by 46
  10. 2023StepFormer: Self-supervised Step Discovery and Localization in Instructional Videoscited by 37
  11. 2021Predicting the Future from First Person (Egocentric) Vision: A Surveycited by 36
  12. 2022Rethinking Learning Approaches for Long-Term Action Anticipationcited by 29
31 more
  1. 2023Pretrained Language Models as Visual Planners for Human Assistancecited by 28
  2. 2023Learning to Ground Instructional Articles in Videos through Narrationscited by 28
  3. 2021Multi-Modal Temporal Convolutional Network for Anticipating Actions in Egocentric Videoscited by 23
  4. 2024VideoLLaMB: Long Streaming Video Understanding with Recurrent Memory Bridgescited by 20
  5. 2023Every Mistake Counts in Assemblycited by 18
  6. 2025Memory-efficient Streaming VideoLLMs for Real-time Procedural Video Understandingcited by 18
  7. 2021Induce, Edit, Retrieve: Language Grounded Multimodal Schema for Instructional Video Retrievalcited by 13
  8. 2023United We Stand, Divided We Fall: UnityGraph for Unsupervised Procedure Learning from Videoscited by 13
  9. 2022Multimedia Generative Script Learning for Task Planningcited by 12
  10. 2024Propose, Assess, Search: Harnessing LLMs for Goal-Oriented Planning in Instructional Videoscited by 12
  11. 2021Embedding Symbolic Temporal Knowledge into Deep Sequential Modelscited by 11
  12. 2021Transferring Knowledge from Text to Video: Zero-Shot Anticipation for Procedural Actionscited by 10
  13. 2023GePSAn: Generative Procedure Step Anticipation in Cooking Videoscited by 9
  14. 2024Human Action Anticipation: A Surveycited by 8
  15. 2023Generating Illustrated Instructionscited by 7
  16. 2022Text-Derived Knowledge Helps Vision: A Simple Cross-modal Distillation for Video-based Action Anticipationcited by 6
  17. 2024Simulating Task-Oriented Dialogues with State Transition Graphs and Large Language Modelscited by 6
  18. 2022FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observationscited by 5
  19. 2023What You Say Is What You Show: Visual Narration Detection in Instructional Videoscited by 5
  20. 2022Cross-modal Contrastive Distillation for Instructional Activity Anticipationcited by 4
  21. 2022A Closer Look at Temporal Ordering in the Segmentation of Instructional Videoscited by 4
  22. 2022Graph2Vid: Flow graph to Video Grounding for Weakly-supervised Multi-Step Localizationcited by 4
  23. 2023Non-Sequential Graph Script Induction via Multimedia Groundingcited by 4
  24. 2023Knowledge-Guided Short-Context Action Anticipation in Human-Centric Videoscited by 4
  25. 2024Show and Guide: Instructional-Plan Grounded Vision and Language Modelcited by 2
  26. 2022Finding Islands of Predictability in Action Forecastingcited by 0
  27. 2023Efficient Pre-training for Localized Instruction Generation of Videoscited by 0
  28. 2024Directed Domain Fine-Tuning: Tailoring Separate Modalities for Specific Training Taskscited by 0
  29. 2024Multimodal Language Models for Domain-Specific Procedural Video Summarizationcited by 0
  30. 2025Learning to Generate Long-term Future Narrations Describing Activities of Daily Livingcited by 0
  31. 2026VIGiA: Instructional Video Guidance via Dialogue Reasoning and Retrievalcited by 0