mlsanityData to 7 Oct 2026

Winning the CVPR'2022 AQTC Challenge: A Two-stage Function-centric Approach

arXiv 2206.09597 ↗First posted 20 Jun 2022cs.CV cs.MMcited by 3

Not on the map. Why?

Abstract (arXiv)

Affordance-centric Question-driven Task Completion for Egocentric Assistant(AQTC) is a novel task which helps AI assistant learn from instructional videos and scripts and guide the user step-by-step. In this paper, we deal with the AQTC via a two-stage Function-centric approach, which consists of Question2Function Module to ground the question with the related function and Function2Answer Module to predict the action based on the historical steps. We evaluated several possible solutions in each module and obtained significant gains compared to the given baselines. Our code is available at urlhttps://github.com/starsholic/LOVEU-CVPR22-AQTC.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 14 of its 22 references that are in the map. First posted on arXiv 20 Jun 2022.

Builds on

14 of 22 references.

References · 14

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  4. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  5. 2015VQA: Visual Question Answeringcited by 2,489
  6. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  7. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  8. 2018TVQA: Localized, Compositional Video Question Answeringcited by 463
  9. 2016Visual Dialogcited by 377
  10. 2018Bilinear Attention Networkscited by 277
  11. 2015Stacked Attention Networks for Image Question Answeringcited by 273
  12. 2016Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Groundingcited by 243
2 more
  1. 2022AssistQ: Affordance-centric Question-driven Task Completion for Egocentric Assistantcited by 32
  2. 2021Unified Learning Approach for Egocentric Hand Gesture Recognition and Fingertip Detectioncited by 6

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 1.

3 papers.

  1. 2024VideoLLM-MoD: Efficient Video-Language Streaming with Mixture-of-Depths Vision Computationcited by 55
  2. 2023First Place Solution to the CVPR'2023 AQTC Challenge: A Function-Interaction Centric Approach with Spatiotemporal Visual-Language Alignmentcited by 3
  3. 2023A Solution to CVPR'2023 AQTC Challenge: Video Alignment for Multi-Step Inferencecited by 0