mlsanityData to 14 Oct 2026

A Solution to CVPR'2023 AQTC Challenge: Video Alignment for Multi-Step Inference

arXiv 2306.14412 ↗First posted 26 Jun 2023cs.CV cs.MMcited by 0

Not on the map. Why?

Abstract (arXiv)

Affordance-centric Question-driven Task Completion (AQTC) for Egocentric Assistant introduces a groundbreaking scenario. In this scenario, through learning instructional videos, AI assistants provide users with step-by-step guidance on operating devices. In this paper, we present a solution for enhancing video alignment to improve multi-step inference. Specifically, we first utilize VideoCLIP to generate video-script alignment features. Afterwards, we ground the question-relevant content in instructional videos. Then, we reweight the multimodal context to emphasize prominent features. Finally, we adopt GRU to conduct multi-step inference. Through comprehensive experiments, we demonstrate the effectiveness and superiority of our method, which secured the 2nd place in CVPR'2023 AQTC challenge. Our code is available at https://github.com/zcfinal/LOVEU-CVPR23-AQTC.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 12 of its 13 references that are in the map. First posted on arXiv 26 Jun 2023.

Builds on

12 of 13 references.

References · 12

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  5. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,811
  6. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,367
  7. 2017Multimodal Machine Learning: A Survey and Taxonomycited by 1,013
  8. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 602
  9. 2019End-to-End Learning of Visual Representations from Uncurated Instructional Videoscited by 507
  10. 2019Bridging the Gap between Training and Inference for Neural Machine Translationcited by 123
  11. 2022AssistQ: Affordance-centric Question-driven Task Completion for Egocentric Assistantcited by 32
  12. 2022Winning the CVPR'2022 AQTC Challenge: A Two-stage Function-centric Approachcited by 3

Built on by

Not cited yet.