mlsanityData to 14 Oct 2026

A Simple Baseline for Audio-Visual Scene-Aware Dialog

arXiv 1904.05876 ↗First posted 11 Apr 2019cs.CV cs.AI cs.CLcited by 52

Not on the map. Why?

Abstract (arXiv)

The recently proposed audio-visual scene-aware dialog task paves the way to a more data-driven way of learning virtual assistants, smart speakers and car navigation systems. However, very little is known to date about how to effectively extract meaningful information from a plethora of sensors that pound the computational engine of those devices. Therefore, in this paper, we provide and carefully analyze a simple baseline for audio-visual scene-aware dialog which is trained end-to-end. Our method differentiates in a data-driven manner useful signals from distracting ones using an attention mechanism. We evaluate the proposed approach on the recently introduced and challenging audio-visual scene-aware dataset, and demonstrate the key features that permit to outperform the current state-of-the-art by more than 20% on CIDEr.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). First posted on arXiv 11 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 11; 2023: 7; 2024: 16; 2025: 5; 2026: 5.

52 papers.

Sort:

By area:

  1. 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 191
  2. 2021ZeroCap: Zero-Shot Image-to-Text Generation for Visual-Semantic Arithmeticcited by 150
  3. 2022Vision Transformers are Parameter-Efficient Audio-Visual Learnerscited by 72
  4. 2024CAT: Enhancing Multimodal Large Language Model to Answer Questions in Dynamic Audio-Visual Scenarioscited by 53
  5. 2022Learning in Audio-visual Context: A Review, Analysis, and New Perspectivecited by 44
  6. 2021VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputscited by 42
  7. 2024Align Anything: Training All-Modality Models to Follow Instructions with Language Feedbackcited by 39
  8. 2021Perceptual Score: What Data Modalities Does Your Model Perceive?cited by 35
  9. 2024MeaCap: Memory-Augmented Zero-shot Image Captioningcited by 35
  10. 2023ConZIC: Controllable Zero-shot Image Captioning by Sampling-Based Polishingcited by 33
  11. 2022AssistQ: Affordance-centric Question-driven Task Completion for Egocentric Assistantcited by 32
  12. 2024On-the-fly Modulation for Balanced Multimodal Learningcited by 32
38 more
  1. 2023Cross-modal Prompts: Adapting Large Pre-trained Models for Audio-Visual Downstream Taskscited by 31
  2. 2023Progressive Spatio-temporal Perception for Audio-Visual Question Answeringcited by 29
  3. 2024Meerkat: Audio-Visual Large Language Model for Grounding in Space and Timecited by 26
  4. 2022Zero-Shot Video Captioning with Evolving Pseudo-Tokenscited by 25
  5. 2022Vision+X: A Survey on Multimodal Learning in the Light of Datacited by 22
  6. 2023Object-aware Adaptive-Positivity Learning for Audio-Visual Question Answeringcited by 19
  7. 2022Learning to Retrieve Videos by Asking Questionscited by 18
  8. 2023Modality-Independent Teachers Meet Weakly-Supervised Audio-Visual Event Parsercited by 18
  9. 2024Look, Listen, and Answer: Overcoming Biases for Audio-Visual Question Answeringcited by 18
  10. 2024Boosting Audio Visual Question Answering via Key Semantic-Aware Cuescited by 14
  11. 2021Latent Space Explanation by Interventioncited by 13
  12. 2024Patch-level Sounding Object Tracking for Audio-Visual Question Answeringcited by 13
  13. 2025Question-Aware Gaussian Experts for Audio-Visual Question Answeringcited by 12
  14. 2021Learning Reasoning Paths over Semantic Graphs for Video-grounded Dialoguescited by 11
  15. 2021Video and Text Matching with Conditioned Embeddingscited by 10
  16. 2023CAD -- Contextual Multi-modal Alignment for Dynamic AVQAcited by 10
  17. 2024NarrativeBridge: Enhancing Video Captioning with Causal-Temporal Narrativecited by 10
  18. 2023Target-Aware Spatio-Temporal Reasoning via Answering Questions in Dynamics Audio-Visual Scenarioscited by 8
  19. 2024SHMamba: Structured Hyperbolic State Space Model for Audio-Visual Question Answeringcited by 7
  20. 2021Ensemble of MRR and NDCG models for Visual Dialogcited by 5
  21. 2021Saying the Unseen: Video Descriptions via Dialog Agentscited by 5
  22. 2022End-to-End Multimodal Representation Learning for Video Dialogcited by 5
  23. 2024Answering Diverse Questions via Text Attached with Key Audio-Visual Cluescited by 5
  24. 2024CLIP-Powered TASS: Target-Aware Single-Stream Network for Audio-Visual Question Answeringcited by 4
  25. 2024Learning Trimodal Relation for Audio-Visual Question Answering with Missing Modalitycited by 4
  26. 2025Music Audio-Visual Question Answering Requires Specialized Multimodal Designscited by 4
  27. 2024M2K-VDG: Model-Adaptive Multimodal Knowledge Anchor Enhanced Video-grounded Dialogue Generationcited by 3
  28. 2025AV-Master: Dual-Path Comprehensive Perception Makes Better Audio-Visual Question Answeringcited by 3
  29. 2022Dual Decomposition of Convex Optimization Layers for Consistent Attention in Medical Imagescited by 2
  30. 2022Enabling Harmonious Human-Machine Interaction with Visual-Context Augmented Dialogue System: A Reviewcited by 2
  31. 2024SaSR-Net: Source-Aware Semantic Representation Network for Enhancing Audio-Visual Question Answeringcited by 2
  32. 2025MoLT: Mixture of Layer-Wise Tokens for Efficient Audio-Visual Learningcited by 2
  33. 2026Query-Guided Spatial-Temporal-Frequency Interaction for Music Audio-Visual Question Answeringcited by 2
  34. 2026Knowing When to Answer: Adaptive Confidence Refinement for Reliable Audio-Visual Question Answeringcited by 2
  35. 2022Describing Sets of Images with Textual-PCAcited by 1
  36. 2024Social Media Ready Caption Generation for Brandscited by 0
  37. 2025Multi-Modal Scene Graph with Kolmogorov-Arnold Experts for Audio-Visual Question Answeringcited by 0
  38. 2026CAE-AV: Improving Audio-Visual Learning via Cross-modal Interactive Enrichmentcited by 0

50 most cited of 52 All 52 (JSON)