AVTrace: Diagnosing Audio-Visual Temporal Reasoning in Omni Models
On the map: LLM reasoning → Audio-language model reasoning also fits: Multimodal reasoning
Abstract (arXiv)
Omni models can describe video content, but can they locate events in time, preserve event order, and judge audio-visual synchronization? We introduce AVTrace (Audio-Visual Temporal Reasoning Assessment and Capability Evaluation), a silver-standard diagnostic suite spanning onset and span grounding, synchronization, next-step prediction, cross-modal localization, chain parsing, and event-conditioned comprehension. It contains 34,114 training examples and category-balanced development and test splits of 3,500 and 7,000 examples. We evaluate five open omni models under their respective input configurations using reference-blind response normalization followed by deterministic scoring. All five off-the-shelf systems score below the test split's majority-label baseline of 0.556 on synchronization verification, and obtain low scores on chain parsing and event-conditioned grounding and comprehension. Development-set perturbations reveal task-dependent sensitivity in Qwen3-Omni-30B to modality removal and changes in visual input processing, without isolating their underlying causes. Parameter-efficient temporal post-training improves Gemma4-E4B-it on several benchmark metrics. On three external image benchmarks, task metrics change modestly, including some degradations, while teacher-forcing perplexity decreases. Together, these findings show that semantic reference-text overlap should not be treated as a proxy for temporal localization, and that AVTrace can identify task-specific weaknesses while providing a testbed for temporal post-training.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Audio-language model reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 41 references that are in the map. First posted on arXiv 17 Sep 2026.
Builds on
16 of 41 references.
Closest ancestors · 3
Same direction, newest first
- 2026Listen, See and Track: Spatio-Temporal Audio-Visual Sound Event Reasoning for Omni-Modal Language Models10 Aug
- 2025Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalities23 May
- 2025Qwen2.5-Omni Technical Report26 Mar
Other references · 13
- 2025Qwen3-Omni Technical Reportcited by 497
- 2026Gemma 4 Technical Reportcited by 259
- 2022Learning to Answer Questions in Dynamic Audio-Visual Scenarioscited by 191
- 2025WorldSense: Evaluating Real-world Omnimodal Understanding for Multimodal LLMscited by 155
- 2022Audio-Visual Segmentationcited by 151
- 2020CoVoST 2 and Massively Multilingual Speech-to-Text Translationcited by 74
- 2024AVHBench: A Cross-Modal Hallucination Benchmark for Audio-Visual Large Language Modelscited by 56
- 2023VITATECS: A Diagnostic Dataset for Temporal Concept Understanding of Video-Language Modelscited by 55
- 2024LongVALE: Vision-Audio-Language-Event Benchmark Towards Time-Aware Omni-Modal Perception of Long Videoscited by 51
- 2025InteractiveOmni: A Unified Omni-modal Model for Audio-Visual Multi-turn Dialoguecited by 29
- 2025MAVERIX: Multimodal Audio-Visual Evaluation and Recognition IndeXcited by 7
- 2018Audio-Visual Event Localization in Unconstrained Videoscited by 1
1 more
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025OmniVideoBench: Towards Audio-Visual Understanding Evaluation for Omni MLLMs
- 2025AURA: A Fine-Grained Benchmark and Decomposed Metric for Audio-Visual Reasoning
- 2025JointAVBench: A Benchmark for Joint Audio-Visual Reasoning Evaluation
- 2026LatentOmni: Rethinking Omni-Modal Understanding via Unified Audio-Visual Latent Reasoning
- 2026AVI-Bench: Toward Human-like Audio-Visual Intelligence of Omni-MLLMs
- 2026AVRT: Audio-Visual Reasoning Transfer through Single-Modality Teachers
- 2026OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
- 2026StreamOV: Streaming Omni-Video Understanding via Evidence-Guided Memory and Response Triggering