mlsanityData to 14 Oct 2026

CIViC: Evidence Gates for Audio-Conditioned Visual Prediction on Proxy Tasks

arXiv 2610.15327 ↗First posted 11 Oct 2026cs.LG cs.CV cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

Removing audio can shift a visual predictor's output without showing whether the task uses the intended audio information. We introduce Conditional Input and Visual Interpretation Checks (CIViC), a four-gate protocol that links an interpretation to an intervention, held-out task comparison, or endpoint control. We demonstrate CIViC on actor-disjoint face-raster tests, a synthetic audio-feature mixture, and source-video-disjoint SpeakerVid proxy tasks. In the static face-raster test, cross-actor audio replacement meets the registered detection rule with a small effect; on a separate temporal endpoint, constant-audio fits score better than audio-trained fits in four matched seeds. In the synthetic mixture, the condition-augmented fit has higher 1,000-step RMSE than the audio-only fit (0.1807 vs. 0.1707), with the gap shrinking during training. In the SpeakerVid proxy, the five-feature-zero fit (five low-level audio summaries zeroed; identity, dialogue, and condition inputs retained) has lower whole-frame grayscale RMSE than aligned audio across four held-out conditions and five paired seeds, while aligned audio has lower segment-drift error. The results show that output activation, held-out error, and endpoint controls support different interpretations across these proxy tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 11 of its 17 references that are in the map. First posted on arXiv 11 Oct 2026.

Builds on

11 of 17 references.

References · 11

  1. 2019AVA-ActiveSpeaker: An Audio-Visual Dataset for Active Speaker Detectioncited by 83
  2. 2025Let Them Talk: Audio-Driven Multi-Person Conversational Video Generationcited by 66
  3. 2024VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesiscited by 50
  4. 2025DualTalk: Dual-Speaker Interaction for 3D Talking Head Conversationscited by 32
  5. 2025SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human Generationcited by 30
  6. 2019Animating Face using Disentangled Audio Representationscited by 24
  7. 2025AnyTalker: Scaling Multi-Person Talking Video Generation with Interactivity Refinementcited by 20
  8. 2025Audio-Sync Video Generation with Multi-Stream Temporal Controlcited by 15
  9. 2025UniLS: End-to-End Audio-Driven Avatars for Unified Listening and Speakingcited by 11
  10. 2025Silence is Golden: Leveraging Adversarial Examples to Nullify Audio Control in LDM-based Talking-Head Generationcited by 8
  11. 2025Assessing Identity Leakage in Talking Face Generation: Metrics and Evaluation Frameworkcited by 1

Built on by

Not cited yet.