mlsanityData to 14 Oct 2026

How Should a Camera-First AI Agent Work?

arXiv 2610.14772 ↗First posted 10 Oct 2026cs.AI cs.HC cs.LOcited by 0

Not on the map. Why?

Abstract (arXiv)

Camera-first agents must act on the visual world as it unfolds. We argue that the central obstacle for future camera-first agents is not recognition ability - multimodal models (vision-language and live/streaming alike) generalize well through language and remain the right tool for semantic reasoning - but time. Many visual targets are ephemeral: they appear, are query-relevant, and leave within a short window. Binding memory to the same end-to-end path as reasoning means that by the time a monolithic pipeline completes, the evidence is gone. We formalize this as Ephemeral-Target Interaction (ETI) and a Perceptual Deadline Contract separating a hard memory deadline from a soft reasoning deadline, grounded in the bounded, passive nature of sensory and working memory. We propose a two-deadline architecture: an always-on, bounded, non-semantic fast path; an active remembering mechanism that commits a moment to durable memory before it is overwritten (event-driven in v1); and a deferred deliberation layer (multimodal model + tools). We instantiate the contract on real MOT17 track windows, a consistency check, and calibrate the single-path baseline from measured frontier-model latency: even a short answer takes 1.2-9.3 s, and an agentic identify-and-search longer. Answers can be late; evidence cannot be missed.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 14 of its 31 references that are in the map. First posted on arXiv 10 Oct 2026.

Builds on

14 of 31 references.

References · 14

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  2. 2022Token Merging: Your ViT But Fastercited by 1,021
  3. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  4. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 853
  5. 2025Qwen2.5-Omni Technical Reportcited by 736
  6. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 682
  7. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
  8. 2024Moshi: a speech-text foundation model for real-time dialoguecited by 568
  9. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 506
  10. 2024The Platonic Representation Hypothesiscited by 382
  11. 2014Recurrent Models of Visual Attentioncited by 349
  12. 2024Mixture-of-Depths: Dynamically allocating compute in transformer-based language modelscited by 258
2 more
  1. 2023AutoMix: Automatically Mixing Language Modelscited by 125
  2. 2026VisionClaw: Always-On AI Agents through Smart Glassescited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory
  2. 2026MemEye: A Visual-Centric Evaluation Framework for Multimodal Agent Memory
  3. 2026MATEO: A Multimodal Benchmark for Temporal Reasoning and Planning in LVLMs
  4. 2026MNIST-PRO: MNIST is Back as a Partially Observable World for AI Agents
  5. 2026Evaluating Self-Correcting Vision Agents Through Quantitative and Qualitative Metrics
  6. 2026Recompose and Refine Latent Reasoning Flows for Vision-Language-Action Models
  7. 2026Dual-Latent Memory Routing for Vision-Language Reasoning
  8. 2025Agent-X: Evaluating Deep Multimodal Reasoning in Vision-Centric Agentic Tasks