mlsanityData to 7 Oct 2026

TRACER: Verifiable Generative Provenance for Multimodal Tool-Using Agents

arXiv 2605.09934 ↗First posted 11 May 2026cs.CLcited by 3

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

also in 1 more
Abstract (arXiv)

Multimodal large language models increasingly solve vision-centric tasks by calling external tools for visual inspection, OCR, retrieval, calculation, and multi-step reasoning. Current tool-using agents usually expose the executed tool trajectory and the final answer, but they rarely specify which tool observation supports each generated claim. We call this missing claim-level dependency structure the provenance gap. The gap makes tool use hard to verify and hard to optimize, because useful evidence, redundant exploration, and unsupported reasoning are mixed in the same trajectory. We introduce TRACER, a framework for verifiable generative provenance in multimodal tool-using agents. Instead of adding citations after generation, TRACER generates each answer sentence together with a structured provenance record that identifies the supporting tool turn, evidence unit, and semantic support relation. Its relation space contains Quotation, Compression, and Inference, covering direct reuse, faithful condensation, and grounded derivation. TRACER verifies each record through schema checking, tool-turn alignment, source authenticity, and relation rationality, and then converts verified provenance into traceability constraints and provenance-derived local credit for reinforcement learning. We further construct TRACE-Bench, a benchmark for sentence-level provenance reconstruction from coarse multimodal tool trajectories. On TRACE-Bench, simply adding tools often introduces noise. With Qwen3-VL-8B, TRACER reaches 78.23% answer accuracy and 95.72% summary accuracy, outperforming the strongest closed-source tool-augmented baseline by 23.80 percentage points. Compared with tool-only supervised fine-tuning, it also reduces total test-set tool calls from 4949 to 3486. These results show that reliable multimodal tool reasoning depends on provenance-aware use of observations, not on more tool calls alone.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 41 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

35 of 41 references.

References · 35

  1. 2024GPT-4o System Cardcited by 4,467
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2025Qwen3-VL Technical Reportcited by 2,534
  6. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  7. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
  8. 2024VinePPO: Refining Credit Assignment in RL Training of LLMscited by 168
  9. 2025SimpleTIR: End-to-End Reinforcement Learning for Multi-Turn Tool-Integrated Reasoningcited by 145
  10. 2024The Revolution of Multimodal Large Language Models: A Surveycited by 134
  11. 2025MMSearch-R1: Incentivizing LMMs to Searchcited by 120
  12. 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 119
23 more
  1. 2025MiMo: Unlocking the Reasoning Potential of Language Model -- From Pretraining to Posttrainingcited by 117
  2. 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agentcited by 114
  3. 2025Process Reward Models for LLM Agents: Practical Framework and Directionscited by 81
  4. 2025AgentPRM: Process Reward Models for LLM Agents via Step-Wise Promise and Progresscited by 69
  5. 2025SPA-RL: Reinforcing LLM Agents via Stepwise Progress Attributioncited by 68
  6. 2026Hindsight Credit Assignment for Long-Horizon LLM Agentscited by 51
  7. 2025ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understandingcited by 50
  8. 2025STeCa: Step-level Trajectory Calibration for LLM Agent Learningcited by 37
  9. 2026Vision-DeepResearch: Incentivizing DeepResearch Capability in Multimodal Large Language Modelscited by 33
  10. 2026Vision-DeepResearch Benchmark: Rethinking Visual and Textual Search for Multimodal Large Language Modelscited by 30
  11. 2025Agent0-VL: Exploring Self-Evolving Agent for Tool-Integrated Vision-Language Reasoningcited by 28
  12. 2026AgentVista: Evaluating Multimodal Agents in Ultra-Challenging Realistic Visual Scenarioscited by 22
  13. 2024From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learningcited by 21
  14. 2025ToolVQA: A Dataset for Multi-step Reasoning VQA with External Toolscited by 13
  15. 2026Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Modelscited by 13
  16. 2026Towards Long-horizon Agentic Multimodal Searchcited by 9
  17. 2026MTA-Agent: An Open Recipe for Multimodal Deep Search Agentscited by 7
  18. 2026Enhancing LLM-based Search Agents via Contribution Weighted Group Relative Policy Optimizationcited by 7
  19. 2026E3-TIR: Enhanced Experience Exploitation for Tool-Integrated Reasoningcited by 6
  20. 2026Learning from the Irrecoverable: Error-Localized Policy Optimization for Tool-Integrated LLM Reasoningcited by 5
  21. 2025TROVE: A Challenge for Fine-Grained Text Provenance via Source Sentence Tracing and Relationship Classificationcited by 4
  22. 2026GenProve: Learning to Generate Text with Fine-Grained Provenancecited by 2
  23. 2026EpiBench: Benchmarking Multi-turn Research Workflows for Multimodal Agentscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026LEDGERMIND: Provenance-Constrained Multimodal Agentic Reasoning with a Structured Evidence Ledger
  2. 2026ATP-Bench: Towards Agentic Tool Planning for MLLM Interleaved Generation
  3. 2026FaithEyes: Towards Faithful Tool Use via Multi-Agent Process-Image Verification
  4. 2026InSight: A Benchmark for Agentic Claim Verification in Interactive Visualizations
  5. 2026Towards Verifiable Multimodal Deep Research: A Multi-Agent Harness for Interleaved Report Generation
  6. 2026From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents
  7. 2025DART: Leveraging Multi-Agent Disagreement for Tool Recruitment in Multimodal Reasoning
  8. 2026Do Multimodal Agents Really Benefit from Tool Use? A Systematic Study of Capability Gains