mlsanityData to 14 Oct 2026

ECHO: Event-Centric Hypergraph Operations via Multi-Agent Collaboration for Multimedia Event Extraction

arXiv 2603.06683 ↗First posted 4 Mar 2026cs.CVcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Multimedia event extraction (M2E2) aims to predict triggers, ground arguments across text and images, and then assemble them into schema-consistent event records. Recent LLM-based approaches have shown strong potential for M2E2, but their intermediate event hypotheses often remain implicit, and event-argument linking is still tightly coupled with role binding. This leaves little opportunity to inspect or revise intermediate event hypotheses and makes predictions brittle to early errors. To bridge this gap, we present ECHO, a multi-agent framework that reframes M2E2 as iterative refinement over an explicit Multimedia Event Hypergraph (MEHG). Instead of relying on implicit linear generation, ECHO performs auditable atomic updates over a shared hypergraph, making intermediate event structures explicit and revisable. Furthermore, we introduce a Link-then-Bind strategy that decouples event-argument linking from role binding, reducing premature semantic commitment during structured prediction. Extensive experiments on the M2E2 benchmark show that ECHO consistently outperforms prior state-of-the-art approaches, achieving gains of 7.3 and 15.5 F1 points on event mention and argument role, respectively.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 18 of its 28 references that are in the map. First posted on arXiv 4 Mar 2026.

Builds on

18 of 28 references.

References · 18

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  5. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  7. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  8. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 1,011
  9. 2023ChatDev: Communicative Agents for Software Developmentcited by 919
  10. 2018Hypergraph Neural Networkscited by 488
  11. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  12. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 309
6 more
  1. 2019Entity, Relation, and Event Extraction with Contextualized Span Representationscited by 262
  2. 2022CLIP-Event: Connecting Text and Images with Event Structurescited by 86
  3. 2020Cross-media Structured Common Space for Multimedia Event Extractioncited by 61
  4. 2024UMIE: Unified Multimodal Information Extraction with Instruction Tuningcited by 17
  5. 2023Training Multimedia Event Extraction With Generated Images and Captionscited by 11
  6. 2024MMUTF: Multimodal Multimedia Event Argument Extraction with Unified Template Fillingcited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning to Generate and Extract: A Multi-Agent Collaboration Framework For Zero-shot Document-level Event Arguments Extraction
  2. 2026RMPL: Relation-aware Multi-task Progressive Learning with Stage-wise Training for Multimedia Event Extraction
  3. 2025Reflective Agreement: Combining Self-Mixture of Agents with a Sequence Tagger for Robust Event Extraction
  4. 2025Event Extraction in Large Language Model
  5. 2026EventMemAgent: Hierarchical Event-Centric Memory for Online Video Understanding with Adaptive Tool Use
  6. 2026NEST: Narrative Event Structures in Time for Long Video Understanding
  7. 2025Extracting Events Like Code: A Multi-Agent Programming Framework for Zero-Shot Event Extraction
  8. 2025Stepwise Schema-Guided Prompting Framework with Parameter Efficient Instruction Tuning for Multimedia Event Extraction