Benchmarking Agentic Newswriting via Journalistic Workflows
On the map: LLM agents and tool use → Web, search, and research agents
Abstract (arXiv)
Recent advances in autonomous digital agents from industry (e.g., Manus AI and Gemini's research mode) highlight their potential for structured tasks through autonomous decision-making and task decomposition, but it remains unclear how well such systems support real-world information-intensive workflows. We study this question in journalism, where newswriting requires iterative planning, contextual reasoning, and active discovery of missing background to produce a coherent article. We introduce NEWSAGENT, a benchmark for evaluating how agents search raw materials, select relevant information, and iteratively revise drafts through core journalistic functions. Given a writing instruction and partial firsthand materials, agents must identify narrative perspectives, issue keyword-based queries, retrieve historical context, and generate complete news articles. Unlike typical summarization or retrieval tasks, essential context is not directly available and must be actively discovered, reflecting real-world reporting constraints. NEWSAGENT consists of 6k human-verified examples derived from real news. We evaluate open- and closed-sourced LLMs with commonly-used agentic frameworks on NEWSAGENT, which shows that agents are capable of retrieving relevant facts but struggling with planning and narrative integration. We believe that NEWSAGENT serves a realistic testbed for iterating and evaluating agent capabilities in terms of web data manipulation to real-world productivity.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use. Builds on 5 of its 23 references that are in the map. First posted on arXiv 30 Aug 2025.
Builds on
5 of 23 references.
References · 5
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2022Interleaving Retrieval with Chain-of-Thought Reasoning for Knowledge-Intensive Multi-Step Questionscited by 923
- 2018Plan-And-Write: Towards Better Automatic Storytellingcited by 234
- 2025From Mind to Machine: The Rise of Manus AI as a Fully Autonomous Digital Agentcited by 33
- 2025StructEval: Benchmarking LLMs' Capabilities to Generate Structural Outputscited by 25
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025How can AI agents support journalists' work? An experiment with designing an LLM-driven intelligent reporting system
- 2025Can Memory-Augmented LLM Agents Aid Journalism in Interpreting and Framing News for Diverse Audiences?
- 2025InfoAgent: Advancing Autonomous Information-Seeking Agents
- 2025Workflow is All You Need: Escaping the "Statistical Smoothing Trap" via High-Entropy Information Foraging and Adversarial Pacing
- 2026DailyReport: An Open-ended Benchmark for Evaluating Search Agents on Daily Search Tasks
- 2026DynaTree: Dynamic Agentic Retrieval Tree for Time-Sensitive News Retrieval
- 2025From Web Search towards Agentic Deep Research: Incentivizing Search with Reasoning Agents
- 2025Mind2Web 2: Evaluating Agentic Search with Agent-as-a-Judge