mlsanityData to 7 Oct 2026

OrgForge: A Multi-Agent Simulation Framework for Verifiable Synthetic Corporate Corpora

arXiv 2603.14997 ↗First posted 16 Mar 2026cs.CL cs.AI cs.IRcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

also in 1 more
Abstract (arXiv)

Building and evaluating enterprise AI systems requires synthetic organizational corpora that are internally consistent, temporally structured, and cross-artifact traceable. Existing corpora either carry legal constraints or inherit hallucination artifacts from the generating LLMs, silently corrupting results when timestamps or facts contradict across documents and reinforcing those errors during training. We present OrgForge, an open-source multi-agent simulation framework that enforces a strict physics-cognition boundary: a deterministic Python engine maintains a SimEvent ground-truth bus while LLMs generate only surface prose. OrgForge simulates the organizational processes that produce documents, not the documents themselves. Engineers leave mid-sprint, triggering incident handoffs and CRM ownership lapses. Knowledge gaps emerge when under-documented systems break and recover through organic documentation and incident resolution. Customer emails fire only when simulation state warrants contact; silence is verifiable ground truth. A live CRM state machine extends the physics-cognition boundary to the customer boundary, producing cross-system causal cascades spanning engineering incidents, support escalation, deal risk flagging, and SLA-adjusted invoices. The framework generates fifteen interleaved artifact categories traceable to a shared immutable event log. Four graph-dynamic subsystems govern organizational behavior independently of any LLM. An embedding-based ticket assignment system using the Hungarian algorithm makes the simulation domain-agnostic. An empirical evaluation across ten incidents demonstrates a 0.46 absolute improvement in prose-to-ground-truth fidelity over chained LLM baselines, and isolates a consistent hallucination failure mode in which chaining propagates fabricated facts faithfully across documents without correcting them.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 16 of its 38 references that are in the map. First posted on arXiv 16 Mar 2026.

Builds on

16 of 38 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks18 Dec
  2. 2023SOTOPIA: Interactive Evaluation for Social Intelligence in Language Agents18 Oct

Other references · 14

  1. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  3. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,674
  4. 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,378
  5. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,362
  6. 2021DeBERTaV3: Improving DeBERTa using ELECTRA-Style Pre-Training with Gradient-Disentangled Embedding Sharingcited by 1,219
  7. 2025Qwen3 Embedding: Advancing Text Embedding and Reranking Through Foundation Modelscited by 1,175
  8. 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 895
  9. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  10. 2019Evaluating the Factual Consistency of Abstractive Text Summarizationcited by 578
  11. 2023Textbooks Are All You Needcited by 561
  12. 2021A Dataset of Information-Seeking Questions and Answers Anchored in Research Paperscited by 475
2 more
  1. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
  2. 2024MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queriescited by 253

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Can LLM Agents Sustain Long-Horizon Organizational Dynamics?
  2. 2026Synthesis Through Simulation: Generating Coherent Enterprise Data via Scalable Agent-System Interaction
  3. 2026Governed Memory: A Production Architecture for Multi-Agent Workflows
  4. 2026The Agentic Company OS: Substrate Inversion for Sustained Enterprise Agent Deployment
  5. 2026Spec2COBOLRot: An Agentic-AI Degradation Loop for Realistic COBOL Corpus Generation
  6. 2026If You Want Coherence, Orchestrate a Team of Rivals: Multi-Agent Models of Organizational Intelligence
  7. 2026Emergent Misaligned Communication in Long-Horizon Multi-Agent LLM Commerce
  8. 2026ττ-Bench: An Environment for End-To-End, Realistic Agent Construction