# Sibyl: Simple yet Effective Agent Framework for Complex Real-world Reasoning

arXiv: https://arxiv.org/abs/2407.10718 · HTML: https://mlsanity.com/paper/2407.10718/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 15 Jul 2024.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 18 of 18)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [WebGPT: Browser-assisted question-answering with human feedback](https://mlsanity.com/paper/2112.09332/) (2021)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [The Rise and Potential of Large Language Model Based Agents: A Survey](https://mlsanity.com/paper/2309.07864/) (2023)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [Cognitive Architectures for Language Agents](https://mlsanity.com/paper/2309.02427/) (2023)
- [ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving](https://mlsanity.com/paper/2309.17452/) (2023)
- [OpenCodeInterpreter: Integrating Code Generation with Execution and Refinement](https://mlsanity.com/paper/2402.14658/) (2024)
- [S3: Social-network Simulation System with Large Language Model-Empowered Agents](https://mlsanity.com/paper/2307.14984/) (2023)
- [OS-Copilot: Towards Generalist Computer Agents with Self-Improvement](https://mlsanity.com/paper/2402.07456/) (2024)
- [Retrieval meets Long Context Large Language Models](https://mlsanity.com/paper/2310.03025/) (2023)
- [RoleEval: A Bilingual Role Evaluation Benchmark for Large Language Models](https://mlsanity.com/paper/2312.16132/) (2023)
- [Compressing Long Context for Enhancing RAG with AMR-based Concept Distillation](https://mlsanity.com/paper/2405.03085/) (2024)
- [Math Agents: Computational Infrastructure, Mathematical Embedding, and Genomics](https://mlsanity.com/paper/2307.02502/) (2023)
- [X-RiSAWOZ: High-Quality End-to-End Multilingual Dialogue Datasets and Few-shot Agents](https://mlsanity.com/paper/2306.17674/) (2023)
- [Benchmarks Underestimate the Readiness of Multi-lingual Dialogue Agents](https://mlsanity.com/paper/2405.17840/) (2024)

## Built on by (most cited first, 18 of 18)

- [Multi-agent Architecture Search via Agentic Supernet](https://mlsanity.com/paper/2502.04180/) (2025)
- [OASIS: Open Agent Social Interaction Simulations with One Million Agents](https://mlsanity.com/paper/2411.11581/) (2024)
- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usage](https://mlsanity.com/paper/2412.15606/) (2024)
- [Benchmarking Agentic Workflow Generation](https://mlsanity.com/paper/2410.07869/) (2024)
- [DynaSaur: Large Language Agents Beyond Predefined Actions](https://mlsanity.com/paper/2411.01747/) (2024)
- [Iterative Tool Usage Exploration for Multimodal Agents via Step-wise Preference Tuning](https://mlsanity.com/paper/2504.21561/) (2025)
- [Toward Efficient Agents: Memory, Tool learning, and Planning](https://mlsanity.com/paper/2601.14192/) (2026)
- [PlanGEN: A Multi-Agent Framework for Generating Planning and Reasoning Trajectories for Complex Problem Solving](https://mlsanity.com/paper/2502.16111/) (2025)
- [Synergizing Logical Reasoning, Knowledge Management and Collaboration in Multi-Agent LLM System](https://mlsanity.com/paper/2507.02170/) (2025)
- [COLA: A Scalable Multi-Agent Framework For Windows UI Task Automation](https://mlsanity.com/paper/2503.09263/) (2025)
- [Cache Mechanism for Agent RAG Systems](https://mlsanity.com/paper/2511.02919/) (2025)
- [Evaluating Theory of Mind and Internal Beliefs in LLM-Based Multi-Agent Systems](https://mlsanity.com/paper/2603.00142/) (2026)
- [T^2Agent A Tool-augmented Multimodal Misinformation Detection Agent with Monte Carlo Tree Search](https://mlsanity.com/paper/2505.19768/) (2025)
- [LLM Library Learning Fails: A LEGO-Prover Case Study](https://mlsanity.com/paper/2504.03048/) (2025)
- [Communication and Verification in LLM Agents towards Collaboration under Information Asymmetry](https://mlsanity.com/paper/2510.25595/) (2025)
- [MATRIX: Multimodal Agent Tuning for Robust Tool-Use Reasoning](https://mlsanity.com/paper/2510.08567/) (2025)
- [CB-Orchestrator: Adaptive Workflow Optimization for LLM Agents via Contextual Bandits](https://mlsanity.com/paper/2610.13064/) (2026)

## Abstract (arXiv)

Existing agents based on large language models (LLMs) demonstrate robust problem-solving capabilities by integrating LLMs' inherent knowledge, strong in-context learning and zero-shot capabilities, and the use of tools combined with intricately designed LLM invocation workflows by humans. However, these agents still exhibit shortcomings in long-term reasoning and under-use the potential of existing tools, leading to noticeable deficiencies in complex real-world reasoning scenarios. To address these limitations, we introduce Sibyl, a simple yet powerful LLM-based agent framework designed to tackle complex reasoning tasks by efficiently leveraging a minimal set of tools. Drawing inspiration from Global Workspace Theory, Sibyl incorporates a global workspace to enhance the management and sharing of knowledge and conversation history throughout the system. Furthermore, guided by Society of Mind Theory, Sibyl implements a multi-agent debate-based jury to self-refine the final answers, ensuring a comprehensive and balanced approach. This approach aims to reduce system complexity while expanding the scope of problems solvable-from matters typically resolved by humans in minutes to those requiring hours or even days, thus facilitating a shift from System-1 to System-2 thinking. Sibyl has been designed with a focus on scalability and ease of debugging by incorporating the concept of reentrancy from functional programming from its inception, with the aim of seamless and low effort integration in other LLM applications to improve capabilities. Our experimental results on the GAIA benchmark test set reveal that the Sibyl agent instantiated with GPT-4 achieves state-of-the-art performance with an average score of 34.55%, compared to other agents based on GPT-4. We hope that Sibyl can inspire more reliable and reusable LLM-based agent solutions to address complex real-world reasoning tasks.