SDialog: A Python Toolkit for End-to-End Agent Building, User Simulation, Dialog Generation, and Evaluation
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
We present SDialog, an MIT-licensed open-source Python toolkit that unifies dialog generation, evaluation and mechanistic interpretability into a single end-to-end framework for building and analyzing LLM-based conversational agents. Built around a standardized textttDialog representation, SDialog provides: (1) persona-driven multi-agent simulation with composable orchestration for controlled, synthetic dialog generation, (2) comprehensive evaluation combining linguistic metrics, LLM-as-a-judge and functional correctness validators, (3) mechanistic interpretability tools for activation inspection and steering via feature ablation and induction, and (4) audio generation with full acoustic simulation including 3D room modeling and microphone effects. The toolkit integrates with all major LLM backends, enabling mixed-backend experiments under a unified API. By coupling generation, evaluation, and interpretability in a dialog-centric architecture, SDialog enables researchers to build, benchmark and understand conversational systems more systematically.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 14 of its 27 references that are in the map. First posted on arXiv 9 Dec 2025.
Builds on
14 of 27 references.
Closest ancestors · 1
Same direction, newest first
Other references · 13
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
- 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
- 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
- 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
- 2017ParlAI: A Dialog Research Software Platformcited by 139
- 2017Pyroomacoustics: A Python package for audio room simulations and array processing algorithmscited by 131
- 2017Rasa: Open Source Language Understanding and Dialogue Managementcited by 67
- 2024Who's asking? User personas and the mechanics of latent misalignmentcited by 40
- 2021Assessing Dialogue Systems with Distribution Distancescited by 16
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025SDialog: A Python Toolkit for End-to-End Agent Building, User Simulation, Dialog Generation, and Evaluation
- 2024DialSim: A Dialogue Simulator for Evaluating Long-Term Multi-Party Dialogue Understanding of Conversational Agents
- 2026ToolRACER: A Robust Agentic Conversation Emulation Resource for Agent Training and Evaluation
- 2024Cohesive Conversations: Enhancing Authenticity in Multi-Agent Simulated Dialogues
- 2025APIDA-Chat: Structured Synthesis of API Search Dialogues to Bootstrap Conversational Agents
- 2026ATOD: An Evaluation Framework and Benchmark for Agentic Task-Oriented Dialogue Systems
- 2025MEDAL: A Framework for Benchmarking LLMs as Multilingual Open-Domain Dialogue Evaluators
- 2025Human vs. Agent in Task-Oriented Conversations