mlsanityData to 14 Oct 2026

SDialog: A Python Toolkit for End-to-End Agent Building, User Simulation, Dialog Generation, and Evaluation

arXiv 2506.10622 ↗First posted 12 Jun 2025cs.CL cs.AI cs.LGcited by 3

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

also in 1 more
Abstract (arXiv)

We present SDialog, an MIT-licensed open-source Python toolkit that unifies dialog generation, evaluation and mechanistic interpretability into a single end-to-end framework for building and analyzing LLM-based conversational agents. Built around a standardized Dialog representation, SDialog provides: (1) persona-driven multi-agent simulation with composable orchestration for controlled, synthetic dialog generation, (2) comprehensive evaluation combining linguistic metrics, LLM-as-a-judge and functional correctness validators, (3) mechanistic interpretability tools for activation inspection and steering via feature ablation and induction, and (4) audio generation with full acoustic simulation including 3D room modeling and microphone effects. The toolkit integrates with all major LLM backends, enabling mixed-backend experiments under a unified API. By coupling generation, evaluation, and interpretability in a dialog-centric architecture, SDialog enables researchers to build, benchmark and understand conversational systems more systematically.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 14 of its 27 references that are in the map. First posted on arXiv 12 Jun 2025.

Builds on

14 of 27 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debate14 Aug

Other references · 13

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  3. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  5. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 828
  6. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  7. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 451
  8. 2017ParlAI: A Dialog Research Software Platformcited by 139
  9. 2017Pyroomacoustics: A Python package for audio room simulations and array processing algorithmscited by 132
  10. 2017Rasa: Open Source Language Understanding and Dialogue Managementcited by 67
  11. 2024Who's asking? User personas and the mechanics of latent misalignmentcited by 40
  12. 2021Assessing Dialogue Systems with Distribution Distancescited by 16
1 more
  1. 2024Dialog2Flow: Pre-training Soft-Contrastive Action-Driven Sentence Embeddings for Automatic Dialog Flow Extractioncited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025SDialog: A Python Toolkit for End-to-End Agent Building, User Simulation, Dialog Generation, and Evaluation
  2. 2024DialSim: A Dialogue Simulator for Evaluating Long-Term Multi-Party Dialogue Understanding of Conversational Agents
  3. 2024Cohesive Conversations: Enhancing Authenticity in Multi-Agent Simulated Dialogues
  4. 2025MEDAL: A Framework for Benchmarking LLMs as Multilingual Open-Domain Dialogue Evaluators
  5. 2025Evaluating LLM-based Agents for Multi-Turn Conversations: A Survey
  6. 2026ToolRACER: A Robust Agentic Conversation Emulation Resource for Agent Training and Evaluation
  7. 2025Human vs. Agent in Task-Oriented Conversations
  8. 2026ATOD: An Evaluation Framework and Benchmark for Agentic Task-Oriented Dialogue Systems