mlsanityData to 14 Oct 2026

The DeepSpeak-Agentic Dataset

arXiv 2606.03686 ↗First posted 2 Jun 2026cs.AIcited by 0

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

We present DeepSpeak-Agentic, a dataset of videos comprising over 37 hours of semi-structured conversations between a human and an embodied AI agent. We use this dataset to evaluate the automatic forensic identification (audio, video, or text) of AI agents, study the nature of human-agent interactions, and provide a benchmark for future advances in the large-language models and AI-generated voices and faces that power embodied AI agents. We also contribute a scalable data-capture system that creates agents, automatically pairs them with human crowd workers, records audiovisual conversations across specified scenarios, and identifies and separates the human and agent in the combined stream.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 17 of its 26 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

17 of 26 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Evaluation and Benchmarking of LLM Agents: A Survey29 Jul
  2. 2023AgentBench: Evaluating LLMs as Agents7 Aug

Other references · 15

  1. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  2. 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 575
  3. 2024Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Textcited by 263
  4. 2021FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Datasetcited by 181
  5. 2024DF40: Toward Next-Generation Deepfake Detectioncited by 129
  6. 2022ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wildcited by 121
  7. 2021WaveFake: A Data Set to Facilitate Audio Deepfake Detectioncited by 78
  8. 2025Characterizing AI Agents for Alignment and Governancecited by 30
  9. 2025Deepfake Detection that Generalizes Across Benchmarkscited by 21
  10. 2025Speech DF Arena: A Leaderboard for Speech DeepFake Detection Modelscited by 21
  11. 2023GenConViT: Deepfake Video Detection Using Generative Convolutional Vision Transformercited by 13
  12. 2025Unlocking the Hidden Potential of CLIP in Generalizable Deepfake Detectioncited by 10
3 more
  1. 2025Can Vision-Language Models Answer Face to Face Questions in the Real-World?cited by 7
  2. 2024The DeepSpeak Datasetcited by 6
  3. 2025Diversity Boosts AI-Generated Text Detectioncited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026OpenThoughts-Agent: Data Recipes for Agentic Models
  2. 2025Everything Counts: The Managed Omnirelevance of Speech in Human-Voice Agent Interaction
  3. 2025AMUSE: Audio-Visual Benchmark and Alignment Framework for Agentic Multi-Speaker Understanding
  4. 2026Multi-Agent Forensic Reasoning for Generalizable Deepfake Video Detection
  5. 2025VoiceAssistant-Eval: Benchmarking AI Assistants across Listening, Speaking, and Viewing
  6. 2026HumanForge: A Human-Centric Deepfake Video Benchmark with Multi-Agent Forgery Rationales
  7. 2026Agentic and Generative AI for Open-Source Intelligence and Cyber Investigations: Taxonomy, Evaluation, Challenges, and Future Directions
  8. 2025Advancing Embodied Agent Security: From Safety Benchmarks to Input Moderation