mlsanityData to 7 Oct 2026

Multimodal and embodied agents

1,469 papers19 this weekLow confidence in this grouping: the papers may belong to more than one topic, or the group may changeLow confidence in this grouping: the papers may belong to more than one topic, or the group may change · About

Keywords: visual, reasoning, multimodal, framework, agent

Key facts. Multimodal and embodied agents: 1,469 papers on arXiv since 2022 (data through 7 Oct 2026). 74 of its 1,469 mapped papers (5%) cite “Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents” (2022, arXiv 2201.07207). 69 of its 1,469 mapped papers (5%) cite “VideoAgent: Long-form Video Understanding with Large Language Model as Agent” (2024, arXiv 2403.10517). 113 new papers 17 Sep–7 Oct.

What’s new

17 Sep – 7 Oct 2026

113 new papers, 17 Sep – 7 Oct; 19 this week.

  1. This week · 19
  2. iAm.md: Robot Skill Self-Assessment through Agentic Introspection for Unknown Open-Vocabulary Domains
  3. NavGPT-3: Harnessing Context in a Hierarchical Navigation Runtime also fits: Reinforcement learning for LLM agents
  4. Speaking the Navigator's Language: Trajectory-Grounded Instruction Translation for Frozen Aerial VLN Agents
  5. Agentic RSR: Real-to-Sim-to-Real through Scene Reconstruction and Execution-Grounded Robot Policies
  6. RoboQuest: Generalist Physical Agents that Search, Inspect and Test
  7. HarnessIR: Harnessing Multimodal Foundation Models for Universal Real-World Image Restoration
  8. DUDA-Bench: Benchmarking LLM Agents on Multimodal Data-Driven Urban Diagnosis also fits: Web, search, and research agents
  9. Adversarial Images Hijack Web Agents from Visual Grounding to Browser Execution also fits: GUI and mobile agents
  10. RACER: Reflective Agent Coupling Query Interpretation and Tool-Based Retrieval for Frame Selection in Long Video Understanding
  11. WorldSolver: Can LLM Agents Simulate the Physical Dynamics via Solver Generation? also fits: General LLM agent frameworks and benchmarks
All 113
  1. Visual Orchestration Tax in Agentic VLM Pipelines: Auditing and Certifying Visual Evidence Reuse
  2. M3SunAgent: Monocular 3D Spatial Understanding Agent for Metric Depth Estimation and 3D Visual Grounding
  3. CueRator: Agentic Search for Symbolic Rules to Adapt Frozen Multimodal Encoders
  4. OntoPlan: An Ontology-Grounded Scene Representation and Agentic Framework for Scalable Robot Task Planning
  5. CIRRA: Dual-Level Continual Instruction Reconciliation with Ongoing Execution for Embodied Robot Agents in Interactive Household Tasks also fits: General LLM agent frameworks and benchmarks
  6. Internalizing Agent Experience into Diffusion Model Weights via On-Policy Context Distillation
  7. One Figure, Every Canvas: Editable Flowchart Relayout via Agentic Pipeline
  8. Recursive Video In-Context Learning for Agentic Robot
  9. From Social Reasoning to Embodied Interaction: An Agentic Framework for Social Robots
  10. Earlier · 94
  11. IRSTD-Agent: Agentic Infrared Small Target Detection via Zoom-Guided Interaction Learning
  12. Code2Games: Enabling Coding Agents for Gaming World Generation also fits: Software engineering coding agents
  13. PreAct-Nav: Agentic Reasoning Before Action for Urban Navigation
  14. VideoResearchAgent: Grounded Task Synthesis and Sim-to-Real RL for Open-Web Video Research also fits: Web, search, and research agents
  15. Video2World: Benchmarking Coding Agents for Interactive World Modeling from Embodied Videos
  16. LMBuild: Evaluating LLM Agents for Generating Buildable and Functional Structures also fits: Software engineering coding agents
  17. CADForge: Agentic Single-View CAD Reconstruction with Explicit Geometry Reasoning
  18. Agentic AI with Structured CoT for Enhancing AI's Spatial Intelligence: Visualization and Reasoning of Rotation
  19. WebFovea: When the Model Is Right but the Click Is Wrong -- Reliable Round Trips for Vision-Based Web Agents on Live Websites also fits: Web, search, and research agents
  20. WAMJET: A Harness for World Action Model Acceleration
  21. MeshQuery: Agentic Seam Planning for UV Parametrization
  22. OpenRUA: Robot-Use Agents Are Zero-Shot Visuomotor Policies also fits: Software engineering coding agents
  23. A Simulation-Grounded Agentic VLM Framework for Wildfire Monitoring and Reporting
  24. EditHero: A Benchmark for Long-Horizon Part-Level 3D Editing and Vibe Modeling
  25. Awomo-SimDataEngine: Agentic Simulation-ReadyWorld Generation
  26. OmniSeek: Native Tool Integration for Multi-turn Audio-Visual Reasoning
  27. Form and Void: Entangled Composition through an Autonomous AI Agent
  28. Fewer Tokens, Better Action: GPT-6 Astra Robot Agents with 14% Higher Success Rate but 65% Fewer Tokens
  29. LiteReality-Agent: An Agentic System for Interactable 3D Indoor Scene Reconstruction
  30. VideoEvolve: Evolving Agent Harnesses for Video Temporal Grounding
  31. Are Frontier VLM Agents Ready to Be Robot Generalists? An Empirical Study with the Embodied Agent Arena
  32. Spatial Strategies, Not Actions: Vector-Quantized Geodesics as Tools for LLM-Driven Agents
  33. WorldAuditBench: Interactive 3D World Auditing with Multimodal Agents
  34. EviRover: Reinforcing Agentic Perception Beyond a Glance
  35. CoEvoWhen: Policy-Tool Coevolution for Ultra-Long Video Temporal Grounding
  36. NavHarness: Adaptive Goals for Agentic Vision-Language Navigation
  37. LIBERO-Agent: Evaluating General-Purpose Agents for Direct Embodied Manipulation
  38. Scale and Selection: What Makes Automatic Harness Evolution Work for Visual-Interface Robot Agents also fits: Reinforcement learning for LLM agents
  39. ASENA: Self-evolving Agents for Embodied Navigation
  40. Asking the World: Generalist Physical Reasoning through Agentic World Modeling and Probing
  41. CamAgent: An LLM-Agent Framework for Multi-Species Camera-Trap Workflows
  42. Agentic Tool-Augmented Reasoning for Explainable Image Forgery Detection
  43. Learning to Route in Visual Space via Multi-Step Embedding Retrieval
  44. Text2Sim: Agentic Physics-Based Simulation Generation with Distilled Expertise
  45. DuplexSpeechBench-Document Grounding: Benchmarking Document Grounding and Hallucinations in Voice Agents also fits: General LLM agent frameworks and benchmarks
  46. Vision-Language-Action Autonomous Driving Agent with Language-based Memory
  47. VidHarness: Evolving Agent Harnesses for Cost-Efficient Long Video Understanding
  48. MotorMind: Scaffolding General Vision Language Models for Zero-Shot Robot Manipulation
  49. Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution
  50. Exemplar2VQA: A Scalable Exemplar-Driven Visual Question Answering Generation Framework via Multi-Agent Coding
  51. Encore: Few-Shot Agentic Discovery of Manipulation Strategies also fits: General LLM agent frameworks and benchmarks
  52. HaPRL: Human-Anchored Process Reinforcement Learning for Visual Search Agent also fits: Reinforcement learning for LLM agents
  53. Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes
  54. DualManip: Agentic Dynamic Manipulation via Dual-Path Semantic Reasoning and Geometric Adaptation
  55. Systematic Multi-Agent Vision-and-Language Navigation: Formulation, Benchmark, and Method
  56. VehicleArena: A Realistic Urban Environment for Multi-Agent Driving
  57. AutoRef: Harness Optimization for Agentic Multi-Reference Image Generation
  58. A.D.A.M.O. (Agent for language-Driven Actions with Multimodal Observations): A Visual-Symbolic Framework for Virtual Humans
  59. DexAgent: An Agentic Human2Sim2Robot Framework for Dexterous Manipulation with Self-Evolving Tool Library
  60. Timeline-Bench: Evaluating Agents on Realistic Video-Editing Tasks, from Raw Footage to Final Cut
  61. JRDB-AVR: An Active Visual Reasoning Benchmark for Embodied Agents in Real-World Environments
  62. V-Gym: Enhancing Agentic Visual Reasoning via Skill-Data Co-Evolution
  63. PainterBench: A Figural Divergent-Thinking Benchmark for Tool-Using Language Models
  64. CodeActionBench: Evaluating Agentic Code-as-Policy for Embodied Manipulation
  65. StoryEngine: A State-Grounded Agentic Framework for Video Storytelling
  66. DataMagic: Authoring Data Videos through Declarative Multi-Agent Orchestration
  67. WorldAgent: Verification-Guided Agentic Physical World Construction
  68. Vision-Language Agents for Active Perception in Optics Laboratories
  69. OmniSmartHome: A Multimodal Reasoning Benchmark for Smart-Home Agents
  70. CaptchaArena: A Large-Scale, Fine-Grained Dataset for Training Computer-Use Agents on Interactive CAPTCHAs also fits: Reinforcement learning for LLM agents
  71. CueKFS: Agentic Cue-Driven Keyframe Selection for Long Video Understanding
  72. TemplateCraft: Agentic Visual Template Generation
  73. CognitiveReality: Robot-Agnostic Semantic Gaussian Mapping with an LLM Agent for Immersive Collaborative VR Teleoperation
  74. SciHorizon-eLab: An Agentic Protocol-to-Task Compiler for Scalable Benchmarking of Scientific Embodied Agents also fits: Agent skill learning and evolution
  75. MVAgent: Multi-Agent Video Generation via Consistent Condition Construction and Shot-Level Policy Optimization
  76. RAPID: Robot Agentic Programming from Demonstrations
  77. World Action Agent: Harnessing VLMs for Robot Manipulation via World Action Rehearsal
  78. Robo-Harness K1: Harnessing Robot-Use Agents via Perception Augmentation
  79. From Passive Execution to Active Exploration: Agentic Embodied Manipulation in Realistic Environments
  80. Beneath the Scores: Rethinking Hallucination Evaluation for Video Understanding Models
  81. Signals of AI Hallucination: Designing Hallucination-Aware Cues for Embodied Conversational Agents in VR also fits: Social simulation and multi-agent debate
  82. AstraLOD3: Zero-shot multimodal agentic reconstruction of LOD3 building models
  83. EmbodiedSWE: Coding Agents for Long Horizon Dexterous Robotics
  84. Generalizing Manipulation Skills with a Local Coding Agent
  85. RoboFollow: Unveiling the Instruction Following Mirage in Embodied Agents
  86. Qwen3.8-Omni: Towards Native Omni-Modal Agents
  87. Agentic Building-Aware Satellite Gaussian Splatting for Auditable Urban DSM Reconstruction
  88. Agentic Video Understanding: A Survey
  89. ARSTAG: An Agentic Real2Sim2Real System for Task-Specific Robot Data Generation
  90. ME-VLM: A Unified VLM for Embodied Cognition and Agent Coordination
  91. VLM-in-Sandbox: Visual Workspaces for Agentic Visual Reasoning
  92. A2Safe: Counterfactual Evidence-Aligned Adaptive Agent Collaboration for Safe and Effective Visual Question Answering also fits: Medical and scientific domain agents
  93. WidgetVA: A Widget-Centric Framework and Benchmark for Agentic Visual Analytics
  94. Structured World-State Reasoning for Agentic Robotic Search
  95. AR-WAM: A Visual-Conditioned Agent-Ready World Action Model for Robotic Manipulation
  96. MM-ContextFold: Context Folding for Multimodal Agentic Retrieval
  97. DiagGen: Agentic Generation of Deformable Assets with Sim-based Diagnostics for Robotic Simulation
  98. An Evolutionary Agentic Approach for Open-ended Image Quality Perception
  99. "Dear LLaVA, Please Drive": A Depth-Aware Vision-Language Agent for Closed-Loop Robotic Control
  100. SkillIR: Evolving Scene-Aware Skills for Agentic Image Restoration
  101. AgentVidBench: A Multi-Hop Video Question Answering Benchmark for Evaluating MLLM Agents
  102. StageGuard: Learning Stage Transitions for Long-Horizon Robot Tasks via Agentic Distillation also fits: Reinforcement learning for LLM agents
  103. Semantic Action Graph: A Shared Representation for Agent Grounding and Human Interpretation of Sports Highlights
  104. RoboFind: Multi-Agent Personalized Object Search for People Who Are Blind or Have Low Vision

Foundations

Most cited by papers here

  1. Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agents2022cited by 74 here
  2. VideoAgent: Long-form Video Understanding with Large Language Model as Agent2024cited by 69 here
  3. VideoAgent: A Memory-augmented Multimodal Agent for Video Understanding2024cited by 61 here
  4. LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Models2022cited by 51 here
  5. EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents2025cited by 40 here

Lineage

The papers this direction builds on, oldest first.

Each paper on the line cites the one before.

  1. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 133 here
  2. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 119 here
  3. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 47 here
  4. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 106 here
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 93 here
  6. 2023A Survey on Large Language Model based Autonomous Agentscited by 89 here
  7. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 67 here
  8. 2024VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Taskscited by 39 here
  9. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 44 here
  10. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 36 here
14 more it builds on, not in this chain
  1. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 74 here
  2. 2022LLM-Planner: Few-Shot Grounded Planning for Embodied Agents with Large Language Modelscited by 51 here
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 146 here
  4. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 186 here
  5. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 62 here
  6. 2023Gorilla: Large Language Model Connected with Massive APIscited by 37 here
  7. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 63 here
  8. 2023AgentBench: Evaluating LLMs as Agentscited by 38 here
  9. 2023CogAgent: A Visual Language Model for GUI Agentscited by 39 here
  10. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 69 here
  11. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 61 here
  12. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 40 here
  13. 2025Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understandingcited by 36 here
  14. 2025WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agentcited by 37 here

The 24 most cited of 1,984.

Browse

30 of 1,469, ranked 6–35

  1. 6WebWatcher: Breaking New Frontier of Vision-Language Deep Research Agent202537
  2. 7Deep Video Discovery: Agentic Search with Tool Use for Long-form Video Understanding202536
  3. 8Embodied Agent Interface: Benchmarking LLMs for Embodied Decision Making202432
  4. 9DeepEyesV2: Toward Agentic Multimodal Model202530
  5. 10SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Code202430
  6. 11GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editing202429
  7. 12Kimi K2.5: Visual Agentic Intelligence202628
  8. 13RS-Agent: Automating Remote Sensing Tasks through Intelligent Agent202424
  9. 14ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks202522
  10. 15LVAgent: Long Video Understanding by Multi-Round Dynamical Collaboration of MLLM Agents202520
All 30
  1. 16Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agents202520
  2. 17CaP-X: A Framework for Benchmarking and Improving Coding Agents for Robot Manipulation202619
  3. 18An Intelligent Agentic System for Complex Image Restoration Problems202419
  4. 19PyVision: Agentic Vision with Dynamic Tooling202518
  5. 20Seeing, Listening, Remembering, and Reasoning: A Multimodal Agent with Long-Term Memory202518
  6. 21Large Language Models as Urban Residents: An LLM Agent Framework for Personal Mobility Generation202418
  7. 22VCA: Video Curious Agent for Long Video Understanding202418
  8. 23Automated Movie Generation via Multi-Agent CoT Planning202517
  9. 24Agent AI: Surveying the Horizons of Multimodal Interaction202417
  10. 25MatPlotAgent: Method and Evaluation for LLM-Based Agentic Scientific Data Visualization202417
  11. 26EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents202516
  12. 27Agentic Robot: A Brain-Inspired Framework for Vision-Language-Action Models in Embodied Agents202515
  13. 28LoTa-Bench: Benchmarking Language-oriented Task Planners for Embodied Agents202415
  14. 29RestoreAgent: Autonomous Image Restoration Agent via Multimodal Large Language Models202415
  15. 30An Embodied Generalist Agent in 3D World202315
  16. 31LLMLight: Large Language Models as Traffic Signal Control Agents202315
  17. 32SAGE: Scalable Agentic 3D Scene Generation for Embodied AI202614
  18. 33Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generation202514
  19. 34LongVT: Incentivizing "Thinking with Long Videos" via Native Tool Calling202514
  20. 35LLaVA-Plus: Learning to Use Tools for Creating Multimodal Agents202314