mlsanityData to 14 Oct 2026

RTL-BenchMT: Dynamic Maintenance of RTL Generation Benchmark Through Agent-Assisted Analysis and Revision

arXiv 2605.15537 ↗First posted 15 May 2026cs.AIcited by 2

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

This paper introduces RTL-BenchMT, an agentic framework for dynamically maintaining RTL generation benchmarks. Large Language Models (LLMs) assisted automated RTL generation is one of the most important directions in EDA research. However, current RTL benchmarks face two critical challenges: (1) flawed cases in the benchmarks and (2) overfitting to the benchmarks. Both challenges are difficult to resolve purely by manual engineering effort. To address these issues and systematically reduce human maintenance costs, we propose an automated agentic framework, RTL-BenchMT. RTL-BenchMT focuses on two key applications: (1) automatically identifying and revising flawed benchmark cases and (2) automatically detecting and updating overfitting cases. With the assistance of RTL-BenchMT, we conduct a thorough, in-depth analysis of flawed and overfitting cases and produce a refined benchmark suite that will be open-sourced to the community.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 9 of its 18 references that are in the map. First posted on arXiv 15 May 2026.

Builds on

9 of 18 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024MAGE: A Multi-Agent Engine for Automated RTL Code Generation10 Dec
  2. 2024VerilogCoder: Autonomous Verilog Coding Agents with Graph-based Planning and Abstract Syntax Tree (AST)-based Waveform Tracing Tool15 Aug

Other references · 7

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  2. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  3. 2023VerilogEval: Evaluating Large Language Models for Verilog Code Generationcited by 252
  4. 2024BetterV: Controlled Verilog Generation with Discriminative Guidancecited by 100
  5. 2024VerilogReader: LLM-Aided Hardware Test Generationcited by 17
  6. 2025Forget What You Know about LLMs Evaluations -- LLMs are Like a Chameleoncited by 13
  7. 2025DecoRTL: A Run-time Decoding Framework for RTL Code Generation with LLMscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Agentic Electronic Design Automation: A Handoff Perspectivecited by 0
  2. 2026iCoder-27B: Recursive AI-Led Development of Frontier Industrial Coding Modelcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024EDA-Aware RTL Generation with Large Language Models
  2. 2026Dr. RTL: Autonomous Agentic RTL Optimization through Tool-Grounded Self-Improvement
  3. 2025PEFA-AI: Advancing Open-source LLMs for RTL generation using Progressive Error Feedback Agentic-AI
  4. 2026EvalBuilder: Can LLMs Build Efficient Benchmarks?
  5. 2026ACE-RTL: When Agentic Context Evolution Meets RTL-Specialized LLMs
  6. 2025TuRTLe: A Unified Evaluation of LLMs for RTL Generation
  7. 2026RTL-BenchLS: A Large-Scale Benchmark for RTL Reasoning and Generation with Large Language Models
  8. 2026ChipCraftBrain: Validation-First RTL Generation via Multi-Agent Orchestration