VEHBench: A Stage-Local Diagnostic Benchmark for LLM-Assisted Vibration Energy Harvester Design
Not on the map. Why?
Abstract (arXiv)
Battery-free Internet of Things (IoT) requires iterative design of vibration energy harvesters (VEHs) under coupled physical constraints, while LLMs are emerging as interface layers for engineering workflows. However, existing engineering benchmarks primarily assess final artifact validity, offering limited insights into how LLMs behave across different stages of coupled physical design. We introduce VEHBench, an engineering-native diagnostic benchmark for LLM-assisted VEH design, featuring 763 literature-grounded tasks scored by an analytical physical oracle. VEHBench evaluates four design roles: specification triage, verifier-guided search, corrupted-state recovery, and policy-conditioned selection. Experimental results reveal that LLM capability is strongly stage-dependent: no single model consistently dominates the entire workflow, and response-control profiles expose distinct behavioral patterns across design roles. VEHBench thus provides a stage-aware foundation for evaluating, selecting, routing, and improving verifier-grounded engineering LLMs. The benchmark artifact is available at https://huggingface.co/datasets/AnonymousVehbench/vehbench
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 10 of its 37 references that are in the map. First posted on arXiv 20 Jul 2026.
Builds on
10 of 37 references.
References · 10
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
- 2016The CMA Evolution Strategy: A Tutorialcited by 377
- 2022Holistic Evaluation of Language Modelscited by 131
- 2024Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the datacited by 51
- 2024DesignQA: A Multimodal Benchmark for Evaluating Large Language Models' Understanding of Engineering Documentationcited by 32
- 2024Large Language Model Agent as a Mechanical Designercited by 21
- 2025Toward Engineering AGI: Benchmarking the Engineering Design Capabilities of LLMscited by 20
- 2025Probe-Rewrite-Evaluate: A Workflow for Reliable Benchmarks and Quantifying Evaluation Awarenesscited by 9
- 2025BuildArena: A Physics-Aligned Interactive Benchmark of LLMs for Engineering Constructioncited by 9
- 2024MSEval: A Dataset for Material Selection in Conceptual Design to Evaluate Algorithmic Modelscited by 1
Built on by
Not cited yet.