# Skilled AI Agents for Embedded and IoT Systems Development

arXiv: https://arxiv.org/abs/2603.19583 · HTML: https://mlsanity.com/paper/2603.19583/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 10 of its 24 references that are in the map.
- First posted on arXiv 20 Mar 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 10 of 10)

- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance](https://mlsanity.com/paper/2305.05176/) (2023)
- [BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions](https://mlsanity.com/paper/2406.15877/) (2024)
- [SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks](https://mlsanity.com/paper/2602.12670/) (2026)
- [AutoIOT: LLM-Driven Automated Natural Language Programming for AIoT Applications](https://mlsanity.com/paper/2503.05346/) (2025)
- [FreshStack: Building Realistic Benchmarks for Evaluating Retrieval on Technical Documents](https://mlsanity.com/paper/2504.13128/) (2025)
- [GPIoT: Tailoring Small Language Models for IoT Program Synthesis and Development](https://mlsanity.com/paper/2503.00686/) (2025)
- [IoT-MCP: Bridging LLMs and IoT Systems Through Model Context Protocol](https://mlsanity.com/paper/2510.01260/) (2025)
- [EmbedGenius: Towards Automated Software Development for Generic Embedded IoT Systems](https://mlsanity.com/paper/2412.09058/) (2024)
- [EmbedAgent: Benchmarking Large Language Models in Embedded System Development](https://mlsanity.com/paper/2506.11003/) (2025)

## Built on by (most cited first, 3 of 3)

- [Octopus Protocol: One-Shot Hardware Discovery and Control for AI Agents via Infrastructure-as-Prompts](https://mlsanity.com/paper/2605.09055/) (2026)
- [Device Context Protocol: A Compact, Safety-First Architecture for LLM-Driven Control of Constrained Devices](https://mlsanity.com/paper/2605.26159/) (2026)
- [Embedded Arena: Iterative Optimization via Hardware Feedback](https://mlsanity.com/paper/2606.16190/) (2026)

## Abstract (arXiv)

Large language models (LLMs) and agentic systems have shown promise for automated software development, but applying them to hardware-in-the-loop (HIL) embedded and Internet-of-Things (IoT) systems remains challenging due to the tight coupling between software logic and physical hardware behavior. Code that compiles successfully may still fail when deployed on real devices because of timing constraints, peripheral initialization requirements, or hardware-specific behaviors. To address this challenge, we introduce a skills-based agentic framework for HIL embedded development together with IoT-SkillsBench, a benchmark designed to systematically evaluate AI agents in real embedded programming environments. IoT-SkillsBench spans three representative embedded platforms, 23 peripherals, and 42 tasks across three difficulty levels, where each task is evaluated under three agent configurations (no-skills, LLM-generated skills, and human-expert skills) and validated through real hardware execution. Across 378 hardware validated experiments, we show that concise human-expert skills with structured expert knowledge enable near-perfect success rates across platforms.