# On the Viability of using LLMs for SW/HW Co-Design: An Example in Designing CiM DNN Accelerators

arXiv: https://arxiv.org/abs/2306.06923 · HTML: https://mlsanity.com/paper/2306.06923/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 20 references that are in the map.
- First posted on arXiv 12 Jun 2023.

## Builds on (most cited first, 13 of 13)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [ChipGPT: How far are we from natural language hardware design](https://mlsanity.com/paper/2305.14019/) (2023)
- [Can GPT-4 Perform Neural Architecture Search?](https://mlsanity.com/paper/2304.10970/) (2023)
- [Hardware/Software Co-Exploration of Neural Architectures](https://mlsanity.com/paper/1907.04650/) (2019)
- [Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Search](https://mlsanity.com/paper/1901.11211/) (2019)
- [Device-Circuit-Architecture Co-Exploration for Computing-in-Memory Neural Accelerators](https://mlsanity.com/paper/1911.00139/) (2019)
- [Uncertainty Modeling of Emerging Device-based Computing-in-Memory Neural Accelerators with Application to Neural Architecture Search](https://mlsanity.com/paper/2107.06871/) (2021)
- [SWIM: Selective Write-Verify for Computing-in-Memory Neural Accelerators](https://mlsanity.com/paper/2202.08395/) (2022)
- [When Single Event Upset Meets Deep Neural Networks: Observations, Explorations, and Remedies](https://mlsanity.com/paper/1909.04697/) (2019)
- [Computing-In-Memory Neural Network Accelerators for Safety-Critical Systems: Can Small Device Variations Be Disastrous?](https://mlsanity.com/paper/2207.07626/) (2022)
- [Design Automation for Efficient Deep Learning Computing](https://mlsanity.com/paper/1904.10616/) (2019)

## Built on by (most cited first, 11 of 11)

- [On-Device Language Models: A Comprehensive Review](https://mlsanity.com/paper/2409.00088/) (2024)
- [MG-Verilog: Multi-grained Dataset Towards Enhanced LLM-assisted Verilog Generation](https://mlsanity.com/paper/2407.01910/) (2024)
- [A Survey of Research in Large Language Models for Electronic Design Automation](https://mlsanity.com/paper/2501.09655/) (2025)
- [AMSbench: A Comprehensive Benchmark for Evaluating MLLM Capabilities in AMS Circuits](https://mlsanity.com/paper/2505.24138/) (2025)
- [RealBench: Benchmarking Verilog Generation Models with Real-World IP Designs](https://mlsanity.com/paper/2507.16200/) (2025)
- [VeriCoder: Enhancing LLM-Based RTL Code Generation through Functional Correctness Validation](https://mlsanity.com/paper/2504.15659/) (2025)
- [Natural language is not enough: Benchmarking multi-modal generative AI for Verilog generation](https://mlsanity.com/paper/2407.08473/) (2024)
- [FL-NAS: Towards Fairness of NAS for Resource Constrained Devices via Large Language Models](https://mlsanity.com/paper/2402.06696/) (2024)
- [Muffin: A Framework Toward Multi-Dimension AI Fairness by Uniting Off-the-Shelf Models](https://mlsanity.com/paper/2308.13730/) (2023)
- [Towards Safer Heuristics With XPlain](https://mlsanity.com/paper/2410.15086/) (2024)
- [Energy Efficient Software Hardware CoDesign for Machine Learning: From TinyML to Large Language Models](https://mlsanity.com/paper/2603.23668/) (2026)

## Abstract (arXiv)

Deep Neural Networks (DNNs) have demonstrated impressive performance across a wide range of tasks. However, deploying DNNs on edge devices poses significant challenges due to stringent power and computational budgets. An effective solution to this issue is software-hardware (SW-HW) co-design, which allows for the tailored creation of DNN models and hardware architectures that optimally utilize available resources. However, SW-HW co-design traditionally suffers from slow optimization speeds because their optimizers do not make use of heuristic knowledge, also known as the ``cold start'' problem. In this study, we present a novel approach that leverages Large Language Models (LLMs) to address this issue. By utilizing the abundant knowledge of pre-trained LLMs in the co-design optimization process, we effectively bypass the cold start problem, substantially accelerating the design process. The proposed method achieves a significant speedup of 25x. This advancement paves the way for the rapid and efficient deployment of DNNs on edge devices.