# Hardware Phi-1.5B: A Large Language Model Encodes Hardware Domain Specific Knowledge

arXiv: https://arxiv.org/abs/2402.01728 · HTML: https://mlsanity.com/paper/2402.01728/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 37 references that are in the map.
- First posted on arXiv 27 Jan 2024.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [Code Llama: Open Foundation Models for Code](https://mlsanity.com/paper/2308.12950/) (2023)
- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesis](https://mlsanity.com/paper/2203.13474/) (2022)
- [Textbooks Are All You Need II: phi-1.5 technical report](https://mlsanity.com/paper/2309.05463/) (2023)
- [The Stack: 3 TB of permissively licensed source code](https://mlsanity.com/paper/2211.15533/) (2022)
- [VeriGen: A Large Language Model for Verilog Code Generation](https://mlsanity.com/paper/2308.00708/) (2023)
- [Benchmarking Large Language Models for Automated Verilog RTL Code Generation](https://mlsanity.com/paper/2212.11140/) (2022)
- [On Joint Learning for Solving Placement and Routing in Chip Design](https://mlsanity.com/paper/2111.00234/) (2021)
- [ChiPFormer: Transferable Chip Placement via Offline Decision Transformer](https://mlsanity.com/paper/2306.14744/) (2023)
- [(Security) Assertions by Large Language Models](https://mlsanity.com/paper/2306.14027/) (2023)
- [Automatic Cross-Replica Sharding of Weight Update in Data-Parallel Training](https://mlsanity.com/paper/2004.13336/) (2020)
- [Fixing Hardware Security Bugs with Large Language Models](https://mlsanity.com/paper/2302.01215/) (2023)
- [LLM4DV: Using Large Language Models for Hardware Test Stimuli Generation](https://mlsanity.com/paper/2310.04535/) (2023)

## Built on by (most cited first, 10 of 10)

- [MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Context](https://mlsanity.com/paper/2407.02730/) (2024)
- [Cross-Modality Program Representation Learning for Electronic Design Automation with High-Level Synthesis](https://mlsanity.com/paper/2406.09606/) (2024)
- [Location is Key: Leveraging Large Language Model for Functional Bug Localization in Verilog](https://mlsanity.com/paper/2409.15186/) (2024)
- [Enhancing Large Language Models for Hardware Verification: A Novel SystemVerilog Assertion Dataset](https://mlsanity.com/paper/2503.08923/) (2025)
- [Large Language Model for Verilog Code Generation: Literature Review and the Road Ahead](https://mlsanity.com/paper/2512.00020/) (2025)
- [LLMPirate: LLMs for Black-box Hardware IP Piracy](https://mlsanity.com/paper/2411.16111/) (2024)
- [Revealing the Power of Post-Training for Small Language Models via Knowledge Distillation](https://mlsanity.com/paper/2509.26497/) (2025)
- [Natural Language to Verilog: Design of a Recurrent Spiking Neural Network using Large Language Models and ChatGPT](https://mlsanity.com/paper/2405.01419/) (2024)
- [Assessing Economic Viability: A Comparative Analysis of Total Cost of Ownership for Domain-Adapted Large Language Models versus State-of-the-art Counterparts in Chip Design Coding Assistance](https://mlsanity.com/paper/2404.08850/) (2024)
- [SecureRAG-RTL: A Retrieval-Augmented, Multi-Agent, Zero-Shot LLM-Driven Framework for Hardware Vulnerability Detection](https://mlsanity.com/paper/2603.05689/) (2026)

## Abstract (arXiv)

In the rapidly evolving semiconductor industry, where research, design, verification, and manufacturing are intricately linked, the potential of Large Language Models to revolutionize hardware design and security verification is immense. The primary challenge, however, lies in the complexity of hardware specific issues that are not adequately addressed by the natural language or software code knowledge typically acquired during the pretraining stage. Additionally, the scarcity of datasets specific to the hardware domain poses a significant hurdle in developing a foundational model. Addressing these challenges, this paper introduces Hardware Phi 1.5B, an innovative large language model specifically tailored for the hardware domain of the semiconductor industry. We have developed a specialized, tiered dataset comprising small, medium, and large subsets and focused our efforts on pretraining using the medium dataset. This approach harnesses the compact yet efficient architecture of the Phi 1.5B model. The creation of this first pretrained, hardware domain specific large language model marks a significant advancement, offering improved performance in hardware design and verification tasks and illustrating a promising path forward for AI applications in the semiconductor sector.