mlsanityData to 7 Oct 2026

Hardware Phi-1.5B: A Large Language Model Encodes Hardware Domain Specific Knowledge

arXiv 2402.01728 ↗First posted 27 Jan 2024cs.CL cs.AI cs.ARcited by 10

Not on the map. Why?

Abstract (arXiv)

In the rapidly evolving semiconductor industry, where research, design, verification, and manufacturing are intricately linked, the potential of Large Language Models to revolutionize hardware design and security verification is immense. The primary challenge, however, lies in the complexity of hardware specific issues that are not adequately addressed by the natural language or software code knowledge typically acquired during the pretraining stage. Additionally, the scarcity of datasets specific to the hardware domain poses a significant hurdle in developing a foundational model. Addressing these challenges, this paper introduces Hardware Phi 1.5B, an innovative large language model specifically tailored for the hardware domain of the semiconductor industry. We have developed a specialized, tiered dataset comprising small, medium, and large subsets and focused our efforts on pretraining using the medium dataset. This approach harnesses the compact yet efficient architecture of the Phi 1.5B model. The creation of this first pretrained, hardware domain specific large language model marks a significant advancement, offering improved performance in hardware design and verification tasks and illustrating a promising path forward for AI applications in the semiconductor sector.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 19 of its 37 references that are in the map. First posted on arXiv 27 Jan 2024.

Builds on

19 of 37 references.

References · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  7. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  8. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  9. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 876
  10. 2023Textbooks Are All You Need II: phi-1.5 technical reportcited by 559
  11. 2022The Stack: 3 TB of permissively licensed source codecited by 354
  12. 2023VeriGen: A Large Language Model for Verilog Code Generationcited by 197
7 more
  1. 2022Benchmarking Large Language Models for Automated Verilog RTL Code Generationcited by 112
  2. 2021On Joint Learning for Solving Placement and Routing in Chip Designcited by 44
  3. 2023ChiPFormer: Transferable Chip Placement via Offline Decision Transformercited by 42
  4. 2023(Security) Assertions by Large Language Modelscited by 31
  5. 2020Automatic Cross-Replica Sharding of Weight Update in Data-Parallel Trainingcited by 30
  6. 2023Fixing Hardware Security Bugs with Large Language Modelscited by 24
  7. 2023LLM4DV: Using Large Language Models for Hardware Test Stimuli Generationcited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 3; 2026: 1.

10 papers.

  1. 2024MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Contextcited by 19
  2. 2024Cross-Modality Program Representation Learning for Electronic Design Automation with High-Level Synthesiscited by 10
  3. 2025Enhancing Large Language Models for Hardware Verification: A Novel SystemVerilog Assertion Datasetcited by 10
  4. 2024Location is Key: Leveraging Large Language Model for Functional Bug Localization in Verilogcited by 8
  5. 2024LLMPirate: LLMs for Black-box Hardware IP Piracycited by 7
  6. 2025Large Language Model for Verilog Code Generation: Literature Review and the Road Aheadcited by 7
  7. 2025Revealing the Power of Post-Training for Small Language Models via Knowledge Distillationcited by 6
  8. 2024Natural Language to Verilog: Design of a Recurrent Spiking Neural Network using Large Language Models and ChatGPTcited by 1
  9. 2024Assessing Economic Viability: A Comparative Analysis of Total Cost of Ownership for Domain-Adapted Large Language Models versus State-of-the-art Counterparts in Chip Design Coding Assistancecited by 0
  10. 2026SecureRAG-RTL: A Retrieval-Augmented, Multi-Agent, Zero-Shot LLM-Driven Framework for Hardware Vulnerability Detectioncited by 0