# Foundations of Large Language Models

arXiv: https://arxiv.org/abs/2501.09223 · HTML: https://mlsanity.com/paper/2501.09223/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jan 2025.

## Built on by (most cited first, 14 of 14)

- [GRAM: A Generative Foundation Reward Model for Reward Generalization](https://mlsanity.com/paper/2506.14175/) (2025)
- [Large Language Models for Computer-Aided Design: A Survey](https://mlsanity.com/paper/2505.08137/) (2025)
- [Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation](https://mlsanity.com/paper/2503.06594/) (2025)
- [Breaking Down and Building Up: Mixture of Skill-Based Vision-and-Language Navigation Agents](https://mlsanity.com/paper/2508.07642/) (2025)
- [MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learning](https://mlsanity.com/paper/2603.25108/) (2026)
- [Efficiency vs. Alignment: Investigating Safety and Fairness Risks in Parameter-Efficient Fine-Tuning of LLMs](https://mlsanity.com/paper/2511.00382/) (2025)
- [SquarechiPO: Differentially Private and Robust chi2-Preference Optimization in Offline Direct Alignment](https://mlsanity.com/paper/2505.21395/) (2025)
- [Probing Preference Representations: A Multi-Dimensional Evaluation and Analysis Method for Reward Models](https://mlsanity.com/paper/2511.12464/) (2025)
- [Learning to Shop Like Humans: A Review-driven Retrieval-Augmented Recommendation Framework with LLMs](https://mlsanity.com/paper/2509.00698/) (2025)
- [SasAgent: Multi-Agent AI System for Small-Angle Scattering Data Analysis](https://mlsanity.com/paper/2509.05363/) (2025)
- [Generating Privacy Stories From Software Documentation](https://mlsanity.com/paper/2506.23014/) (2025)
- [LSM-MS2: A Foundation Model Bridging Spectral Identification and Biological Interpretation](https://mlsanity.com/paper/2510.26715/) (2025)
- [LLMs Can Assist with Proposal Selection at Large User Facilities](https://mlsanity.com/paper/2512.10895/) (2025)
- [LARAG: Link-Aware Retrieval Strategy for RAG Systems in Hyperlinked Technical Documentation](https://mlsanity.com/paper/2605.07517/) (2026)

## Abstract (arXiv)

This is a book about large language models. As indicated by the title, it primarily focuses on foundational concepts rather than comprehensive coverage of all cutting-edge technologies. The book is structured into six main chapters, each exploring a key area: pre-training, generative models, prompting, alignment, inference, and reasoning. It is intended for college students, professionals, and practitioners in natural language processing and related fields, and can serve as a reference for anyone interested in large language models.