# BaseRT: Best-in-Class LLM Inference on Apple Silicon via Native Metal

arXiv: https://arxiv.org/abs/2607.00501 · HTML: https://mlsanity.com/paper/2607.00501/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 25 references that are in the map.
- First posted on arXiv 1 Jul 2026.

## Builds on (most cited first, 6 of 6)

- [Intelligence per Watt: Measuring Intelligence Efficiency of Local AI](https://mlsanity.com/paper/2511.07885/) (2025)
- [Agentic Artificial Intelligence (AI): Architectures, Taxonomies, and Evaluation of Large Language Model Agents](https://mlsanity.com/paper/2601.12560/) (2026)
- [DaDu-Corki: Algorithm-Architecture Co-Design for Embodied AI-powered Robotic Manipulation](https://mlsanity.com/paper/2407.04292/) (2024)
- [Edge-First Language Model Inference: Models, Metrics, and Tradeoffs](https://mlsanity.com/paper/2505.16508/) (2025)
- [Sherlock: Reliable and Efficient Agentic Workflow Execution](https://mlsanity.com/paper/2511.00330/) (2025)
- [Native LLM and MLLM Inference at Scale on Apple Silicon](https://mlsanity.com/paper/2601.19139/) (2026)

## Built on by (most cited first, 1 of 1)

- [BaseRT: Advancing Best-in-Class LLM Inference with Apple M5 Neural Accelerators](https://mlsanity.com/paper/2607.19438/) (2026)

## Abstract (arXiv)

We present BaseRT, a native Metal inference runtime for large language models (LLMs) on Apple Silicon, and report the highest inference throughput on this hardware to date. Existing runtimes, including llama.cpp and MLX-based frameworks, incur overhead from abstractions not designed for Metal's execution model or Apple Silicon's unified memory topology. By building natively on Metal with chip-specific kernel fusion, unified memory-aware optimisation, and custom dispatch logic, BaseRT recovers performance that framework-based approaches leave on the table. BaseRT supports a wide range of model families across eight quantisation formats (Q2 to FP16) on all Apple M-series devices. In this paper, we evaluate the Qwen3, Llama 3.2, and Gemma 4 families at Q4 and Q8 quantisation on M3 and M4 Pro devices. BaseRT achieves up to 1.56x higher decode throughput than llama.cpp and up to 1.35x higher than MLX, with substantially larger margins on prefill for mixture-of-experts models, delivering consistent best-in-class throughput from sub-1B to 30B parameter models. These results establish Apple Silicon as a more capable inference platform than previously reported, with direct implications for the emerging edge inference paradigm: as privacy requirements, latency constraints, and cloud cost pressures drive inference toward on-device deployment, performance-optimised local runtimes are a critical enabling layer for this transition. BaseRT is publicly available at https://github.com/basecompute/baseRT