# TPP: Transparent Page Placement for CXL-Enabled Tiered-Memory

arXiv: https://arxiv.org/abs/2206.02878 · HTML: https://mlsanity.com/paper/2206.02878/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2022.

## Built on by (most cited first, 10 of 10)

- [COMET: A Comprehensive Cluster Design Methodology for Distributed Deep Learning Training](https://mlsanity.com/paper/2211.16648/) (2022)
- [Herding LLaMaS: Using LLMs as an OS Module](https://mlsanity.com/paper/2401.08908/) (2024)
- [PIFS-Rec: Process-In-Fabric-Switch for Large-Scale Recommendation System Inferences](https://mlsanity.com/paper/2409.16633/) (2024)
- [Harmonia: Enhancing Data Placement and Migration in Hybrid Storage Systems via Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2503.20507/) (2025)
- [Vulcan: Instance-specialized, Verifiable Systems Heuristics Through LLM-driven Search](https://mlsanity.com/paper/2512.25065/) (2025)
- [A CXL Memory Rack for Multi-Turn LLM Serving](https://mlsanity.com/paper/2607.18141/) (2026)
- [Analysis and Optimization of GNN-Based Recommender Systems on Persistent Memory](https://mlsanity.com/paper/2207.11918/) (2022)
- [Topology-Aware Data Movement for Disaggregated GPU Inference](https://mlsanity.com/paper/2607.28633/) (2026)
- [Heterogeneous LLM Serving with General-Purpose Processing-Near-Memory for Retrieval-Based Sparse Attention](https://mlsanity.com/paper/2608.03555/) (2026)
- [Composable CXL Memory as a Kubernetes-Native Shared Memory for LLM Serving](https://mlsanity.com/paper/2609.10790/) (2026)

## Abstract (arXiv)

The increasing demand for memory in hyperscale applications has led to memory becoming a large portion of the overall datacenter spend. The emergence of coherent interfaces like CXL enables main memory expansion and offers an efficient solution to this problem. In such systems, the main memory can constitute different memory technologies with varied characteristics. In this paper, we characterize memory usage patterns of a wide range of datacenter applications across the server fleet of Meta. We, therefore, demonstrate the opportunities to offload colder pages to slower memory tiers for these applications. Without efficient memory management, however, such systems can significantly degrade performance. We propose a novel OS-level application-transparent page placement mechanism (TPP) for CXL-enabled memory. TPP employs a lightweight mechanism to identify and place hot/cold pages to appropriate memory tiers. It enables a proactive page demotion from local memory to CXL-Memory. This technique ensures a memory headroom for new page allocations that are often related to request processing and tend to be short-lived and hot. At the same time, TPP can promptly promote performance-critical hot pages trapped in the slow CXL-Memory to the fast local memory, while minimizing both sampling overhead and unnecessary migrations. TPP works transparently without any application-specific knowledge and can be deployed globally as a kernel release. We evaluate TPP in the production server fleet with early samples of new x86 CPUs with CXL 1.1 support. TPP makes a tiered memory system performant as an ideal baseline (<1% gap) that has all the memory in the local tier. It is 18% better than today's Linux, and 5-17% better than existing solutions including NUMA Balancing and AutoTiering. Most of the TPP patches have been merged in the Linux v5.18 release.