# Perceptual Rasterization for Head-mounted Display Image Synthesis

arXiv: https://arxiv.org/abs/1806.05385 · HTML: https://mlsanity.com/paper/1806.05385/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jun 2018.

## Built on by (most cited first, 3 of 3)

- [VR-Splatting: Foveated Radiance Field Rendering via 3D Gaussian Splatting and Neural Points](https://mlsanity.com/paper/2410.17932/) (2024)
- [GazeGPT: Augmenting Human Capabilities using Gaze-contingent Contextual AI for Smart Eyewear](https://mlsanity.com/paper/2401.17217/) (2024)
- [Visual Acuity Consistent Foveated Rendering towards Retinal Resolution](https://mlsanity.com/paper/2503.23410/) (2025)

## Abstract (arXiv)

We suggest a rasterization pipeline tailored towards the need of head-mounted displays (HMD), where latency and field-of-view requirements pose new challenges beyond those of traditional desktop displays. Instead of rendering and warping for low latency, or using multiple passes for foveation, we show how both can be produced directly in a single perceptual rasterization pass. We do this with per-fragment ray-casting. This is enabled by derivations of tight space-time-fovea pixel bounds, introducing just enough flexibility for requisite geometric tests, but retaining most of the the simplicity and efficiency of the traditional rasterizaton pipeline. To produce foveated images, we rasterize to an image with spatially varying pixel density. To reduce latency, we extend the image formation model to directly produce "rolling" images where the time at each pixel depends on its display location. Our approach overcomes limitations of warping with respect to disocclusions, object motion and view-dependent shading, as well as geometric aliasing artifacts in other foveated rendering techniques. A set of perceptual user studies demonstrates the efficacy of our approach.