Perceptual Rasterization: a novel patented method for image synthesis in head-mounted displays resulting in high field-of-view foveated images with low latency
Head-mounted displays (HMDs) have requirements beyond typical desktop display-based systems: HMDs must maintain low and predictable latency and must cover a significant proportion of the user's field of view at high resolution.
Today’s HMD technology has challenges which result in breaks-in-presence, simulator sickness, and reduced performance:
Current HMD graphics pipelines struggle to produce the images required. They are limited by the number of pixels that can be pushed from the GPU to the display, which strains the raw bandwidth of even the latest GPUs.
Traditional graphics pipelines compute images at a single snapshot time and ignore how the display is driven image will be perceived. Many displays support low persistence by scanning the image illumination, but this results in some parts of the screen appearing at lower latency than others.
The single pass perceptual rasterization method presented offers an improved approach for a graphics pipeline to deliver efficient, high-resolution foveated images with low latency. Solutions such as this may help further market adoption and realize the impactful future of virtual and augmented reality by achieving more comfortable viewing.
The unique combination of techniques included in this patented method include:
Rolling Rasterization: a method for GPUs to synthesize rolling images where the time at each pixel depends on its display location.
Foveation: a method to synthesize images with spatially varying pixel density. Important areas are larger and cover more pixels while less relevant areas are smaller, with fewer pixels.