Foveated sparse rendering for immersive displays

Purpose and goal:

Flexible and efficient image synthesis is the most important component in the development of visually interesting interactive applications for immersive displays in virtual reality (VR) and augmented reality (AR). This project will introduce a step change in image synthesis for head mounted displays and enable significant boosts in both rendering speed, as well as image quality.

We couple our patent-pending sparse rendering algorithms with so-called foveated rendering to develop foveated sparse rendering suitable for light-weight HMDs and other handheld devices. Expected results and effects:

Foveated sparse rendering will enable a speed up of up to 10x, which can be leveraged to either significantly reduce energy footprint and increase battery time, or to leave computational headroom to other important tasks running in parallel. For maximum applicability, we will develop both a software layer for application developers that can be incorporated into any renderer for head mounted displays, as well as a low-level PoC implementation that demonstrates how this, deep technology, can be integrated at GPU hardware level.

Approach and implementation:

This project will be carried out by Sparsit AB through four work packages in collaboration with Norrköping Visualisering AB and Lutra Interactive AB. We will also collaborate within the Wallenberg Autonomous Systems and Software Program Research Arena Media and Language and East Sweden Game. The work packages aim to develop: new perceptual methods for VR/AR rendering, development of foveated sparse rendering, a software library for integration into applications, and two demonstrators where the technology can be evaluated and validated in real world applications.