Cutting Jank and Load Times in the Cloud
Games are an interesting medium, where visualization, controls and sound become an extension of your experience — until your vision stutters and you jump 2 feet forward from where you were standing. As an immersive experience, we also expect a smooth visual experience and transitions between areas to happen quickly. It can feel intolerable if a load screen takes a minute or two to complete!
Janky gameplay due to pipeline cache misses. Ubisoft worked hard to make their game look great, and that included shipping their game with a comprehensive pipeline cache. To force this stuttering, we disabled their pipeline cache.
A game is made up of many kinds of files; movies, audio, the game binary itself, textures (images and descriptions of objects), etc… Some of these artifacts aren’t actually pre-generated, though, at least not fully — notably graphics pipelines, graphics shader workloads, and pipeline caches.
In this post, we’ll be talking about graphics pipelines and pipeline caches. Compilation of graphics pipelines is one of the common reasons that load screens can take a long time, and why your visuals may jump and stutter (a.k.a. jank). Jank can occur for a number of reasons, but jank resulting from compiling pipelines can be fixed by storing the compiled pipelines in a pipeline cache, which is essentially just an object that stores the compiled pipelines for a given system. We'll provide a peek into how Stadia works to provide pipeline caches for players running games on Stadia and, thanks to cloud data aggregation, how Stadia can remove this kind of stutter across all users over time. If you notice a game running better than it used to, it may be thanks to players like you doing your part by playing the game!
Stadia uses the Vulkan graphics API, which uses compute and graphics pipelines to run a game. The core of a pipeline is the shader, which is a small program for something in the game. Some simple games have only a handful of pipelines, while large titles can have hundreds of thousands. The time that it takes to compile a pipeline also varies wildly, but let’s assume they take 100ms on average. If a game needs to build 600 pipelines in a load screen, then there's about a minute of work to be done. A more powerful GPU can help, but it only gets you so far and at significant expense.
The compilation problem can also arise during gameplay when a new element is encountered. Say you're exploring a world and a new monster comes along that needs 10 pipelines to be compiled so that it can be rendered. Some games will drop all frames until it's ready, which results in stuttering, hitching or other jank. Others will continue to render the rest of the scene until the monster is ready, which can also be jarring when the monster suddenly pops into the scene.
A pipeline cache addresses these issues by storing the compiled pipelines. The 100ms compilation time for a pipeline becomes a <1ms lookup when it's found in the pipeline cache, so that the one minute of load screen work now takes under a second, and the pipelines for the monster are ready in the same frame as where they were invoked.
Compiled pipelines in a pipeline cache are only valid on hardware and graphics drivers that are compatible with the ones where the pipeline cache was created. When either of these changes, the compiled pipelines may no longer work and the pipeline will need to be recompiled. On a console, many of the compiled pipelines can be collected and shipped with the game since the hardware and drivers are stable, but can experience cache misses. For PC games, pipelines are typically compiled during the game installation process or on a pipeline cache miss, in which case they are written to the PC's local storage to avoid future cache miss issues for that user.
Graphics Pipelines and Stadia
For cloud gaming, we know we can do better to prevent pipeline cache misses before a player plays a game. Stadia leverages the capabilities of more modern graphics architectures to generate pre-compiled graphics pipelines. Game developers can include the data needed to construct a pipeline cache, which we call pipeline cache source data. This data includes the shader programs, as well as the data used to build render passes, descriptor sets, and other objects used to build the pipelines for their games. With this source data, we can ensure that we can compile the pipelines to populate a game's pipeline cache ahead of updates to Stadia’s hardware fleet and graphics drivers. This way, issues resulting from pipeline compilation are prevented for players launching games on any configuration of hardware and graphics drivers in the fleet.
Capturing all pipelines for a game can be a monumental or even impossible task. Pipelines may have computer generated properties, where parts of different systems combine at runtime to generate a new pipeline. This can be evident in games like Destiny 2 where a player can mix and match parts and color combinations on their avatar, but the armor itself has shine and reflection and coarse textures. The combination of all of these static shaders with the dynamic shaders that get added during gameplay (like on Destiny 2 running around on Titan while standing in a Solar Grenade) make up the graphics pipelines that need to be compiled. Each of these combinations are hard to generate ahead of time so there are still some hiccups caused by pipelines compiling while playing.
We solved this by building our systems to look more like a PC but with a distributed pipeline cache directory. When players encounter a pipeline that needs to be compiled at runtime, we’ll try to collect the source data for those pipelines at the end of the session. As many of these pipelines are dynamically generated, we set up several differential privacy systems to merge only pipelines that are identical across users and automatically delete those that aren’t. These pipelines are later merged (while employing differential privacy and ensuring game stability) with the pipelines provided by the developer, and the new augmented pipeline caches are deployed back into the fleet. The next time any player runs into the same scenario, they won’t experience the same graphical stutter or long load times. This effectively leverages the compute power of users playing games on Stadia to provide savings to users who follow. As time goes on, we’ll be able to eliminate graphical stutters from pipeline compilation altogether thanks to users playing our games in the cloud.
-- Derek Bulner, Software Engineer and Bob Fraser, Technical Lead Manager