Improving Graphics Interactivity - It's All in the Timing.

C3 | Thu 24 Jan | 10:40 a.m.–11:25 a.m.


Presented by

  • Keith Packard
    @keith_x11
    https://keithp.com

    Keith Packard has been developing free software since 1986, working on the X Window System, Linux, and rocketry electronics. He is currently a Distinguished Linux Technologist at Hewlett Packard Labs and a consultant on Linux Graphics for Valve Corporation. Keith received a Usenix Lifetime Achievement award in 1999, an O'Reilly Open Source award in 2011 and sits on the X.org foundation board. He has spoken at numerous free software events around the world, including Linux Con, the Plumber's Conference, Linux Conf Australia, FOSDEM, FISL, Guadec, Akademy, OSCON and many others. Keith would appreciate being referred to with he, him and his pronouns.

Abstract

Interactive applications, which includes everything from real time games through flight simulators and virtual reality environments, place strong real-time requirements on the whole computing environment to ensure that the correct data are presented to the user at the correct time. This requires two things; the first is that the time when the information will be displayed be known to the application so that the correct contents can be computed, and second that the frame actually be displayed at that time. These two pieces of information are managed inconsistently through the graphics stack, making it difficult for applications to provide a smooth animation experience to users. And because of the many APIs which lie between the application rendering using OpenGL or Vulkan and the underlying hardware, a failure to correctly handle things at any point along the chain will result in stuttering results. Fixing this requires changes throughout the system, from making the kernel provide better control and information about the queuing and presentation of images through changes in composited window systems to ensure that images are displayed at the target time, and that the actual time of presentation is reported back to applications and finally additions to rendering APIs like Vulkan to expose control over image presentation times and feedback about when images ended up being visible to the user. This presentation will first demonstrate the effects of poor display timing support inherent in the current graphics stack, talk about the different solutions required at each level of the system and finally show the working system.