WebGL Techniques and Performance

I gave a talk at Google I/O 2011 on WebGL Techniques and Performance. If that sounds interesting to you then check it out. Answers to some of the questions from the Moderator below.

Note: These answers are my own personal opinions and do not represent my employer nor the Khronos WebGL Working Group.

  • Question: Mouse exclusivity and pointer hiding is essential when trying to write first person games and the like. Do you see any way that this obstacle to WebGL gaming can be overcome?
  • Answer: The various browsers are all experimenting with the best way to implement this and still prevent websites from -stealing- the mouse when you don’t want them too. Full screen support will also help this issue. See next question.
  • Question: Flash has the capacity to offer fullscreen display. Molehill in flash is coming. Do you see a way to offer fullscreen support for the WebGL canvas?
  • Answer: WebKit has fullscreen support today. Chrome is adding it asap. I suspect Firefox is working on it as well.
  • Question: Any plans to support multiple webgl canvases sharing the same context (which would alleviate problems with data sharing and multi-context overhead)?
  • Answer: There is desire to implement shared resources, not shared contexts but it was decided to do it after WebGL 1.0. Now that the 1.0 spec has been ratified I’m sure the Khronos WebGL group will start working on it. A good question though is what’s the use case? In some ways it seems like a ‘nice to have’ but not essential. I’m not saying it’s not important. I’m just suggesting that most apps will use only 1 context. How many flash apps share resources? How many pages use 2 or more canvases and share resources? So, it will happen but I suspect it’s not a huge priority right now.
  • Question: Matrix operations are very math intensive. 3d drawing and physics simulations require a lot of matrix math. JS implemented matrix libraries can never reach the speed of C implemented routines. Any plans to include browser support for matrix APIs?
  • Answer: Some people on the WebKit team are considering exposing CSSMatrix. The Matrix math that is used for CSS 3D transforms. On other fronts making JavaScript 3d math libraries using TypedArrays is getting heavily optimized. I suspect there is another 2x to 4x more optimization to be had over the next few months if we are luck. Finally work is being done to make it efficient to pass TypedArrays to and from Web Workers. If and when that makes it into browsers you’ll be able to move much of your 3d math to another process.
  • Question: Draw instanced is supported by 91% of peoples hardware (according to steam surveys). Draw instanced helps greatly in getting more performance. WebGL has no support for this feature. Can we expect this to change?
  • Answer: The short answers is probably “not in the short term”. The long answer is one of WebGL’s specific goals is that it should be able to run anywhere that OpenGL ES 2.0 runs. That includes iOS, Android and WebOS devices for example. None of those devices support DrawInstanced AFAIK. If the feature can be emulated relatively easily on hardware that does not support it there is a higher possibility it will be added. If you’d like to see it added you can lobby for it on the Public WebGL mailing list.
  • Question: Rendering runs at about 60fps, which is a frame every 16ms. If any objects are allocated (new, [], {}) in that loop, the GC is going to halt everything between 80-180ms. This makes for a very jerky user-experience. How is this going to be addressed?
  • Answer: Browser vendors are aware of this issue. All of them are working on making this less of an issue by optimizing their GC algorithms. Many games and demos already run pretty smoothly today.
  • Question: No WebGL implementation supports multiple render targets. The WebGL spec allows the use of this via gl_FragData[i] and DX9 supports the feature. Why is this not implemented?
  • Answer: One of WebGL’s specific goals is that it should be able to run anywhere that OpenGL ES 2.0 runs. That includes iOS, Android and WebOS devices for example. None of those support multiple render targets and while it might be possible to emulate it would be a lot of work and be extremely slow. If and when OpenGL ES 3.0 comes out, if it supports multiple render targets then maybe WebGL 2.0 will support them. If you feel strongly about this bring it up on the Public WebGL mailing list.
  • Question: New opengl features (like tessellation and geometry shaders) are supported by 23% of peoples hardware (according to steam surveys). Is there plans to make these features accessible in WebGL?
  • Answer: See previous answer.
  • Question: Game audio requires real-time streaming of computed audio data (for doppler, environmental and 3d-positional audio effects). JS would be slow to compute such streams and Browsers do not support them. Is WebGL a feasible games platform without audio?
  • Answer: The WebKit contributors have proposed and implemented the Web Audio API to address this issue. It is shipping and available behind a flag in the Chrome Dev version on all platforms. If you’d like to discuss what will finally become the cross browser standard you can join the Web Audio Working Group.
  • Question: How do we address the issue that 3d applications require lots of data. But web-pages do not have a download metaphor, therefore making it infeasible to offer gigabytes of data?
  • Answer: There are multiple ways to handle this. One is the HTML5 App Cache. Another is the HTML5 FileSystem.
  • Question: DX9 supports vertex shader texture lookups and WebGL does not restrict texture* calls to fragment shaders. Why does ANGLE not support Vertex Shader texture lookups?
  • Answer: The ANGLE project is in the process of adding support for Vertex Shader texture lookups. You can follow the progress here. Note that the latest version of ANGLE is faster than native OpenGL on the same machine in many cases!!!
  • Question: Joysticks and gamepads are good devices to play games (such as flight simulators) and enable it to go to the couch (not possible with a keyboard). How do you see WebGL overcoming its limitation of not being able to talk to these devices?
  • Answer: The W3C is working on the Device API to address this issue. If you’d like to participate in the discussion join the Device APIs and Policy Working Group public mailing list.
  • Question: When webgl will be available on android plateform ? ( mobile and tablet)
  • Answer: WebGL is already available on Android in Firefox beta. As of 2011/05/13 it is still pretty beta but I’m sure we’ll see improvements over the following months.
  • Question: DX9 supports floating point textures and OES_texture_float allows use of them in WebGL, why does ANGLE not support them?
  • Answer: Request support on the ANGLE project and see what they say.
  • Question: Could you recommend me a good site where to learn all about shaders from scratch? Thanks
  • Answer: Sorry I don’t know a place to recommend. Maybe someone else will leave an answer below. You might consider searching for shaders on Amazon.
  •  Thanks!

  • Toofar

    I saw your GDC 2011 presentation. Brillant. Do you think that the  GL_CULL_FACE will be include in Webgl and will it usefull? 

  • glCullFace is supported and of course it’s useful.

  • Article is very interesting,thanks for your sharing. I will necessarily add it in the selected works and I will visit this site.
    wedding dress

  • Anonymous

    RE: context sharelist.     The desired usecase is to support multithreading.   For example, resource prep threads with own gl context sharelisted with primary compositing thread, all referencing the same resource handle space.     Otherwise, webgl will be largely restricted to single thread or worse,  access to single gl context in multiple threads effectively serializing the threads.   As mobile silicon moves to multicore, webgl won’t follow on except as single gl context per canvas operating on unique resource space, no way to have multi thread collaborative ops on single target gl context.   Like, offscreen rendering resource prep threads preparing texture resources for final output/compositor output main thread, presenting to canvas/context.    Maybe that is the complication with webgl:  the contexts are scarfed from canvas elements (similar to X or windows from associated contextr window element.)  But in desktop opengl, for example, it is possible to declare 1×1 ‘offscreen’ windows for the ‘worker’ thread contexts, sharelist these with a main GUI window that is the context for the final compositor thread, and multithread composited resources into the final context/window.    On multicore machines, this is much more performant than putting all of the resource texture prep and final compostiing in the same thread.

    In Webgl, there would need to be the concept of an ‘offscreen’ canvas element, that could still serve up its context(say, serviced by a web worker), but some would want to ‘sharelist’ those contexts with a main canvas element context so that these could collaborate on resource prep.  (How do they efficiently do that if they all have independent resource handle space?  That is where the desire to sharelist comes from–so multithreads can collaborate on resource prep with a final compositing thread.  This is quite doable on desktop opengl.   I once developed a c++ class, CTO (Context Thread Object) that was designed to be a pool element (of CTOs) that could all be sharelisted.   A main CTO managed the main composting thread, was just one of the CTOs in the pool.  These were also designed to be attachable/detachable during their lifetime– client code would attach opengl processing to a CTO, which was called during its windows pump(if currently attached), and this was dynamic.  It was very performant.)  

    There are some restrictions in using sharelisted contexts — like, no context can allocate a resource handle prior to all the sharelisting.  That might be the current hang-up with the canvas element based context serving/sharelist issue; how to guarantee that.

    Until that is supported, webgl better tread lightly around ‘web workers’ multithreading, and/or be limited to totally independent contexts/canvas elements.      

  • You should join the Khronos public WebGL mailing list and ask for this feature

  • Johnny Scott

    Googling WebGl performance led me to your I/O talk which was fantastic. I’d l-o-v-e to get the code from the examples you walked through (your pseudo code is a bit hard for me to parse given I’m quite the newb to OpenGl). I have a WebGl game engine I wrote this weekend I intend to use as a prototyping tool for Android and iOS dev (maybe eventually hopefully Chrome Web Store will allow Webgl apps if it doesn’t already). Thanks in advance for your help.

  • Daosheng Mu

    In your talk, you refer to a particle library in webGL wiki.  I can’t find it. Would you please provide this link for me? Thanks.

  • Daosheng Mu

       In your talk, you refer to a particle library in webGL wiki, but I can’t find it. Would you please provide this link for me? Thanks.

  • There’s 2 versions

    This one which is more stand alone

    And this one which is integrated with the TDL library

    I think I fixed a bug or 2 in the shaders in the TDL one (I should probably back port that)