Search

Categories

WebGL: How It Works

This is a continuation from WebGL Fundamentals. Before we continue I think we need to discuss at a basic level what WebGL and your GPU actually do. There are basically 2 parts to this GPU thing. The first part processes vertices (or streams of data) into clipspace vertices. The second part draws pixels based on the first part.

When you call

    gl.drawArrays(gl.TRIANGLE, 0, 9);

The 9 there means “process 9 vertices” so here are 9 vertices being processed.

On the left is the data you provide. The vertex shader is a function you write. It gets called once for each vertex. You do some math and set the special variable “gl_Position” and the GPU takes your result and stores it internally.

Assuming you’re drawing TRIANGLES, every time this first part generates 3 vertices the GPU uses them to make a triangle. It figures out which pixels the 3 points of the triangle correspond to, and then rasterizes the triangle which is a fancy word for “draws it with pixels”. For each pixel it will call your fragment shader asking you what color to make that pixel.

That’s all very interesting but as you can see in our examples to up this point the fragment shader has very little info per pixel. Fortunately we can pass it more info. We define “varyings” for each value we want to pass from the vertex shader to the fragment shader.

As a simple example, lets just pass the clipspace coordinates we computed directly from the vertex shader to the fragment shader.

We’ll draw with a simple triangle. Continuing from our previous example let’s change our F to a triangle.

// Fill the buffer with the values that define a rectangle.
function setGeometry(gl) {
  gl.bufferData(
      gl.ARRAY_BUFFER,
      new Float32Array([
             0, -100,
           150,  125,
          -175,  100]),
      gl.STATIC_DRAW);
}

And we have to only draw 3 vertices.

  // Draw the scene.
  function drawScene() {
    ...
    // Draw the geometry.
    gl.drawArrays(gl.TRIANGLES, 0, 3);
  }

Then in our vertex shader we declare a *varying* to pass data to the fragment shader.

varying vec4 v_color;
...
void main() {
  // Multiply the position by the matrix.
  gl_Position = vec4((u_matrix * vec3(a_position, 1)).xy, 0, 1);

  // Convert from clipspace to colorspace.
  // Clipspace goes -1.0 to +1.0
  // Colorspace goes from 0.0 to 1.0
  v_color = gl_Position * 0.5 + 0.5;
}

And then we declare the same *varying* in the fragment shader.

precision mediump float;

varying vec4 v_color;

void main() {
  gl_FragColor = v_color;
}

WebGL will connect the varying in the vertex shader to the varying of the same name and type in the fragment shader.

Here’s the working version.


click here to open in a separate window

Move, scale and rotate the rectangle. Notice that since the colors are computed from clipspace they don’t move with the rectangle. They are relative to the background.

Now think about it. We only compute 3 vertices. Our vertex shader only gets called 3 times therefore it’s only computing 3 colors yet our triangle is many colors. This is why it’s called a *varying*.

WebGL takes the 3 values we computed for each vertex and as it rasterizes the triangle it interpolates between the values we computed for the vertices. For each pixel it calls our fragment shader with the interpolated value for that pixel.

In the example above we start out with the 3 vertices

Vertices
0 -100
150 125
-175 100

Our vertex shader applies a matrix to translate, rotate, scale and convert to clipspace. The defaults for translation, rotation and scale are translation = 200, 150, rotation = 0, scale = 1,1 so that’s really only translation. Given our backbuffer is 400×300 our vertex shader applies the matrix and then computes the following 3 clipspace vertices.

values written to gl_Position
0.000 0.660
0.750 -0.830
-0.875 -0.660

It also converts those to colorspace and writes them to the *varying* v_color that we declared.

values written to v_color
0.5000 0.750 0.5
0.8750 0.915 0.5
0.0625 0.170 0.5

Those 3 values written to v_color are then interpolated and passed to the fragment shader for each pixel.

v_color is interpolated between v0, v1 and v2

We can also pass in more data to the vertex shader which we can then pass on to the fragment shader. So for example lets draw a rectangle, that consists of 2 triangles, in 2 colors. To do this we’ll add another attribute to the vertex shader so we can pass it more data and we’ll pass that data directly to the fragment shader.

attribute vec2 a_position;
attribute vec4 a_color;
...
varying vec4 v_color;

void main() {
   ...
  // Copy the color from the attribute to the varying.
  v_color = a_color;
}

We now have to supply colors for WebGL to use.

  // look up where the vertex data needs to go.
  var positionLocation = gl.getAttribLocation(program, "a_position");
  var colorLocation = gl.getAttribLocation(program, "a_color");
  ...
  // Create a buffer for the colors.
  var buffer = gl.createBuffer();
  gl.bindBuffer(gl.ARRAY_BUFFER, buffer);
  gl.enableVertexAttribArray(colorLocation);
  gl.vertexAttribPointer(colorLocation, 4, gl.FLOAT, false, 0, 0);

  // Set the colors.
  setColors(gl);
  ...

// Fill the buffer with colors for the 2 triangles
// that make the rectangle.
function setColors(gl) {
  // Pick 2 random colors.
  var r1 = Math.random();
  var b1 = Math.random();
  var g1 = Math.random();

  var r2 = Math.random();
  var b2 = Math.random();
  var g2 = Math.random();

  gl.bufferData(
      gl.ARRAY_BUFFER,
      new Float32Array(
        [ r1, b1, g1, 1,
          r1, b1, g1, 1,
          r1, b1, g1, 1,
          r2, b2, g2, 1,
          r2, b2, g2, 1,
          r2, b2, g2, 1]),
      gl.STATIC_DRAW);
}

And here’s the result.


click here to open in a separate window

Notice that we have 2 solid color triangles. Yet we’re passing the values in a *varying* so they are being varied or interpolated across the triangle. It’s just that we used the same color on each of the 3 vertices of each triangle. If we make each color different we’ll see the interpolation.

// Fill the buffer with colors for the 2 triangles
// that make the rectangle.
function setColors(gl) {
  // Make every vertex a different color.
  gl.bufferData(
      gl.ARRAY_BUFFER,
      new Float32Array(
        [ Math.random(), Math.random(), Math.random(), 1,
          Math.random(), Math.random(), Math.random(), 1,
          Math.random(), Math.random(), Math.random(), 1,
          Math.random(), Math.random(), Math.random(), 1,
          Math.random(), Math.random(), Math.random(), 1,
          Math.random(), Math.random(), Math.random(), 1]),
      gl.STATIC_DRAW);
}

And now we see the interpolated *varying*.


click here to open in a separate window

Not very exciting I suppose but it does demonstrate using more than one attribute and passing data from a vertex shader to a fragment shader. If you check out the image processing examples you’ll see they also use an extra attribute to pass in texture coordinates.

Before we move on just one more thing.

What do these buffer and attibute commands do?

Buffers are the way of getting vertex and other per vertex data onto the GPU. gl.createBuffer creates a buffer. gl.bindBuffer sets that buffer as the buffer to be worked on. gl.bufferData copies data into the buffer.

Once the data is in the buffer we need to tell WebGL how to get data out of it and provide it to the vertex shader’s attributes.

To do this, first we ask WebGL what locations it assigned to the attributes. For example in the code above we have

  // look up where the vertex data needs to go.
  var positionLocation = gl.getAttribLocation(program, "a_position");
  var colorLocation = gl.getAttribLocation(program, "a_color");

Once we know the location of the attribute we then issue 2 commands.

    gl.enableVertexAttribArray(location);

This command tells WebGL we want to supply data from a buffer.

    gl.vertexAttribPointer(
        location,
        numComponents,
        typeOfData,
        normalizeFlag,
        strideToNextPieceOfData,
        offsetIntoBuffer);

And this command tells WebGL to get data from the buffer that’s was last bound with gl.bindBuffer, how many components per vertex (1 – 4), what the type of data is (BYTE, FLOAT, INT, UNSIGNED_SHORT, etc…), the stride which means how many bytes to skip to get from one piece of data to the next piece of data, and an offset for how far into the buffer our data is.

Number of components is always 1 to 4.

If you are using 1 buffer per type of data then both stride and offset can always be 0. 0 for stride means “use a stride that matches the type and size”. 0 for offset means start at the beginning of the buffer. Setting them to values other than 0 is more complicated and though it has some benefits in terms of performance it’s not worth the complication unless you are trying to push WebGL to its absolute limits.

I hope that clears up buffers and attributes.

What’s normalizeFlag for in vertexAttribPointer?

The normalize flag is for all the non floating point types. If you pass in false then values will be interpreted as the type they are. BYTE goes from -128 to 127, UNSIGNED_BYTE goes from 0 to 255, SHORT goes from -32768 to 32776 etc…

If you set the normalize flag to true then the values of a BYTE (-128 to 127) represent the values -1.0 to +1.0, UNSIGNED_BYTE (0 to 255) become 0.0 to +1.0, A normalized SHORT also goes from -1.0 to +1.0 it just has more resolution than a BYTE.

The most common use for normalized data is for colors. Most of the time colors only go from 0.0 to 1.0. Using a full float each for red, green, blue and alpha would use 16 bytes per vertex per color. If you have complicated geometry that can add up to a lot of bytes. Instead you could convert your colors to UNSIGNED_BYTEs where 0 represnets 0.0 and 255 represents 1.0. Now you’d only need 4 bytes per color per vertex, a 75% savings.

Let’s change our code to do this. When we tell WebGL how to extract our colors we’d use

  gl.vertexAttribPointer(colorLocation, 4, gl.UNSIGNED_BYTE, true, 0, 0);

And when we fill out our buffer with colors we’d use

// Fill the buffer with colors for the 2 triangles
// that make the rectangle.
function setColors(gl) {
  // Pick 2 random colors.
  var r1 = Math.random() * 256; // 0 to 255.99999
  var b1 = Math.random() * 256; // these values
  var g1 = Math.random() * 256; // will be truncated
  var r2 = Math.random() * 256; // when stored in the
  var b2 = Math.random() * 256; // Uint8Array
  var g2 = Math.random() * 256;

  gl.bufferData(
      gl.ARRAY_BUFFER,
      new Uint8Array(   // Uint8Array
        [ r1, b1, g1, 255,
          r1, b1, g1, 255,
          r1, b1, g1, 255,
          r2, b2, g2, 255,
          r2, b2, g2, 255,
          r2, b2, g2, 255]),
      gl.STATIC_DRAW);
}

Here’s that sample.


click here to open in a separate window

  • kurt

    Hey.
    These are really useful tutorials.
    I chose not to set my origin to top-left corner, but to leave it at the bottom left.
    But, what happens is, that the x and y coordinates are getting interchanged.

    In the setGeometry function, I just verified that the vertices are as they are supposed to be. But when drawn, I find that x and y vertices are interchanged.
    (e.g., I pass [-10, -5, -10, 5, 10, 5, 10, -5] as vertices, to draw a flat rectangle with a long resting base, only to find that what is drawn is a tall rectangle.

    Can you please tell me what could be wrong?
    Again, I assure you, the vertices going into setGeometry are perfectly correct.
    So they should be going to the buffer correctly.

  • http://greggman.com greggman

    Can you post your code on http://jsfiddle.net or some place else that I can take a look?

  • kurt

    http://jsfiddle.net/DwB7P/1/

    I put just the portion that is enough to draw a rectangle on the screen.
    I’ve removed the animation there was, so it can be looked into easily.
    The same problem exists.

    Thanks a lot!

    p.s. : There is nothing shown on jsFiddle by the way, probably because of the missing webgl-utils.js. Anyway, my code is there.

  • http://greggman.com greggman

    The problem is you had sin and cos reversed.
    http://jsfiddle.net/greggman/LYXE5/1/

  • kurt

    Oh :( Sorry about that. Thank you greggman!

  • kurt

    Can you suggest a good way to draw more than 1 shape in different places in an animation based WebGL page?

    My animation occurs via the requestAnimFrame function.
    What I could think of is to hold an array of objects corresponding to my different distinct hapes.

    In every call of my “refresh” function, I first clear the canvas.
    Then, I loop over all the elements of that array and
    call “initBuffers” (the same one in my JSFiddle page), which has bindBuffer and setGeometry calls. Then I call the drawScene function which has a drawArrays call after setting the rotation and translations in the shader program.
    Then happens my objects’ parameters updating and the same cycle repeats.

    So, I reset the buffers every time for each object and redraw every single frame

    I notice that my animation is quite slow at times, and gives a discontinuous appearance. Could you please tell me how it should be actually done?

  • http://greggman.com greggman

    It really depends on what you are trying to do.

    Most GL programs (ie, games) try to create all of their shapes when a start up (or when they start a level). After that they don’t change the shapes. Other apps have different needs so it really depends on your app but ideally you’d try to change the buffers as little as possible.

    Generally you need some kind of class or object that represents your shape (the buffers). You need some kind of object that represent a shader program. You need some kind of object that represents the uniform settings for that shader program (what color for example). And you need some kind of object that represents a specific collection of those to actually draw. Shape X + ShaderProgram Y + Settings Z = ?? VisualObject? Model?

    So for example you might make an object called a Shape

    Shape = function(positions) {
      this.buffer = gl.createBuffer(); 
      gl.bindBuffer(gl.ARRAY_BUFFER, this.buffer);
      gl.bufferData(gl.ARRAY_BUFFER, positions, gl.STATIC_DRAW);
    }
    
    
    Shape.prototype.setup = function() {
      gl.bindBuffer(gl.ARRAY_BUFFER, this.buffer);
      gl.enableVertexAttribArray(positionLocation);
      gl.vertexAttribPointer(positionLocation, ..);
    };
    

    And similar objects for programs and uniform settings.

    Then at init time you can do things like

       var shape1 = new Shape([data for shape1]);
       var shape2 = new Shape([data for shape2]);
    

    and at draw time you can do things like

    function render() {
       ...
       shape1.setup();
       program1.setup();
       uniforms2.setup();
       draw();
       shape2.setup();
       program2.setup();
       uniforms2.setup();
       draw();
    }
    

    I’m sorry if that’s not very detailed but the question you’re asking seems like a general programming question and so would require a really long answer.

    I’d suggest looking at some of the existing WebGL frameworks as examples
    http://www.khronos.org/webgl/wiki/User_Contributions#Frameworks

  • kurt

    Thanks! :)

    So we’re saving those “bufferData” calls by keeping separate buffers for each shape and calling “createBuffer”, “bindBuffer”, “bufferData” only once during init.

    Everytime we have refresh the screen,
    we only have to call “enableVertexAttribArray” and “vertexAttribPointer” after binding the required shape’s buffer(using “bindBuffer”).

    My shapes don’t change with time.
    There are just a variety of them (like polygons, circles, triangles) that stay from beginning to end.

    So this is the best way to do it I suppose. Hope I got it right. Did I ?

  • http://greggman.com greggman

    Yes, that’s the normal way to do it.

  • Geo877

    This tutorial is brilliant, everything has been so clearly explained, and I love the interactive demos throughout, thanks very much! :)