Categories

WebGL Fundamentals

WebGL makes it possible to display amazing realtime 3D graphics in your browser but what many people don’t know is that WebGL is actually a kind of 2D API, not a 3D API. Let me explain.

WebGL only cares about 2 things. Clipspace coordinates in 2D and colors. Your job as a programmer using WebGL is to provide WebGL with those 2 things. You provide 2 “shaders” to do this. A Vertex shader which provides the clipspace coordinates and a fragment shader that provides the color.

Clipspace coordinates always go from -1 to +1 no matter what size your canvas is. Here is a simple WebGL example that shows WebGL in its simplest form.

// Get A WebGL context
var canvas = document.getElementById("canvas");
var gl = canvas.getContext("experimental-webgl");

// setup a GLSL program
var vertexShader = createShaderFromScriptElement(gl, "2d-vertex-shader");
var fragmentShader = createShaderFromScriptElement(gl, "2d-fragment-shader");
var program = createProgram(gl, [vertexShader, fragmentShader]);
gl.useProgram(program);

// look up where the vertex data needs to go.
var positionLocation = gl.getAttribLocation(program, "a_position");

// Create a buffer and put a single clipspace rectangle in
// it (2 triangles)
var buffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, buffer);
gl.bufferData(
    gl.ARRAY_BUFFER, 
    new Float32Array([
        -1.0, -1.0, 
         1.0, -1.0, 
        -1.0,  1.0, 
        -1.0,  1.0, 
         1.0, -1.0, 
         1.0,  1.0]), 
    gl.STATIC_DRAW);
gl.enableVertexAttribArray(positionLocation);
gl.vertexAttribPointer(positionLocation, 2, gl.FLOAT, false, 0, 0);

// draw
gl.drawArrays(gl.TRIANGLES, 0, 6);

Here’s the 2 shaders

<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec2 a_position;

void main() {
  gl_Position = vec4(a_position, 0, 1);
}
</script>

<script id="2d-fragment-shader" type="x-shader/x-fragment">
void main() {
  gl_FragColor = vec4(0,1,0,1);  // green
}
</script>

This will draw a green rectangle the entire size of the canvas. Here it is


click here to open in a separate window

Not very exciting :-p

Again, clipspace coordinates always go from -1 to +1 regardless of the size of the canvas. In the case above you can see we are doing nothing but passing on our position data directly. Since the position data is already in clipspace there is no work to do. If you want 3D it’s up to you to supply shaders that convert from 3D to 2D because WebGL IS A 2D API!!!

For 2D stuff you would probably rather work in pixels than clipspace so let’s change the shader so we can supply rectangles in pixels and have it convert to clipspace for us. Here’s the new vertex shader

<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec2 a_position;

uniform vec2 u_resolution;

void main() {
   // convert the rectangle from pixels to 0.0 to 1.0
   vec2 zeroToOne = a_position / u_resolution;

   // convert from 0->1 to 0->2
   vec2 zeroToTwo = zeroToOne * 2.0;

   // convert from 0->2 to -1->+1 (clipspace)
   vec2 clipSpace = zeroToTwo - 1.0;

   gl_Position = vec4(clipSpace, 0, 1);
}
</script>

Now we can change our data from clipspace to pixels

// set the resolution
var resolutionLocation = gl.getUniformLocation(program, "u_resolution");
gl.uniform2f(resolutionLocation, canvas.width, canvas.height);

// setup a rectangle from 10,20 to 80,30 in pixels
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
    10, 20,
    80, 20,
    10, 30,
    10, 30,
    80, 20,
    80, 30]), gl.STATIC_DRAW);

And here it is


click here to open in a separate window

You might notice the rectangle is near the bottom of that area. WebGL considers the bottom left corner to be 0,0. To get it to be the more traditional top left corner used for 2d graphics APIs we just flip the y coordinate.

   gl_Position = vec4(clipSpace * vec2(1, -1), 0, 1);

And now our rectangle is where we expect it.


click here to open in a separate window

Let’s make the code that defines a rectangle into a function so we can call it for different sized rectangles. While we’re at it we’ll make the color settable.

First we make the fragment shader take a color uniform input.

<script id="2d-fragment-shader" type="x-shader/x-fragment">
precision mediump float;

uniform vec4 u_color;

void main() {
   gl_FragColor = u_color;
}
</script>

And here’s the new code that draws 50 rectangles in random places and random colors.

  var colorLocation = gl.getUniformLocation(program, "u_color");
  ...
  // Create a buffer
  var buffer = gl.createBuffer();
  gl.bindBuffer(gl.ARRAY_BUFFER, buffer);
  gl.enableVertexAttribArray(positionLocation);
  gl.vertexAttribPointer(positionLocation, 2, gl.FLOAT, false, 0, 0);

  // draw 50 random rectangles in random colors
  for (var ii = 0; ii < 50; ++ii) {
    // Setup a random rectangle
    setRectangle(
        gl, randomInt(300), randomInt(300), randomInt(300), randomInt(300));

    // Set a random color.
    gl.uniform4f(colorLocation, Math.random(), Math.random(), Math.random(), 1);

    // Draw the rectangle.
    gl.drawArrays(gl.TRIANGLES, 0, 6);
  }
}

// Returns a random integer from 0 to range - 1.
function randomInt(range) {
  return Math.floor(Math.random() * range);
}

// Fills the buffer with the values that define a rectangle.
function setRectangle(gl, x, y, width, height) {
  var x1 = x;
  var x2 = x + width;
  var y1 = y;
  var y2 = y + height;
  gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
     x1, y1,
     x2, y1,
     x1, y2,
     x1, y2,
     x2, y1,
     x2, y2]), gl.STATIC_DRAW);
}

And here's the rectangles.


click here to open in a separate window

I hope you can see that WebGL is actually a pretty simple API. While it can get more complicated to do 3D that complication is added by you, the programmer, in the form of more complex shaders. The WebGL API itself is 2D and fairly simple.

From here you can go in 2 direction. If you are interested in image procesing I'll show you how to do some 2D image processing. If you are interesting in learning about translation, rotation and scale then start here.

What do type="x-shader/x-vertex" and type="x-shader/x-fragment" mean?

<script> tags default to having JavaScript in them. You can put no type or you can put type="javascript" or type="text/javascript" and the browser will interpret the contents as JavaScript. If you put anything else the browser ignores the contents of the script tag.

We can use this feature to store shaders in script tags. Even better, we can make up our own type and in our javascript look for that to decide whether to compile the shader as a vertex shader or a fragment shader.

In this case the function createShaderFromScriptElement looks for a script with specified id and then looks at the type to decide what type of shader to create.

  • Paul


    WebGL considers the bottom right corner to be 0,0. To get it to be the more traditional top right corner used for 2d graphics we just flip the y coordinate.”

    I think you mean bottom left and top left don’t you?

    And it’s really only “traditional” for 2d graphic APIs. Not so much in the real world. :)

  • http://greggman.com greggman

    yes, thanks. fixed.

  • Marco Falcioni

    Hi Greg, I was following this and I can’t seem to find webgl-utils.js – is this homegrown or distributed by a 3rd party?

  • http://greggman.com greggman

    It’s home grown. You can pull down individual files here
    http://greggman.com/downloads/examples/webgl/

    At some point I’ll make a zip file or put it all on github or something but I’ve got a lot more to write first.

  • emackey

    This is a wonderful tutorial, but I have to disagree about the 2D.  Clip space is 3D: the Z coordinate is also constrained to +/- 1, the near and far planes will clip polygons even if their X and Y coordinates are within range, and the depth buffer will sort out fragments based on their screenspace-Z coordinates.  Certainly, if you don’t supply a projection matrix, then you’ll be looking at an orthographic view, which doesn’t “look 3D” on casual inspection, certainly not as 3D-looking as a perspective view.  But WebGL is certainly not ignoring the 3rd dimension, not in clip-space or screen-space.

  • http://greggman.com greggman

    A depth buffer is not 3d. It’s just a help for drawing in 2d. You can draw 3d in other 2d APIs. See any of the 3d canvas demos around the web.

    The point is, like other drawing APIs, WebGL itself only thinks in 2D. All 3D is added by you. There is no such thing as a “projection” matrix in WebGL anymore than there is in Canvas2D. That’s something your program uses to get a 2D API to draw 3D. In Canvas2D you use it in JavaScript. In WebGL you use it in GLSL. In either case it’s your code, not the API’s.

  • 0rel

    Hi! Sorry to disagree as well… But WebGL is really not 2D API!

    OpenGL and GLSL are certainly capable of doing 3D operations (vector math etc. in GLSL) and handling of 3D data (vertex, normals etc.).

    I agree, that there are no fixed functions for 3D projection and shading…  But that’s all.

    Nice to see these 2D tutorials. – I was using OpenGL, and now I’m trying to get into WebGL/JavaScript a bit. Maybe it would be better to call the series: WebGL is >also< a 2D API? 3D space is the superset… *dr smarty-pant ;P*

  • http://greggman.com greggman

    There is no more 3D in WebGL than there is in any other 2D graphics library.  

    What part of the Canvas2D API is doing the 3D in this sample?http://mrdoob.github.com/three.js/examples/canvas_geometry_hierarchy.html 

    Why, that would be none. The 3d is done by user supplied code using a 2D api to draw 3d. The Canvas API itself has no 3d in it.

    What part of the WebGL API is doing the 3D in this sample?
    http://mrdoob.github.com/three.js/examples/webgl_geometry_hierarchy.html 

    Again, that would be none. All the 3d is done by user supplied code using a 2D api to draw 2D. The WebGL API itself has no 3d in it.

  • 0rel

    WebGL essentially is OpenGL ES, accessed through the browser. And OpenGL is an interface to use 3D graphics hardware. – As I’ve understood it, the 2D canvas element only gives access to the WebGL context, which enables you to render WebGL powered graphics.

    At the end, WebGL draws RGB pixels onto the 2D screen. However, the calculations on the graphics card can be done in 3D, and are supported as that by the API itself. – A 3D vector is a triplet of numbers, actually implemented and optimized in software (OpenGL driver) and hardware as a 3D entity. And operations in GLSL like the cross product of 3d vectors are done in 3D as well, for example.

    Since 3D is the superset of 2D, it is completely fine to do 2D rendering with OpenGL as well… and I’d actually like to learn more about this! – Don’t want to offend anybody, just interested in the subject matter…

  • http://greggman.com greggman

    I don’t see your point? So because an x86 can do vector math (SSE) its a 3d CPU?

    Vector math is Vector math. If you use it for 3d YOU programmed 3D. The processor implementing the vector math just multiplied vectors, it didn’t know that you were using them to do 3d or physics or fluid equations or whatever.

    OpenGL ES 2.0 does exactly as much 3d as any other 2d library. No more, no less which is to say NONE. All 3d is supplied by the programmer, not the API.

    three.js is a 3D API, OpenSceneGraph is a 3D API, Unity3D is a 3D library. All of those actually do 3D with no work on the programmers part. The programmer does not have to provide functions to translate from 2D to 3D as they are built into the API

    WebGL and OpenGL ES 2.0 on the other hand do not do 3D. All 3D is supplied by the programmer. They draw in 2D. They require 2D data. They provide some math libraries. How you use those math libraries is up to you. As long as you eventually supply 2D data WebGL and OpenGL ES 2.0 are happy.

    I don’t understand why that is so hard to grasp.

  • 0rel

    OpenGL ES has a quite generic interface, so that a program can use the functions in may different ways. – But in the end it provides the functionality to manipulate and render 3D geometry. That’s already implemented in the architecture.

    For example, GLSL interpolates vertex positions in 3D by default, after the vertex shader has been executed, and before the 2D pixels are rendered, in the fragment shader. It always requires geometry data in 3D, also if you’re only interested in 2 dimensions.

    http://www.opengl.org/sdk/docs/manglsl/xhtml/gl_Position.xml 
    http://en.wikipedia.org/wiki/Shader#Vertex_shaders: Quote:
    “The purpose is to transform each vertex’s 3D position in virtual space to the 2D coordinate at which it appears on the screen”

    It’s up to the programmer, how this 3D to 2D transformations is done…

    It’s also true, that one can no just load a model, do animations and assign materials to it that easily in OpenGL ES like in 3D engines (e.g. Unity), and I agree that it is a very bare-bones library. But in my opinion, it’s still a 3D library, exactly because it provides the tools to do 3D vector math and handle 3D geometry. It’s very low-level though…

  • http://greggman.com greggman

    Well, we’ll just have to agree to disagree. OpenGL and OpenGL ES 1.1 with fixed function pipelines actually were 3D APIs. You could give them 3D data and with no code written by you they would draw 3D.

    OpenGL ES 2.0 and WebGL on the other hand no such path exists. You have to provide the functions to convert from 3D to 2D same as any other 2D library. There’s no difference between Canvas 2D and WebGL in terms of how much 3D is built in. The only difference is WebGL is faster.

  • 0rel

    Thanks for your answer. Probably, it’s a question of definition to a certain degree… After all, I still want to learn more about WebGL and your tutorials shed a light on an interesting (and probably quite useful) side of it.

  • Denis

    Is it possible to detect on which of the rectangles mouse is focused? Can you give some guidelines?

  • http://greggman.com greggman

    You might be better off asking that question on Stackoverflow for more details but as for some guidelines…

    In the sample above the program has no memory of “rectangles” meaning when it’s done drawing there’s just a WebGL Canvas with a bunch of pixels in it. There’s no structure or data siting around to remember that those pixels were made by drawing rectangles.

    If you want to track rectangles first you’ll need some way of remembering the rectangles. Often this data is called a “scene graph” but it really depends on the app.

    Next you’ll need a way of taking a mouse click and figuring out which entry in your scene graph was just clicked on.

    One common way to do that is to walk through your scene graph and doing the math to figure out whether the click was inside one of your rectangles. This is what the browser itself does to figure out which things on a page you click on.

    Another semi common way, at least when the shapes are far more complicated than rectangles, is to draw the scene to an off screen buffer with each object in the scene drawn in a different color. Then you read the color of the pixel under the mouse and based on that color you know which object was clicked on.

    That technique was used in Google Body.

  • http://greggman.com greggman

    You might be better off asking that question on Stackoverflow for more details but as for some guidelines…

    In the sample above the program has no memory of “rectangles” meaning when it’s done drawing there’s just a WebGL Canvas with a bunch of pixels in it. There’s no structure or data siting around to remember that those pixels were made by drawing rectangles.

    If you want to track rectangles first you’ll need some way of remembering the rectangles. Often this data is called a “scene graph” but it really depends on the app.

    Next you’ll need a way of taking a mouse click and figuring out which entry in your scene graph was just clicked on.

    One common way to do that is to walk through your scene graph and doing the math to figure out whether the click was inside one of your rectangles. This is what the browser itself does to figure out which things on a page you click on.

    Another semi common way, at least when the shapes are far more complicated than rectangles, is to draw the scene to an off screen buffer with each object in the scene drawn in a different color. Then you read the color of the pixel under the mouse and based on that color you know which object was clicked on.

    That technique was used in Google Body.

  • Benoit Jacob

    Great approach, was definitely a big inspiration for this presentation/tutorial I just gave:
    http://people.mozilla.org/~bjacob/webgl-spotlight-js-2012/

    By the way, may I mention that your “random rectangles” example is exactly the only thing one can draw in WebGL *without* using any shaders or buffers?

    gl.enable(gl.SCISSOR_TEST);
    gl.scissor(x,y,w,h);
    gl.clearColor(r,g,b,a);
    gl.clear(gl.COLOR_BUFFER_BIT);

  • Everton Marques

    Superb introduction.

  • Adrian M.

    No, webgl is a 3D API, and that is why and why is different from SVG or other 2D libraries:

    1) Projections:
    The first difference is that a 3D library must be able to project from 3D to 2D, that is one of the most important difference between a 3D and 2D API. WebGL is projecting all what you are drawing, and that is why your coords, by default use value from -1 to 1 and not from 0 to your screen.width. Happily, you may lets the Z coord to his default value (0) to avoid 2D drawing, but that is a programmer choise, not an API limitation.

    2) Allow the 3th dimension
    Any 2D library only allow 2 numbers to define positions or movement operations, 3D library accept 3. OpenGL and WebGL accept 3.
    Example:
    SVG: transform=’translate(2, 3)’ // there is 2
    WebGL: mat4.translate(mvMatrix, [-1.5, 0.0, -7.0]);
    // A point it that a matrix of 4*4 is only needed to project in 3D

    3) Depth buffer
    A depth buffer is different from a Z-Index, a 2D API use z-index to “simulate” the 3th dimension, but do not allow that a same object had part to cross over an other object.

    The depth buffer allow to draw a triangle going from far away to very near the viewer, others triangles may be partially hidden by this one, and that is near impossible in a 2D API.

    4) There is more difference, but i will extends my text much more.

    Just a point to peoples that say WebGL is 2D: try to implement a real 3D scene with SVG or Canvas and you will get an extremly complex code: simulate blending (not easy), 3D projections(easy), deph buffer (hard), clipping (easy), texturing (imposible, these type of filters are just not allowed), etc.

    I hope that throw information to this controversy

  • http://greggman.com greggman

    We’ll just have to agree to disagree.

    My definition of the 3D library is one I can have give 3D data and without having to provide rendering functions it draws that 3D data. Examples, OpenGL 1.0, OpenGL ES 1.1, OpenInventor, OpenSceneGraph, ThreeJS.

    A library that rasterizes triangles but for which I have to provide all of the math to actually render a 3D scene is not a 3D library. It’s just a fast 2D library with special functions that made it particularly good for rendering 2D once I provide all the math to convert from 3D to 2D.

    I think there is strong distinction between old fixed function libraries that would draw 3D for you new new ones that don’t.

    With old fixed function 3D libraries you gave them 3D data, a color, some lights and called Draw and you got a 3d scene.

    With current programmable pipelines they’ll do nothing with 3D data unless you supply all the math. That’s like arguing that C++ is a graphics library just because I can implement everything in it. C++ doesn’t do graphics out of the box which is why it’s not called a graphics library. WebGL doesn’t do 3D out of the box which is why it’s not a 3D library.

  • Adrian M.

    ” I have to provide all of the math to actually render a 3D scene is not a 3D library”

    Well, all math are here, maybe you just don’t know how to use them ;-)
    Example: mat4.translate(mvMatrix, [-1.5, 0.0, -7.0]);
    Example2:
    gl.bindBuffer(gl.ARRAY_BUFFER, squareVertexPositionBuffer);
    vertices = [
    1.0, 1.0, 1.0,
    -1.0, 1.0, -1.0,
    1.0, -1.0, 0.5,
    -1.0, -1.0, -0.2
    ];
    gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(vertices), gl.STATIC_DRAW);
    squareVertexPositionBuffer.itemSize = 3;
    squareVertexPositionBuffer.numItems = 4;

  • http://greggman.com greggman

    Yep, that’s it. I don’t know how to use it. Maybe you should take a look at who you’re talking to instead of being a dick.

    Your example does absolutely nothing. Did it get cut off?

  • Adrian M.

    mat4.translate(mvMatrix, [-1.5, 0.0, -7.0]);

    This example show how to implement a matrix 4*4 multiplication with only one call. This matrix is used to translate the scene in a 3D dimension. If that is not Math with 3 dimensions, I really want to know what do you expect, maybe your own new Final fantasy just ready to publish on your website?

    The seconds example show how to pass to GL data to represent 1 square in 3D.

    “Maybe you should take a look at who you’re talking to instead of be a dick.” In your case, did you know to who you are talking? I am talking to a guy that don’t know what is to respect. Put down your ego.

    “Did it get cut off?” Off course, you may check the entire code there: http://www.jlabstudio.com/webgl/2011/10/webgl-tutorial-1-un-triangulo-y-un-cuadrado/

  • http://greggman.com greggman

    Your example does nothing. “mat4.translate(…)” is not part of WebGL. That’s some JavaScript library.

    It’s not about ego. It’s about disrespect. Telling people they don’t know what they’re doing is un called for. Especially when that person implements WebGL in one of the major browsers, writes the WebGL spec, and has written more than 70% of all the WebGL conformance tests.

  • Adrian M.

    I never hoped to been the truth incarnated.

    But i “lost time” writing int YOUR web, and what you say me ” Maybe you should take a look at who you’re talking to instead of being a dick”: that is injurious and disrespect.

    If you are a so good person that has invented all the world and nobody know anything that you may learn, I am so sorry for you: that should be terribly ennoying.

  • xmypuk

    Thank you for this tutorial. It was very helpful for me.

    Can somebody explain please one thing? At the last example if the loop is replaced by the setTimeout or setInterval calls then the canvas area is being cleared at each iteration. Why is this happens and how can I avoid this behaviour?

  • http://greggman.com greggman

    don’t use setTimeout or setInterval for animation. It’s evil as it DOSes the user’s machine and prevents the browser from giving the user an optimal experience. Instead use requestAnimationFrame.

    As for not clearing the canvas. When you create the webgl context pass in { preserveDrawingbuffer: true }. In the program above it would be

    
    var gl = getWebGLContext(canvas, {
        preserveDrawingbuffer: true,
    });
    
    

    In a WebGL program without a helper functions it would be

    
    var gl = someCanvasWelement.getContext("experimental-webgl", {
        preserveDrawingbuffer: true,
    });
    
    

    Hope that helps.

  • xmypuk

    Yes, it works! (but Buffer with the capital letter)

    As for requestAnimationFrame callbacks. I’m not sure that in my case it will give some advantages because my script may doing calculations permanently all the time and animation frame callback will be skipped (or defered) anyway since thread is busy.

    But I’ll try optimize all this stuff later, may be with the Workers or somehow else.

    Thank you again!

  • Jono Brandel

    This set of posts is fantastic. It kind of led me down a rabbit hole to drawing more and different kinds of polygons. I posted an expanded question concerning this subject matter over on Stackoverflow which I think you might enjoy.

  • http://twitter.com/johnslegers John Slegers

    You’re discussing semantics here.

    If there’s anything I’ve learned from online discussions, it’s the pointlessness of semantic discussions.

    There are valid arguments for calling WebGL a 3D API.

    There are also valid arguments for calling WebGL a 2D API.

    In the end, it depends on how you define “3D API” or “2D API”.

    Without first agreeing on the definition, any discussion on the matter is pointless.

  • http://greggman.com greggman

    I wrote a post about my definition:

    http://games.greggman.com/game/webgl-2d-vs-3d-libraries/

    The short if it is, a library’s purpose is make it so you don’t have to have any knowledge of how it does the thing it does. You don’t need to know how physics work to use a physics library. You just give it shapes, masses and forces and it handles the rest. You don’t know how a sorting library works to use it. You just tell it to sort and it sorts. So, similarly, a 3D library does 3D for you without you having to know 3D. That’s true of Three.js, Unreal, Unity, Even OpenGL 1.0 but it is not true for OpenGL ES 2.0 nor WebGL.

  • shannah78

    I’m just getting started with OGL. I’ve read a few books on the subject and have gone through countless tutorials, but it was your suggestion of viewing it as a 2D API that really made it click for me. Thanks for the great tutorials.