Friday, April 12, 2013

Thoughts about the Course Intermediate Computer Graphics INFR 2350U

Computer Graphics INFR 2350U

The Computer Graphics INFR 2350U course at UOIT introduces and its students to the amazing would of shader based computer graphics. Its prerequisite is intro to computer graphics where the course teaches about openGL and its fixed pipeline. Computer Graphics teaches students about the openGL programmable pipeline works and how to utilize it. Having just had the last class for the semester i can say without a doubt that the course is superb! 

An average class consists of our teacher Dr. Andrew Hogue talking about a post process method that could pull off a certain effect. He would teach it such that by the end of the lecture you could implement that new shader in the same day.

The approach to home work was neat and game like (being in game dev it seems fitting). All the homework questions are given at the beginning of the year and are arranged such that questions have upgrades to them. Each question is basically implementing a shader algorithm and is worth 5xp for the base and additional 5xp per upgrade on them with the exception of a few that were insane upgrades worth 10xp. The homework mark is graded as your xp/75xp with the ability to get more than the 75xp which was caped at 95xp. At the last class you need a total of 40xp to write the final exam. However there was a restriction of 8 base questions which would equal 40xp just enough to write the exam but to get perfect or above you would have to do multiple upgrades. 

The TA for the class was also very good because while the teacher taught us theory the TA tough us practical work. The TA was very knowledgeable in the topics and taught with confidence which made up a great teaching environment.

At the end of the day, if i had to rate the course i would give it a 10/10 absolutely fantastic course.

A Rant about Level-Up

A Rant about level up

Level up is yearly event held in Toronto where universities and collages from around Ontario come and show off there games to the public. At the end of the event there are prizes and awards given out. The awards are for generic game awards like best game-play and stuff like that. The awards are voted on by the public but only for some and the others are based on judges that went around undercover.

The turn out for Level up was around 50 teams my groups included. It was neat seeing all the work from different students in the same field as me. One thing I could notice is that almost every one with the exception of UOIT (the university I study at) had made games through existing engines. At UOIT we had to make our games from scratch. For example my groups game was made from openGL, Fmod and windows code only. Where other teams made a game through unity, Cry-engine or unreal engine. In effect, of this there was a noticeable difference between UOIT teams and the other teams where the other teams had to only focus on game-play and maybe assets where UOIT teams had to make everything and did not have enough time in the year to polish every aspect of their games.

When it came time to hand out awards the only games that one were the ones from the teams that used a industrial game engine. I think to make the awards more fair there should have been more categories so that teams that had to make every thing have a chance in competing with engines that make the game for you.

All in all it was a fantastic event and I had a great time. Got a lot of feed back from the public and made changes to better my groups team.

Non-Photo realistic shading

Non-Photo realistic shading
There are different ways of shading a scene in a game. The scene does not always have to be realistic. Toon shading give a great effect if used properly. Toon shading is a post process that makes the colours have a ranges of large jumps and through another pass adds in a black border line around edges. The final result of the toon shader looks like a toon like image with outlines.

http://en.wikipedia.org/wiki/Cel_shading

On the first pass the we add in the toon shading part. This is done easily by sending in the fragments colour to a texture look up which will map that value to another. An easy but slow way to remap the frag colour is to send each component r,g,b floats in to the texture1D look-up.

Texture look up

On the second pass we want to add in the black borders. This is done by using any edge detection algorithm like the Sobel algorithm. You want to run this algorithm on both your normals and depth of the original scene to catch all of the edges. We use the normals and depth because large changes in their colour actually means an edge while using original images colour makes edges where colour change is present so not all the edges picked up would actually be an edge.


http://marctenbosch.com/npr_shading/

Bloom

Bloom

Bloom is a post processing effect used to make an image pop by having the bright colors in a seen bleed over there surroundings. this effect is a three/four pass algorithm depending on if the blur is 2 pass or 1. The passes involved are: the bright pass, the blur pass and the composite pass.

http://commons.wikimedia.org/wiki/File:Frangipani_flowers.jpg  -  Original Photo 


The bright pass is used to make the bright colors in a scene brighter and the dark colors darker. there are plenty of ways that this can be done. I would recommend looking at how photoshops does it's colour levels adjustment.


Bright pass

The next pass is to take the bright pass texture and blur it so that the bright colors will bleed out on to the dark colors.

Blurred image


The final pass is the composite pass which takes the blurred bright pass and puts it onto the original scene giving the Bloom effect.


Bloomed image

Ambient Occlusion

Ambient Occlusion

Ambient occlusion is a great way to give a scene more definition in a scene and therefore making it look better. Ambient occlusion is a process that tries to simulate the absents of light in hard to get places like a sharp corner in a room. Theres basically two ways you can do ambient occlusion in a game. one is have the AO (Ambient Occlusion) baked into the texture or have it done dynamically at run-time on a shader in the game.

http://www.interstation3d.com/tutorials/making_slow_decay/slow_decay_take07.htm


Baking in the AO is not a hard as programs like Maya can bake in the AO onto a texture and it's sample rate and texture size will determine how good that will look. Also putting that in a game is even easier because all you have to do is apply that texture to the mesh and BAM AO. But this also has some draw back the AO is only correct if the scene is completely static. If an object moves then the AO could look incorrect.

http://features.cgsociety.org/cgfilms/cgfilm.php?story_id=3572


Dynamically doing AO on a shader is good because of the AO looks correct when objects move around. The quality of the AO is based on how many samples the shader takes and if it is downscale. The draw back of the SSAO (Screen Space Ambient Occlusion) shader is that it can get really expensive fast. It is common practice to downscale the scene and take low samples to try to cut on expensive operations.

http://commons.wikimedia.org/wiki/File:Screen_space_ambient_occlusion.jpg 


Thursday, April 11, 2013

2 Pass Gaussian Blur

2 Pass Gaussian Blur

In computer graphics you sometimes need to blur the image. Many Shader algorithms require a blur for example Bloom, SSAO, Motion Blur, Depth of Field, etc. 2 pass Gaussian blur is a simple way to blur an image. This method is better than 2D Gaussian blur and box blur as it has less texture reads and doesn't have the aliasing that box blur does. Although it does requires another FBO and FBO texture.

The algorithm is just as simple as the 2D Gaussian blur but this is only done in one dimension in stead of two. Which is why it requires two passes, one pass for vertical blur and one for horizontal blur. For the horizontal you sample the pixel colour at the fragment multiply that by a weighting. Then sample pixel to the left of that multiply that colour by a weighting and add it to the sum. Finally sample the color from the pixel to the right and multiply that by a weighting and add that to the sum. The sum is now your outColour. Take that resulting FBO texture and apply the vertical pass. For the vertical pass do the same but in a vertical fashion sampling above and below the current fragment.

Here are some sample of a 2 pass Gaussian Blur which look identical to the 2D Gaussian blur

http://vimeo.com/20226360
http://www.jhlabs.com/ip/blurring.html


2D Gaussian Blur

2D Gaussian Blur (3x3)

In computer graphics you sometimes need to blur the image. Many Shader algorithms require a blur for example Bloom, SSAO, Motion Blur, Depth of Field, etc. 2D Gausian blur is a simple way to blur an image although it has some problems. It is inefficient as it has to do a lot of texture reads. Although a obvious better choice then Box Blur because Gaussian can have the same amount of texture reads but doesn't have the nasty aliasing box blur does.

The algorithm is just like Box blurs where you add the color contribution of each adjacent pixel is a 3x3 matrix (where the current fragment is in the center) but this time instead of dividing by 9 at the end we multiply each pixel by a weighting before adding it to the color sum. the sum of the weights have to add up to 1 and the weights are calculated by either the Gaussian graph or pascals triangle (gives a nice approximation). The weights for each pixel for a 3x3 2D Gaussian blur is below.

Not normalized
1  2  1
2  4  2
1  2  1

normalized
0.0625  0.125  0.0625 
0.125   0.25    0.125
0.0625  0.125 0.0625

here are some examples of a Gaussian Blur
http://www.borisfx.com/graffiti/new_filters.php
http://www.java2s.com/Code/Java/Advanced-Graphics/GaussianBlurDemo.htm

http://www.jhlabs.com/ip/blurring.html

Box Blur

 Box Blur (3x3)

In computer graphics you sometimes need to blur the image. Many Shader algorithms require a blur for example Bloom, SSAO, Motion Blur, Depth of Field, etc. Box blur is a simple way to blur an image although it has some problems. first it is inefficient as it has to do a lot of texture reads. Secondly it causes aliasing.

the Box blur algorithm is a very simple one. you sample the color at the pixel the fragment shader is at then add the one to the left of it to it. Add the one to the right of it. Add the one above it. Add the one above and to the left. Add the one above and to the right. Add the one below. Add the one below and to the left and add the one below and to the right. basically add each of the pixels colors in a 3x3 matrix such that the current fragment is the middle of that matrix. Then divide the sum of the pixels' color by 9. and that is your resulting fragment color.

here are some examples of box Blur.

http://codecave.org/image-processing/chap_area.html

http://www.jhlabs.com/ip/blurring.html

http://javaimageprocessingtutorial.blogspot.ca/

Wednesday, February 27, 2013

Intro to FBOs


Frame Buffer Objects
Frame buffer objects (FBOs) are OpenGL objects that hold texture data which can be filled by rendering off screen, or as one of my course’s TA calls it rendering in Magic Land. FBOs are really important for graphics because they can be used to apply post processing effects, which is the act of applying an effect to an image or texture. A FBO will hold user defined textures and you can have more than one texture or even no textures attached. To apply a post process you have to render a full screen quad and run a shader that does the screen process. In the end you will have a 2D texture that is essentially a screen picture of that frame of your application. Then you just render another full screen quad and attach that texture to it too display it to the screen.

Setting up an FBO is pretty easy. You just have to create a handle then set some parameters and check if the FBO has correctly been made by quarrying openGL. Below I will go over a bare bone Create FBO function that has the capability to set up a multiple render targets (MRTs). So this function is pretty bad in the sense that it has very little flexibility and may not do what you’d want it to do. The function call accepts a Boolean to tell it which FBO attachment types to make. Although there are only 2 types the function can make, depth and colour, the normal option is really just another colour frame buffer. The function accepts pointers to texture handles and the width and height of the screen. The rest of the code below will be explained through the comments in the code. For explanations on some of the openGL code used you can just Google them for their full break down.

GLuint Game::CreateFBO(bool useColour, bool useDepth, bool useNormal, GLuint *colourTextureOut,                 GLuint *depthTextureOut, GLuint *normalTextureOut, unsigned int width, unsigned int height)
{
//this is a simple check to see if we are creating any of the frame buffers and if we are not  //return 0.
            if (!useColour && !useDepth && !useNormal)
                        return 0;
            //this is our fbo handle which is much like a texture handle.
            unsigned int fboHandle;

            // generate FBO in graphics memory
            glGenFramebuffers(1, &fboHandle);
            glBindFramebuffer(GL_FRAMEBUFFER, fboHandle);

            // create a texture for colour storage
            if (useColour && colourTextureOut)
            {
                        //create and bind fbo
                        glGenTextures(1, colourTextureOut);
                        glBindTexture(GL_TEXTURE_2D, *colourTextureOut);
//Create texture notice how we made it using GL_RGBA8 which is basically for color as //well as GL_RGBA as parameters.
                    glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA8, width, height, 0, GL_RGBA,                    GL_UNSIGNED_BYTE, 0);

                        //set up parameters
                        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
                        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
                        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
                        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
                       
//Notice how we use the color attachment 0 colour attachments are important for //using MRT as each frame buffer must use its own colour attachment.
                        glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, *colourTextureOut, 0);
            }

            // create a texture for depth storage
            if (useDepth && depthTextureOut)
            {
                        //Create fbo
                        glGenTextures(1, depthTextureOut);
                        glBindTexture(GL_TEXTURE_2D, *depthTextureOut);
                        //Create texture
                        glTexImage2D(GL_TEXTURE_2D, 0, GL_DEPTH_COMPONENT24, width, height, 0, GL_DEPTH_COMPONENT, GL_UNSIGNED_INT, 0);
                        //set up parameters
                        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_NEAREST);
                        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_NEAREST);
                        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
                        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

                        // we use the depth attachment for depth
                        glFramebufferTexture2D(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_TEXTURE_2D, *depthTextureOut, 0);
            }

            if (useNormal && normalTextureOut)
            {
                        //create and bind fbo
                        glGenTextures(1, normalTextureOut);
                        glBindTexture(GL_TEXTURE_2D, *normalTextureOut);
                        //Create texture
                        glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA8, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, 0);
                        //set up parameters
                        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
                        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
                        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
                        glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
                       
                        // as the color attachment 0 has been used we need to use the next one.
                        glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT1, GL_TEXTURE_2D, *normalTextureOut, 0);
            }
           
            // so now that we created the FBO we need to check the status of it
            int status = glCheckFramebufferStatus(GL_FRAMEBUFFER);
            if (status != GL_FRAMEBUFFER_COMPLETE)
            {          //if the FBO fails we need to delete it and all the frame buffers created
                        std::cout<<"FBO Failure" << std::endl;
                        glBindFramebuffer(GL_FRAMEBUFFER, 0);
                        glBindTexture(GL_TEXTURE_2D, 0);
                        glDeleteFramebuffers(1, &fboHandle);
                        glDeleteTextures(1, colourTextureOut);
                        glDeleteTextures(1, depthTextureOut);
                        glDeleteTextures(1, normalTextureOut);
                        *colourTextureOut       = 0;
                        *depthTextureOut       = 0;
                        *normalTextureOut      = 0;
                        return 0;
            }

            //disable some stuff
            glBindTexture(GL_TEXTURE_2D, NULL);
            glBindFramebuffer(GL_FRAMEBUFFER, NULL);
            return fboHandle;
}

                 Now that you know how to make an FBO, let me show you really quickly how you would use one. It’s simple, just like most openGL code you have to tell openGL what you’re doing. First off the GLenum is an enumeration of the color attachments that the FBO has attached. From above you’ll see that we had the color attachments 0 and 1. The next thing we do is tell openGL that we are binding a Frame Buffer and then pass it the type and the FBO handle created by the function above. Next we tell openGL that we are drawing to buffers and tell it how many by passing it the GLenum we just created. The next step is to draw the scene. Once done, you can unbind the frame buffer by binding with NULL.

void Game::DeferredFirstPass(float elapsed, float time)
{
            GLenum Buffers[] = {GL_COLOR_ATTACHMENT0, GL_COLOR_ATTACHMENT1};
           
            glBindFramebuffer(GL_FRAMEBUFFER, GbufferFBO);
            glDrawBuffers(2, Buffers);

            glViewport(0, 0, windowWidth, windowHeight);
            glClear(GL_DEPTH_BUFFER_BIT | GL_COLOR_BUFFER_BIT);

            this->RenderScene(0.0f, 0.0f);

            glBindFramebuffer(GL_FRAMEBUFFER, NULL);
            glViewport(0, 0, windowWidth, windowHeight);
}

                The Last smidgen of information I will give is how to use the MRT s in the shader. This is extremely easy and it really is just making your out colour an array like the code below shows. Out putting to a different index will output to a different texture in the current FBO.
#version 140

uniform sampler2D TextureData;

in vec2 texcoord;
in vec3 normal;

out vec4 outColor[2];

void main ()
{          //render to two targets save packed normals in one and textured object in the other
            outColor[1] = vec4(normal * 0.5f + 0.5f ,1.0f);
            outColor[0] = texture(TextureData, texcoord.st);
}

                The code snippets have been from a tech demo I’ve made that shows off deferred lighting, just one of the many post-processing effects that use FBOs and MTRs. Below is a screen shot of my tech demo.

Monday, February 25, 2013

Intro To Black Magic... Also Know as Shaders


Intro to Shaders
                Shaders are like mystical black magic in a text file, the visual effects they can achieve is incredible. Shaders are responsible for all most all of the graphical advancements in games. A shader manipulates data in the graphics pipeline at certain programmable steps. I cannot stress this enough: what is passed to a shader is just DATA and you can do whatever you want to DATA! There are currently 5 programmable stages in the pipeline the vertex shader, Tessellation Control & Evaluation Shaders, Geometry Shaders, Fragment Shaders and Compute Shaders. Below is a diagram of the full openGL graphics pipeline.
From http://img.hexus.net/v2/lowe/News/Khronos/OGLCS.jpg
 Most games can get away with only using the vertex and fragment shaders to create Interesting visual effects. This blog will be focused on the vertex and fragment shader as they are the easiest to understand. Below is a simplified version of the Graphics pipeline that makes it easier to understand what happens.
From http://goanna.cs.rmit.edu.au/`gl/teaching/Interactive3D/2012/images/pipline.png
                Shaders are written in a couple of programming languages such as HLSL, GLSL and CG. These languages have different syntax but they all do the same stuff. The program language I use is GLSL because I found it to be congruent with openGL, the graphics APU I use. A Vertex has a specific job to process individual vertices and send out its new position to the pipeline. Here is a simple Vertex shader in GLSL.

#version 150
uniform mat4 uni_ModelViewProjection;
in vec3 in_vertex;

void main(void)
{
                gl_Position = uni_ModelViewProjection * vec4(in_vertex, 1.0);
}

All this Shader does is takes a vertex -in_vertex- and transforms it by the modelVeiwProjection matrix which both are variables passed in from the c++ program. It then outputs the new vertex in the system variable gl_Position. A vertex Shader must set gl_Position! If it doesn’t set it the shader will not compile.

The Fragment shader is next which manipulates the pixel colour of an object per fragment. The only thing the following example does, is set every output color to red.  A fragment shader must send out a colour vec4 or vec3.

#version 150
out vec4 out_color;

void main(void)
{
                out_color =  vec4 (1.0f, 0.0f, 0.0f, 0.0f);
}

                With vertex and fragment shaders you can make amazing visual effects from the extremely easy one I showed to very nice and complex ones like these:
From http://www.userinter.net/wp-content/uploads/2011/04/Shader-Effects.jpg

From http://devmaster.net/assets/post/3021/featured_image/sized_for_post_17ced3ab-a06c-4edd-9b0d-a1ef4be7c4b7.jpg