What is the difference between a Library and a Framework?

Library
A library contains many pieces of functionality that can be individually selected and used. The pieces do not depend upon one another and so you are not locked into functionality you do not want. Ultimately this means you are not bound to a workflow and the code base is more adaptable to change.

Framework
A framework unlike a library sets out strictly how you will be working. It provides a workflow that can add value however is hard to change. Frameworks can speed up development allowing rapid development however they can result in changes required being impossible to implement.

Anonymous Functions – Closure Callbacks

Here is an example of anonymous functions in unity, below is a very helpful technique around loading assets and then setting a delegate to grab that loaded asset. Below, take note of the ‘closure’ delegate, which is being passed into the asset bundle manager load function as the second parameter. Because the delegate was defined inside the loop it forms closure around the variable ‘i’ which would usually not exist. When each bundle is loaded the callback will trigger and the asset will be placed at the correct index in the array.


for(int i=0; i < things.Length; i++)
{
    AssetBundleManager.Load(thingNames[i], delegate (AssetBundle bundle)
    {
        characters[i] = bundle.Load(thingNames[i] + "LOD0")
    });
}

Optimising Unity games for Mobile

Optimise for CPU and GPU

CPU

CPU is often limited by the number of batches that need to be rendered. “Batching” is where the engine attempts to combine the rendering of multiple objects into a chunk of memory in order to reduce CPU overhead due to resources switching.

To draw an object on the screen, the engine has to issue a draw call to the graphics API (e.g. OpenGL or Direct3D). Draw calls are often expensive, with the graphics API doing significant work for every draw call, causing performance overhead on the CPU side. This is mostly caused by the state changes done between the draw calls (e.g. switching to a different material), which causes expensive validation and translation steps in the graphics driver.

Basically draw calls are the commands that tells the GPU to render a certain set of vertices as triangles with a certain state (shaders, blend state and so on). It should be noted that draw calls aren’t necessarily expensive. In older versions of Direct3D, many calls required a context switch, which was expensive, but this isn’t true in newer versions. The main reason to make fewer draw calls is that graphics hardware can transform and render triangles much faster than you can submit them. If you submit few triangles with each call, you will be completely bound by the CPU and the GPU will be mostly idle. The CPU won’t be able to feed the GPU fast enough. Making a single draw call with two triangles is cheap, but if you submit too little data with each call, you won’t have enough CPU time to submit as much geometry to the GPU as you could have.

There are some real costs with making draw calls, it requires setting up a bunch of state (which set of vertices to use, what shader to use and so on), and state changes have a cost both on the hardware side (updating a bunch of registers) and on the driver side (validating and translating your calls that set state).

Unity uses static batching and dynamic batching to address this.

  • Static Batching: combine static (i.e. not moving) objects into big meshes, and render them in a faster way.

Internally, static batching works by transforming the static objects into world space and building a big vertex + index buffer for them. Then for visible objects in the same batch, a series of “cheap” draw calls are done, with almost no state changes in between. So technically it does not save “3D API draw calls”, but it saves on state changes done between them (which is the expensive part).

  • Dynamic Batching: for small enough meshes, transform their vertices on the CPU, group many similar ones together, and draw in one go.

Built-in batching has several benefits compared to manually merging objects together (most notably, the objects can still be culled individually). But it also has some downsides too (static batching incurs memory and storage overhead; and dynamic batching incurs some CPU overhead). Only objects sharing the same material can be batched together. Therefore, if you want to achieve good batching, you need to share as many materials among different objects as possible.

If you have two identical materials which differ only in textures, you can combine those textures into a single big texture – a process often called texture atlasing. Once textures are in the same atlas, you can use a single material instead.

Currently, only Mesh Renderers are batched. This means that skinned meshes, cloth, trail renderers and other types of rendering components are not batched.

Semitransparent shaders most often require objects to be rendered in back-to-front order for transparency to work. Unity first orders objects in this order, and then tries to batch them – but because the order must be strictly satisfied, this often means less batching can be achieved than with opaque objects.

Manually combining objects that are close to each other might be a very good alternative to draw call batching. For example, a static cupboard with lots of drawers often makes sense to just combine into a single mesh, either in a 3D modeling application or using Mesh.CombineMeshes.

 

GPU

GPU is often limited by fillrate or memory bandwidth. If running the game at a lower display resolution makes it faster then you’re most likely limited by fillrate on the GPU. Fill rate refers to the number of pixels that a video card can render or write to memory every second. It is measured in megapixels or gigapixels per second, which is obtained by multiplying the clock frequency of the graphics processing unit (GPU) by the number of raster operations (ROPs).

 

Textures – Texture Size, Compression, Atlases and MipMaps

Optimal Texture Type – PNG is the lesser of many evils. It does lossless image compression compared to lossy JPEG compression, it doesn’t do alpha as great as TGA does – but it does do compression and alpha mapping good enough to make it better than the other file types.

Texture Compression –  ETC texture compression, however doesn’t support alpha channels. If alpha then go with uncompressed.

You should always have mipmaps checked if you’re using 3D, because otherwise you get awful artifacts when the camera moves, plus it runs faster since it doesn’t have to calculate so many pixels for distant objects. Other than looking slightly blurry compared to not having mipmaps, there shouldn’t be any downsides, and the slight blurriness is more than compensated for by the lack of flickering texture artifacts. You can use trilinear filtering so that the transition between mipmap levels is smooth. If you have any serious degradation with mipmaps, that’s not normal

To create Texture Atlases use Texture Packer Tool with the standalone version

http://www.codeandweb.com/texturepacker/download

https://www.assetstore.unity3d.com/en/#!/content/8905

 

Models – Triangle Count and UV Map

  • Don’t use any more triangles than necessary
  • Try to keep the number of UV mapping seams and hard edges (doubled-up vertices) as low as possible

You should use only a single skinned mesh renderer for each character. Unity optimizes animation using visibility culling and bounding volume updates and these optimizations are only activated if you use one animation component and one skinned mesh renderer in conjunction. The rendering time for a model could roughly double as a result of using two skinned meshes in place of a single mesh and there is seldom any practical advantage in using multiple meshes.

When animating use as few bones as possible

A bone hierarchy in a typical desktop game uses somewhere between fifteen and sixty bones. The fewer bones you use, the better the performance will be. You can achieve very good quality on desktop platforms and fairly good quality on mobile platforms with about thirty bones. Ideally, keep the number below thirty for mobile devices and don’t go too far above thirty for desktop games.

Use as few materials as possible

You should also keep the number of materials on each mesh as low as possible. The only reason why you might want to have more than one material on a character is that you need to use different shaders for different parts (eg, a special shader for the eyes). However, two or three materials per character should be sufficient in almost all cases.

 

Culling and LOD (Level of Detail)

Occlusion Culling is a feature that disables rendering of objects when they are not currently seen by the camera because they are obscured (occluded) by other objects. This does not happen automatically in 3D computer graphics since most of the time objects farthest away from the camera are drawn first and closer objects are drawn over the top of them (this is called “overdraw”). Occlusion Culling is different from Frustum Culling. Frustum Culling only disables the renderers for objects that are outside the camera’s viewing area but does not disable anything hidden from view by overdraw. Note that when you use Occlusion Culling you will still benefit from Frustum Culling.

The occlusion culling process will go through the scene using a virtual camera to build a hierarchy of potentially visible sets of objects. This data is used at runtime by each camera to identify what is visible and what is not. Equipped with this information, Unity will ensure only visible objects get sent to be rendered. This reduces the number of draw calls and increases the performance of the game.

 

Fog and Lighting Effects

The solution we came up with is the use of simple mesh faces with a transparent texture (Fog planes) instead of global fog. Once a player comes too close to a fog plane, it fades out and moreover, vertices of the fog plane are pulled away (because even a fully transparent alpha surface still consumes lot of render time).

 

Debug Performance – Rendering Statistics, and Frame Debugger

Rendering Statistics

The Game View has a Stats button in the top right corner. When the button is pressed, an overlay window is displayed which shows realtime rendering statistics, which are useful for optimizing performance. The exact statistics displayed vary according to the build target.

Frame Debugger

The Frame Debugger lets you freeze playback for a running game on a particular frame and view the individual draw calls that are used to render that frame. As well as listing the drawcalls, the debugger also lets you step through them one-by-one so you can see in great detail how the scene is constructed from its graphical elements.

 

Extra Tips

  • Set Static property on a non-moving objects to allow internal optimizations like static batching.
  • Do not use dynamic lights when it is not necessary – choose to bake lighting instead.
  • Use compressed texture formats when possible, otherwise prefer 16bit textures over 32bit.
  • Use pixel shaders or texture combiners to mix several textures instead of a multi-pass approach.
  • CG: Use half precision variables when possible.
  • Do not use Pixel Lights when it is not necessary – choose to have only a single (preferably directional) pixel light affecting your geometry.
  • Alpha blending is ruthless on mobile.
  • Use occlusion culling.
  • Use texture atlases and pay attention to texture memory.
  • Limit particle emission count, use fast mobile shaders.
  • Use lightmapping, baked shadows, and blob shadows.

 

 

 

 

 

Shader Writing for Unity

Before we begin creating our own shaders we need to understand some basics.


 

What are Shaders?

Shaders in Unity – small scripts that contain the mathematical calculations and algorithms for calculating the colour of each pixel rendered, based on the lighting input and the material configuration.
A shader is simply code, a set of instructions that will be executed on the GPU. It is a program for one of the stages of the graphics rendering pipeline. All shaders can be divided into two groups: vertices and fragment(pixel). In a nutshell shaders are special programs which represents how different materials are renderered.

What is a Material?
Materials are wrappers which contain a shader and the values for its properties. Hence, different materials can share the same shader, feeding it with different data.
Another way of describing Materials is that they are definitions of how a surface should be rendered, including references to textures used, tiling information, colour tints and more. The available options for a material depend on which shader the material is using.
In general materials are not much more than containers for shaders and textures that can be applied to 3D models. Most of the customization of materials depends on which shader is chosen for it, although all shaders have some common functionality. Basically a material determines object appearance and includes a reference to a shader that is used to render geometry or particles.
In summary a shader’s job is to take in 3D geometry, convert it to pixels and render it on the 2D screen. A shader can define a number of properties that you will use to affect what is displayed when your model is rendered – the settings of those properties when stored are a material.

What is the Graphics Pipeline
The Graphics Pipeline or Rendering Pipeline refers to the sequence of steps used to create a 2D raster representation of a 3D scene.

Input Data
Data is sent in to the pipeline in the Input Assembler and processed all the way through the stages until it is displayed as a pixel on your monitor. The data typically being a 3D model (vert position, normal direction, tangents, texture coordinates and color).
Even sprites, particles, and textures in your game world are usually rendered using vertices just like a 3D model.

What came before?
“The fixed pipeline” – Pre DirectX 8 and OpenGL Arb assembly language. Fixed way to transform pixels and vertices. Impossible for developer to change how pixels and verts were transformed
and processed after passing them to the GPU.

Stages of the Graphics Pipeline
Vertex Shader Stage
This stage is executed per vertex and is mostly used to transform the vertex, do per vertex calculations or make calculations for use later down pipeline.
Hull Shader Stage (Only used for tessellation)
Takes the vert as input control points and convert it in to control points that make up a patch (a fraction of a surface)
Domain shader stage (Only used for tessellation)
This stage calculates a vertex position of a point in the patch created by the Hull Shader
Geometry Shader Stage
A geometry shader is an optoinal program that takes the primitives (a point, line, triangle etc) as an input and can modify remove or add geometry to it.
Pixel Shader Stage
The pixel shader (also known as fragment shaders in the openGL world) is executed once per pixel giving color to a pixel. It gets its input from the earlier stages in the pipeline
and is mostly used for calculating surface properties, lighting, and post-process effects.

Optimize!
Each of the stages above is usually executed thousands of times per frame, and can be a bottleneck in the graphics pipeline. A simple cube made from triangles usually has around 36 verts. This means that the vertex shader stage will be executed 36 times every frame, and if you aim for 60 fps, this will be executed 2160 times per second. Optimize as much as you can.

Unity’s Rendering Pipeline
So with shaders we can define how our object will appear in the game world and how it will react to lighting. How these lights will react on the objects depend on the passes of the shader and which rendering path is used. The rendering path can be changed through Unity’s Player Settings. Or it can be overridden in the camera’s ‘Rendering Path’ setting in the inspector. In Unity there are 3 rendering paths: Vertex Lit,Forward Rendering and Deferred Rendering. If the graphics card can’t handle the current selected render path it will fallback and use another one. So for example if deferred rendering isn’t supported by the graphics card, Unity will automatically use Forward Rendering. If forward rendering is not supported it will change to Vertex Lit. Since all shaders are influenced by the rendering path that is set I will briefly describe what each rendering path does.

Vertex Lit
Vertex Lit is the simplest lighting mode available. It has no support for real-time shadows. It is commonly used on old computers with limited hardware. Internally it will calculate lighting from all lights at the object vertices in one pass. Since lighting is done on a per-vertex level, per-pixel effects are not supported.

Forward Rendering
Forward rendering renders each object in one or more passes, depending on the lights that affect the object. All lights are treated differently depending on the settings and intensity being set by the user. When forward rendering is used, the amount of pixel lights set from the quality menu that affect the object will be rendering using full per-pixel lighting. Additionally 4 point lights are calculated per-vertex and all other lights are computed as Spherical Harmonics which is an approximation. A light can be per-pixel lit depending on several situations. Lights with the render mode set to Not Important, are always per-vertex or spherical harmonics. Brighter lights are always calculated per-pixel also when the render mode is set to Important. Forward rendering is the default selected rendering path in Unity.

Deferred Rendering
In Deferred rendering there is no limit on the number of lights that affect an object and all lights are calculated on a per-pixel base. This means that all lights interact with normal maps etc. Lights can also have cookies and shadows. Since all lights are calculated per-pixel it works great on big polygons. Deferred rendering is only available in Unity Pro.

Creating a Shader in Unity
1.) Firstly we need a 3D model with a material on it which will use our new shader. (Add Sphere)
2.) Create Shader – surface shader
3.) Add material, Set the shader this material is using to our new shader, now set 3D model mesh renderer material to this new material.
4.) This is what our ShaderLab shader is structured like at start:
Shader “Category/ShaderName” {
     Properties{}
     SubShader {
          Pass {
             CGPROGRAM
             // your shaders here
             ENDCG
          }
     }
    SubShader {
    }
    SubShader {
    }
    SubShader {
    }
    FallBack “FallbackShaderName”
}
The category is used to place the shader in the shader dropdown, and the name is used to identify it. Next each shader can have many properties. These can be normal numbers and floats, color data or textures. ShaderLab got a way of defining these so it looks good and user friendly in the Unity inspector.
Now we need to define at least one sub shader so our object can be displayed. We can have more than one sub shader, Unity will pick the first sub shader that runs on the graphics card. Each sub shaders defines a list of rendering passes. Each pass causes the geometry to be rendered once. Generally speaking you would like to use the minimum amount of passes possible since which every added pass our performance goes down because of the object being rendered again. A pass can defined in 3 ways, a regular pass, use pass or a grab pass.
The ‘UsePass’ command is used when we want to use another pass from another shader. This can help by reducing code duplication.
The ‘GrabPass’ is a special pass. It grabs the content of the screen where the object is to be drawn into a texture. This texture can then be used for more advanced image based processing effects. A regular pass sets various states for the graphics hardware. For example we could turn on/off vertex lighting, set blending mode, or set fog parameters.
Inside each subshader, there needs to be a pass, as a shader ca be executed in multiple passes. Try ot keep the number of passes to a minimum for performance reasons but a pass will render the geometry object once and then move on to the next pass. Most shaders will only need one pass.
Your shader implementation will be inside the pass, surrounded by CGPROGRAM or GLSLPROGRAM and ENDGLSL if you want to use GLSL. Unity will cross compile CG to optimized GLSL or HLSL depending on the platform.
Then we have the fallback. If none of the shaders will work, we can fallback to another simple shader like the diffuse shader.
Here we have an example of a shader that takes in ambient light.
1.) Category and name, can be whatever you want.
Shader “UnityShaderExample/SimpleAmbientLight” 
2.) Properties, first the name of the property, then a display name that will show up in the Unity Editor, a prop type and default value
  Properties {
        _AmbientLightColor (“Ambient Light Color”, Color) = (1,1,1,1)
        _AmbientLighIntensity(“Ambient Light Intensity”, Range(0.0, 1.0)) = 1.0
    }
3.) Sub Shaders
    SubShader 
    {
4.) Passes per Sub Shader
        Pass 
        {
            CGPROGRAM
5.) Define Shader Compilation Target
#pragma target 2.0
6.) Define the name of the function that will be used as the vertex shader
#pragma vertex vertexShader 
7.)  Define name of function to be used as fragment shader
#pragma fragment fragmentShader
8.) Define our variables that the property is pointing at, these must be same as property name above
    fixed4 _AmbientLightColor; 
            float _AmbientLighIntensity;
9.) Vertex Shader
            float4 vertexShader(float4 v:POSITION) : SV_POSITION
            {
                return mul(UNITY_MATRIX_MVP, v);
            }
10.) Pixel Shader
            fixed4 fragmentShader() : SV_Target
            {
                return _AmbientLightColor * _AmbientLighIntensity;
            }
            ENDCG
        }
    }
}

What is this Shader doing?
The Vertex Shader
The Vertex Shader is doing one thing only, and that is a matrix calculation. This function takes one input, and that is the vertex position only, and it got one output, the transformed position of the vertex (SV_POSITION) in screen space, the position of the vertex on the screen, stored by the return value of this function. This value is obtained by multiplying the vertex position (currently in local space) with the Model, View and Projection matrices easily obtained by Unity’s’ built-in state variable.
This is done to position the vertices at the correct place on your monitor, based on where the camera is (view) and the projection.
The SV_POSITION is a semantic as is used to pass data between different shader stages in the programmable pipeline. The SV_POSITION is interpreted by the rasterizer stage. Think if this as one of many registers on the GPU you can store values in. This semantic can store a vector value (XYZW), and since it is stored in SV_POSITION, the GPU knows that the intended use for this data is for positioning.

The Pixel Shader
This is where all the coloring is happening, and our algorithm is implemented. This algorithm doesn’t need any input as we won’t do any advanced lighting calculations yet (we will learn this in the next tutorial). The output is the RGBA value of our pixel color stored in SV_Target (a render target, our final output).

Unity Shaders Reference Material
This table of mathematical functions from the Nvidia Developer Zone is a great help