[SoC 2014] Material System Improvements

Threads related to Google Summer of Code
Post Reply
scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 216

[SoC 2014] Material System Improvements

Post by scrawl »

This is a heavily updated version of the material system draft I had posted earlier. It was reworked a little based on your feedback (but mostly rewritten and reworded).


Overview

The proposal in summary:
  • Add much-needed improvements to the material system
  • Add a Sample demonstrating the new features
  • Create a default "uber shader" used by the Sample Browser that includes some of the most commonly used shader techniques
  • Port several materials and shaders from existing Components and Samples to the new system where appropriate
What's wrong with the current system?

The Ogre material system works great for fixed function materials.
Unfortunately, using lots of different shader effects is incredibly tedious right now.
When coding shaders, you typically need to compile a new shader for each combination of features that are in use. Every new feature that you want to be able to toggle individually (such as normal mapping, hardware instancing, etc) essentially doubles the number of potential shaders. It's an O(2^n) problem.
A commonly used technique to mitigate this is to use an "uber shader" - that is, a single shader file that includes all features, guarded by "#if XYZ" directives. This is currently not supported well in Ogre: preprocessor defines need to be set up in advance in a GPU program definition. A material is unable to impact this definition in any way. If a different combination of defines is needed, you'll need another program definition for it - and we're back to the O(2^n) problem.

Another issue is that shader files (even with a C-style preprocessor) lack flexibility. Often, factors such as the number of texturing layers for an alpha splatting shader, number of lights in a multi light shader etc are not known in advance.

A "last resort" taken by many Ogre users (and even Ogre itself, in the current version of the Terrain component and the Deferred Shading sample) is to manually assemble the shader code from within C++ code. This practice is highly prone to error, frustrating to debug, makes it harder to grasp what the shader does, and in general hardcodes something that clearly should not be hardcoded. An indicator that we have a serious problem.

Design proposal

Note: The concepts below are completely optional. Existing "low level" materials will continue to function as before. Low level material access is still useful in areas where permutations are not needed (e.g. most compositors).

Properties

We add the concept of abstract material properties. A property can be any kind of string key-value pair. The only special property is the "generator" property - it specifies the type of material generator to use (see further below).
For materials with a generator, the techniques section would usually be left blank in the material script, as the techniques will be generated at runtime later.

Inheriting properties from other materials works great in a material script:

Examples

Code: Select all

material foo
{
   generator main
   diffuse_map foo.png
   diffuse 1.0 1.0 1.0
}

material foo1 : foo
{
   normal_map foo_n.png
}

material foo2 : foo1
{
   diffuse 0.5 0.5 0.5
}

material ocean
{
   generator water
   wave_speed 20
   choppiness 10
}

material lake : ocean
{
   wave_speed 2
   choppiness 5
}
Material generator

The material generator's job is to fill in the techniques of a material when it's loaded. It does so by looking at the material's properties and then creating techniques, passes, texture units, and shaders (invoking the shader generator, see below) as appropriate. This can also be affected by external factors, such as graphics settings, current number of lights in the scene etc. These factors can also change during runtime - to respond to this, the materials can be reloaded so that the generator is invoked again.

Since there's no material generator that can be truly generic, it should be a pure virtual class in the Ogre core, left to be implemented by users. A sample implementation including some of the most common shader effects should be added to the sample browser, so the majority of users can just copy that and modify it for their purposes.
Specialised Components such as the Terrain component should ship with a ready-made MaterialGenerator subclass and its associated shader file(s).

To create a material generator, a class should be derived from the pure virtual Ogre::MaterialGenerator class. Instances of that class can then be registered with the MaterialManager.
Usually one instance per class should be sufficient. A use case for multiple instances of the same material generator is when you'd like to store extra per-material information in the material generator object. This would usually be done for materials created programatically, as it's rather tedious to store information in the property map (and would involve lots of unnecessary string conversions).

An example: storing layer information in material generator objects

Code: Select all

class TerrainMaterialGenerator : public MaterialGenerator
{
   struct LayerInfo
   {
      std::string colourMap;
      std::string normalMap;
      uint8 flags;
   }
   std::vector<LayerInfo> layers;

   virtual void generate (MaterialPtr material);
}
A stripped-down example of a very basic material generator

Code: Select all

MyMaterialGenerator::generate(MaterialPtr mat)
{
   const map<string,string>& properties = mat->getProperties();
   mat->removeAllTechniques();
   Technique* tech = mat->createTechnique();
   tech->getPass(0)->createTextureUnitState(properties["diffuse_map"]);
   if (!properties["normal_map"].empty() && mNormalMappingEnabledInUserSettings)
   {
      tech->getPass(0)->createTextureUnitState(properties["normal_map"]);
   }
   tech->getPass(0)->setVertexProgram (mVertexShaderGenerator->getShader(properties));
   tech->getPass(0)->setFragmentProgram (mFragmentShaderGenerator->getShader(properties));
}
Shader generator

The default shader generator uses plain old shader files (typically containing GLSL or HLSL code) combined with a couple of macros for added flexibility (see "Macro overview" section). Custom shader generators can also be plugged in, allowing for fancier setups, e.g. building shaders in Lua or another scripting language.

Any number of shader generators can be registered, and it's up to the material generator to decide which shader generator(s) should be used. Example: one shader generator for Deferred materials, one for regular Forward materials, one for Terrain, etc.

To reduce the amount of shaders compiled, shader generators track the permutations of compiled shaders and just return an already existing shader if the same permutation is requested again. A permutation basically comes down to the combination of values from any macros (see below) that are retrieved.

Preprocessor

The shader generator needs a very fast, C-like preprocessor to handle includes, flow control, line directives, expressions and macros (object-like and function-like). Speed is more important here than handling every possible edge use case.
Directives intended for the shader compiler such as #version, #extension need to be kept obviously.

Macro overview


To give a satisfying degree of flexibility, several macros are implemented in the default shader generator, serving as a kind of "extension" to the shader language and as interface to certain Ogre facilities from within the shader file.

OGRE_SHADER_LANG

Gets expanded to the shader language currently being used. Useful for handling GLSL and GLSLES in the same file since they're very similar.

Example:

Code: Select all

#if OGRE_SHADER_LANG == OGRE_LANG_GLSLES
// Add precision qualifiers which are only used in GLSL ES
precision highp int;
#endif
OGRE_VERTEX_SHADER, OGRE_FRAGMENT_SHADER, ...

Expanded to "1" or "0" depending on the type of shader currently being compiled. Makes it possible to handle vertex and fragment shader in the same file, which is useful to get the inputs/outputs to match correctly.

Example:

Code: Select all

// declare Inputs/outputs here
#if OGRE_VERTEX_SHADER
// Vertex shader code here
#else
// Fragment shader code here
#endif
@property(propertyName)

Properties are passed to the shader generator upon invocation.
Expanded to "0" if the property with this name does not exist or is 0, otherwise expanded to "1".
Example:

Code: Select all

#if @property(normal_map)
// Declare the normal map sampler only if we use a normal map
uniform sampler2D normalMap;
#endif
#foreach(symbol, count)

Code block gets repeated count times. Useful for multiple lights in a single pass, multi layer texture splatting, etc.
Current iteration can be retrieved via @(symbol).
The count parameter can be a literal or a @property macro.
#line directives for the shader compiler will be inserted automatically to keep the line numbers valid when debugging compile errors.

Example:

Code: Select all

#foreach(n, 2)
This is iteration number @{n}
#endforeach
Output:

Code: Select all

#line 2
This is iteration number 0
#line 2
This is iteration number 1
@param_named(...)
@param_named_auto(...)

Identical to param_named / param_named_auto in a program definition. There are several reasons this macro is very useful:
• Do not bind the parameter if in a dead branch (as the parameter would not even exist)
• Bind indexed parameters (e.g. per light parameters) in a foreach construct

@counter

Unfortunately, some shader languages (such as HLSL and CG) only provide indexed float4 TEXCOORD<n> interpolator registers (with 0 <= n < 8 ), rather than a variable like concept (such as 'varying' in GLSL).
This is rather inconvenient when dealing with permutations - you keep having to change the indices as you add and remove features.
A very simple but effective fix is to add a counter macro. Only invocations that survive the preprocessing step will add to the count.

Example

Code: Select all

out float4 normal : TEXCOORD@counter
#if NORMAL_MAP
out float4 tangent : TEXCOORD@counter
#endif
out float4 position : TEXCOORD@counter
Output (for NORMAL_MAP = 0)

Code: Select all

out float4 normal : TEXCOORD0
out float4 position : TEXCOORD1
Precedence:

1. @property
2. #foreach
3. Run the preprocessor - eliminates dead branches
5. @counter
4. @param_named[_auto]

Schedule
Core changes (~4 weeks)
  • Implement the preprocessor and create associated unit tests
  • Add MaterialGenerator class and integrate with MaterialManager and MaterialSerializer
  • Implement the default shader generator and create unit tests for its macros
Implement a Sample demonstrating the system (~4 weeks)
  • Create a sample with a few entities and lights that will serve as a testbed
  • Write an uber shader and an associated MaterialGenerator with some commonly used techniques such as normal mapping, HW skinning, HW instancing, [PSSM] shadows, specular maps, parallax, environment mapping, point / directional / spot lights, variable amount of lights, multiple lighting techniques (phong, BDRFs, etc)
  • Add buttons/controls to toggle each shader feature individually, and to change the current shader language
  • Demonstrate automatic generation of LOD techniques
  • Demonstrate automatic generation of scheme techniques (e.g. a small RTT box in the corner with slightly altered shaders)
First pull request to the main repository. Interested users can start playing with the new system and provide feedback.

Port existing materials & shaders to the generator where appropriate (~5 weeks)
  • Port the Terrain material. Current implementation is ~2.3k lines of code, so this will take a while. The refactored version will be much smaller and more efficient.
  • Port the Deferred Shading Sample
  • Refactor Sample Browser materials in general to use the uber shader and material generator where appropriate; delete no longer used shaders. This will also help with getting more consistent, better looking materials in many samples.
Pull request to the main repository.

Documentation and cleanup (~1 weeks)
  • Write a manual section about the added features
  • Make sure doxygen comments are complete
  • Make sure helpful exceptions are thrown in case of user error
  • Double check for any code quality issues
  • Run static & dynamic analysis
Final pull request.

If there's any time left, work on some additional features (see below), otherwise move them post-GSOC.

Optional tasks

I have selected a few other tasks related to the material system I would like to work on. They are by no means essential so can be moved post-GSOC if there's no time left.

Terrain material improvements

After the terrain component is ported, working on it will be much easier and nicer. Time to put that to use! I have a few ideas for improving the terrain material, which I've already implemented in personal projects long ago, so will be rather straightforward to add:
• Triplanar texturing support
• Multiple lights support and point light support
• Multi-pass splatting, essentially removing the layer limit

Improve microcode caching

Since Ogre 1.8, microcode caching can be used to speed up shader load times.
The main issue with this (and probably the reason it has not been enabled in the sample browser yet) is that editing a shader file (or pulling in a new version of it) has no effect if a cache has already been saved.
It would be nice to be able to benefit from shader caching not just in a final product, but also during development. The way I deal with it in my own projects is to save the last write time of the shader source files along with the cache, and check if a file has been modified before trying to load its cache. It should be investigated if this solution can be added to the Ogre core in a clean way. If not, we should at least add it to the Sample Browser.
Note that to get change tracking to work properly with #include, a dependency list needs to be built for each shader.

Faster shader editing

The current process for editing shaders appears to be: make adjustments, launch program, wait for it to load, check the result - and repeat.
A much nicer way would be to have the application automatically react to shader changes and reload them. Therefore you would not have to restart the application every time to test something out.
I am not sure if the file monitoring part is Core-worthy. It would need to be optional in any case, of course. If not in the core, the sample browser would be a good place.
It also seems there is currently a bug affecting this, which would need to be fixed first: https://ogre3d.atlassian.net/browse/OGRE-325

Get rid of CG sample materials

The reason we used CG in the first place was to write shaders once and use them in both OpenGL and DX renderers.
Unfortunately, due to CG not being free software and being unavailable (or having limited support) on several platforms / hardware, GLSL alternatives for the sample materials have to be provided anyway.
A lot of work on this has been done already. Compiling Ogre without CG at this moment has most samples working. I would like to complete this by adding the missing GLSL shaders and then removing the CG materials completely, as it makes no sense to maintain them any longer.
The CG plugin itself can be kept for now, though maybe it should be marked as deprecated at some point - the new GL3 render system doesn't support CG anyway.

A promising alternative to CG could be mojoshader: "construction of a full HLSL compiler is in progress".

Integrate modular shaders

With GLSL, Ogre supports the use of modular shaders. Frequently used functions can be put into modules compiled separately and then attached/linked to individual programs.
This should be integrated with the default shader generator. Obviously making each included header into a module automatically is problematic, and impossible if the header itself needs permutations. A better way would be to add a macro that turns the current file into a module.
All this does is reduce compile time, so it's a rather low priority feature.

Why this is an important project

That was a pretty long post, so let us recap what we would gain from the project:

Ogre users are given a set of powerful and versatile shaders they can start with.
Shader writers are given all the tools they need to write reusable shaders quickly and efficiently.
Artists are given a more abstract material system interface and do not have to worry about shader implementation details.
Ogre projects adopting the new system (including our own Sample Browser) benefit from more efficient, easier to maintain code.

Why I am the person for this project

I have 3 years of experience in using Ogre. I've only contributed small patches so far - but I would like to get involved more, so this GSOC is a great chance.

I already worked with the Ogre::Terrain component a lot, but I also wrote my own terrain system for large worlds from scratch, which can be seen here: http://www.ogre3d.org/forums/viewtopic.php?f=11&t=79026

I have written the shader pipelines for two projects I use Ogre in. As I kept adding more features, writing all permutations by hand became extremely tedious. I went for generating the shaders entirely in C++ code next, but quickly realized that every new feature or update I wanted to do would be rather painful with such a system.
To improve this, I needed some sort of helper library to deal with the permutation problem - and the shiny material system was born. In hindsight, I am not 100% happy with it. If I had to rewrite it from scratch, I would do a few things different. But it was a valuable experience none the less because I have learned a lot about material/shader generation systems which will be useful for this GSOC.
Last edited by scrawl on Tue Feb 25, 2014 5:00 pm, edited 2 times in total.
Owen
Google Summer of Code Student
Google Summer of Code Student
Posts: 91
Joined: Mon May 01, 2006 11:36 am
x 21

Re: [GSOC 2014] Material System Improvements

Post by Owen »

scrawl wrote:

Code: Select all

MyMaterialGenerator::generate(MaterialPtr mat)
{
   const map<string,string>& properties = mat->getProperties();
   mat->removeAllTechniques();
   Technique* tech = mat->createTechnique();
   tech->getPass(0)->createTextureUnitState(properties["diffuse_map"]);
   if (!properties["normal_map"].empty() && mNormalMappingEnabledInUserSettings)
   {
      tech->getPass(0)->createTextureUnitState(properties["normal_map"]);
   }
   tech->getPass(0)->setVertexProgram (mVertexShaderGenerator->getShader(properties));
   tech->getPass(0)->setFragmentProgram (mFragmentShaderGenerator->getShader(properties));
}
The generate method should support asynchronous completion. Otherwise, you're going to have some kind of interesting hell when the generator depends upon resources which haven't yet loaded.

How are properties defined?

How are unique shader instances determined?

You may be interested in this document I cooked up while proscastinating from revising ( :roll: ) regarding the intelligent "piecemeal" shading language I was cooking up. There are some of the same issues to cover. Notably, we basically have the same property idea; I fleshed out there the matter of how properties get passed from material system to shader. Especially important to consider is the matter of texture coordinate set(s).

Microcode caching fixes will probably go in when I do the finishing work on the RS redesign. It isn't much incrementally on top of fixing it to work again after I broke the world, anyway.

The shader generator will probably want to overload the ResourceLoader::openCache method so it can manage the backend of such caching (It'll probably want to stash the cache data into some directory based upon a hash of the shader permutation or whatever)
scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 216

Re: [GSOC 2014] Material System Improvements

Post by scrawl »

The shader generator will probably want to overload the ResourceLoader::openCache method so it can manage the backend of such caching (It'll probably want to stash the cache data into some directory based upon a hash of the shader permutation or whatever)
I don't think this should actually be a resource loader - it makes sense to build the internals on top of existing facilities, that is, the HighLevelGpuProgramManager which manages the shader resources for us.
Doing it that way, caching generated shaders is also no different from caching hand-coded ones - the generated shader would be created as a regular shader with a name based on a unique hash. The shader generator itself doesn't do any caching. The HighLevelGpuProgramManager already has caching built into it.
You may be interested in this document
Wrong link? It's asking me to sign in.
How are unique shader instances determined?
Quoting myself:
A permutation basically comes down to the combination of values from any macros that are retrieved.
If you meant implementation wise, this is how I'm currently doing it:
On startup, for each shader file, build a list of properties affecting this shader file.
When attempting to create a shader instance, a hash is built based on the specific property values (as defined by the material and current settings). I use boost::hash_combine for this.
The generate method should support asynchronous completion. Otherwise, you're going to have some kind of interesting hell when the generator depends upon resources which haven't yet loaded.
Hm, that is an interesting point. Off the top of my head, though, I can't think of any scenario where it'd depend on loading a resource. Do you have an example in mind?
How are properties defined?
Each material generator defines its own set of properties.
Owen
Google Summer of Code Student
Google Summer of Code Student
Posts: 91
Joined: Mon May 01, 2006 11:36 am
x 21

Re: [GSOC 2014] Material System Improvements

Post by Owen »

scrawl wrote:
The shader generator will probably want to overload the ResourceLoader::openCache method so it can manage the backend of such caching (It'll probably want to stash the cache data into some directory based upon a hash of the shader permutation or whatever)
I don't think this should actually be a resource loader - it makes sense to build the internals on top of existing facilities, that is, the HighLevelGpuProgramManager which manages the shader resources for us.
Doing it that way, caching generated shaders is also no different from caching hand-coded ones - the generated shader would be created as a regular shader with a name based on a unique hash. The shader generator itself doesn't do any caching. The HighLevelGpuProgramManager already has caching built into it.
Check my GSOC branch & thread. HLGPM doesn't exist any more. Caching is handled directly by the ResourceLoader. Every resource must have an associated resource loader, and resources no longer have names and there is no general way to look up a resource by its name (That depends upon what resource system you are using)
You may be interested in this document
Wrong link? It's asking me to sign in.
Boo! Of course the share link is different... here
How are unique shader instances determined?
Quoting myself:
A permutation basically comes down to the combination of values from any macros that are retrieved.
If you meant implementation wise, this is how I'm currently doing it:
On startup, for each shader file, build a list of properties affecting this shader file.
When attempting to create a shader instance, a hash is built based on the specific property values (as defined by the material and current settings). I use boost::hash_combine for this.
The generate method should support asynchronous completion. Otherwise, you're going to have some kind of interesting hell when the generator depends upon resources which haven't yet loaded.
Hm, that is an interesting point. Off the top of my head, though, I can't think of any scenario where it'd depend on loading a resource. Do you have an example in mind?
Can't pick the right (whatever)-mapping shader until you know whether you have a 2D texture, a 3D texture, a TextureArray...
How are properties defined?
Each material generator defines its own set of properties.
I understood that much. How does that correlate with the shader file when using the default generator?
scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 216

Re: [GSoC 2014] Material System Improvements

Post by scrawl »

I think it would be a good idea to keep the proposal free of any resource system details. The reason is that I am not sure yet what the target branch is. The new resource system will go into 2.0. As for this project, it involves no interface breaks of any kind, so if possible, it should be merged to 1.10 or 1.11 (assuming 1.11 will exist).
Anyway, I'm very excited to try out your work when it gets merged! Custom resource loaders are a feature I've been dying for.
and resources no longer have names
Hell yeah :D
I understood that much. How does that correlate with the shader file when using the default generator?
It's still just a string map.
Can't pick the right (whatever)-mapping shader until you know whether you have a 2D texture, a 3D texture, a TextureArray...
Most texture uses would be predefined. If a texture layer accepts multiple types (e.g. either a cubemap, or a sphere map for environment map purposes) then these can be bound to separate properties, respectively.
al2950
OGRE Expert User
OGRE Expert User
Posts: 1227
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 157

Re: [GSoC 2014] Material System Improvements

Post by al2950 »

First off that is a great proposal, and if you do get picked your hardest task is going to be dealing with everyones, often conflicting, opinions on what this material system should do :). Having said that here are a few of mine after a quick read!
  • I am not to sure how the shader generator is intended to work with different material types/properties. From your proposal it looks like each generator will be a single uber shader, albeit better organised with the help of the macros. It will not plug different sharers together (eg PPL + PSSM), like RTSS does? Not saying this is a bad idea just want to know how it works. Although I like the concept of this in RTSS, I think it is trying to be too flexible and is causing some complicated problems! :(
  • I would like to see the ability of applying a material generator or material property to all materials without having to change/edit every material file. A good example would be adding PPL with PSSM to all objects/materials (This was called a material template in RTSS). However you might want to have something in a material which prevents this global generator/property being applied to it. Perhaps have a concept of material groups, to which you can apply settings to.
  • I REALLY like the sound of the "Faster shader editing" I would therefore like it to be move into the main proposal :D
  • Maybe out of the scope of this proposal, but it would be good to have an editor which supported the shader macros which allowed you to hide code via enabling/disabling properties.
Thats it for now! I am very excited in seeing this proposal evolve :D
scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 216

Re: [GSoC 2014] Material System Improvements

Post by scrawl »

I would like to see the ability of applying a material generator or material property to all materials without having to change/edit every material file.
That is actually really simple: You define a new property with a default value of "true". All materials that do not explicitely specify that property will then have that feature enabled. Materials that set the property to "false" will have the feature disabled.
Perhaps have a concept of material groups, to which you can apply settings to.
Actually you can do this with inheritance. You can also have any number of inheritance levels. I'll edit the example to clarify that. The way you would define your "groups" is to have several base materials and then derive each specific material from one of the base materials.
It will not plug different sharers together (eg PPL + PSSM),
No. But in your example, PPL and PSSM would be features of the uber shader that can be toggled individually.
Although I like the concept of this in RTSS, I think it is trying to be too flexible and is causing some complicated problems! :(
Fully agreed.
al2950
OGRE Expert User
OGRE Expert User
Posts: 1227
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 157

Re: [GSoC 2014] Material System Improvements

Post by al2950 »

scrawl wrote:
I would like to see the ability of applying a material generator or material property to all materials without having to change/edit every material file.
That is actually really simple: You define a new property with a default value of "true". All materials that do not explicitely specify that property will then have that feature enabled. Materials that set the property to "false" will have the feature disabled.
All sounds very promising. :D Just to clarify the point above, I have a lot of materials generated by an exporter from 3D max by artists, using the above method I could apply a material generator to all of them using a couple of lines of code, and not touch the materials?
User avatar
Mako_energy
Greenskin
Posts: 125
Joined: Mon Feb 22, 2010 7:48 pm
x 9

Re: [GSOC 2014] Material System Improvements

Post by Mako_energy »

This looks like a really great proposal I'd like to see accepted. I only have one small question about a clarification you made.
scrawl wrote:When attempting to create a shader instance, a hash is built based on the specific property values (as defined by the material and current settings). I use boost::hash_combine for this.
In the actual Ogre implementation you'd be working on in this GSoC, would you be using Boost? Is making Boost mandatory for using this component a part of your proposal, or would you be able to dodge that issue?
scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 216

Re: [GSoC 2014] Material System Improvements

Post by scrawl »

No I won't be using boost. It would be silly to introduce a dependency for such a small function.
All sounds very promising. :D Just to clarify the point above, I have a lot of materials generated by an exporter from 3D max by artists, using the above method I could apply a material generator to all of them using a couple of lines of code, and not touch the materials?
Well sure, you could loop over all your materials in code and apply the generator to them.
scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 216

Re: [GSoC 2014] Material System Improvements

Post by scrawl »

OGRE was sadly not accepted for GSOC this year :( (for the first time since, when?)

However, I am really dedicated to this project and I will work on it anyway.

I can start coding in a few weeks. All work will be based on the default branch (at least for now). If there are any remaining issues with the proposal, please speak up now.
User avatar
spacegaier
OGRE Team Member
OGRE Team Member
Posts: 4304
Joined: Mon Feb 04, 2008 2:02 pm
Location: Germany
x 135
Contact:

Re: [GSoC 2014] Material System Improvements

Post by spacegaier »

scrawl wrote:OGRE was sadly not accepted for GSOC this year :( (for the first time since, when?)
From 2006 on, we only did not make it in 2010. Other than that one year, we always managed to get at least one slot. But now back to topic...
Ogre Admin [Admin, Dev, PR, Finance, Wiki, etc.] | BasicOgreFramework | AdvancedOgreFramework
Don't know what to do in your spare time? Help the Ogre wiki grow! Or squash a bug...
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5292
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [SoC 2014] Material System Improvements

Post by dark_sylinc »

As you know, we have not been accepted in GSoC 2014. Nontheless:

I just had the time to review your proposal. It's much better than I thought and I like it.
There are a few things I would like to discuss with you at some point:
  • Ability of the shader include "pieces" (snippets of code that generator will copy paste) so that users can customize shader code supplied by Ogre without having to modify the whole code (specially when they only want to make minor changes). We can also take advantage of this system for snippets that can be "modularized". Basically, it works like defining a multi-line macro without the ugly '\' escape on every new line.
i.e. for example, user's shader code:

Code: Select all

#piece CustomLighting @parameters
    texture2D normalMapTex : register(s@counter);
#endpiece

#piece CustomLighting @code
    float3 normalTangentSpace = tex2D( normalMapTex, uv0 );
    normal = tangentToWorldSpace( normalTangentSpace, normal, tangent, binormal );
#endpiece
Our pixel shader code will look like this:

Code: Select all

    /* ... */
    in float3 normal;
    in float3 tangent;
    in float3 binormal;
    uniform float4 paramXX;
    uniform float4 paramYY;
    #insertpiece CustomLighting@parameters
    /* ... */
 )
{
   float4 diffuseCol = tex2D( diffuseMap, uv0 );
    #insertpiece CustomLighting@code
    diffuseCol *= dot( normal, viewDir );
    /* ... */
}
The one writting the custom piece would have to know that variables like normal binormal & tangent, uv0 & normalMapTex are already declared. We could abide to some conventions, or allow some enforcing so the user has something guaranteed. I haven't thought that far. I want to keep that simple though.
Note that a generator could take advantage of pieces so that it generates the shader by gluging pieces together, or by choosing a set of presets known to compile.

The 2nd thing I want to discuss, actually much more important:
  • Shader generator living inside the Renderable.
Here's what's planned for 2.x: The user will take a material, let's call it "LeatherSofa", and will assigned to multiple different Sofa meshes.
Some of these Sofa will be auto instanced, other Sofas have skeletal animations, AND/OR other Sofas cast shadows.
So you can have up to 8 different LeatherSofa materials (I talk about that in this other post), but the user just cares about the one and only "LeatherSofa".
The RenderQueue will group all entities using the same settings of LeatherSofa (i.e. all that use skeletal anim, auto instancing, and cast shadows together; then all that use auto instancing and cast shadows; then all that use skel. anim and cast shadows, etc) request the Shader Generator to generate on the fly (obviously cached) the shader for each group of renderables.
Take in mind that if for example, auto instancing & skeletal animations are used together, then an extra texture unit is needed to pass the matrices (and if shadow casting is on, the shadow caster material also needs this texture unit).

If the ShaderGenerator doesn't support a particular feature (i.e. no auto instancing) then assigning a LeaderSofa to an auto instanced renderable will throw an error (or turning auto instancing on for that renderable after being assigned LeaderSofa).
When no ShaderGenerators are used, turning on auto instancing will throw an error. If the renderable is skeletally animated but the material doesn't support it, again it will throw.

This way the burden of knowledge to write shaders is taken completely off from inexperienced users (or experienced users that don't want to bother about this). And experienced users can write/modify their own shader generator (or the shader files that the generator uses) to fit their needs without much hassle.
Right now each feature requires you to write one material for each; and ensure you've paired the right material with the right entity (and veryfying your shadow casters have an additional texture unit if that's needed for instancing); not to mention instancing today is explicit rather than automatic. To that, add all the disadvantage you've already mentioned in your first post.
All of this is Very error prone and user hostile.

Edit: And auto instancing will improve performance to levels that will put us in league with AAA engines.
scrawl
OGRE Expert User
OGRE Expert User
Posts: 1119
Joined: Sat Jan 01, 2011 7:57 pm
x 216

Re: [SoC 2014] Material System Improvements

Post by scrawl »

Both good points. We can have a look at that when the groundwork is done.

As for generating from the RenderQueue, the concern I have is does that actually take into account the generation of materials that are not visible in the frustum (yet)? Otherwise you do get lag when turning the camera after a scene has been loaded. I notice this when generating materials from the handleSchemeNotFound callback. It is not called for culled renderables. You do not get this problem when generating from Material::load() (which is done as soon as you create the entity)
other Sofas have skeletal animations
Beware of the walking Sofas :lol:
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5292
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [SoC 2014] Material System Improvements

Post by dark_sylinc »

scrawl wrote:As for generating from the RenderQueue, the concern I have is does that actually take into account the generation of materials that are not visible in the frustum (yet)? Otherwise you do get lag when turning the camera after a scene has been loaded.
I was thinking that materials get generated as they're needed, and then cached; this means that:
  • Culled entities won't generate a material until they enter the camera for the first time
  • Hickups/lag/spikes when materials that were never used are requested (I've heard this is clearly happening in Battlefield 4 unless using Mantle)
  • A simple interface needs to be made to pregenerate (in foreground or background) material combinations that you know will be used in the near future (i.e. foreground: loading screens, background: corridor to a different area that hides the loading screen)
Take note that what causes hickups is shader generation, not material creation (due to compilation and API hidden stalls/syncs). Although "LeatherSofa" and "FabricShirt" use different textures and parameters, their vertex & fragment shaders will probably be the same and be already cached (as long as they use the same settings: auto instancing, skeletal animation, shadow casting); thus the stall shouldn't happen twice, but rather only once.
scrawl wrote:I notice this when generating materials from the handleSchemeNotFound callback. It is not called for culled renderables. You do not get this problem when generating from Material::load() (which is done as soon as you create the entity)
I'm not experienced with this callback so I do not know why you would want/need this or why this is a problem or not.
scrawl wrote:I
other Sofas have skeletal animations
Beware of the walking Sofas :lol:
:lol: I thought "ok... weird choice of example"
User avatar
Zonder
Ogre Magi
Posts: 1168
Joined: Mon Aug 04, 2008 7:51 pm
Location: Manchester - England
x 73

Re: [SoC 2014] Material System Improvements

Post by Zonder »

I do like the idea of been able to define fragments of shader to be included. I do think some strict naming should be used if possible or you have macros for the names of parts you need so the shader generator can insert the correct one.
There are 10 types of people in the world: Those who understand binary, and those who don't...
al2950
OGRE Expert User
OGRE Expert User
Posts: 1227
Joined: Thu Dec 11, 2008 7:56 pm
Location: Bristol, UK
x 157

Re: [SoC 2014] Material System Improvements

Post by al2950 »

Dark_sylinc wrote: Ability of the shader include "pieces" (snippets of code that generator will copy paste) so that users can customize shader code supplied by Ogre without having to modify the whole code (specially when they only want to make minor changes). We can also take advantage of this system for snippets that can be "modularized". Basically, it works like defining a multi-line macro without the ugly '\' escape on every new line.
Zonder wrote:I do like the idea of been able to define fragments of shader to be included. I do think some strict naming should be used if possible or you have macros for the names of parts you need so the shader generator can insert the correct one.
Agreed, I think this would be very good. It just so happens, because they got awarded GSOC slots, i was looking at the JMonkey engine and their material system. In terms of these shader "pieces" I see them being simliar to JMonkey's ShdaerNodes, which define inputs, outputs and some code.
http://hub.jmonkeyengine.org/wiki/doku. ... hadernodes
I do quite like the idea (I am not suggesting creating a node based shader system, but using it as a way to group shader techniques so they can be easily re-used in a different shader generator). NB RTSS was trying to do this, although it was trying to resolve the inputs & outputs automatically which actually ends up being more confusing than helpful!

@dark_sylinc
One point that slightly confused me was your second point about the render queue driving what the material system does, or at least giving it helpful hints. I think it sounds a great idea, but not sure how it would work, as I see this to be an extension to the current system and not replacing it. IE There are a number of people who still want to create shaders the 'old way'. If the render queue is heavily tide in to the material system, how to people use the 'old way'?
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5292
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: [SoC 2014] Material System Improvements

Post by dark_sylinc »

al2950 wrote:I think it sounds a great idea, but not sure how it would work, as I see this to be an extension to the current system and not replacing it.
You're correct in that it isn't just an extension, but rather a Core compoenent.
However since there could be multiple shader generators (i.e. one for photorealistic rendering, another for non-photorealistic-rendering, a custom one, etc) the generic class would live in Core while specializations added as components (like the current RenderSystem vs D3D 9RS & GL RS).
I'm trying to stay away from adding more virtuals, however one has to acknowledge that there would be multiple shader generators to fit user's dynamic needs. But we just have to put virtuals only when necessary (i.e. retrieving cached shaders doesn't have to be virtual, only the generator itself) and try to avoid resolving the virtual address once per renderable (Data Oriented Design principles). It's not easy though.
But we'll worry about that kind of optimization a bit later; as that is implementation specific rather than a problem of the algorithm itself.
al2950 wrote: IE There are a number of people who still want to create shaders the 'old way'. If the render queue is heavily tide in to the material system, how to people use the 'old way'?
When regular material systems are being used, the RenderQueue will use the provided material instead of requesting a new/cached one to the shader gen. Some integrated features (i.e. now I can just think of auto instancing) cannot be turned on for those renderables.
The RenderQueue will ask, and the ShaderGenerator shall spit out a Material (i.e. Gen_LeatherSofa_Instanced_Skeleton_Shadows) that fits the needs for that renderable.
When using the 'old way', the RenderQueue won't ask the ShaderGenerator and the already set material will be used.
The 'old way' will still be very useful for a few tasks, i.e. postprocessing effects.
Post Reply