Page 1 of 2

[SoC 2008] Ogre Material and Shader Library

Posted: Mon Mar 31, 2008 10:25 am
by Tenttu
Ogre has supported shader rendering for a quite while now. However the current collection of shaders in Ogre isn't at the same level as the rest of the engine. The shaders are hard to reuse and duplicate a lot of common functionality.

The main goals of this project is to solve those issues by refactoring, rewriting and adding new shader functionality to the Ogre3D.

Application

I'd be interested to hear what shaders you think should be implemented by default with Ogre. I did make a preliminary list, but imo the final list should be a result of the discussion here.

Posted: Mon Mar 31, 2008 10:39 am
by tuan kuranes
What would really make that shader library usable is that all those shader would be implemented template materials.

That would end in template Based Material+shader library for ogre using latest script capabilities, and refactor therefore all material from ogre samples with only template inherited materials.

I would even propose a 'material switch' sample, where user can switch between different material+shader library (normal+specular+n diffuse light, normal+specular+3light integrated, deferred shading, cell-shaded, etc.)

That would make library really reusable as well as demonstrating material script capabilities. (template, scheme, technique, dynamic scripting, advanced compiler)

Posted: Mon Mar 31, 2008 11:53 am
by Tenttu
Of course the goal would be to make the shaders as reusable as possible and currently thats easiest to achieve with the material "templates" (it used to be called material inheriting?). The new script compiler should also play an important role for implementing the material scripts in a best possible way.

The lighting should be abstracted behind a common interface within the shaders in a such way that different kind of methods for passing the light data would be possible. Such as per pass iteration, light information array or even reading light information from special textures (Light indexed rendering).

Posted: Mon Mar 31, 2008 12:45 pm
by tuan kuranes
the material "templates" (it used to be called material inheriting?).
Sorry, you're right, have to use exact terms : manual state that as Material copying and texture alias
If you intend to do that material work too, you should make it clearer in your application document and title. (material and shader library)

Posted: Mon Mar 31, 2008 12:51 pm
by Tenttu
Oh, ok I kind of thought it would be obvious that the material script work would be part of it. Thanks for the tip.

Posted: Mon Mar 31, 2008 1:54 pm
by syedhs
Just in case you didn't realize it, the dateline is today and I am afraid that if you wait Ogre community for the shader list, it is already too late. Maybe you can simply type in your personal list and have the application submitted, the details can be filled in later. :wink:

Posted: Mon Mar 31, 2008 2:19 pm
by Tenttu
The application was submitted couple days ago already, I didn't immediately create the topic. So don't worry :)

[edit]Some modifications to the .pdf application and to the topic title.[/edit]

Posted: Fri Apr 04, 2008 5:37 pm
by Praetor
I think this is actually an important project. It isn't in the "core" of Ogre, but it is part of how people view the engine. Being able to pick and go with Ogre having great shaders can really help Ogre in the long run. I've been toying with the new compilers for a while now (obviously....) writing reusable materials in various ways. The one thing that is not clear is a timetable. Writing a "library" means doing as many as possible throughout the summer. What kind of timeframe do you think each of these will take, and can we prioritize them so that we get the most useful done first?

Posted: Fri Apr 04, 2008 6:42 pm
by KungFooMasta
I also think this would be a very good project. Community Shaders for everybody to use and modify if they wanted. There needs to be some concrete goals, like X shaders created throughout the summer, and a schedule to outline it.

Posted: Fri Apr 04, 2008 7:27 pm
by FrameFever
wow this project is really really interesting for Ogre :!:

After using Ogre for one year, the biggest problem in my opinion is, there is no shader library. I hope this library will solve this problem.

how many lights will each shader support?
Do you have enough experience do write all this shaders? I hope you have.
good luck!

Posted: Fri Apr 04, 2008 11:00 pm
by Tenttu
I will create a more detailed schedule during the weekend so that it's ready before the Mondays submission deadline. Good to see that I am not the only one who thinks that this kind of library would be important for Ogre.
how many lights will each shader support?
The target should be to support all possible lighting conditions ogre can offer, either automatically or through user selection, combined with fallback system to reduce functionality if hardware can't support certain situation.
Do you have enough experience do write all this shaders? I hope you have.
I have used ogre for about four years and have written tons of shaders during that time. So that shouldn't be a problem.
I think this is actually an important project. It isn't in the "core" of Ogre, but it is part of how people view the engine.
Actually I disagree :) Imo the core of current generation graphics or atleast a part of it are the shaders.

Posted: Sat Apr 05, 2008 3:17 am
by Kencho
I think such library is important to have things working right out of the box. That's part of every engine marketing, but also consider mandatory to let the final user to decide whether to use the default shaders or their own. Hence why I'm supporting the idea of this not being "core". A bundled solution, yes, but not something the end user must use (or modify the engine shaders code itself, and thus being forced to publish their shaders' sources ;))

Right now the shaders and materials are examples, and (unless I'm wrong here), not LGPL, so you're free to use them or modify them. These materials/shaders library should be of the same nature, IMHO :)

Posted: Sat Apr 05, 2008 10:11 am
by tuan kuranes
Don't forget that the library should also be something of a 'material power showcase', as material scripts are very powerful, but rarely used at its best.

Ogre samples material would be better example than now (a they were written before all that power, before 'material copying', before 'texture alias', before cg becomes '2.0', before ...)

The core part of this gsoc is about making all ogre samples material uses the library, but that doesn't mean its mandatory to use it.

Posted: Sat Apr 05, 2008 11:10 am
by Kencho
Yes, of course ^_^ I was mostly stating that, while out-of-the-box high quality assets are mandatory for any engine, they aren't core functionality themselves :) Of course these materials should showcase the true Ogre power :)

Posted: Sat Apr 05, 2008 11:15 am
by Tenttu
By core I didn't mean mandatory, but (the most) important part of graphics rendering.

Ogre's goal is to offer solutions so that you don't have to write it on your own. Shouldn't this also be for the shaders? The collection of shaders and materials should be so good that there would be no need for anyone to write their own. Just like there is no need to anyone to use Graphics API's directly when you can use Ogre. Only thing you'd need to do is extend the current system.

Well that should be the eventual goal atleast, of course it's not that easy to achieve with current shader languages.

Posted: Sun Apr 06, 2008 12:30 pm
by Vectrex
Check out the Crysis shader solution. It's modular

Posted: Sun Apr 06, 2008 6:48 pm
by Tenttu
Yea, I've seen the system and actually talked to a guy who was part of designing it.

Posted: Sun Apr 06, 2008 7:33 pm
by spookyboo
This is a really great project and an excellent addition to Ogre. About not being part of the core, I think you have to look at Ogre as a layered library which consists of the real core and a moduled layer on top of it. The top level layer makes life a little easier. I think this is on the same level as the particleFX and terrain plugins.

Posted: Sun Apr 06, 2008 8:07 pm
by hellcatv
I'm personally developing something very similar and am happy to share my intermediate results. I've written a very extensive uebershader such that every option *I* am interested in is togglable.
The shader has been quite extensively tested and works with the power set of the following features:
offset mapping
normal mapping
integrated shadows (coded, not tested)
per-light OR many lights in one pass calculations
ambient occlusion maps
ambient diffuse cubemaps
environment maps (+shininess support)
2 separate detail maps with diff uv sets (and code to reproject first uv-set tangents into the new uv spaces using ddx and ddy functions)
skinning with N bones
hacky subsurface effects from Dawn demo (not yet complete)
GLSL, HLSL, Cg support
Win,Lin,Mac support (only ATI tested so far, but they're pickier)
ps_2_0 (1 light per pass) ps_2b (4 lights per pass), ps_3_0 (6 lights per pass) arbfp1 support

I'm just in the middle of commenting it now... you can download the uncommented version here (I will update it as I have time) ... I'm putting it in the public domain
http://graphics.stanford.edu/~danielrh/UeberShader.hlsl
(and a material that uses it http://graphics.stanford.edu/~danielrh/Man.material )
hopefully this would assist you in your quest to create the "perfect" material for everyone to use. I'm aiming to make a technique autogenerator that will spit out a useful subset of the above technique then users would just select from one of my prebaked combos of above events and choose
set_texture_alias or set_named_param_alias (not yet existent ) to setup the shaders...

Posted: Sun Apr 06, 2008 10:28 pm
by Tenttu
While using preprocessor definitions that extensively makes the system to reuse lots of it's code. It becomes very cumbersome to read, understand and extend. Imo "ubershaders" are a thing at the past and a more "proper" programming methods should be studied and used for shader development. This is for example what games like Crysis has done.

I'd go with a similar route I used for refactoring the Ogre's Offset Mapping shader a while ago.
http://ogre.cvs.sourceforge.net/ogre/og ... iew=markup
http://ogre.cvs.sourceforge.net/ogre/og ... iew=markup

In addition the .pdf now also contains somekind of timetable for the project.

Posted: Sun Apr 06, 2008 11:45 pm
by FrameFever
ambient occlusion is really interesting, when you have enough time, I would like to see this in Ogre :)

I have a question what direction will you go when you (re)write this shaders?Quality or Speed?

Posted: Mon Apr 07, 2008 1:09 am
by Tenttu
If there is enough interest, I'd love to experiment with Screen Space Ambient Occlusion at the end of the project.
I have a question what direction will you go when you (re)write this shaders?Quality or Speed?
With the basic shaders it's rather easy to achieve quality with speed. But the performance might become a problem with the more demanding shaders, so I'd say it's important to go for speed over quality, with certain limits.

Posted: Mon Apr 07, 2008 1:38 am
by Kencho
Well, if possible, I think the best solution would be to provide different schemes ("faster", "prettier") for the most demanding materials. The reason: That maybe the hardware is able to handle it, but they mean too much cost overall. The chance to choose is the way to go IMHO.

Posted: Tue Apr 08, 2008 12:42 pm
by tuan kuranes
Please update your PDF with resulting 'Materials' really included in planning, and how you'll implement materials (copy, scheme, etc), giving a use-case would help. (and make sure we know which version of the pdf, in order to know when to re-read applications...)

Posted: Tue Apr 22, 2008 10:49 am
by FrameFever
what is with this project?
Is it accepted?