RT Shader System Component (RTSS)

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
Post Reply
Vectrex
Ogre Magi
Posts: 1266
Joined: Tue Aug 12, 2003 1:53 am
Location: Melbourne, Australia
x 1
Contact:

Re: RT Shader System Component

Post by Vectrex »

Ah ok, so if I generated the materials with code I could then generate the shaders automatically? I want to avoid having a million variations to maintain, so in code I'd simply generate the materials with a few if statements. Wouldn't it be cool to have an if statement in material files?? :twisted:
User avatar
Mattan Furst
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 260
Joined: Tue Jan 01, 2008 11:28 am
Location: Israel
x 32

Re: RT Shader System Component

Post by Mattan Furst »

Here's my two cents for the day.

One of the arguments I keep hearing against real time shader generation systems is that the code they will produce is very generic and therefore long and unoptimized. And you can always write better optimized shaders by hand.

One of the ways the RT Shader System can circumvent this problem is to allow the user to define which parameters are constant. For instance when defining a light source the attenuation parameters and light color usually do not change during runtime (at least not per specific light). You can then generate a shader with the constant parameters written directly into the shader. Furthermore calculations involving multiplication by one or zero and additions of zero values can be removed completely.

This way you can generate shaders that are so specific that only the most feverish of optimization buffs can compete with.
it's turtles all the way down
User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
x 2
Contact:

Re: RT Shader System Component

Post by Nir Hasson »

Ah ok, so if I generated the materials with code I could then generate the shaders automatically?
Sure you can, that was one of the main goals of this system and I use it in this scenario - I have bunch of materials that generated thru code and now I can generate shaders for them at runtime, nothing is precompiled and of course nothing to maintain... (Just the code of the system itself)
Wouldn't it be cool to have an if statement in material files??
Regarding the material files - I work now on extending the property set in order to make the RT Shader System deal with scripts as well beside the option to explicitly generate shaders using its API. So you could also use the current material script system and in a few lines of script you can cause to existing FFP material to be processed by the system and have shader based techniques..
Vectrex
Ogre Magi
Posts: 1266
Joined: Tue Aug 12, 2003 1:53 am
Location: Melbourne, Australia
x 1
Contact:

Re: RT Shader System Component

Post by Vectrex »

Maybe material scripts should support C# style attributes?
User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
x 2
Contact:

Re: RT Shader System Component

Post by Nir Hasson »

Maybe material scripts should support C# style attributes?
Don't know what this means - I didn't work on C# too much (thank god :D )
I thoght about something like this

Code: Select all

pass
{

	texture_unit
	{
		texture RustySteel.jpg
	}

	texture_unit
	{
		texture flare.png
		colour_op_ex add src_texture src_current
		colour_op_multipass_fallback one one
		env_map spherical
	}
			
	texture_unit
	{
		texture BeachStones.jpg
	}
			
	rtshadersystem DefaultShaderGenerator
	{				
		light_model		per_pixel
	}
}
This will generate a new tehcnique with the scheme name DefaultShaderGenerator that will do per pixel lighting..
If you have other suggestion maybe put it in this thread http://www.ogre3d.org/forums/viewtopic.php?f=4&t=52417 - cause we are getting out of this thread scope...
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: RT Shader System Component

Post by masterfalcon »

I'm seeing a bunch of errors similar to this on OS X:

Code: Select all

/Users/davidrogers/Documents/Development/Framework-Sources/ogre/Components/RTShaderSystem/src/OgreShaderFFPFog.cpp:77:0 /Users/davidrogers/Documents/Development/Framework-Sources/ogre/Components/RTShaderSystem/src/OgreShaderFFPFog.cpp:77: error: no matching function for call to 'sh_hash_combine(Ogre::uint32&, Ogre::uint32)'
Is this just me?
User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
x 2
Contact:

Re: RT Shader System Component

Post by Nir Hasson »

I'm seeing a bunch of errors similar to this on OS X:
I don't have the OS X env here - so I can not check it.
However I guess it is related to some compiler casts that are done implicitly on Microsoft and Linux env.
I think the compiler dont perform implict cast of the enum there to int.
Try replacing

Code: Select all

sh_hash_combine(hashCode, mFogMode);
sh_hash_combine(hashCode, mCalcMode);
With

Code: Select all

sh_hash_combine(hashCode, (int)mFogMode);
sh_hash_combine(hashCode, (int)mCalcMode);
Keep me update in order to add it to next patch
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: RT Shader System Component

Post by masterfalcon »

Nir Hasson wrote:
I'm seeing a bunch of errors similar to this on OS X:
I don't have the OS X env here - so I can not check it.
However I guess it is related to some compiler casts that are done implicitly on Microsoft and Linux env.
I think the compiler dont perform implict cast of the enum there to int.
Try replacing

Code: Select all

sh_hash_combine(hashCode, mFogMode);
sh_hash_combine(hashCode, mCalcMode);
With

Code: Select all

sh_hash_combine(hashCode, (int)mFogMode);
sh_hash_combine(hashCode, (int)mCalcMode);
Keep me update in order to add it to next patch
casting hashCode to (size_t &) solved the errors for me. I haven't tried running yet as I'm at work right now.
User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
x 2
Contact:

Re: RT Shader System Component

Post by Nir Hasson »

O.K - that is strange since all other implementations in other source defines hashCode as uint32 and nothing went wrong there...
Anyways I'll just modify the combine function protoype from

Code: Select all

inline void sh_hash_combine(std::size_t& seed, T const& v)
To this code

Code: Select all

inline void sh_hash_combine(uint32& seed, T const& v)
This is internal function used by the RT Shader System and it is declared in OgreShaderPrerequisites.h.
Can you verify that this solution works for you ? I prefer it over solving it just in the FFPFog class...
User avatar
masterfalcon
OGRE Team Member
OGRE Team Member
Posts: 4270
Joined: Sun Feb 25, 2007 4:56 am
Location: Bloomington, MN
x 126
Contact:

Re: RT Shader System Component

Post by masterfalcon »

Nir Hasson wrote:O.K - that is strange since all other implementations in other source defines hashCode as uint32 and nothing went wrong there...
Anyways I'll just modify the combine function protoype from

Code: Select all

inline void sh_hash_combine(std::size_t& seed, T const& v)
To this code

Code: Select all

inline void sh_hash_combine(uint32& seed, T const& v)
This is internal function used by the RT Shader System and it is declared in OgreShaderPrerequisites.h.
Can you verify that this solution works for you ? I prefer it over solving it just in the FFPFog class...
Yeah, that appears to work great!
User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
x 2
Contact:

Re: RT Shader System Component

Post by Nir Hasson »

Yeah, that appears to work great!
Thanks man :D WIll be in next patch...
User avatar
boyamer
Orc
Posts: 459
Joined: Sat Jan 24, 2009 11:16 am
Location: Italy
x 6

Re: RT Shader System Component

Post by boyamer »

Is possible to add an example on how to set up Ogre with the RT Shader system?

Thanks
User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
x 2
Contact:

Re: RT Shader System Component

Post by Nir Hasson »

Is possible to add an example on how to set up Ogre with the RT Shader system?
Sorry for my late response - had a long weekend.
The trunk code already has the RT Shader System embedded example in all samples code that based on the ExampleApplication frame work.
Make sure you build the RT Shader System component – using the OGRE_BUILD_COMPONENT_RTSHADERSYSTEM variable in the CMake script.
Then take a look at the ExampleFrameListener.h and take a look at the code sections between the USE_RTSHADER_SYSTEM defines.
Basically pressing F3 will switch the main viewport scheme to the default scheme of the ShaderGenerator and therefore all materials will run using shader generated techniques.
Pressing F2 will switch back to default material scheme.

Hope it helps you…
User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
x 2
Contact:

Re: RT Shader System Component

Post by Nir Hasson »

Hi –
Just finished some work on the normal map side.
Here is the original material script –

Code: Select all

material Examples/DarkMaterial
{
technique
{
   pass
   {
     ambient 0.1 0.1 0.1
     texture_unit
     {
        texture BeachStones.jpg
     }						
   }
}
}
And this is the script that generates normal map shader

Code: Select all

material Examples/DarkMaterial
{
technique
{
   pass
   {
     ambient 0.1 0.1 0.1
     texture_unit
     {
        texture BeachStones.jpg
     }					
     rtshader_system
     {
        light_model sgx_normal_map BeachStones.dds
     }
  }
}
}
This is the result when rendering with the default material scheme -
Default material scheme
Default material scheme
SG_normal_map_off.jpg (82.39 KiB) Viewed 6383 times
This is the result when rendering with the shader generator material scheme -
BeachStonesNormalMap.zip
The normal map I used
(155.11 KiB) Downloaded 218 times
Attachments
Shader generator material scheme
Shader generator material scheme
SG_normal_map_on.jpg (131.07 KiB) Viewed 6383 times
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: RT Shader System Component

Post by Assaf Raman »

Watch out for my OGRE related tweets here.
User avatar
boyamer
Orc
Posts: 459
Joined: Sat Jan 24, 2009 11:16 am
Location: Italy
x 6

Re: RT Shader System Component

Post by boyamer »

Wow,looks very nice!
User avatar
xadhoom
Minaton
Posts: 973
Joined: Fri Dec 28, 2007 4:35 pm
Location: Germany
x 1

Re: RT Shader System Component

Post by xadhoom »

Nice advertisement ;-)

Is this functionality already in the trunk?

xad
Vectrex
Ogre Magi
Posts: 1266
Joined: Tue Aug 12, 2003 1:53 am
Location: Melbourne, Australia
x 1
Contact:

Re: RT Shader System Component

Post by Vectrex »

xadhoom wrote:Nice advertisement ;-)

Is this functionality already in the trunk?

xad
Yep, press F3 and F2 to swap between fixed function and shader in the ogre demos
User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
x 2
Contact:

Re: RT Shader System Component

Post by Nir Hasson »

Nice advertisement :D

Is this functionality already in the trunk?

xad
I have to finish the point light implementation, spot light and linear depth first - then I'll merge it in.
I guess around the end of this week - just building the tension till there :wink:
In the mean while the version in the trunk contains only the FFP emulation - so you won't find there any special effects :D
Nir
User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
x 2
Contact:

Re: RT Shader System Component

Post by Nir Hasson »

Hi –

I just commited the new version for the RT Shader System.

This version includes the following changes –
1. Built in per pixel and normal map custom sub render states implementation. Enables generating shaders that perform per pixel lighting calculations instead.
Current implementation supports all kind of light type but can be used only in a single pass since it always adds the ambient term. I'll might implement option to separate it in order to user in per light iteration passes.

2. Script support for the system. In each pass in the material script users can add now a section called rtshader_system and within it define custom properties.
Currently the only one property is built in supported:
light_model sgx_per_pixel // Per pixel lighting model.
Light_model sgx_normal_map normal_map_texture_name // Normal map lighting model.

3. Ability to implement custom sub render state outside the RT Shader System project scope. I implemented a custom PSSM receiver sub render state and used it in my own application. I did it also in the PlayPen frame work but didn’t include it in this patch.

4. Sample application changes -
F2 – set the main viewport material scheme to default material manager scheme.
F3 – set the main viewport material scheme to default shader generator scheme (This can be referred as turning the shader generator on).
F4 – Toggle the light model of the default scheme render state of the shader generator between per vertex to per pixel.

5. Demo_Dot3Bump – added another material that demonstrates using bump map lighting model of the RT Shader system. The material is added to the Examples-Advanced.material and called Examples/RTShaderSystem/BumpMapping.
In order to see it in action toggle to it pressing m and then press F3 to enable the shader generating.

6. Material changes – particle system material uses track vertex colour property on in order to give same results when rendering via shaders.

7. Core changes are only in the Pass::setUserAny method which I added previously. Now it takes a key parameter in order to allow associating multiple user data objects to a single pass.
User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 99
Contact:

Re: RT Shader System Component

Post by Wolfmanfx »

Hi
Are there any plans to document how the system (i mean in an abstract way) suppose to work? I know the source code is well documented and right now i am reading it but you know ;)
User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
x 2
Contact:

Re: RT Shader System Component

Post by Nir Hasson »

Hi -

I guess you got a point there. Documentation isn’t my favorite part - but I guess it is somewhat helpful :)
I’ll add it to my task list and submit it where it belongs. I’ll try to attach some high level design documents I made as well.
Where do you suggest me to place the documents? In the wiki or general system documents?.
Beside, I'm working on a dedicated sample as well as cross samples integration that will demonstrate its capabilities.
User avatar
Wolfmanfx
OGRE Team Member
OGRE Team Member
Posts: 1525
Joined: Fri Feb 03, 2006 10:37 pm
Location: Austria - Leoben
x 99
Contact:

Re: RT Shader System Component

Post by Wolfmanfx »

The place is not so importanted for me :)
Describe the SubRenderState because its a little bit mixed up for me in one place you use it to supply the system with basic functions like transform in another class you build up the whole NormalMap extension in it which make use of other subrenderstates.
User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
x 2
Contact:

Re: RT Shader System Component

Post by Nir Hasson »

Ok – I'll figure out where to put it…

In the meanwhile – just a short answer for you:
The SubRenderState represents an independent piece of rendering pipeline starting from the vertex shader until the pixel shader. It is done in this way in order to allow users to extend the system capabilities in a seamless way. The FFP emulation is built upon sub classing this interface so there you see the Transform part, Lighting per vertex etc.
In addition I added some common extension such as per pixel and normal mapping both again are sub classes of SubRenderState…
User avatar
Nir Hasson
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 363
Joined: Wed Nov 05, 2008 4:40 pm
Location: TLV - Israel
x 2
Contact:

Re: RT Shader System Component

Post by Nir Hasson »

Hi –

I just commit a new dedicated RT Shader System sample to main brunch.
It will demonstrate the main system capabilities – FFP Emulation, built in lighting models etc.
I also merged the system to the new samples frame work (I like it ) and now you can see exactly how many shaders have been generated during each sample runtime.
F2 – Toggles the system.
F3 - Toggles global lighting model (Vertex / Pixel).
F4 – Toggle the info panel.

I’d like to get feedback for Linux and Mac users since I can’t test it in these environments.

Art credits goes to Michael Grosberg.
Object space normal map generated using xNormal- check it out here…
http://www.xnormal.net/

Here is a screen shot from the new sample -
Shader System Sample
Shader System Sample
RTShaderSystemSample.jpg (110.33 KiB) Viewed 6064 times
Post Reply