ETM Normal Mapping - Code Review

SongOfTheWeave

08-04-2008 11:32:52

Okay, I've gotten some decent looking normal mapping working on the terrain.

While it works, some of it is stupid. However, I'm not entirely sure how to go about improving it. I would very much appreciate a code review on these shaders.

I've diverged significantly from the Dot3Bump example on the wiki because... well, it seemed ridiculous. It had some cubic texture that was strange gradients and I didn't see what that had to do with me, so I dropped it entirely and did it this way. And, as I mentioned, it works, but I am highly suspicious that I'm doing (a lot?) more work than is neccessary in certain spots.

I also suspect that a good number of the calculations could be moved into the vertex shader with negligible impact on the perceived effect, however, it seems that whenever I move one thing, I end up having to move everything else.

I'm looking for input and critique here.
Oh! And feel free to use my shaders (though you might want to wait until we get some good input to improve them a touch first hehe.)

Thanks guys!

[edit] P.S. I've added code to ETM to generate tangents and binormals in case you're wondering where those came from. In interests of full disclosure, I will post my additions to ETM at a later date, and in another thread. [/edit]



Vertex Shader:
void ETDynLighting_vp
(
float4 position : POSITION,
float3 normal : NORMAL,
float3 tangent : TANGENT,
float2 uv : TEXCOORD0,
float delta : BLENDWEIGHT,

uniform float4 lightPosition,
uniform float3 eyePosition,
uniform float4x4 worldviewproj,
uniform float morphFactor,

out float4 oWorldPos : POSITION,

out float2 oUV : TEXCOORD0,
out float4 oPos : TEXCOORD1,
out float3 oNorm : TEXCOORD2,
out float4 oLightPos : TEXCOORD3,
out float3 oEyePos : TEXCOORD4,
out float3 oTangent : TEXCOORD5
)
{
position.y = position.y + (delta.x * morphFactor);
oPos = position;
oWorldPos = mul(worldviewproj, position);
oUV = uv;
oNorm = normal;
oLightPos = lightPosition;
oEyePos = eyePosition;
oTangent = tangent;
}


Pixel Shader:
void ETDynLighting_fp
(
float2 uv : TEXCOORD0,
float4 position : TEXCOORD1,
float3 normal : TEXCOORD2,
float4 lightPos : TEXCOORD3,
float3 eyePos : TEXCOORD4,
float3 tangent : TEXCOORD5,

uniform sampler2D covMap1,
uniform sampler2D covMap2,
uniform sampler2D splat1,
uniform sampler2D splat2,
uniform sampler2D splat3,
uniform sampler2D splat4,
uniform sampler2D splat5,
uniform sampler2D splat6,

uniform float splatScaleX,
uniform float splatScaleZ,

uniform float4 lightDiffuse,
uniform float4 lightSpecular,
uniform float exponent,
uniform float4 ambient,

out float4 oColor : COLOR
)
{
float3 cov1 = tex2D(covMap1, uv).rgb;
float3 cov2 = tex2D(covMap2, uv).rgb;
uv.x *= splatScaleX;
uv.y *= splatScaleZ;
float3 normModifier = tex2D(splat1, uv) * cov1.x
+ tex2D(splat2, uv) * cov1.y
+ tex2D(splat3, uv) * cov1.z
+ tex2D(splat4, uv) * cov2.x
+ tex2D(splat5, uv) * cov2.y
+ tex2D(splat6, uv) * cov2.z;

normModifier = normModifier.xzy;

float3 binormal = cross(tangent, normal);
// Form a rotation matrix out of the vectors
float3x3 rotation = float3x3(tangent, binormal, normal);
normModifier = mul(rotation, normModifier);

normModifier = normalize(normModifier);

float3 EyeDir = normalize(eyePos - position.xyz);
float3 LightDir = lightPos.xyz - (position * lightPos.w);
float dist = length(LightDir);
// Normalize this way since we already found the magnitude
LightDir = LightDir / dist;
float3 HalfAngle = normalize(LightDir + EyeDir);

normal = normalize(normal);
normal = normal + normModifier;
normal = normalize(normal);
float NdotL = max(dot(LightDir, normal), 0.0);
float NdotH = max(dot(HalfAngle, normal), 0.0);
//float NdotL = max(dot(LightDir, normModifier), 0.0);
//float NdotH = max(dot(HalfAngle, normModifier), 0.0);
float4 Lit = lit(NdotL, NdotH, exponent);

oColor = lightDiffuse * Lit.y + lightSpecular * Lit.z + ambient;
// Taking the ambient component out here and doing an ambient pass first,
// then doing additive lighting passes looks like shit for some reason.
// Look into this later... this method only looks good with only 1 light.
//oColor = lightDiffuse * Lit.y + lightSpecular * Lit.z;



// Attenuation stuff. Probably put this in the vp if I ever go back to it
//float fLuminence = 1 / (atten.y + atten.z * dist + atten.w * dist * dist);
//oColor = ambient + (fLuminence * (lightDiffuse * Lit.y + lightSpecular * Lit.z));
//oColor = ambient + (fLuminence * (lightDiffuse * NdotL + lightSpecular * NdotH));

}


Declarations:
vertex_program ET/Programs/VSDynLighting cg
{
source ETDynLighting.cg
entry_point ETDynLighting_vp
profiles vs_1_1 arbvp1

default_params
{
param_named_auto lightPosition light_position_object_space 0
param_named_auto eyePosition camera_position_object_space
param_named_auto worldviewproj worldviewproj_matrix
param_named_auto morphFactor custom 77
}
}

fragment_program ET/Programs/PSDynLighting cg
{
source ETDynLighting.cg
entry_point ETDynLighting_fp
profiles ps_1_1 arbfp1

default_params
{
param_named_auto lightDiffuse light_diffuse_colour 0
param_named_auto lightSpecular light_specular_colour 0
param_named exponent float 33
param_named_auto ambient ambient_light_colour 0
//param_named_auto atten light_attenuation
}
}


Material (omitting fallback technique as this post is TMI already):
material ETTerrainMaterial
{
technique
{
// primary splatting technique, requires PS 2.0
pass Splatting
{
lighting off

vertex_program_ref ET/Programs/VSLodMorph2
{
}

fragment_program_ref ET/Programs/PSSplat2
{
param_named splatScaleX float 20
param_named splatScaleZ float 20
}
}

pass Lighting
{
scene_blend modulate
iteration once_per_light

vertex_program_ref ET/Programs/VSDynLighting
{
}

fragment_program_ref ET/Programs/PSDynLighting
{
param_named splatScaleX float 20
param_named splatScaleZ float 20
}
}

//Decal pass
pass Decal
{
lighting off
depth_bias 0.6 0.3
scene_blend alpha_blend

texture_unit
{
texture decalBase.png
}
}
}
}

NoodlesOnMyBack

08-04-2008 15:16:20

Decent? Holy crap that looks awesome! Can you post more screenshots please? Im very interested on this.
First questions:
How much FPS you got in release mode?
Size of the terrain?
How is the performance with alpha splatting?

SongOfTheWeave

09-04-2008 09:52:34

Here are some more screenshots:





While standing somewhere where I get a wide view as in the 2nd screenshot I'm getting between 40 and 50 FPS with a release build. Which is... bad. But I just built the little FPS display widget to check this today, so I'm not sure how many FPS I was getting before. I have a suspicion that my PhysX implementation in my client is sucking a lot of time each frame.

I'll stick the FPS widget into my toolset in a bit and get a more "pure" measurement.

The areas in the screenshots are only 128x128 (129 verticies), but as I said, it's my suspicion that the FPS I'm getting are quite so bad because of other factors in addition to the normal mapping.

Oh, I was running the app at 1680x1050, OpenGL on this machine:

Windows XP
AMD 64x2 running at 2.35ghz
2gigs DDR2 memory
GeForce 7600 GS w/256 MB

with uh.. Visual Studio 2005 and Firefox open in the background <.<


-------

On another note, see the washed out lighting on the (currently unanimated) player model?

I recently changed my world scale by a factor of 0.01 (in other words, my world scale is now 1/100th the size it was before. A lot smaller.) After the changed, I set the scale (in ogre, as in, I applied a scale to the node the entity was attached to) down on some of my models that were made to fit the previous scale.

Why does applying a massive scale adjustment to a mesh fj0rk the fixed function lighting?

I realize the ideal response is to just scale down the meshes in a 3D authoring tool like blender or 3DSMax... but... I'm an idiot when it comes to that stuff and I can't get the damn thing to reflect my changes when I export it. Some global/local coordinate issue I'm sure, but it frusterates me.

Regardless, the lighting shouldn't be affected by mesh scale, right? In theory?

SongOfTheWeave

09-04-2008 11:19:30

Blarg, my framerates were quite so bad because the CEGUI static text widget is apparently horrendous for displaying multiline text. I drop 20-30 frames when I start displaying the FPS widget I made.

I'll remake my fps widget in a way that isn't terrible and get back with better numbers :P

[Edit]
Happens with all the CEGUI text widgets. I think the issue here is that it sums the pixel width of each glyph in the line when you call setText on a text widget, to determine whether it needs to wordwrap.

Since I'm changing the text every frame, I'm doing that calculation every frame... still, it's ridiculous, my frames dropped from 160 (application running, campaign loaded, no areas displaying) to 33 just from turning on the FPS widget (which updates it's contents every frame atm.)

I need a stupider text widget! hehe.
[/Edit]

nullsquared

09-04-2008 11:20:08

The normals are transformed by the same object transformation matrix as the positions - scale, too. So, here is some code:

// fixed function pipes
entity->setNormaliseNormals(true);
// shader pipes
normal = normalize(normal);

SongOfTheWeave

09-04-2008 11:36:31

The normals are transformed by the same object transformation matrix as the positions - scale, too. So, here is some code:

// fixed function pipes
entity->setNormaliseNormals(true);
// shader pipes
normal = normalize(normal);


Ah! Beauty.

entity->setNormaliseNormals(true);

Fixed the overlighting issue.

SongOfTheWeave

09-04-2008 12:01:41

[Edit]
Happens with all the CEGUI text widgets. I think the issue here is that it sums the pixel width of each glyph in the line when you call setText on a text widget, to determine whether it needs to wordwrap.

Since I'm changing the text every frame, I'm doing that calculation every frame... still, it's ridiculous, my frames dropped from 160 (application running, campaign loaded, no areas displaying) to 33 just from turning on the FPS widget (which updates it's contents every frame atm.)

I need a stupider text widget! hehe.
[/Edit]


It looks like the FPS killing abilities of CEGUI text widgets may be a debug mode only phenomenon.

I get around 50 FPS when looking out across the terrain in release mode, obviously, my performance increases a great deal when I look straight up or straight down, at the sky or the ground. This is at 1680x1050 fullscreen. It jumps up to around 80 if I run it at 1280x960 in windowed mode.

In conclusion, the lighting/normal mapping shaders could use some optimisation.

jjp

09-04-2008 15:51:36

The cubic texture in the example is there because with shader model 1 you don't have the function normalize() available in the pixel shader. So that cubic texture lookup is just a replacement for normalize().

Some parts of your pixel shader can be moved to the vertex shader with no impact on accuracy whatsoever :)

- Binormal can be calculated per vertex
- Light- and eye-direction as well
- Creating the rotation matrix and half angle vector per vertex can be done as well but will give more inaccurate results if camera or light are close to the surface
..

NoodlesOnMyBack

09-04-2008 15:59:37

Nice screenshots SongOfTheWeave! I think i'll wait until you optimize the shader programs before starting to play with this, but looks very promising, also the fps are not bad.

[OFF TOPIC]
Using a scaling factor for the new "world scale" as you are doing its not very performance friendly, and that scaling thingy you have to do it also to bones of the skeletons present on entities right? or did you find out some more efficient way of scaling everything up?
By the moment im using: meshmagick -verbose transform -scale=0.01/0.01/0.01 ninja.mesh
[/OFF TOPIC]

SongOfTheWeave

09-04-2008 22:39:42

Nice screenshots SongOfTheWeave! I think i'll wait until you optimize the shader programs before starting to play with this, but looks very promising, also the fps are not bad.

Thank you!

[OFF TOPIC]
Using a scaling factor for the new "world scale" as you are doing its not very performance friendly, and that scaling thingy you have to do it also to bones of the skeletons present on entities right? or did you find out some more efficient way of scaling everything up?
By the moment im using: meshmagick -verbose transform -scale=0.01/0.01/0.01 ninja.mesh
[/OFF TOPIC]


Yes, what I -should- do, is modify my meshes and re-export them so I don't have to do extra work to scale them. I just haven't done it.

As for how I'm doing the scaling, I just put a scale transform on the node the entities are attached to.

What is this meshmagick thing you referred to?

kungfoomasta

09-04-2008 23:13:52

You don't know meshmagick?! Its a cool tool (which I haven't gotten the opportunity to use yet) that can scale or reposition meshes and their skeletons. Back when I was exporting models from Maya to Ogre I had a lot of scaling problems also. I actually made a tool that did scaling and repositioning, but its not as mature as meshmagick.

http://www.ogre3d.org/phpBB2/viewtopic.php?t=32352&highlight=meshmagick

SongOfTheWeave

10-04-2008 01:12:44

You don't know meshmagick?! Its a cool tool (which I haven't gotten the opportunity to use yet) that can scale or reposition meshes and their skeletons. Back when I was exporting models from Maya to Ogre I had a lot of scaling problems also. I actually made a tool that did scaling and repositioning, but its not as mature as meshmagick.

http://www.ogre3d.org/phpBB2/viewtopic.php?t=32352&highlight=meshmagick


This makes my life so much less frusterating!

SongOfTheWeave

11-04-2008 09:53:44

I've optimised the shaders a touch, code to follow.

Here's a link to an article on my blog where I discuss dynamic skies and normal mapping. It has a bunch of screenshots too. http://www.cutthroatstudios.com/blog/2008/04/dynamicskies-normalmapping

void ETDynLighting_vp
(
float4 position : POSITION,
float3 normal : NORMAL,
float3 tangent : TANGENT,
//float3 binormal : BINORMAL,
float2 uv : TEXCOORD0,
float delta : BLENDWEIGHT,

uniform float4 lightPosition,
uniform float3 eyePosition,
uniform float4x4 worldviewproj,
uniform float morphFactor,

out float4 oWorldPos : POSITION,

out float2 oUV : TEXCOORD0,
out float3 oNorm : TEXCOORD1,
out float3 oTangent : TEXCOORD2,
out float3 oBinormal : TEXCOORD3,
out float3 oLightDir : TEXCOORD4,
out float3 oEyeDir : TEXCOORD5

)
{
position.y = position.y + (delta.x * morphFactor);
oWorldPos = mul(worldviewproj, position);
oUV = uv;
oNorm = normal;
oTangent = tangent;
//oBinormal = binormal;
oBinormal = cross(tangent, normal);

oEyeDir = eyePosition - position.xyz;
oLightDir = lightPosition.xyz - (position * lightPosition.w);
}


void ETDynLighting_fp
(
float2 uv : TEXCOORD0,
float3 normal : TEXCOORD1,
float3 tangent : TEXCOORD2,
float3 binormal : TEXCOORD3,
float3 lightDir : TEXCOORD4,
float3 eyeDir : TEXCOORD5,


uniform sampler2D covMap1,
uniform sampler2D covMap2,
uniform sampler2D splat1,
uniform sampler2D splat2,
uniform sampler2D splat3,
uniform sampler2D splat4,
uniform sampler2D splat5,
uniform sampler2D splat6,

uniform float splatScaleX,
uniform float splatScaleZ,

uniform float4 lightDiffuse,
uniform float4 lightSpecular,
uniform float exponent,
uniform float4 ambient,

out float4 oColor : COLOR
)
{
float3 cov1 = tex2D(covMap1, uv).rgb;
float3 cov2 = tex2D(covMap2, uv).rgb;
uv.x *= splatScaleX;
uv.y *= splatScaleZ;
float3 normModifier = tex2D(splat1, uv) * cov1.x
+ tex2D(splat2, uv) * cov1.y
+ tex2D(splat3, uv) * cov1.z
+ tex2D(splat4, uv) * cov2.x
+ tex2D(splat5, uv) * cov2.y
+ tex2D(splat6, uv) * cov2.z;

// The nvidia normal map generator uses z as the up direction
normModifier = normalize(expand(normModifier.xzy));

// Form a rotation matrix out of the vectors
tangent = normalize(tangent);
binormal = normalize(binormal);
normal = normalize(normal);
float3x3 rotation = float3x3(tangent, binormal, normal);

normModifier = mul(rotation, normModifier);
normModifier = normalize(normModifier);

lightDir = normalize(lightDir);
eyeDir = normalize(eyeDir);
float3 halfAngle = normalize(lightDir + eyeDir);

normal = normal + normModifier;
normal = normalize(normal);

float NdotL = max(dot(lightDir, normal), 0.0);
float NdotH = dot(halfAngle, normal);
float4 Lit = lit(NdotL, NdotH, exponent);

oColor = lightDiffuse * Lit.y + lightSpecular * Lit.z + ambient;
// Taking the ambient component out here and doing an ambient pass first,
// then doing additive lighting passes looks like shit for some reason.
// Look into this later... this method only looks good with only 1 light.
//oColor = lightDiffuse * Lit.y + lightSpecular * Lit.z;
}

Luth

11-04-2008 11:02:00

Good god. Those pictures look awesome. :shock:

If i ever manage to finish my terrain generator, i'll try to use that kind of mapping/shading too. Well ... at least if i'm able to understand how you did it when the time comes. Together with those day/night cycles and dynamic skies this looks really amazing. And may i ask how you did your colour/splatt mapping? Did you use a generic approach or did you create the coverage map / drawed the textures manually? Since those textures in the picture with that river seem to be placed mostly independent of the terrain height and slope.

SongOfTheWeave

11-04-2008 11:19:52

Good god. Those pictures look awesome. :shock:

If i ever manage to finish my terrain generator, i'll try to use that kind of mapping/shading too. Well ... at least if i'm able to understand how you did it when the time comes. Together with those day/night cycles and dynamic skies this looks really amazing. And may i ask how you did your colour/splatt mapping? Did you use a generic approach or did you create the coverage map / drawed the textures manually? Since those textures in the picture with that river seem to be placed mostly independent of the terrain height and slope.


Thank you very much. Everybody needs some encouragement every now and again!

To answer your question, you paint the textures on the terrain manually in the toolset (similar to the ETM demo app.)

This is an old shot so it doesn't have all the fancy pants stuff I've been doing lately.



Bigger: http://www.cutthroatstudios.com/images/toolsetpreview01.jpg

[edit] BTW, how do you put a link around an image? It doesn't seem to want to let me... but it works for those fileshare thumbnails that people often post.

Luth

11-04-2008 12:36:24

I think to have those kind of links, you need a separate thumbnail somewhere. The code you get from imageshack for posting into a forum looks something like:


[URL=http://blabla.com/my.php?image=terrain.jpg][img]http://blabla.com/terrainthumb.jpg[/img][/URL]


And learning that stuff in that little time is quite fast. Sometimes i wish i wouldn't have a job leaving such little time for coding after work. :mrgreen:

[Edit]
Hmmm .... now your Edit2 is gone. Ah ... whatever.
[/Edit]

SongOfTheWeave

11-04-2008 13:11:02

I think to have those kind of links, you need a separate thumbnail somewhere. The code you get from imageshack for posting into a forum looks something like:


[URL=http://blabla.com/my.php?image=terrain.jpg][img]http://blabla.com/terrainthumb.jpg[/img][/URL]


And learning that stuff in that little time is quite fast. Sometimes i wish i wouldn't have a job leaving such little time for coding after work. :mrgreen:

[Edit]
Hmmm .... now your Edit2 is gone. Ah ... whatever.
[/Edit]


I tried that and it didn't seem to work. Maybe I did something stupid.

And yah, I deleted my Edit2. Decided it sounded too much like bragging. :roll: