Alpha mapping + normal mapping material script

Problems building or running the engine, queries about how to use features etc.
Post Reply
RSAgent007
Halfling
Posts: 40
Joined: Thu Apr 06, 2006 6:40 am

Alpha mapping + normal mapping material script

Post by RSAgent007 »

Hello everyone,

I have a .mesh that uses both an alpha map and a normal map. I've written material scripts that do them separately, but not together. What I have below does not work...

My alpha map is included in my diffuse texture as the alpha layer ("alphaDiffuse.tga"). My normal map is called "normal.tga".

Code: Select all

material ___Default
{

    // This is the preferred technique which uses both vertex and
    // fragment programs, supports coloured lights
    technique
    {
        // Base ambient pass
        pass
        {
            // base colours, not needed for rendering, but as information
            // to lighting pass categorisation routine
            ambient 1 1 1
            diffuse 0 0 0 
            specular 0 0 0 0 
            // Really basic vertex program
            // NB we don't use fixed function here because GL does not like
            // mixing fixed function and vertex programs, depth fighting can
            // be an issue
            vertex_program_ref Ogre/BasicVertexPrograms/AmbientOneTexture
            {
                param_named_auto worldViewProj worldviewproj_matrix
                param_named_auto ambient ambient_light_colour
            }
            
        }
        // Now do the lighting pass
        // NB we don't do decal texture here because this is repeated per light
        pass
        {
            // base colours, not needed for rendering, but as information
            // to lighting pass categorisation routine
            ambient 0 0 0 
            
            // do this for each light
            iteration once_per_light

        
            scene_blend add

            // Vertex program reference
            vertex_program_ref Examples/BumpMapVP
            {
                param_named_auto lightPosition light_position_object_space 0
                param_named_auto worldViewProj worldviewproj_matrix
            }

            // Fragment program
            fragment_program_ref Examples/BumpMapFP
            {
                param_named_auto lightDiffuse light_diffuse_colour 0 
            }

       
            // Base bump map
            texture_unit
            {
                texture normal.tga
                colour_op replace
            }
            // Normalisation cube map
            texture_unit
            {
                cubic_texture nm.png combinedUVW
                tex_coord_set 1
                tex_address_mode clamp
            }
        }
        
        // Decal pass
        pass
        {
            // base colours, not needed for rendering, but as information
            // to lighting pass categorisation routine
            lighting off
            // Really basic vertex program
            // NB we don't use fixed function here because GL does not like
            // mixing fixed function and vertex programs, depth fighting can
            // be an issue
            vertex_program_ref Ogre/BasicVertexPrograms/AmbientOneTexture
            {
                param_named_auto worldViewProj worldviewproj_matrix
                param_named ambient float4 1 1 1 1
            }
            
            // The alpha_rejection value is the alpha tolerance
            cull_hardware none
            scene_blend dest_colour zero
            shading phong
            alpha_rejection greater 200            
            
            texture_unit
            {
                texture alphaDiffuse.tga
                tex_coord_set 0
                colour_op modulate
                scale 1 1
                scroll 0 0
                rotate 0                
            }
            
        }
    }
}
If I change "scene_blend dest_colour zero" to "scene_blend alpha_blend", only the backside is alpha-blended (not sure why), but then now all my colors are washed out (presumbly).

What do I need to do to get my normal+alpha map working?

Thanks!!
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

It's because you're doing it in multiple passes, and the intermediate passes to combine lighting aren't playing nice with your final alpha compositing.

You're going to have to rewrite this material so that it does everything in one pass, so that the final compositing is based on alpha alone. Your middle pass has 'iteration once per light' which deals with any number of lights - you'll have to lose that flexibility and pick a maximum number of lights to do in the one pass, and make your shader iterate them instead.
RSAgent007
Halfling
Posts: 40
Joined: Thu Apr 06, 2006 6:40 am

Post by RSAgent007 »

I tried to change it so that it does everything in one pass, but now when I view it, the model has no color (there's a sense of depth however).

This is what I have:

Code: Select all

material ___Default
{		
	technique
	{

		pass
		{
			// base colours, not needed for rendering, but as information
			// to lighting pass categorisation routine
			ambient 0.58 0.58 0.58
			diffuse 0 0 0
			specular 0 0 0 0
			// Really basic vertex program
			// NB we don't use fixed function here because GL does not like
			// mixing fixed function and vertex programs, depth fighting can
			// be an issue
			vertex_program_ref Ogre/BasicVertexPrograms/AmbientOneTexture
			{
                		param_named_auto worldViewProj worldviewproj_matrix
                		param_named_auto ambient ambient_light_colour
            		} 		
		
			// base colours, not needed for rendering, but as information
			// to lighting pass categorisation routine
			scene_blend add

			ambient 0 0 0 
			// Vertex program reference
			vertex_program_ref Examples/BumpMapVP
			{
				param_named_auto lightPosition light_position_object_space 0
				param_named_auto worldViewProj worldviewproj_matrix
			}

			// Fragment program
			fragment_program_ref Examples/BumpMapFP
			{
				param_named_auto lightDiffuse light_diffuse_colour 0 
			}
			
			// Base bump map
			texture_unit
			{
				texture Sphere01NormalsMap.tga
				colour_op replace 
			}
			// Normalisation cube map
			texture_unit
			{
				cubic_texture nm.png combinedUVW
				tex_coord_set 1
				tex_address_mode clamp
			}

			lighting off
			// Really basic vertex program
			// NB we don't use fixed function here because GL does not like
			// mixing fixed function and vertex programs, depth fighting can
			// be an issue
			vertex_program_ref Ogre/BasicVertexPrograms/AmbientOneTexture
			{
				param_named_auto worldViewProj worldviewproj_matrix
				param_named ambient float4 1 1 1 1
			} 

	    		cull_hardware none
	    		scene_blend alpha_blend
			shading phong
			alpha_rejection greater 150 
			
			texture_unit
			{
				texture alphaNormalTest.tga
				tex_coord_set 2
				colour_op modulate
				scale 1 1
				scroll 0 0
				rotate 0				
			}
		}
	}
}
Is there anything wrong with this?
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

Yes, plenty ;)

You've compressed the material into one pass but since you haven't altered the shader, or changed some of the material commands, it's not going to work.

Think about how the original worked. It performed a single ambient pass, then added a lighting pass per light, then modulated the whole thing with a decal / albedo pass (the surface texture). You have to combine all those things into one shader, add in your setting of the alpha component in your pixel shader, and your scene_blend needs to be alpha_blend if you want to make the object take into account that alpha result.
RSAgent007
Halfling
Posts: 40
Joined: Thu Apr 06, 2006 6:40 am

Post by RSAgent007 »

Sinbad, thank you very much for your help. I am learning a lot as an Ogre fledging.

I think I am getting closer (but it's still not looking right :(). I changed the scene_blend to alpha_blend in the last .material script I posted, and I have slightly altered the pixel shader as follow:

my .program file has this:

Code: Select all

// Bump map fragment program, support for this is optional
fragment_program Examples/BumpMapFP cg
{
    source Example_BumpMapping.cg
    entry_point main_fp
    profiles ps_2_0 arbfp1
}
And my .cg file has this:

Code: Select all

void main_fp( float2 uv         : TEXCOORD0,
              float3 TSlightDir : TEXCOORD1,

              out float4 colour : COLOR,

              uniform float4 lightDiffuse,
              uniform sampler2D   normalMap : register(s0),
              uniform samplerCUBE normalCubeMap : register(s1) ,
              uniform sampler2D alphaMap : register(s2)
              )
{
    // retrieve normalised light vector, expand from range-compressed
    float3 lightVec = expand(texCUBE(normalCubeMap, TSlightDir).xyz);

    // get bump map vector, again expand from range-compressed
    float3 bumpVec = expand(tex2D(normalMap, uv).xyz);

    float alpha = tex2D(alphaMap, uv).a;
    
    colour.a = alpha;

    // Calculate dot product
    colour = lightDiffuse * dot(bumpVec, lightVec) * alpha;
}
You mentioned that I need to combine the ambient pass, lighting pass, and then decal pass into one shader. But I am not sure how to do that...

Also, I imagine that the pixel shader could not be combined with the vertex shader. This is how I see the flow:

material {
vertex shader
pixel shader
specify texture units
}

Is that right? Sorry for all the beginner questions =/
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

Quick reply since I'm short of time:

Code: Select all

void main_fp( float2 uv         : TEXCOORD0,
              float3 TSlightDir : TEXCOORD1,

              out float4 colour : COLOR,

              uniform float4 lightDiffuse,
			  uniform float4 ambient,
              uniform sampler2D   normalMap : register(s0),
              uniform samplerCUBE normalCubeMap : register(s1) ,
              uniform sampler2D alphaMap : register(s2)
              )
{
    // retrieve normalised light vector, expand from range-compressed
    float3 lightVec = expand(texCUBE(normalCubeMap, TSlightDir).xyz);

    // get bump map vector, again expand from range-compressed
    float3 bumpVec = expand(tex2D(normalMap, uv).xyz);

    float alpha = tex2D(alphaMap, uv).a;
   

    // Calculate dot product
    colour = float4(ambient.rgb + lightDiffuse * saturate(dot(bumpVec, lightVec)), alpha);
}

Make sure you add

Code: Select all

param_named_auto ambient ambient_light_colour
to your fragment program params in the material too.
RSAgent007
Halfling
Posts: 40
Joined: Thu Apr 06, 2006 6:40 am

Post by RSAgent007 »

Hello, thanks again. I'm learning a lot.

I tried that out, and this is what I get now:

Image

I can see what is going on, but I'm still not sure why these results would produce.

Here are my .cg, .program, and .material file (i renamed it to .txt for the web):

http://www.finalaeon.com/_temp/z/test.cg.txt
http://www.finalaeon.com/_temp/z/test.program.txt
http://www.finalaeon.com/_temp/z/test.material.txt

Thank you very much for your help so far :)
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

Code: Select all

         fragment_program_ref Examples/BumpMapFP
         {
            // param_named_auto lightDiffuse light_diffuse_colour 0
            param_named_auto ambient ambient_light_colour
         }
You need to uncomment that lightDiffuse param setup.

What is your alpha map's format? Is it actually RGBA with alpha values in the alpha component, or is it greyscale? If it's greyscale you need to either change your shader to refer to the '.r' component of the result from tex2D instead of the .a component (because the values aren't loaded into the alpha channel of the texture), or change the material definition and add the option 'alpha' onto the 'texture' directive of the unit referring to the alpha texture.

You've also got too much in your material definition. You can't put multiple vertex_program_refs in there, only one can be used per pass and in your case only the last one is being used. I've trimmed it and fixed the problems above, try this:

Code: Select all

material ___Default
{      
   technique
   {

      pass
      {
         cull_hardware none
         scene_blend alpha_blend
         shading phong
         alpha_rejection greater 150
         
         // Vertex program reference
         vertex_program_ref Examples/BumpMapVP
         {
            param_named_auto lightPosition light_position_object_space 0
            param_named_auto worldViewProj worldviewproj_matrix
         }

         // Fragment program
         fragment_program_ref Examples/BumpMapFP
         {
            param_named_auto lightDiffuse light_diffuse_colour 0
            param_named_auto ambient ambient_light_colour
         }
         
         // Base bump map
         texture_unit
         {
            texture Sphere01NormalsMap.tga
            tex_coord_set 0
            colour_op replace
         }
         // Normalisation cube map
         texture_unit
         {
            cubic_texture nm.png combinedUVW
            tex_coord_set 1
            tex_address_mode clamp
         }

         texture_unit
         {
            texture alphaNormalTest.tga alpha
            tex_coord_set 2
            colour_op modulate
            scale 1 1
            scroll 0 0
            rotate 0            
         }
      }
   }
}
RSAgent007
Halfling
Posts: 40
Joined: Thu Apr 06, 2006 6:40 am

Post by RSAgent007 »

Once again, I appreciate your help!

I'm pretty sure the alpha map is not greyscale, but contains an alpha value. You can double check here (contains normal map + alpha map + mesh):
http://www.finalaeon.com/_temp/z/test.zip

I'm using the alpha layer in the diffuse texture map. I removed the "alpha" option you had in the texture unit directive in the material. However, whether or not I left it there had no effect. Just in case I was wrong about the alpha map, I changed the ".a" to ".r" in the pixel shader as you mentioned. I now produce this:

Image

I'm not sure what those "patches" indicate, as there isn't any diffuse colors from the diffuse texture map as well.

It is being clear where it should be (near the bottom of the hair), so it seems that the alpha map is there. I can also see the where the normal map is affecting the lighting, albeit with those weird patches.
RSAgent007
Halfling
Posts: 40
Joined: Thu Apr 06, 2006 6:40 am

Post by RSAgent007 »

I have made some progress.

I modified the code that Sinbad suggested to this:

Code: Select all

void main_fp( float2 uv         : TEXCOORD0,
              float3 TSlightDir : TEXCOORD1,

              out float4 colour : COLOR,

              uniform float4 lightDiffuse,
              uniform float4 ambient,
              uniform sampler2D   normalMap : register(s0),
              uniform samplerCUBE normalCubeMap : register(s1) ,
              uniform sampler2D alphaMap : register(s2)
              )
{
    // retrieve normalised light vector, expand from range-compressed
    float3 lightVec = expand(texCUBE(normalCubeMap, TSlightDir).xyz);

    // get bump map vector, again expand from range-compressed
    float3 bumpVec = expand(tex2D(normalMap, uv).xyz);

    float alpha = tex2D(alphaMap, uv).a;   
    float4 colourValues = tex2D(alphaMap, uv);

    // Calculate dot product
    colour = float4(colourValues.rgb * lightDiffuse * saturate(dot(bumpVec, lightVec)), alpha);
    //colour = float4(ambient.rgb + lightDiffuse * saturate(dot(bumpVec, lightVec)), alpha);
} 
That now takes the diffuse color, but I'm still left with patches, like this:

Image

Could it be because of the mesh tangents? I used the Maya 8.0 exporter.
RSAgent007
Halfling
Posts: 40
Joined: Thu Apr 06, 2006 6:40 am

Post by RSAgent007 »

Is there a way to check the mesh tangents?
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

You can use OgreMeshUpgrade.exe in 'interactive' mode to display the mesh format and reorganise it if you want.

Really to solve your problems you need to go over the shader with a fine toothcomb, disabling and reenabling bits as you go until you discover the part that's not behaving as you expect. Isolate each component and display it on its own, to see which are correct and which are not. E.g. colour the mesh using the tangents, then just the normal map (without using it for lighting, just slap the raw colour on there), then the basic light vectors etc etc. It's the only way to find out which element is not working.
Yin
Gnoblar
Posts: 4
Joined: Fri Mar 30, 2007 3:07 pm

Post by Yin »

Was this problem solved? I'm a graphic artist, not a programmer so I don't have the same capabilities in coding materials. But I would need a material like that combined this, got normal maps working after a while but the thought of transparency combined with normals being a problem didn't cross my mind. Played around a bit with the code posted here, got some weird results (normal mapped model, all white texture, dark shading was transparent :D).

From a graphic artist viewpoint, OGRE is very difficult. Takes so much to do(at least the first time) simple things like adding basic maps to a material :/
Post Reply