Cel Shading and Texture

Problems building or running the engine, queries about how to use features etc.
Post Reply
Liessa
Gnoblar
Posts: 24
Joined: Fri Mar 09, 2007 1:34 pm
x 1

Cel Shading and Texture

Post by Liessa »

Hi,

I'm trying to use the Cel Shading Example on a Mesh with existing texture.
In CelShading.material I added another texture_unit containing my imagefile with tex_coord_set 3.

In main_vp I just pass the corresponding texture-UV-Coords to main_fp:

Code: Select all

float2 texCoord3 : TEXCOORD3
out float2 uv3	: TEXCOORD3
uv3 = texCoord3;
In main_fp I calculate the output-color:

Code: Select all

float4 texture_color = tex2D(texture, uv3);
colour = edge * ((texture_color * diffuseIn) + (texture_color * specularIn));
The Cel Shading works, but my texture isn't shown. the whole Mesh is shown in green - in fact one color contained in my imagefile.

Did I forget anything? Any Idea? Is there anything wrong with tex_coord_set 3 or TEXCOORD3?[/code]
claustre
Goblin
Posts: 278
Joined: Thu Mar 23, 2006 10:35 am
Location: Toulouse, France
Contact:

Post by claustre »

Well generally TEXCOORD0 is used to pass the texture coordinates in a mesh, I really doubt your mesh uses 4 texture coordinates...
Liessa
Gnoblar
Posts: 24
Joined: Fri Mar 09, 2007 1:34 pm
x 1

Post by Liessa »

TEXCOORD0 - TEXCOORD2 (and tex_coord_set 0-2) are already in use by Cel Shading Example for cel_shading_diffuse.png, cel_shading_specular.png and cel_shading_edge.png.
claustre
Goblin
Posts: 278
Joined: Thu Mar 23, 2006 10:35 am
Location: Toulouse, France
Contact:

Post by claustre »

Yes they are, but as OUTPUT registers to store some values necessary to the computation. However, as INPUT register of your mesh data it must be TEXCOORD0 that is used for storing texture coordinates.

So your code should be :

Code: Select all

float2 texCoord3 : TEXCOORD0
out float2 uv3   : TEXCOORD3 
uv3 = texCoord3; 
Liessa
Gnoblar
Posts: 24
Joined: Fri Mar 09, 2007 1:34 pm
x 1

Post by Liessa »

Thank you for your quick answers!
You are right! I changed it and now it works. I really got confused about the whole TEXCOORD# in and out and tex_coord_set thing...
DzX
Gnoblar
Posts: 2
Joined: Tue Nov 20, 2007 6:14 pm

Post by DzX »

Can you attach the 2 files, .material and .cg ?
Thanks a lot!
DzX
Gnoblar
Posts: 2
Joined: Tue Nov 20, 2007 6:14 pm

Post by DzX »

please..can anyone send me the shader modified for texture?
Liessa
Gnoblar
Posts: 24
Joined: Fri Mar 09, 2007 1:34 pm
x 1

Post by Liessa »

If you want to use a texture color in your Cel Shader you just have to modify the Cel Shading example in Ogre:
- material file: add a texture_unit containing your texture
- cg file, main_vp: simply copy input texture uv to output

Code: Select all

void main_vp(
...
float2 texCoord3 : TEXCOORD0,
out float uv3 : TEXCOORD3,
...
)
{
  uv3 = texCoord3;
}
- cg file, main_fp: get texture color

Code: Select all

void main_fp(
...
float2 uv3 : TEXCOORD3,
out float4 colour : COLOR,
...
uniform sampler2D texture : register(s3)
)
{
  float4 texture_color = tex2D( texture, uv3 );
  colour = ( texture_colour * diffuseIn );
  colour.a = texture_colour.a;
}
tobaccomonk
Gnoblar
Posts: 5
Joined: Sat Nov 03, 2007 4:30 pm

Post by tobaccomonk »

Just a minor correction to main_vp,
the line reading:

Code: Select all

out float uv3 : TEXCOORD3,
should actually read:

Code: Select all

out float2 uv3 : TEXCOORD3,
Also in the .material file, make sure you add

Code: Select all

iteration once_per_light
in your pass section, if you have more than one light source.

Well, here is how it looks like (it works for me, please notify me if I'm missing something).

In the .material file:

Code: Select all

vertex_program Ogre/CelShadingTexturedVP cg
{
	source Example_CelShadingTextured.cg
	entry_point main_vp
	profiles vs_1_1 arbvp1

	default_params
	{
		param_named_auto lightPosition light_position_object_space 0
		param_named_auto eyePosition camera_position_object_space
		param_named_auto worldViewProj worldviewproj_matrix
		param_named shininess float 10 
	}
}

fragment_program Ogre/CelShadingFP cg
{
	source Example_CelShading.cg
	entry_point main_fp
	profiles ps_2_0 arbfp1 fp20
}

material Examples/CelShadingTextured
{
	technique
	{
		pass
		{
			iteration once_per_light
			vertex_program_ref Ogre/CelShadingTexturedVP
			{
				param_named shininess float 1.0 // you can change this as well
			}
			fragment_program_ref Ogre/CelShadingTexturedFP
			{
			}
			texture_unit
			{
				texture cel_shading_diffuse.png 1d
				tex_address_mode clamp
				filtering none
			}
			texture_unit
			{
				texture cel_shading_specular.png 1d
				tex_address_mode clamp
				filtering none
				tex_coord_set 1
			}
			texture_unit
			{
				texture cel_shading_edge.png 1d
				tex_address_mode clamp
				filtering none
				tex_coord_set 2
			}
			texture_unit
			{
				texture diffuse_map.png //add your diffuse map here
			}
			texture_unit
			{
				texture specular_map.png //add your specular map here
			}
		}
	}
	
}
Please notice that this requires ps_2_0 (to handle more than 4 input maps).

And Example_CelShadingTextured.cg:

Code: Select all

void main_vp(float4 position : POSITION,
			 float3 normal : NORMAL,
			 // outputs
			 out float4 oPosition : POSITION,
			 out float  diffuse : TEXCOORD0,
			 out float  specular : TEXCOORD1,
			 out float  edge : TEXCOORD2,
			 float2 texCoord3 : TEXCOORD0, 
			 out float2 uv3 : TEXCOORD3,
			 // parameters
			 uniform float3 lightPosition, // object space
			 uniform float3 eyePosition,   // object space
			 uniform float4  shininess,
			 uniform float4x4 worldViewProj)
{
	// calculate output position
	oPosition = mul(worldViewProj, position);

	// calculate light vector
	float3 N = normalize(normal);
	float3 L = normalize(lightPosition - position.xyz);
	
	// Calculate diffuse component
	diffuse = max(dot(N, L) , 0);

	// Calculate specular component
	float3 E = normalize(eyePosition - position.xyz);
	float3 H = normalize(L + E);
	specular = pow(max(dot(N, H), 0), shininess);
	// Mask off specular if diffuse is 0
	if (diffuse == 0) specular = 0;

	// Edge detection, dot eye and normal vectors
	edge = max(dot(N, E), 0);
	uv3 = texCoord3;
}

void main_fp(float diffuseIn : TEXCOORD0,
			 float specularIn : TEXCOORD1,
			 float edge	: TEXCOORD2,
			 
			 out float4 colour : COLOR,

			 float2 uv3	: TEXCOORD3,
			 
			 uniform sampler1D diffuseRamp,
			 uniform sampler1D specularRamp,
			 uniform sampler1D edgeRamp,
			 uniform sampler2D diffmap : register(s3),
			 uniform sampler2D specmap : register(s4)
)
{

	// Step functions from textures
	diffuseIn = tex1D(diffuseRamp, diffuseIn).x;
	edge = tex1D(edgeRamp, edge).x;


	float4 diffuse_map = tex2D( diffmap, uv3 );
	colour = edge * ( ( diffuse_map * diffuseIn ) + ( tex2D( specmap, uv3 ) * specularIn) ); 
	colour.a = diffuse_map.a; 
}
Edit reason: Forgot to multiply by the "edge" factor in the final fragment shader output
Last edited by tobaccomonk on Sat Apr 12, 2008 2:40 am, edited 2 times in total.
Lhor
Gnoblar
Posts: 3
Joined: Wed Feb 27, 2008 8:02 pm

Post by Lhor »

Hi,

I still can't make it work. I've tried both examples but with no luck. I have managed to apply the cel shader, and play with the colors, but haven't been able to apply a texture.
The test scene is simple, 1 mesh, 1 plane and 1 light.
The mesh is properly mapped and, just in case, have tried it with 3 mapping channels.

Is there something I could be missing from the previous example?

Here are 2 images, left the working shader (the outline is the same color as the shadow on purpouse), the right image is using the above material and cg program.
I'd appreciate any input on the matter. Thanks!


Image
Lhor
Gnoblar
Posts: 3
Joined: Wed Feb 27, 2008 8:02 pm

Fixed

Post by Lhor »

Ok, I've found the problem. First, I made a mistake when copyng the shader. Still that render me a Black mesh. After some playing around I added to the material file two:

colour_op add

at the texture unit of the "cel_shading_diffuse.png" and at the "cel_shading_specular.png"

Code: Select all

texture_unit
         {
            texture cel_shading_diffuse.png 1d
            tex_address_mode clamp
            filtering none
		colour_op add
         }
         texture_unit
         {
            texture cel_shading_specular.png 1d
            tex_address_mode clamp
            filtering none
            tex_coord_set 1
		colour_op add

         }
and this is how it looks:
left---> no "colour_op" add
center--> "colour_op add" in diffuse
right ---> "colour_op add" in diffuse and shading

Image

Now it just needs a bit of adjusting. But works great :)

Hope this helps out.

Thanks tobaccomonk, Liessa and everyone who's worked on this subject!!
Ustaz
Gnoblar
Posts: 1
Joined: Thu Apr 03, 2008 7:01 pm

Post by Ustaz »

Hi!

I can't get it to look like cel shading?

So I wondering if someone could give me an example how the specular map and the diffuse map should look like?
Lhor
Gnoblar
Posts: 3
Joined: Wed Feb 27, 2008 8:02 pm

Post by Lhor »

well, the specular, diffuse and edge maps (png), are nothing more than a 16x1 pixel ramp image. U can find all 3 in the Ogre demos.

It took me several tries till I found what I was doing wrong, but if u use the previous examples it should work.
grillamesh
Gnoblar
Posts: 1
Joined: Mon Apr 07, 2008 12:38 am

Post by grillamesh »

Code: Select all

        texture_unit
         {
            texture diffuse_map.png //add your diffuse map here
         }
         texture_unit
         {
            texture specular_map.png //add your specular map here
         } 

I can't get the cel shading to work!!

Can someone give me a example how the texture of the diffuse_map.png and specular_map.png should look like så i can get the cel shading to work.
tobaccomonk
Gnoblar
Posts: 5
Joined: Sat Nov 03, 2007 4:30 pm

Post by tobaccomonk »

Ustaz wrote: I can't get it to look like cel shading?

So I wondering if someone could give me an example how the specular map and the diffuse map should look like?
In order to get a nice cartoonish look of a cel-shaded scene, you will need to use proper diffuse and specular maps. Using photorealistic, or semi-photorealistic textures probably won't do the trick.

You might find some clues in this post about the game XIII
grillamesh wrote: Can someone give me a example how the texture of the diffuse_map.png and specular_map.png should look like så i can get the cel shading to work.
They are basically the diffuse and specular maps you'd use for your mesh in general. They don't need to have any particular specification except that they should reflect how you would your mesh to look like.

Also you might want to check the above URL regarding XIII.
abwing@abwing.com
Gnoblar
Posts: 1
Joined: Thu Mar 13, 2008 5:23 am

Post by abwing@abwing.com »

Hi, I have some problem with it. When I set my entity to use your material above all of my mesh with texture go wrong mapping. Can you tell me how to fix it. or it wrong since I have export the model(some of my model have blinn.cg file when export)
Post Reply