2D Shadows

Get answers to all your basic programming questions. No Ogre questions, please!
Post Reply
User avatar
calsmurf2904
Orc
Posts: 401
Joined: Tue Sep 16, 2008 9:39 pm
Location: Netherlands

2D Shadows

Post by calsmurf2904 »

Hello,

I'm currently in the process of learning D3D while still creating my projects with Ogre. (I think its some kind of plus to know the lower-level api's).
My current code uses the ID3DXSprite class to draw the 2d elements, but how could one add shadows in this way?
I found a few articles, like this:
http://www.gamedev.net/reference/progra ... oftshadow/
http://www.catalinzima.com/?page_id=313

But they are all for when you are using vertices, which I'm clearly not. (Well, ID3DXSprite uses it internal maybe, but I'm not using it directly).
How could one handle 2d shadows when the only resources for the light is the position, range, color and the pixels of the screen?

Thanks in advance,
Calsmurf2904
Visit my blog at http://calsmurf2904.wordpress.com !
Got a Google Wave account? Add me as contact! (projectxgame <at> gmail <dot> com)
Image
User avatar
vitefalcon
Orc
Posts: 438
Joined: Tue Sep 18, 2007 5:28 pm
Location: Seattle, USA
x 13

Re: 2D Shadows

Post by vitefalcon »

What I've done is to make a material which uses a shader to do the first pass and draw a shadow, while the second pass draws the sprite.

Shadow2D.cg

Code: Select all

void Shadow2D_VP(
	float4 position : POSITION,
	float2 uv		: TEXCOORD0,

	out float4 outPos : POSITION,
	out float2 outUv : TEXCOORD0,
	
	uniform float4x4 worldViewProj,
	uniform float4 shadowShift
	)
{
	outPos = mul(worldViewProj,position) + shadowShift;
	outUv = uv;
}


void Shadow2D_FP(
	float2 uv : TEXCOORD0,
	
	out float4 outColour : COLOR,
	
	uniform sampler2D panelTexture,
	uniform float4 shadowColour
	)
{
	float4 colour = tex2D(panelTexture, uv);
	float4 alpha = float4(colour.a, colour.a, colour.a, colour.a);
	outColour = shadowColour * alpha;
}
Material File

Code: Select all

vertex_program Shadow2D/CasterVP cg
{
	source Shadow2D.cg
	entry_point Shadow2D_VP
	profiles vs_2_x arbvp1
	
	default_params
	{
		param_named_auto worldViewProj worldviewproj_matrix
		// The shadow shift is in projection coordinates.
		param_named shadowShift float4 0.05 -0.05 0.0 0
	}
}

fragment_program Shadow2D/CasterFP cg
{
	source Shadow2D.cg
	entry_point Shadow2D_FP
	profiles ps_2_x arbfp1
	
	default_params
	{
		param_named shadowColour float4 0.0 0.0 0.0 0.5
	}
}

material Shadow2D/Template
{
	transparency_casts_shadows on
	
	technique
	{
		pass ShadowPass
		{
			lighting off
			scene_blend alpha_blend
			depth_write off
			depth_check on
			
			texture_unit ShadowMask
			{
				texture DEFAULT_SHADOW_MASK.png
				tex_address_mode clamp
			}
			
			vertex_program_ref Touchscape/Shadow2D/CasterVP
			{
			}
			
			fragment_program_ref Touchscape/Shadow2D/CasterFP
			{
			}
		}
		pass ObjectRender
		{
			lighting off
			depth_write on
			depth_check on
			separate_scene_blend one zero src_alpha one_minus_src_alpha
			alpha_rejection greater_equal 128
			
			texture_unit Image2D
			{
				texture DEFAULT_IMAGE.png
				tex_address_mode clamp
			}
		}
	}
}
Now if you want to have a material with shadows, do this:

Code: Select all

material MySpriteMaterial : Shadow2D/Template
{
	set_texture_alias ShadowMask MySpriteShadowMask.png
	set_texture_alias Image2D MySpriteImage.png
}
This is a very cheap way of doing it and should work even if you're not explicitly using vertices because I believe underneath ID3DXSprite it does use a quad to represent the sprite (two triangles to be precise). The code above is in terms of Ogre material. Note that the light direction is hard-coded in the material file. However, you should be able to change them from your code. Also, if you're not using Ogre3D, I hope the Cg program will still spark some ideas to help you resolve your issue.

[Edit]You can see this shadow working in one of the videos I've posted to explain my issue with matrices: here.[/Edit]
Image
User avatar
calsmurf2904
Orc
Posts: 401
Joined: Tue Sep 16, 2008 9:39 pm
Location: Netherlands

Re: 2D Shadows

Post by calsmurf2904 »

Yea, I saw that method somewhere else...but it didn't really fit my needs.
What I now did was create a new rendertarget, create the vertexes of the sprite that ID3DXSprite uses. (So just four vertexes). Pass these over to the shadow function and let it render to the render target.
Then blend the render target with the scene.

This works (well kinda, no soft shadows :() but for some reason the shadow "jumps" over when moving from a specific angle from the light to another angle from the light. Perhaps someone could help me with this? I'll post the code later.
Visit my blog at http://calsmurf2904.wordpress.com !
Got a Google Wave account? Add me as contact! (projectxgame <at> gmail <dot> com)
Image
User avatar
calsmurf2904
Orc
Posts: 401
Joined: Tue Sep 16, 2008 9:39 pm
Location: Netherlands

Re: 2D Shadows

Post by calsmurf2904 »

Alright, fixed it all. My ShadowMap generation code is: (kinda messy, and has to be optimized, but it works):

Code: Select all

if(lightMap && alphaOne && lights.size() > 0)
	{
		LPDIRECT3DSURFACE9 backBuffer;
		LPDIRECT3DSURFACE9 lightmapSurface;
		dxDevice->GetRenderTarget(0, &backBuffer);

		lightMap->GetSurfaceLevel(0, &lightmapSurface);
		dxDevice->SetRenderTarget(0, lightmapSurface);
		dxDevice->SetFVF(D3DFVF_VERTEX);
		dxDevice->SetRenderState(D3DRS_CULLMODE, D3DCULL_NONE);

		//render the lightmap with 2d lights only
		dxDevice->Clear(0, 0, D3DCLEAR_TARGET | D3DCLEAR_ZBUFFER, D3DCOLOR_XRGB(35, 35, 35), 1.0f, 0);
		dxDevice->BeginScene();

		HRESULT hr = dxSprite->Begin(D3DXSPRITE_ALPHABLEND);
		if(hr != S_OK)
			std::cout << "D3D error: 0x" << std::hex << hr << std::endl;

		for(int l = 0;l<lights.size();l++)
		{
			D3DLight* light = lights[l];
			if(!light->Get2D())
				continue;

			ClearAlphaToOne(dxDevice, alphaOne, windows[0]->GetWidth(), windows[0]->GetHeight());
			dxDevice->SetRenderState(D3DRS_DESTBLEND, D3DBLEND_ZERO);
			dxDevice->SetRenderState(D3DRS_SRCBLEND, D3DBLEND_ONE);
			dxDevice->SetRenderState(D3DRS_ALPHABLENDENABLE, TRUE);
			dxDevice->SetRenderState(D3DRS_COLORWRITEENABLE, D3DCOLORWRITEENABLE_ALPHA);

			for(int i = 0;i<sprites.size();i++)
			{
				EngineSprite* sprite = sprites[i];
				if(!strcmp(sprite->GetTypeName(), "Text"))
					continue;

				if(sprite->GetZOrder() != light->GetPosition().absZ)
					continue;

				if(sprite->GetSize().absX == windows[0]->GetWidth())
					continue;

				if(!sprite->GetVisible())
					continue;

				bool backFacing[4];
				memset(backFacing, 0, 4);

				Vector2 vertices[4];
				memset(vertices, 0, 4*sizeof(Vector2));

				vertices[0].absX = -(sprite->GetSize().absX/2);
				vertices[0].absY = -(sprite->GetSize().absY/2);

				vertices[1].absX = sprite->GetSize().absX/2;
				vertices[1].absY = -(sprite->GetSize().absY/2);

				vertices[2].absX = sprite->GetSize().absX/2;
				vertices[2].absY = sprite->GetSize().absY/2;

				vertices[3].absX = -(sprite->GetSize().absX/2);
				vertices[3].absY = sprite->GetSize().absY/2;

				for (int i2 = 0; i2 < 4; i2++)
				{
					sprite = sprites[i];
					Vector2 firstVertex = Vector2(vertices[i2].absX, vertices[i2].absY) + sprite->GetPosition();
					int secondIndex = (i2 - 1) % 4;
					if(secondIndex < 0)
						secondIndex = 3;
					else if(secondIndex > 3)
						secondIndex = 0;
					Vector2 secondVertex = Vector2(vertices[secondIndex].absX, vertices[secondIndex].absY) + sprite->GetPosition();
					Vector2 middle = (firstVertex + secondVertex);
					middle.absX /= 2;
					middle.absY /= 2;

					Vector2 L = Vector2(light->GetPosition().absX, light->GetPosition().absY) - middle;

					Vector2 N = Vector2();
					N.absX = - (secondVertex.absY - firstVertex.absY);
					N.absY = secondVertex.absX - firstVertex.absX;
	                
					if (Vector2DotProduct(N, L) > 0)
						backFacing[i2] = false;
					else
						backFacing[i2] = true;
				}

				int startingIndex=0;
				int endingIndex=0;
				for (int i3 = 0; i3 < 4; i3++)
				{
					int currentEdge = i3;
					int nextEdge = (i3 - 1) % 4;
					if(nextEdge < 0)
						nextEdge = 3;
					else if(nextEdge > 3)
						nextEdge = 0;

					if (backFacing[currentEdge] && !backFacing[nextEdge])
						endingIndex = nextEdge;

					if (!backFacing[currentEdge] && backFacing[nextEdge])
						startingIndex = nextEdge;
				}

				int shadowVertexCount = 0;
				if(endingIndex > startingIndex)
					shadowVertexCount = endingIndex - startingIndex + 1;
				else
					shadowVertexCount = 5 - startingIndex + endingIndex + 1;

				InternalVertex* shadowVertices = new InternalVertex[shadowVertexCount * 2];

				//create a triangle strip that has the shape of the shadow
				int currentIndex = startingIndex;
				int svCount = 0;
				while (svCount != shadowVertexCount*2)
				{
					sprite = sprites[i];
					Vector2 vertexPos = vertices[currentIndex] + sprite->GetPosition();
	                
					//one vertex on the hull
					shadowVertices[svCount].color = 0x11000000;
					shadowVertices[svCount].x = vertexPos.absX;
					shadowVertices[svCount].y = vertexPos.absY;
					shadowVertices[svCount].z = 0;

					//one extruded by the light direction
					shadowVertices[svCount + 1].color = 0x11000000;
					Vector2 L2P = vertexPos - Vector2(light->GetPosition().absX, light->GetPosition().absY);
					L2P.Normalize();
					Vector2 finalPos = vertexPos + L2P * 128000;
					shadowVertices[svCount + 1].x = finalPos.absX;
					shadowVertices[svCount + 1].y = finalPos.absY;
					shadowVertices[svCount + 1].z = 0;
	              
					svCount+=2;
					currentIndex = (currentIndex + 1) % 4;
				}

				dxDevice->SetTexture(0, NULL);
				HRESULT hr = dxDevice->DrawPrimitiveUP(D3DPT_TRIANGLESTRIP, shadowVertexCount*2-2, shadowVertices, sizeof(InternalVertex));
				if(hr != S_OK)
					std::cout << "D3D error: 0x" << std::hex << hr << std::endl;

				delete shadowVertices;
			}

			dxDevice->SetRenderState(D3DRS_DESTBLEND, D3DBLEND_ONE);
			dxDevice->SetRenderState(D3DRS_SRCBLEND, D3DBLEND_DESTALPHA);
			dxDevice->SetRenderState(D3DRS_BLENDOP,  D3DBLENDOP_ADD);
			dxDevice->SetRenderState(D3DRS_SEPARATEALPHABLENDENABLE, TRUE);
			dxDevice->SetRenderState(D3DRS_COLORWRITEENABLE, D3DCOLORWRITEENABLE_ALPHA | D3DCOLORWRITEENABLE_RED | D3DCOLORWRITEENABLE_GREEN | D3DCOLORWRITEENABLE_BLUE);

			int vertexCount = (360/(360.0f/32.0f))+3.5f;
			InternalVertex* vertexes = new InternalVertex[vertexCount];

			vertexes[0].x = light->GetPosition().absX;
			vertexes[0].y = light->GetPosition().absY;
			vertexes[0].z = 0;
			vertexes[0].color = D3DCOLOR_ARGB(((int)((light->GetIntensity()*255)+0.5)), light->GetColor().r, light->GetColor().g, light->GetColor().b);

			int count = 1;
			for (float angle=0; angle<360; angle+=(360/32) )
			{
				vertexes[count].x = light->GetRange()*cos((M_PI/180)*angle)+light->GetPosition().absX;
				vertexes[count].y = light->GetRange()*sin((M_PI/180)*angle)+light->GetPosition().absY;
				vertexes[count].z = 0;
				vertexes[count].color = D3DCOLOR_ARGB(0, 0, 0, 0);
				count++; 
			}

			vertexes[count].x = light->GetRange()*cos((M_PI/180)*0)+light->GetPosition().absX;
			vertexes[count].y = light->GetRange()*sin((M_PI/180)*0)+light->GetPosition().absY;
			vertexes[count].z = 0;
			vertexes[count].color = D3DCOLOR_ARGB(0, 0, 0, 0);

			dxDevice->SetTexture(0, NULL);
			HRESULT hr = dxDevice->DrawPrimitiveUP(D3DPT_TRIANGLEFAN, vertexCount-2, vertexes, sizeof(InternalVertex));
			if(hr != S_OK)
				std::cout << "D3D error: 0x" << std::hex << hr << std::endl;

			delete vertexes;
		}

		ClearAlphaToOne(dxDevice, alphaOne, windows[0]->GetWidth(), windows[0]->GetHeight());

		dxSprite->End();
		dxDevice->EndScene();
		dxDevice->Present(NULL, NULL, NULL, NULL);

		dxDevice->SetRenderTarget(0, backBuffer);
	}
Where ClearAlphaToOne does this:

Code: Select all

void ClearAlphaToOne(LPDIRECT3DDEVICE9 dxDevice, LPDIRECT3DTEXTURE9 texture, int width, int height)
{
	dxDevice->SetRenderState(D3DRS_COLORWRITEENABLE, D3DCOLORWRITEENABLE_ALPHA);

	InternalVertex* vertexes = new InternalVertex[6];
	vertexes[0].x = vertexes[3].x = vertexes[5].x = 0.0f;
	vertexes[0].y = vertexes[1].y = vertexes[3].y = 0.0f;
	vertexes[1].x = vertexes[2].x = vertexes[4].x = width;
	vertexes[2].y = vertexes[4].y = vertexes[5].y = height;
	vertexes[0].z = vertexes[1].z = vertexes[2].z = vertexes[3].z = vertexes[4].z = vertexes[5].z = 0.0f;

	vertexes[0].u = vertexes[3].u = vertexes[5].u = 0.0f;
	vertexes[0].v = vertexes[1].v = vertexes[3].v = 0.0f;
	vertexes[1].u = vertexes[2].u = vertexes[4].u = 1.0f;
	vertexes[2].v = vertexes[4].v = vertexes[5].v = 1.0f;

	dxDevice->SetFVF(D3DFVF_VERTEX);
	dxDevice->SetTexture(0, texture);
	dxDevice->DrawPrimitiveUP(D3DPT_TRIANGLELIST, 2, vertexes, sizeof(InternalVertex));

	delete vertexes;
	dxDevice->SetRenderState(D3DRS_COLORWRITEENABLE, D3DCOLORWRITEENABLE_ALPHA | D3DCOLORWRITEENABLE_RED | D3DCOLORWRITEENABLE_GREEN | D3DCOLORWRITEENABLE_BLUE);
}
And the texture alphaOne is just a texture the size of the screen filled with 255, 255, 255, 255 ARGB pixels.
The only downside is that it doesn't work with rotation yet, I need to transform every vertex position to reflect the rotation, but haven't done this yet.

This looks like this:
http://img705.imageshack.us/img705/6239/2dshadows4.png

PS:
For anybody wondering, I am drawing the shadow map with this:

Code: Select all

if(lightMap)
	{
		dxDevice->SetRenderState(D3DRS_DESTBLEND, D3DBLEND_SRCCOLOR);
		dxDevice->SetRenderState(D3DRS_SRCBLEND, D3DBLEND_ZERO);
		dxDevice->SetRenderState(D3DRS_BLENDOP, D3DBLENDOP_ADD);

		D3DXMATRIX matrix;
		D3DXMatrixTranslation(&matrix, 0, 0, 0.1f);
			
		dxDevice->SetTexture(0, lightMap);
		dxDevice->SetFVF(D3DFVF_VERTEX);
		dxDevice->SetTransform(D3DTS_WORLD, &matrix);
		dxDevice->SetStreamSource(0, shadowTexture, 0, sizeof(InternalVertex));
		dxDevice->DrawPrimitive(D3DPT_TRIANGLELIST, 0, 2);

		dxDevice->SetRenderState(D3DRS_DESTBLEND, D3DBLEND_ZERO);
		dxDevice->SetRenderState(D3DRS_SRCBLEND, D3DBLEND_ONE);
		dxDevice->SetRenderState(D3DRS_BLENDOP,  D3DBLENDOP_MIN);
	}
Next thing to do is getting soft shadows working. And specular mapping.

How I got this code is just porting over this XNA C# code over to C++.
http://www.catalinzima.com/?page_id=313

As seen in the screenshot it isn't really correct, since I assume the sprite is a rectangle instead of any other shape.

[edit]
Edited it all a bit to make the lights support colors, and the shadows aren't transparent anymore. (They just block the light where the shadow came from fully now, which is realistic).

@vitefalcon: Your example looks cool, and works with rotation. But for a 2d top-down game the approach above is more fitting then your approach. (Yours is good with a RPG though and possibly a platformer). It mostly just depends on the game :) Thanks for the help anyway!
Visit my blog at http://calsmurf2904.wordpress.com !
Got a Google Wave account? Add me as contact! (projectxgame <at> gmail <dot> com)
Image
Post Reply