2D Shadows

Get answers to all your basic programming questions. No Ogre questions, please!

2D Shadows

Postby calsmurf2904 » Mon Mar 15, 2010 6:17 pm

Hello,

I'm currently in the process of learning D3D while still creating my projects with Ogre. (I think its some kind of plus to know the lower-level api's).
My current code uses the ID3DXSprite class to draw the 2d elements, but how could one add shadows in this way?
I found a few articles, like this:
http://www.gamedev.net/reference/programming/features/2dsoftshadow/
http://www.catalinzima.com/?page_id=313

But they are all for when you are using vertices, which I'm clearly not. (Well, ID3DXSprite uses it internal maybe, but I'm not using it directly).
How could one handle 2d shadows when the only resources for the light is the position, range, color and the pixels of the screen?

Thanks in advance,
Calsmurf2904
Visit my blog at http://calsmurf2904.wordpress.com !
Got a Google Wave account? Add me as contact! (projectxgame <at> gmail <dot> com)
Image
User avatar
calsmurf2904
Orc
 
Posts: 401
Kudos: 0
Joined: 16 Sep 2008
Location: Netherlands

Re: 2D Shadows

Postby vitefalcon » Tue Mar 16, 2010 8:02 pm

What I've done is to make a material which uses a shader to do the first pass and draw a shadow, while the second pass draws the sprite.

Shadow2D.cg
Code: Select all
void Shadow2D_VP(
   float4 position : POSITION,
   float2 uv      : TEXCOORD0,

   out float4 outPos : POSITION,
   out float2 outUv : TEXCOORD0,
   
   uniform float4x4 worldViewProj,
   uniform float4 shadowShift
   )
{
   outPos = mul(worldViewProj,position) + shadowShift;
   outUv = uv;
}


void Shadow2D_FP(
   float2 uv : TEXCOORD0,
   
   out float4 outColour : COLOR,
   
   uniform sampler2D panelTexture,
   uniform float4 shadowColour
   )
{
   float4 colour = tex2D(panelTexture, uv);
   float4 alpha = float4(colour.a, colour.a, colour.a, colour.a);
   outColour = shadowColour * alpha;
}


Material File
Code: Select all
vertex_program Shadow2D/CasterVP cg
{
   source Shadow2D.cg
   entry_point Shadow2D_VP
   profiles vs_2_x arbvp1
   
   default_params
   {
      param_named_auto worldViewProj worldviewproj_matrix
      // The shadow shift is in projection coordinates.
      param_named shadowShift float4 0.05 -0.05 0.0 0
   }
}

fragment_program Shadow2D/CasterFP cg
{
   source Shadow2D.cg
   entry_point Shadow2D_FP
   profiles ps_2_x arbfp1
   
   default_params
   {
      param_named shadowColour float4 0.0 0.0 0.0 0.5
   }
}

material Shadow2D/Template
{
   transparency_casts_shadows on
   
   technique
   {
      pass ShadowPass
      {
         lighting off
         scene_blend alpha_blend
         depth_write off
         depth_check on
         
         texture_unit ShadowMask
         {
            texture DEFAULT_SHADOW_MASK.png
            tex_address_mode clamp
         }
         
         vertex_program_ref Touchscape/Shadow2D/CasterVP
         {
         }
         
         fragment_program_ref Touchscape/Shadow2D/CasterFP
         {
         }
      }
      pass ObjectRender
      {
         lighting off
         depth_write on
         depth_check on
         separate_scene_blend one zero src_alpha one_minus_src_alpha
         alpha_rejection greater_equal 128
         
         texture_unit Image2D
         {
            texture DEFAULT_IMAGE.png
            tex_address_mode clamp
         }
      }
   }
}


Now if you want to have a material with shadows, do this:
Code: Select all
material MySpriteMaterial : Shadow2D/Template
{
   set_texture_alias ShadowMask MySpriteShadowMask.png
   set_texture_alias Image2D MySpriteImage.png
}


This is a very cheap way of doing it and should work even if you're not explicitly using vertices because I believe underneath ID3DXSprite it does use a quad to represent the sprite (two triangles to be precise). The code above is in terms of Ogre material. Note that the light direction is hard-coded in the material file. However, you should be able to change them from your code. Also, if you're not using Ogre3D, I hope the Cg program will still spark some ideas to help you resolve your issue.

[Edit]You can see this shadow working in one of the videos I've posted to explain my issue with matrices: here.[/Edit]
Image
User avatar
vitefalcon
Orc
 
Posts: 438
Kudos: 13
Joined: 18 Sep 2007
Location: Seattle, USA

Re: 2D Shadows

Postby calsmurf2904 » Wed Mar 17, 2010 6:49 am

Yea, I saw that method somewhere else...but it didn't really fit my needs.
What I now did was create a new rendertarget, create the vertexes of the sprite that ID3DXSprite uses. (So just four vertexes). Pass these over to the shadow function and let it render to the render target.
Then blend the render target with the scene.

This works (well kinda, no soft shadows :() but for some reason the shadow "jumps" over when moving from a specific angle from the light to another angle from the light. Perhaps someone could help me with this? I'll post the code later.
Visit my blog at http://calsmurf2904.wordpress.com !
Got a Google Wave account? Add me as contact! (projectxgame <at> gmail <dot> com)
Image
User avatar
calsmurf2904
Orc
 
Posts: 401
Kudos: 0
Joined: 16 Sep 2008
Location: Netherlands

Re: 2D Shadows

Postby calsmurf2904 » Wed Mar 17, 2010 5:50 pm

Alright, fixed it all. My ShadowMap generation code is: (kinda messy, and has to be optimized, but it works):
Code: Select all
if(lightMap && alphaOne && lights.size() > 0)
   {
      LPDIRECT3DSURFACE9 backBuffer;
      LPDIRECT3DSURFACE9 lightmapSurface;
      dxDevice->GetRenderTarget(0, &backBuffer);

      lightMap->GetSurfaceLevel(0, &lightmapSurface);
      dxDevice->SetRenderTarget(0, lightmapSurface);
      dxDevice->SetFVF(D3DFVF_VERTEX);
      dxDevice->SetRenderState(D3DRS_CULLMODE, D3DCULL_NONE);

      //render the lightmap with 2d lights only
      dxDevice->Clear(0, 0, D3DCLEAR_TARGET | D3DCLEAR_ZBUFFER, D3DCOLOR_XRGB(35, 35, 35), 1.0f, 0);
      dxDevice->BeginScene();

      HRESULT hr = dxSprite->Begin(D3DXSPRITE_ALPHABLEND);
      if(hr != S_OK)
         std::cout << "D3D error: 0x" << std::hex << hr << std::endl;

      for(int l = 0;l<lights.size();l++)
      {
         D3DLight* light = lights[l];
         if(!light->Get2D())
            continue;

         ClearAlphaToOne(dxDevice, alphaOne, windows[0]->GetWidth(), windows[0]->GetHeight());
         dxDevice->SetRenderState(D3DRS_DESTBLEND, D3DBLEND_ZERO);
         dxDevice->SetRenderState(D3DRS_SRCBLEND, D3DBLEND_ONE);
         dxDevice->SetRenderState(D3DRS_ALPHABLENDENABLE, TRUE);
         dxDevice->SetRenderState(D3DRS_COLORWRITEENABLE, D3DCOLORWRITEENABLE_ALPHA);

         for(int i = 0;i<sprites.size();i++)
         {
            EngineSprite* sprite = sprites[i];
            if(!strcmp(sprite->GetTypeName(), "Text"))
               continue;

            if(sprite->GetZOrder() != light->GetPosition().absZ)
               continue;

            if(sprite->GetSize().absX == windows[0]->GetWidth())
               continue;

            if(!sprite->GetVisible())
               continue;

            bool backFacing[4];
            memset(backFacing, 0, 4);

            Vector2 vertices[4];
            memset(vertices, 0, 4*sizeof(Vector2));

            vertices[0].absX = -(sprite->GetSize().absX/2);
            vertices[0].absY = -(sprite->GetSize().absY/2);

            vertices[1].absX = sprite->GetSize().absX/2;
            vertices[1].absY = -(sprite->GetSize().absY/2);

            vertices[2].absX = sprite->GetSize().absX/2;
            vertices[2].absY = sprite->GetSize().absY/2;

            vertices[3].absX = -(sprite->GetSize().absX/2);
            vertices[3].absY = sprite->GetSize().absY/2;

            for (int i2 = 0; i2 < 4; i2++)
            {
               sprite = sprites[i];
               Vector2 firstVertex = Vector2(vertices[i2].absX, vertices[i2].absY) + sprite->GetPosition();
               int secondIndex = (i2 - 1) % 4;
               if(secondIndex < 0)
                  secondIndex = 3;
               else if(secondIndex > 3)
                  secondIndex = 0;
               Vector2 secondVertex = Vector2(vertices[secondIndex].absX, vertices[secondIndex].absY) + sprite->GetPosition();
               Vector2 middle = (firstVertex + secondVertex);
               middle.absX /= 2;
               middle.absY /= 2;

               Vector2 L = Vector2(light->GetPosition().absX, light->GetPosition().absY) - middle;

               Vector2 N = Vector2();
               N.absX = - (secondVertex.absY - firstVertex.absY);
               N.absY = secondVertex.absX - firstVertex.absX;
                  
               if (Vector2DotProduct(N, L) > 0)
                  backFacing[i2] = false;
               else
                  backFacing[i2] = true;
            }

            int startingIndex=0;
            int endingIndex=0;
            for (int i3 = 0; i3 < 4; i3++)
            {
               int currentEdge = i3;
               int nextEdge = (i3 - 1) % 4;
               if(nextEdge < 0)
                  nextEdge = 3;
               else if(nextEdge > 3)
                  nextEdge = 0;

               if (backFacing[currentEdge] && !backFacing[nextEdge])
                  endingIndex = nextEdge;

               if (!backFacing[currentEdge] && backFacing[nextEdge])
                  startingIndex = nextEdge;
            }

            int shadowVertexCount = 0;
            if(endingIndex > startingIndex)
               shadowVertexCount = endingIndex - startingIndex + 1;
            else
               shadowVertexCount = 5 - startingIndex + endingIndex + 1;

            InternalVertex* shadowVertices = new InternalVertex[shadowVertexCount * 2];

            //create a triangle strip that has the shape of the shadow
            int currentIndex = startingIndex;
            int svCount = 0;
            while (svCount != shadowVertexCount*2)
            {
               sprite = sprites[i];
               Vector2 vertexPos = vertices[currentIndex] + sprite->GetPosition();
                  
               //one vertex on the hull
               shadowVertices[svCount].color = 0x11000000;
               shadowVertices[svCount].x = vertexPos.absX;
               shadowVertices[svCount].y = vertexPos.absY;
               shadowVertices[svCount].z = 0;

               //one extruded by the light direction
               shadowVertices[svCount + 1].color = 0x11000000;
               Vector2 L2P = vertexPos - Vector2(light->GetPosition().absX, light->GetPosition().absY);
               L2P.Normalize();
               Vector2 finalPos = vertexPos + L2P * 128000;
               shadowVertices[svCount + 1].x = finalPos.absX;
               shadowVertices[svCount + 1].y = finalPos.absY;
               shadowVertices[svCount + 1].z = 0;
                
               svCount+=2;
               currentIndex = (currentIndex + 1) % 4;
            }

            dxDevice->SetTexture(0, NULL);
            HRESULT hr = dxDevice->DrawPrimitiveUP(D3DPT_TRIANGLESTRIP, shadowVertexCount*2-2, shadowVertices, sizeof(InternalVertex));
            if(hr != S_OK)
               std::cout << "D3D error: 0x" << std::hex << hr << std::endl;

            delete shadowVertices;
         }

         dxDevice->SetRenderState(D3DRS_DESTBLEND, D3DBLEND_ONE);
         dxDevice->SetRenderState(D3DRS_SRCBLEND, D3DBLEND_DESTALPHA);
         dxDevice->SetRenderState(D3DRS_BLENDOP,  D3DBLENDOP_ADD);
         dxDevice->SetRenderState(D3DRS_SEPARATEALPHABLENDENABLE, TRUE);
         dxDevice->SetRenderState(D3DRS_COLORWRITEENABLE, D3DCOLORWRITEENABLE_ALPHA | D3DCOLORWRITEENABLE_RED | D3DCOLORWRITEENABLE_GREEN | D3DCOLORWRITEENABLE_BLUE);

         int vertexCount = (360/(360.0f/32.0f))+3.5f;
         InternalVertex* vertexes = new InternalVertex[vertexCount];

         vertexes[0].x = light->GetPosition().absX;
         vertexes[0].y = light->GetPosition().absY;
         vertexes[0].z = 0;
         vertexes[0].color = D3DCOLOR_ARGB(((int)((light->GetIntensity()*255)+0.5)), light->GetColor().r, light->GetColor().g, light->GetColor().b);

         int count = 1;
         for (float angle=0; angle<360; angle+=(360/32) )
         {
            vertexes[count].x = light->GetRange()*cos((M_PI/180)*angle)+light->GetPosition().absX;
            vertexes[count].y = light->GetRange()*sin((M_PI/180)*angle)+light->GetPosition().absY;
            vertexes[count].z = 0;
            vertexes[count].color = D3DCOLOR_ARGB(0, 0, 0, 0);
            count++;
         }

         vertexes[count].x = light->GetRange()*cos((M_PI/180)*0)+light->GetPosition().absX;
         vertexes[count].y = light->GetRange()*sin((M_PI/180)*0)+light->GetPosition().absY;
         vertexes[count].z = 0;
         vertexes[count].color = D3DCOLOR_ARGB(0, 0, 0, 0);

         dxDevice->SetTexture(0, NULL);
         HRESULT hr = dxDevice->DrawPrimitiveUP(D3DPT_TRIANGLEFAN, vertexCount-2, vertexes, sizeof(InternalVertex));
         if(hr != S_OK)
            std::cout << "D3D error: 0x" << std::hex << hr << std::endl;

         delete vertexes;
      }

      ClearAlphaToOne(dxDevice, alphaOne, windows[0]->GetWidth(), windows[0]->GetHeight());

      dxSprite->End();
      dxDevice->EndScene();
      dxDevice->Present(NULL, NULL, NULL, NULL);

      dxDevice->SetRenderTarget(0, backBuffer);
   }


Where ClearAlphaToOne does this:
Code: Select all
void ClearAlphaToOne(LPDIRECT3DDEVICE9 dxDevice, LPDIRECT3DTEXTURE9 texture, int width, int height)
{
   dxDevice->SetRenderState(D3DRS_COLORWRITEENABLE, D3DCOLORWRITEENABLE_ALPHA);

   InternalVertex* vertexes = new InternalVertex[6];
   vertexes[0].x = vertexes[3].x = vertexes[5].x = 0.0f;
   vertexes[0].y = vertexes[1].y = vertexes[3].y = 0.0f;
   vertexes[1].x = vertexes[2].x = vertexes[4].x = width;
   vertexes[2].y = vertexes[4].y = vertexes[5].y = height;
   vertexes[0].z = vertexes[1].z = vertexes[2].z = vertexes[3].z = vertexes[4].z = vertexes[5].z = 0.0f;

   vertexes[0].u = vertexes[3].u = vertexes[5].u = 0.0f;
   vertexes[0].v = vertexes[1].v = vertexes[3].v = 0.0f;
   vertexes[1].u = vertexes[2].u = vertexes[4].u = 1.0f;
   vertexes[2].v = vertexes[4].v = vertexes[5].v = 1.0f;

   dxDevice->SetFVF(D3DFVF_VERTEX);
   dxDevice->SetTexture(0, texture);
   dxDevice->DrawPrimitiveUP(D3DPT_TRIANGLELIST, 2, vertexes, sizeof(InternalVertex));

   delete vertexes;
   dxDevice->SetRenderState(D3DRS_COLORWRITEENABLE, D3DCOLORWRITEENABLE_ALPHA | D3DCOLORWRITEENABLE_RED | D3DCOLORWRITEENABLE_GREEN | D3DCOLORWRITEENABLE_BLUE);
}


And the texture alphaOne is just a texture the size of the screen filled with 255, 255, 255, 255 ARGB pixels.
The only downside is that it doesn't work with rotation yet, I need to transform every vertex position to reflect the rotation, but haven't done this yet.

This looks like this:
http://img705.imageshack.us/img705/6239/2dshadows4.png

PS:
For anybody wondering, I am drawing the shadow map with this:
Code: Select all
if(lightMap)
   {
      dxDevice->SetRenderState(D3DRS_DESTBLEND, D3DBLEND_SRCCOLOR);
      dxDevice->SetRenderState(D3DRS_SRCBLEND, D3DBLEND_ZERO);
      dxDevice->SetRenderState(D3DRS_BLENDOP, D3DBLENDOP_ADD);

      D3DXMATRIX matrix;
      D3DXMatrixTranslation(&matrix, 0, 0, 0.1f);
         
      dxDevice->SetTexture(0, lightMap);
      dxDevice->SetFVF(D3DFVF_VERTEX);
      dxDevice->SetTransform(D3DTS_WORLD, &matrix);
      dxDevice->SetStreamSource(0, shadowTexture, 0, sizeof(InternalVertex));
      dxDevice->DrawPrimitive(D3DPT_TRIANGLELIST, 0, 2);

      dxDevice->SetRenderState(D3DRS_DESTBLEND, D3DBLEND_ZERO);
      dxDevice->SetRenderState(D3DRS_SRCBLEND, D3DBLEND_ONE);
      dxDevice->SetRenderState(D3DRS_BLENDOP,  D3DBLENDOP_MIN);
   }


Next thing to do is getting soft shadows working. And specular mapping.

How I got this code is just porting over this XNA C# code over to C++.
http://www.catalinzima.com/?page_id=313

As seen in the screenshot it isn't really correct, since I assume the sprite is a rectangle instead of any other shape.

[edit]
Edited it all a bit to make the lights support colors, and the shadows aren't transparent anymore. (They just block the light where the shadow came from fully now, which is realistic).

@vitefalcon: Your example looks cool, and works with rotation. But for a 2d top-down game the approach above is more fitting then your approach. (Yours is good with a RPG though and possibly a platformer). It mostly just depends on the game :) Thanks for the help anyway!
Visit my blog at http://calsmurf2904.wordpress.com !
Got a Google Wave account? Add me as contact! (projectxgame <at> gmail <dot> com)
Image
User avatar
calsmurf2904
Orc
 
Posts: 401
Kudos: 0
Joined: 16 Sep 2008
Location: Netherlands


Return to Back to Basics

Who is online

Users browsing this forum: Yahoo [Bot] and 1 guest