Shadows - Differences between 1.9 and 2.1

Discussion area about developing with Ogre-Next (2.1, 2.2 and beyond)


Post Reply
ChicChic
Kobold
Posts: 26
Joined: Tue Sep 05, 2017 10:19 am
x 7

Shadows - Differences between 1.9 and 2.1

Post by ChicChic »

Hi,

We're upgrading our Ogre version in our software from 1.9 to 2.1.
Everything goes OK except for two things :
- First, PSSM shadow map differs even if the exact same settings and points of view are applied to both 1.9 and 2.1.
It seems that the camera's shadow map view is wider in 2.1 which make the shadow a lot uglier.

- If I put a point light with DPSM shadow compositor as described in the Shadow sample behind a transparent glass there are huge artifact on the shadow of this transparent mesh...

Is there someone experiencing the same issues ?
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Shadows - Differences between 1.9 and 2.1

Post by dark_sylinc »

ChicChic wrote:Hi,

We're upgrading our Ogre version in our software from 1.9 to 2.1.
Everything goes OK except for two things :
- First, PSSM shadow map differs even if the exact same settings and points of view are applied to both 1.9 and 2.1.
It seems that the camera's shadow map view is wider in 2.1 which make the shadow a lot uglier.
PSSM shadows in 1.9 were using LiPSM as its backend which in certain specific scenarios could improve quality, however it was full of bugs which were causing lots of shadow artifacts (very noticeable missing or clipped shadow, flickering shadows, entire regions suddenly covered in shadow when they shouldn't, extreme quality problems with certain lighting angles) which is why we changed to using an improved Focused setup. Thus comparing 1:1 is not possible.

I suggest you tweak the parameters again until you find something that satisfies you. Resolution, shadow extrusion distance and pssm lambda tends to have a huge impact on quality.
If your scene is very big, it's possible the terrain casting shadows is forcing PSSM to get very wide, which is why I recommend alternative options for shadowing the terrain (the terrain sample provides an example)
ChicChic wrote: - If I put a point light with DPSM shadow compositor as described in the Shadow sample behind a transparent glass there are huge artifact on the shadow of this transparent mesh...

Is there someone experiencing the same issues ?
Could you provide a screenshot and/or a way to reproduce? It's weird because transparent meshes shouldn't be casting shadows.
ChicChic
Kobold
Posts: 26
Joined: Tue Sep 05, 2017 10:19 am
x 7

Re: Shadows - Differences between 1.9 and 2.1

Post by ChicChic »

PSSM shadows in 1.9 were using LiPSM as its backend which in certain specific scenarios could improve quality, however it was full of bugs which were causing lots of shadow artifacts (very noticeable missing or clipped shadow, flickering shadows, entire regions suddenly covered in shadow when they shouldn't, extreme quality problems with certain lighting angles) which is why we changed to using an improved Focused setup. Thus comparing 1:1 is not possible.
I understand what you say but there is a huge difference ! Here is an exemple of 1.9 vs 2.1 difference (img shadow_1 and shadow_2), it seems that all the issues you are telling me appears in 2.1 for my case instead of 1.9 ?!?! First plan is not very beautiful and second plan (roofs of houses, the distant road sign) contain a lot of incoherency.
I suggest you tweak the parameters again until you find something that satisfies you. Resolution, shadow extrusion distance and pssm lambda tends to have a huge impact on quality.
If your scene is very big, it's possible the terrain casting shadows is forcing PSSM to get very wide, which is why I recommend alternative options for shadowing the terrain (the terrain sample provides an example)
I did tried a lot of tweaks before posting... :cry: PSSM lambda, Far/Near shadow distance, resolution, etc. It even seems that the light's node orientation without changing light's direction vector affect the quality of the shadows :shock: (lots lots lots of try...) !!!!

And i'm not making a video game, I'm building a modular professional robotic simulator so the terrain example/heightmap can not apply for our needs... And output camera's picture are used by scientific robotic algorithms so they must be realistic enough to provide the same result in real than in simulation.
Could you provide a screenshot and/or a way to reproduce? It's weird because transparent meshes shouldn't be casting shadows.
here it is : file point_shadow. Already tried some HLMS/PBS transparent/not transparent etc. combination... maybe I forgot some things...

Anyway, thanks for giving me some of your time, and thanks a lot for your future advice :wink:
Attachments
2.1 Shadows
2.1 Shadows
1.9 Shadows
1.9 Shadows
point_shadow.jpg
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Shadows - Differences between 1.9 and 2.1

Post by dark_sylinc »

Hi!
ChicChic wrote: I did tried a lot of tweaks before posting... :cry: PSSM lambda, Far/Near shadow distance, resolution, etc. It even seems that the light's node orientation without changing light's direction vector affect the quality of the shadows :shock: (lots lots lots of try...) !!!!
If you're using compositor scripts, could you paste the contents of the shadow nodes you're using?

Additionally, I'm not sure if you're doing this, as it may not be obvious... shadow distance is controlled via these two calls:

Code: Select all

//Set sane defaults for proper shadow mapping
mSceneManager->setShadowDirectionalLightExtrusionDistance( 500.0f );
mSceneManager->setShadowFarDistance( 500.0f );
These two can have a profound change in quality.

Regarding the point shadow... yeah that looks weird. Perhaps something obvious pops up when you post the compositor script. Otherwise the best I can think of is to take a RenderDoc capture to see what's being rendered onto the point's shadow map. It looks like it's either uninitialized, or some weird geometry is being drawn into it.
ChicChic
Kobold
Posts: 26
Joined: Tue Sep 05, 2017 10:19 am
x 7

Re: Shadows - Differences between 1.9 and 2.1

Post by ChicChic »

Hi dark_sylinc,
If you're using compositor scripts, could you paste the contents of the shadow nodes you're using?
Here it is !

Code: Select all

compositor_node MainNode1
{
	in 0 rt_output
	texture rt_visual target_width target_height PF_R8G8B8A8 no_fsaa depth_texture depth_format PF_D32_FLOAT depth_pool 1
	texture rt_debug target_width target_height PF_R8G8B8A8 no_fsaa depth_pool 1
	target rt_visual
	{
		pass clear
		{
			colour_value 0.0 0.0 0.0 0.0
		}
		pass render_quad
		{
			quad_normals camera_direction
			material SkyPostprocess
		}
		pass render_scene
		{
			overlays off
			rq_first 0
			rq_last 100
			shadows ShadowNode1
		}
	}
	target rt_debug
	{
		pass clear
		{
			colour_value 0.0 0.0 0.0 0.0
		}
		pass render_scene
		{
			//renderQueue DEBUG - Yardstick
			lod_update_list	off
			overlays on
			rq_first 100
			rq_last max
		}
	}
	target rt_output
	{
		pass clear
		{
		}
		pass render_quad
		{
			//Blending renderTargets : RT0 + overlayTexture
			material Overlay
			input 0 rt_visual
			input 1 rt_debug
		}
	}
}
compositor_node_shadow ShadowNode1
{
	technique pssm
	texture atlas 2048 3072 PF_D32_FLOAT no_fsaa
	num_splits 3
	pssm_lambda 0.95
	shadow_map 0 atlas uv 0.0 0.0 1.000000000 0.6666666667 light 0 split 0
	shadow_map 1 atlas uv 0.0 0.6666666667 0.5000000000 0.3333333333 light 0 split 1
	shadow_map 2 atlas uv 0.5000000000 0.6666666667 0.5000000000 0.3333333333 light 0 split 2
	target atlas
	{
		pass clear
		{
			colour_value 1.0 1.0 1.0 1.0
		}
	}
	shadow_map_target_type directional 
	{
		shadow_map 0 1 2 
		{
			pass render_scene
			{
				rq_first 0
				rq_last 80
			}
		}
	}
}
workspace RobinCompositorMain1
{
	connect_output MainNode1 0
}
Additionally, I'm not sure if you're doing this, as it may not be obvious... shadow distance is controlled via these two calls:

Code: Select all

//Set sane defaults for proper shadow mapping
    mSceneManager->setShadowDirectionalLightExtrusionDistance( 500.0f );
    mSceneManager->setShadowFarDistance( 500.0f );
These two can have a profound change in quality.
Sadly they were already set up to 200 meters for both and the near clip to 1.0... Also tried to change the camera's if there were any "unexpected contamination". sorry :(
Regarding the point shadow... yeah that looks weird. Perhaps something obvious pops up when you post the compositor script. Otherwise the best I can think of is to take a RenderDoc capture to see what's being rendered onto the point's shadow map. It looks like it's either uninitialized, or some weird geometry is being drawn into it.
Here is the compositor when a point light is present without directional light (our product generates it at start depending of the configuration of all lights)

Code: Select all

compositor_node MainNode1
{
	in 0 rt_output
	texture rt_visual target_width target_height PF_R8G8B8A8 no_fsaa depth_texture depth_format PF_D32_FLOAT depth_pool 1
	texture rt_debug target_width target_height PF_R8G8B8A8 no_fsaa depth_pool 1
	target rt_visual
	{
		pass clear
		{
			colour_value 0.0 0.0 0.0 0.0
		}
		pass render_quad
		{
			quad_normals camera_direction
			material SkyPostprocess
		}
		pass render_scene
		{
			overlays off
			rq_first 0
			rq_last 100
			shadows ShadowNode1
		}
	}
	target rt_debug
	{
		pass clear
		{
			colour_value 0.0 0.0 0.0 0.0
		}
		pass render_scene
		{
			//renderQueue DEBUG - Yardstick
			lod_update_list	off
			overlays on
			rq_first 100
			rq_last max
		}
	}
	target rt_output
	{
		pass clear
		{
		}
		pass render_quad
		{
			//Blending renderTargets : RT0 + overlayTexture
			material Overlay
			input 0 rt_visual
			input 1 rt_debug
		}
	}
}
abstract target cubemap_target_shadow
{
	pass clear
	{
		colour_value 1.0 1.0 1.0 1.0
	}
	pass render_scene
	{
		camera_cubemap_reorient true
		rq_first 0
		rq_last 80
	}
}
compositor_node_shadow ShadowNode1
{
	texture atlas 512 512 PF_D32_FLOAT no_fsaa
	texture tmpCubemap 512 512 PF_FLOAT32_R cubemap no_fsaa
	technique focused
	shadow_map 0 atlas uv 0.000000000 0.000000000 1.000000000 1.000000000 light 0
	target atlas
	{
		pass clear
		{
			colour_value 1.0 1.0 1.0 1.0
		}
	}
	shadow_map_target_type point
	{
		shadow_map_repeat 0 
		{
			target tmpCubemap +X : cubemap_target_shadow {}
			target tmpCubemap -X : cubemap_target_shadow {}
			target tmpCubemap +Y : cubemap_target_shadow {}
			target tmpCubemap -Y : cubemap_target_shadow {}
			target tmpCubemap +Z : cubemap_target_shadow {}
			target tmpCubemap -Z : cubemap_target_shadow {}
			shadow_map
			{
				pass render_quad
				{
					material Ogre/DPSM/CubeToDpsm
					input 0 tmpCubemap
				}
			}
		}
	}
}
workspace RobinCompositorMain1
{
	connect_output MainNode1 0
}
The point is : it's working for spot lights ! but not point lights... At first I have bet on DPSM shader but I don't know what would the problem be !

Again, thanks for your time and all your work !
I didn't knew about RenderDoc capture, I'll check that... 8)
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Shadows - Differences between 1.9 and 2.1

Post by dark_sylinc »

Dang it! I was hoping to spot an obvious error in your script(s).

What I did notice is that for your splits are:
  1. 2048x2048
  2. 1024x1024
  3. 1024x1024
For large scenes this resolution is too low. I have a very large scene where I set all 3 splits to 2048x2048.

One thing I forgot to ask: Do you have objects with infinite AABBs? Infinite AABBs tend to play very bad with shadow mapping code, causing quality to degrade in 2.1; whereas in LiPSM in 1.x; it would cause a lot bugs but also quality to (wrongly) improve a lot by random accident (since the NaNs in certain places would behave like 0s as if there were no casters).

Could you setup the shadow map debugging and post a picture of all the splits? perhaps something obvious gets revealed.

Edit: Additionally use HlmsPbs::setDebugPssmSplits(true) to help you see if the PSSM splits are well distributed.
ChicChic
Kobold
Posts: 26
Joined: Tue Sep 05, 2017 10:19 am
x 7

Re: Shadows - Differences between 1.9 and 2.1

Post by ChicChic »

What I did notice is that for your splits are:

2048x2048
1024x1024
1024x1024


For large scenes this resolution is too low. I have a very large scene where I set all 3 splits to 2048x2048.
I already tried that, of course it is a little more beautiful but still far from 1.9...
Could you setup the shadow map debugging and post a picture of all the splits? perhaps something obvious gets revealed.

Already done but I didn't want to post it at first, I join the shadow map debug for 2.1 and the first split of the 1.9 (don't want to take much more time to show it the same way than 2.1).
You can see why I said that
the camera's shadow map view is wider in 2.1 which make the shadow a lot uglier
.
That's also why I said that
It even seems that the light's node orientation without changing light's direction vector affect the quality of the shadows
: I visually checked it :shock:

I'm curently triing to understand the point light problem with RenderDoc capture but I'm not very familiar to deep OpenGL functions (that's why I use Ogre 3D :mrgreen: ).

Last but not least, I will check for the infinite AABB in my project, this is a pertinent new clue for me to test !
Attachments
Shadow map debug in 2.1 (all split)
Shadow map debug in 2.1 (all split)
First shadow split in 1.9
First shadow split in 1.9
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Shadows - Differences between 1.9 and 2.1

Post by dark_sylinc »

Mmmm... the first split is gigantic. The idea of PSSM is that the first the first split covers close objects, the 2nd split covers what's in your vecinity, and the 3rd split covers what's too distant to care about details, so the third split covers much more.

I'd suggest a breakpoint inside PSSMShadowCameraSetup::getShadowCamera and inspect the values of mSplitPoints. They should be in harmony with what you'd expect from setting the near and far distance e.g. mSplitPoints should contain values like { 0.2, 10.91, 51.66, 500 } where 500 in the value from mSceneManager->setShadowFarDistance( 500 ).
If they're wrong (e.g. it goes up to 1500), then you're incorrectly setting up something.

If they're correct, then I'd suggest increasing pssm_lambda to push the split points closer to you and get better quality on close objects.
ChicChic
Kobold
Posts: 26
Joined: Tue Sep 05, 2017 10:19 am
x 7

Re: Shadows - Differences between 1.9 and 2.1

Post by ChicChic »

I'd suggest a breakpoint inside PSSMShadowCameraSetup::getShadowCamera and inspect the values of mSplitPoints. They should be in harmony with what you'd expect from setting the near and far distance e.g. mSplitPoints should contain values like { 0.2, 10.91, 51.66, 500 } where 500 in the value from mSceneManager->setShadowFarDistance( 500 ).
If they're wrong (e.g. it goes up to 1500), then you're incorrectly setting up something.
I've checked that last week, the values seems good : [0.1 4.53359 21.7486 200] and they are exactly the same in 1.9 :cry:
If they're correct, then I'd suggest increasing pssm_lambda to push the split points closer to you and get better quality on close objects.
PSSM Lamba is set to 0.95 in both 1.9 and 2.1, so already pretty high !

At least it seems that, since the beginning, I'm looking for the mistakes at the right place ! But with no results...
Always look on the bright side of life !!!! :lol:

Haven't tested all AABB for now, I'll check that tomorrow, working reason...
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: Shadows - Differences between 1.9 and 2.1

Post by dark_sylinc »

I just pushed an important fix to shadow mapping precision I've been wanting to correct for quite some time.

Could you please check if this enhancement positively affects your issue in any way?
Thanks
ChicChic
Kobold
Posts: 26
Joined: Tue Sep 05, 2017 10:19 am
x 7

Re: Shadows - Differences between 1.9 and 2.1

Post by ChicChic »

:cry:
I had lots of hopes when looking at your commit but it hasn't changed anything for my directional light...

I also tried my point light problem again (even if its not correlated) and the problem has gone ?!?!? :shock: I don't know if that came from Ogre or me (I don't remember to have changed anything...).

Anyway... thank you !
Post Reply