Terrain Texturing Problem ? [Solved]

You can see some areas render as a white area. Playing with things it is just overly bright, it is actually rendering the correct texture just extremely bright. also I should mention that it is not a page boundary, it is in the middle of a page. Though there is a line in the texturing at the page boundaries as well.

My config file looks like:
GroupName=PLSM2

LandScapeFileName=NewWorld
FileSystem=LandScapeFileName
LandScapeExtension=RAW

Width=16
Height=16
RawWidth=513
RawHeight=513


Data2DFormat=HeightFieldRaw

ScaleX=11250
ScaleY=5500
ScaleZ=11250


NumTextureFormatSupported=1

#TextureFormatSupported0=ImagePaging

#TextureFormat=ImagePaging

TextureFormatSupported0=PLSplattingShaderUnlit
TextureFormat=PLSplattingShaderUnlit

#TextureFormatSupported0=PLSplattingShaderLit
#TextureFormat=PLSplattingShaderLit

NumMatHeightSplat=4

MaterialHeight1=50
MaterialHeight2=70
MaterialHeight3=80

SplatFilename0=grass1-dry.jpg
SplatFilename1=grass1.jpg
SplatFilename2=yellowsand1.jpg
SplatFilename3=grass1-lush.jpg

VisibleRenderables=75

NumRenderablesLoading=10
RenderableLoadInterval=3

MaxAdjacentPages=1
MaxPreloadedPages=2
PageLoadInterval=5

MaxNumRenderables=256
IncrementRenderables=256
MaxNumTiles=256
IncrementTiles=256

HorizonVisibilityComputing=yes

VertexCompression=yes
VertexLit=yes
VertexProgramMorph=yes
ZHorizon=yes

#HeightMapBlurFactor=0.5f

# where to put camera on load.
BaseCameraViewpoint.x=-4702.0f
BaseCameraViewpoint.y=250245.0f
BaseCameraViewpoint.z=-19140.0f

Baselookat.x=0.0f
Baselookat.y=0.0f
Baselookat.z=0.0f


The lit version of the shader does the same and worse so I figure the best place to start is to fix the unlit version first.

Another note for others fighting with PLSM2, the coverage says it expects NewWorld.Coverage.0.0.png etc. but it really needs NewWorld.Coverage.0.0.0.png etc. whare the first one is always 0. Not sure why since the code does not look like it works this way but if you don't Ogre will report that NewWorld.Coverage.0.0.0.png etc can't be loaded.

In the lit version I have noticed that the material uses scene_blend add to add the directional lighting. This doesn't seem right to me ( nor does it look right since the world is rendered as pure white ). I changed it to modulate which looks better but still doesn't seem right. Seems to me Ambient and Directional should be computed at the same time to avoid over lighting. Also it seems the shader adds the color:

float4 color = float4(0.0, 0.0, 0.0, 0.0);

if(nDotL > 0.0)
{

//add diffuse light from material and light
color += lightDiffuse * matDiffuse * nDotL;
}


Which has no effect really since color starts at 0 anyways, this leads me to believe there is a logic problem here.

It appears the problem is from trying to make my own coverage maps. Using the ones from LT3D work better. However if I use PLSplattingShaderLitDecompress the landscape is all washed out and very white. I cannot figure out why but the unlit version is fine.

However if I use PLSplattingShaderLitDecompress the landscape is all washed out and very white
weird.
did you try tweaking the PLSplattingShaderLitDecompress shader code a bit ?
What is your video card ?

Sorry I should have marked this as solved.

It is a problem with the shaders. The shaders are good for what GOOF needs but otherwise kinda useless. I altered the shader to only write to the depth buffer in the Ambient pass and then pass the Ambient into to the directional pass so it can actually be computed correctly. This makes a nice scene, I still get a little dark shading spots on the terrian at certian times of day but I'm convinced I'll fix that eventually.

I understand what Falagard is trying for in the shader but the overall logic is just wrong. Seperating the ambient pass out but not drawing the textures means that the ambient pass only controls how much you want the terrian washed out instead of lit.

He doesn't seem to want or need the ambient lighting but realistically you have to use it. Using a directional light only for light means that you need to have two opposing directional lights with one much dimmer than the main otherwise the side of the entities that is not facing the directional light are just black. It is easier and cheaper to just use ambient lighting to get this effect.

Another weird but unrelated item though is that my frame rate is higher in PLSM2 than it was in the Terrain Scene Manager BUT Suddenly the framerate has dropped dramatically and it says it has loaded sixty some pages and eventually it will crash or the computer will reboot as there is no more video memory available.

PatrickB3 - I want and need to use ambient lighting, it was just done incorrectly in the first iteration of the shaders. The way I wrote the shaders was to use a pass per light. There is an alternative to this, which is to create a shader that does everything in one pass but hard codes the number of lights.

If you're doing a pass per light, the ambient must be done in a first pass, and then the lights in subsequent passes without the ambient also being in these subsequent passes because it'll overbrighten the scene way too much because it's additive.

Using an ambient pass as the first pass is the correct way to do it when doing pass per light. What I didn't do correctly was use the splatting texture (or base texture would be preferrable) in the ambient pass, and since I haven't been using PLM2 for a couple months now, I haven't gone back to fix it.

Use whatever works for you, but don't make assumptions on about whether I need ambient or not. I'm pretty sure I've already described all of this before.

It's not brain surgery - here's a quick attempt - the ambient pass on the first technique of PLSplattingShaderLitDecompress:


pass AmbientPass
{
// These base colors identify the pass to Ogre as an ambient only pass
// but are ignored by the shaders
ambient 1 1 1
emissive 0 0 0
diffuse 0 0 0
specular 0 0 0 0

// Ambient and emissive pass vertex program
vertex_program_ref PLDecompressAmbientVPCG
{
param_named_auto worldViewProj worldviewproj_matrix
param_named_auto globalAmbient ambient_light_colour
param_named matAmbient float4 1 1 1 1
param_named matEmissive float4 0 0 0 0
param_named compressionSettings float4 90000.0 1.37331 90000.0 0.0
}

fragment_program_ref PLDecompressAmbientFP2CG
{
}

shadow_receiver_vertex_program_ref DecompressVertexShadowReceiver/VP
{
param_named_auto worldViewProj worldviewproj_matrix
param_named_auto worldMatrix world_matrix
param_named_auto texViewProjMatrix texture_viewproj_matrix

// Config settings are;
// x scale
// y (height) start
// z scale
//
param_named compressionSettings float4 90000.0 1.37331 90000.0 50000.0
}


PLDecompressAmbientFP2.cg


void main
(
float2 iTexCoord0 : TEXCOORD0,
float4 iColor : COLOR,

out float4 oColor : COLOR,

uniform sampler2D baseMap
)
{

oColor = iColor * tex2D(baseMap, iTexCoord0);
}


Add it to PLShaders.program:


fragment_program PLDecompressAmbientFP2CG cg
{
source PLDecompressAmbientFP2.cg
entry_point main
profiles ps_2_0 arbfp1
}


I'm convinced the dark patches are due to normals on the terrain being incorrect because of what tuan described. Not sure what to do about them.

Didn't mean for it to sound or be taken as a negative about the ambient light. Just that different apps have different needs. I thought about adding the textures to the ambient pass but I assume that would slow things down a bit ( or even a lot ). In my scene I know there is only one directional light so moving the ambient into the directional works for me. But as you stated it would not work if I had multiple directional lights.

I agree about the dark areas as I thought that when I saw his post too, not really sure how to get around that one. I was kinda hoping it was something I had done which would have been easier to fix.

No worries mate, my problem was with your assumption that I didn't want or need ambient, and I have never said that.

Most people won't need multiple directional lights, but some may need multiple point or spot lights (for things like street lamps, camp fires, or casting magical spell effects or gunfire), and the only way to do "N" number of lights is with the following type of material: ambient pass, directional light passes, point light passes, spot light passes. You can always write a shader that hard codes a certain number, and eventually I'd like to do that as well, but I wrote this particular shader to be flexible.

Using an ambient pass, even one with a texture bound to it, will probably speed up your rendering, not slow it down. It would take some testing to determine whether the pixel shader in PLM2 is "heavy" enough to take advantage of early z out rendering.

As I think I have tried to explain before, Ogre's render system is built so that anything that is flagged as the ambient pass gets rendered before all other passes. The purpose of this is to lay down the depth buffer so that subsequent pixel shader heavy passes, (such as the splatting + directional light pass, splatting + point light pass, etc.) will only render the pixels that are absolutely necessary. This is called "early z out". That initial ambient pass may actually speed up tbe rendering of your scene.

As you can see, there are reasons for doing things the way that I did them, I just didn't fix the ambient part of that shader. I'll go fix it next time I get a chance.

Not sure what to do about them
Normal in vertex shaders are to be wrong when using index buffer based Lod.
The only way is to use normal maps.

Ya, I've heard you say that before, tuan, but ... normal maps per pixel on a large terrain? Sounds insane. I'll give it a try some time though and see how it goes.

It's as precise as vertex if at same resolution as terrain.

If you generate it with mapslitter you can get a x4 resolution ending in a better resolution that terrain (and therefore 4x more precise than per vertex normal).

Check the HF terrain + shadow mapping, default terrain is to generate x4 normal map, using some (way too much strong) noise. noise should/could be replaced by "splats detail normal map" either at generation time or runtime.
From what I recall, CryTek used a sand detail normal map. (But may not have use any huge normal map, only a lightmap)

I'll try using the base map in the ambient pass and see what happens. I've been dieing to try the base map creating you added anyways as my base map is incrediably out of date.

I don't really use the base map right now for anything as I'm having an issue with the Texture LOD. The tiles don't seem to update the LOD it uses until I stand on them which isn't pretty at all. Especially since my base map looks nothing like my world lol.

@Tuan, it may be as precise as the vertex normals if at same resolution as the terrain, but it is fetched, normalized, etc. per pixel in the pixel shader, and that's what I'd have a problem with... especially for a terrain that's supposed to go right to the horizon. With the material lod it's nice to be able to switch off pixel shaders and drop down to just vertex shaders + base map for pages that are far away, but that wouldn't be possible if all lighting was done in the pixel shaders with the normal map.

So it seems to me that I'm kinda stuck - there's no solution I've found so far that gets me dynamically lit terrain for the sun + other smaller lights such as point and spot lights.

I appreciate the work you've done on your horizon shadows (or whatever they're called) but I haven't liked the result enough to adopt it myself. Whenever I tried playing with it, it gave splotchy spots all over the terrain, and hey, I'm guessing now that I type this that that's what you're talking about when you mentioned the noise map. If so, get rid of it! It looks like ass. I wonder if I should go looking into it again and see what it looks like without that noise map. Hrm...

@Patrick - there's a material LOD setting in the terrain cfg that specifies when the next material technique should kick in. Set it to something more reasonable for your terrain.

ROFL, this is just embarrassing! I did change that HOWEVER I have a very large world and well I completely did not think about the tile size in that respect. I set the LOD to 1000 which is nice when I start however a tile is like 2000 across so I have to stand on it for Ogre to switch LOD, lol. Sometimes I get so caught up in PLSM2 that I completely forget about the standard Ogre stuff. Just changing the tile size to a smaller size completely fixes that problem. Now I'm gonna try the base generation.

One thing of curiosity, do you know if the triangle count in the Ogre Framerate is busted with PLSM2? It always says like 2 million polys which I find hard to believe? Seems weird that no matter where I am in the world or what I see it is always around this number. On the other hand I can't get to the world edges so maybe it is.

Took me a bit to get the base map generation to work since the save path is hard coded. The base maps are fine but using them in the ambient pass wasn't really pretty on such a large map.

Really? I figured they'd be fine in your ambient pass so it should mostly be in the shadowed areas anyhow. How high is your ambient colour?

@Falagard : Compute lightmap on the CPU when distant... no need of pixel shaders. I'll remove that noise if you wish.
shadown mapping technique comes from :
http://www.ati.com/developer/gdc/2002/G ... eetham.pdf
http://www.ati.com/developer/dx9/ATI-Li ... tering.pdf
but is unfinished (scattering part.)
check their demo here :
http://www.ati.com/developer/sc_dx_arch ... tering.zip

But that's old technique, Crytek released some papers on their work at siggraph :
http://www.ogre3d.org/phpBB2/viewtopic. ... 769#171769
perhaps worth checking that instead.

about my demo, I'll remove/soften the noise then.

@PatrickB3 : Ogre counts Tri of each pass. 3 pass = 3 terrain rendered.

I couldn't get the ati scattering demo to compile/run properly.

The hoffman preetham pdf did look pretty sexy though.

Those papers have everything to do with scattering and nothing to do with your horizon shadowing. Does their demo have the shadow mapping techniuqe?

How is it performed on the CPU when distant? Are you already doing that?

@Falagard : try this one : http://tuan.kuranes.free.fr/scatterShadow.7z
(extract the whole thing, as exe needs vc2005 dll that are inside.)
What compilation problem do you get ?
Does their demo have the shadow mapping technique?
Mine comes exaclty from there. (we even share the same map, HF...)
How is it performed on the CPU when distant? Are you already doing that?
Exactly same code as GPU, but on CPU , and updating less frequently. Code is already PagingLandScapeTexture::computeLightMap ().

Needs work to make it done on CPU when distant as for now it's all on CPU or all on GPU. AFAIK PagingLandScapeTexture would not change mIsShaderShadowed, mIsShadowed upon LOD change.

But it should, or better those variables should be technique dependent... or even better (plsm3) PagingLandScapeTexture should be a PagingLandScapeTechniques manager each being what PagingLandScapeTexture is now...

Ah, I didn't even notice in the ATI zip there was an .exe already, I thought I had to compile. Anyhow, I was getting a linker error on libci.lib and I googled it and someone mentioned adding it to the ignore list, which allowed it to link, but when I tried to run it crashed.

I ran the pre compiled exe of yours and ATI's and now I remember I've downloaded and seen this one before.

If your terrain shadows would look as sexy as this then I'd be happy to use it and abandon directional lighting on the terrain. I think the noise on the terrain (if that's what was causing all the splotchy parts) really threw me off because it didn't look good at all on the terrains that I tested, and I figured if that's the result I didn't want to use it.

I'd still like to be able to use point lights and perhaps spot lights on parts of the terrain very close to the camera, so chances are those areas would be at the highest lod anyhow.

Another problem I have is that for a terrain editor, you need to be able to see lighting in realtime, otherwise it's hard to detect the curves of the terrain. In the editor it might make sense to use my existing directional lighting shader, and then have a button to generate the horizon map. I don't think the horizon map is regenerated in realtime as you modify the terrain, right?

I'd love to see that scattering effect implemented, both for the terrain and sky. I'm going to look into this code and see what I can make of it.

Clay

when I tried to run it crashed
I fixed a stack overflow bug somewhere in their code... cannot recall where, but you can use code in my zip to get a version without segfault.
I don't think the horizon map is regenerated in realtime as you modify the terrain, right?
No, too slow, specially as I use a x4 factor to generate it. (as they do in their demo.)
I'm going to look into this code and see what I can make of it.
It's mainly about creating a sort of "day/night" manager as ogre contributed "caelum" did before. (but with some complex raleigh/mie factor computation.)

Did you read crytek paper about their day/night implementation. (volumetric light using depth)

@Falagard - My ambient varies by time of day from .4 to .9 which is a little extreme but seems to look kinda nice for now because of the way I altered the shader. When I use ambient in a seperate pass it is much too bright that way. When I have more time to play with lighting I'll put back the ambient pass and fix my values accordingly. Using your shader changes the not pretty part wasn't the shadows. It was very bright ( I needed to adjust the ambient and didn't ) which is fixable. However roads seemed missing so it mixed grass with the roads making them greenish. I am uncertian if the roads ( which are just painted dirt ) didn't show on the base because they are too small or if it is that the base generation didn't use the modified files.

@tuan kuranes - I was getting 2 million+ triangles because I had a giant plane in render group 9. When I moved it back to render group 8 the triangle count went pack to normal and it also stopped loading 60+ pages. Now it loads like it should.