Walking around backwards

Here's an interesting thing I've observed (with stock plsm 1.0.5, not anything I've poked at).

I would occasionally see "flashes" when walking, as if the camera was momentarily underground. Now in the stock plsm demo this was often due to a one frame delay between movement and positioning the camera above the terrain. Not a biggie.

But after fixing that (ok, not so stock) I would still occasionally see these changes. When I took some time to look into it (walk around the discontinuity and view it from other angles) it seems that nearby terrain has some discontinuities at various points. Not page/tile boundaries. At one point on a custom terrain set the immediate region was fairly flat in all directions, step backwards and a hill jumps up.

These points are fairly easily found by walking backwards in flat areas, as the terrain changes occur behind the camera.

Has anyone seen anything like this? I think it might be fun to look into this, I have no idea how I'll debug this. Best work these things out before I start coding up my planned texturing change

But after fixing that
How did you fix that ?

At one point on a custom terrain set the immediate region was fairly flat in all directions, step backwards and a hill jumps up.

Could you produce an exact reproducable scenario on Demo with a standard terrain and cfg, giving Camera position before walking, camera position After ?

How did you fix that ?

In my copy of demo, I moved the code which positioned the camera over the terrain towards the end of frameStarted (after the camera movement). Thus the camera was repositioned over the terrain in the same frame. Not a biggie. But it reduced the "flashing" as one runs around a map.

I'd send a patch in, but there are bigger things to do and I want to make sure the patch is worth the time to create.


Could you produce an exact reproducable scenario on Demo with a standard terrain and cfg, giving Camera position before walking, camera position After ?

At some point yes. Right now this is a good excuse for me to figure out how to debug PLSM. The biggest challange would be finding a nice flat spot to play with.

I mentioned it because it is possible that someone has seen it, and is working on a fix. Thus I would be duplicating effort, and could run off in another direction.

thanks for the tip.

if you give me reproducable step, i'll try to debug it.

I've been having fun walking through the plugin in the debugger, and wrote a method to dump renderables (in the hope that the list would change when I took a step).

This may or may not help me track down what is going on, but it does help to understand what Ogre is doing.

I borrowed heavily from SceneManager::renderBasicQueueGroupObjects
when it came to walking the queues.

OgrePagingLandscapeSceneManager should have a list of queue groups:

getRenderQueue()->_getQueueGroupIterator();

Each group in the queue has a number of solid passes, and solid passes with shadows. These passes each have a set of renderables (it appears for the data I am using that each visible tile is represented in each of five passes). There is also a set of renderables for transparent renderables, I'm not sure what they are but suspect they are overlays. There are four transparent renderables, and I believe four overlays in the demo.

OK so far. But what puzzles me is that when the camera is in page (4,2) tile (0,0) the list of renderables doesn't correspond to what I think is in front of the camera. Each renderable corresponds to a single rendered mesh, so I expect that a set of adjacent tiles in front of the camera will be on that render queue.

A sample of logged data (Scene Manager :: setOption being used to poke PLSM )


18:47:44: camera page(x,z) (4,2) tile(x,z) = (0,0)
18:47:44: Solid object list:
18:47:44: 4.4.5.7Rend
18:47:44: 4.4.5.6Rend
18:47:44: 4.4.4.5Rend
18:47:44: 4.4.0.7Rend
18:47:44: 4.4.1.7Rend
18:47:44: 4.4.2.6Rend
18:47:44: 4.4.2.7Rend
18:47:44: 4.4.3.6Rend
18:47:44: 4.4.3.7Rend
18:47:44: 4.4.4.6Rend
18:47:44: 4.4.4.7Rend
18:47:44: Solid object list:
18:47:44: 4.4.7.2Rend
18:47:44: 4.4.5.0Rend
18:47:44: 4.4.1.1Rend
18:47:44: 4.4.0.0Rend
18:47:44: 4.4.4.1Rend
18:47:44: 4.4.1.0Rend
18:47:44: 4.4.2.0Rend
18:47:44: 4.4.2.1Rend
18:47:44: 4.4.3.0Rend
18:47:44: 4.4.3.1Rend
18:47:44: 4.4.4.0Rend
18:47:44: 4.4.3.2Rend


Plotting these values on graph paper shows two blobs of tiles (all on page (4,4).

Any ideas what is going on here? The scene looks fine -- this is before I have taken my little hike.

While it would be easier to let Tuan look into this, I believe I should know the entire render pipeline in order to experiment with Ogre.

I finally reproduced the bug.

It uses LOD 1 whatever lod tiles is. meaning at lod max that it computes height using too far neighbor... which seems correct when on smooth terrain.
And as I use smooth terrain to test precisely height queries it took me some times...

Will look on that strange naming scheme after this one.

Found it.

PagingLandScapeRenderable::init


mName = StringConverter::toString(mInfo->pageX) + "." +
StringConverter::toString(mInfo->pageX) + "." +
StringConverter::toString(mInfo->tileX) + "." +
StringConverter::toString(mInfo->tileZ) + "Rend";


pageX used instead of pageZ above. I walked through tile at (4,1)(5,7) which was named "4.4.5.7Rend".

Nice catch

thanks.
fixed this and the lod thing in cvs.

Still need work on the height query, as it seems that we incorrectly get neigbor vertices some times.
I'll be able to work on this in 2 day only , so anyone that want to have a look can.

Debug plsm2 now display sphere on neighbor vertices .
My guess is that problem comes from paginglandscapedata2dmanager.cpp:842 PagingLandScapeData2DManager::getHeightAtPage not returning good page/tile position there.

just fixed bad page/tile neigbor get in height queries.

still need to get correct tile lod when that getPageHeight changes current page/lod.

temporary fix the height query bug.

Need to find out why previous gettile before entering the getrealworldheight() are wrong, because, re-calling getTile() in that method gives the correct result.

Anyway, that Should work now, if not on a stictching border.

I guess I should switch to the CVS version one of these days.

How in the world did you determine that this was a LOD issue? I'm curious how the thinking went, as I'm trying to learn to troubleshoot things like this on my own when possible.

I did put 4 sphere mesh on neighbor vertices. Then It became evident that neighbor choice was just not the good LOD, as it wasn't taking nearest vertices but nearest + 1, even at max Lod.
generally speaking, graphic debugging needs graphic helpers. Same for collision, physics, scenetree, etc.

Thanks for the tip. I was considering something similar (some towers at boundary points) but I haven't reached that point yet.

How long does it take for the changes in CVS to make it to where I can see them? The copy of CVS head I pulled down yesterday still has the problem.


This should download the most recent publicly available copy of Ogre into a folder called ogrenew within the folder you right-clicked in. Note I say "publicly" available because it can take a while for changes made to the code to propagate to the public servers you will be accessing. Only developers can access code that is 100% up-to-date.

one day should be enough.
In debug mode you should see the spheres.

just fixed another bug on a page border problem.
Now trying to solve query on stitched between tiles of different lod.

I still haven't seen any updates in CVS. Would you mind posting a filename and something to look for so I'll know when the change is pulled down?

I have looked in the web view of the CVS repository on sourceforge, and am pulling from that repository with TortoiseCVS. I notice there seems to be some sort of disagreement between the two, as I have pulled more recent changes down than are shown on the web view.

The last file pulled down was OgrePagingLandScapeTextureCoordinatesManager.cpp (change on the 13th), but this is not shown as changed on the web view. The web shows that rev 1.2 is the latest (on the 7th).

I don't see any spheres, but I do see a nice overlay in the demo which I assume has something to do with visible tiles. And the hills are still popping in.

Changes made are mostly in ogrepaginglandscapedata2Dmanager.
Nearly once a day since the 13rd.

Perhaps it's related to sourceforge design change ?
It prevents anon cvs update ?

you can consult or subscribe to ogre-cvsmail.
http://lists.sourceforge.net/lists/list ... re-cvsmail

you receive mail that contains the patch committed.

Thanks, I'll look into this.

Here is how I've been looking at the repository on the web, perhaps you can see an error:
http://cvs.sourceforge.net/viewcvs.py/o ... te#dirlist

Edit: here is info for data2dmanager.cpp, I am looking to see if I can specify a different branch somewhere.

File: [cvs] / ogre / ogreaddons / paginglandscape / PlugIns / PagingLandScape2 / src / OgrePagingLandScapeData2DManager.cpp (download)
Revision: 1.42, Thu Oct 27 07:34:39 2005 UTC (2 weeks, 6 days ago) by kuranes
Branch: MAIN
CVS Tags: HEAD
Changes since 1.41: +2 -1 lines

Hmm new stuff just appeared on CVS

In the meantime I've been looking at LOD stuff. It looks like distanceLOD is never used (I set breakpoints on all accesses and they weren't hit).

Edit: built and ran, things look great. Thanks for fixing this.

Where is the code for the overlay which used to be in the upper-right? I am interested in studying that for a bit (handy debug tool there).

Edit again: just ran across a null renderable on a tile in getInterpolatedWorldHeight, interesting. I had the client in the background while I was checking the web when the exception came in, so nothing should have been going on. I'll see if I can recreate this.



Plugin_PagingLandScapeSceneManager2.dll!Ogre::PagingLandScapeRenderable::_getNeighbor(Ogre::Neighbor n=NORTH) Line 49 + 0x6 C++
> Plugin_PagingLandScapeSceneManager2.dll!Ogre::PagingLandScapeData2DManager::getInterpolatedWorldHeight(const float x=-6.6500044, const float z=0.44999948) Line 621 + 0xd C++
Plugin_PagingLandScapeSceneManager2.dll!Ogre::PagingLandScapeRaySceneQuery::execute(Ogre::RaySceneQueryListener * listener=0x0580f548) Line 64 + 0x19 C++
OgreMain_d.dll!Ogre::RaySceneQuery::execute() Line 237 + 0x32 C++
Demo_PagingLandScape2.exe!PagingLandScapeFrameListener::frameStarted(const Ogre::FrameEvent & evt={...}) Line 414 + 0x15 C++
OgreMain_d.dll!Ogre::Root::_fireFrameStarted(Ogre::FrameEvent & evt={...}) Line 556 + 0x26 C++
OgreMain_d.dll!Ogre::Root::_fireFrameStarted() Line 601 C++
OgreMain_d.dll!Ogre::Root::renderOneFrame() Line 683 + 0x8 C++
OgreMain_d.dll!Ogre::Root::startRendering() Line 674 + 0x8 C++
Demo_PagingLandScape2.exe!ExampleApplication::go() Line 58 + 0xe C++
Demo_PagingLandScape2.exe!WinMain(HINSTANCE__ * hInst=0x00400000, HINSTANCE__ * __formal=0x00000000, char * strCmdLine=0x00141f2a, HINSTANCE__ * __formal=0x00000000) Line 53 + 0x8 C++
Demo_PagingLandScape2.exe!WinMainCRTStartup() Line 390 + 0x39 C
kernel32.dll!7c816d4f()
ntdll.dll!7c915b4f()
kernel32.dll!7c8399f3()



Here is the tile in question:


- tile 0x0336bdd0 {mWorldBounds={mMinimum={x=-10000.000 y=0.00000000 z=0.00000000 ...} mMaximum={x=1403.5088 y=23882.352 z=11403.509 ...} mNull=false ...} mWorldBoundsExt={mMinimum={x=-15701.754 y=-11941.176 z=-5701.7544 ...} mMaximum={x=7105.2637 y=35823.527 z=17105.264 ...} mNull=false ...} mWorldPosition={x=-4298.2422 y=11941.176 z=5701.7544 ...} ...} Ogre::PagingLandScapeTile * const
+ mWorldBounds {mMinimum={x=-10000.000 y=0.00000000 z=0.00000000 ...} mMaximum={x=1403.5088 y=23882.352 z=11403.509 ...} mNull=false ...} Ogre::AxisAlignedBox
+ mWorldBoundsExt {mMinimum={x=-15701.754 y=-11941.176 z=-5701.7544 ...} mMaximum={x=7105.2637 y=35823.527 z=17105.264 ...} mNull=false ...} Ogre::AxisAlignedBox
+ mWorldPosition {x=-4298.2422 y=11941.176 z=5701.7544 ...} Ogre::Vector3
mInit true bool
mLoaded false bool
+ mRenderable 0x00000000 {mRenderLevel=??? mOpt=0x00be88d0 {data2DFormat={"HeightField"} textureFormat={"Splatting"} LandScape_filename={"s4"} ...} mCurrVertexes=??? ...} Ogre::PagingLandScapeRenderable *
+ mTileSceneNode 0x0594a358 {mLocalAABB={mMinimum={x=78596.492 y=20941.176 z=0.00000000 ...} mMaximum={x=90000.000 y=21764.705 z=11403.509 ...} mNull=false ...} mOctant=0x00000000 {mChildren=0x00000018 mNodes={size=???} mMovingNodes={size=???} ...} mOcclusionBoundingBox=0x00000000 {mRadius=??? } ...} Ogre::SceneNode *
+ mParentSceneNode 0x05886620 {mLocalAABB={mMinimum={x=-0.50000000 y=-0.50000000 z=-0.50000000 ...} mMaximum={x=0.50000000 y=0.50000000 z=0.50000000 ...} mNull=true ...} mOctant=0x00000000 {mChildren=0x00000018 mNodes={size=???} mMovingNodes={size=???} ...} mOcclusionBoundingBox=0x00000000 {mRadius=??? } ...} Ogre::SceneNode *
+ mNeighbors 0x0336bee4 Ogre::PagingLandScapeTile * [4]
+ mInfo {pageX=3 pageZ=2 tileX=7 ...} Ogre::PagingLandScapeTileInfo
mVisible false bool
mTimePreLoaded 0 unsigned int


yes, distanceLOD is a bit deprecated now. I should remove it.

overlay code should be in framelistner for the drawing part and then it use all plsm2 internal "managers" to get info, using the getOption mechanism.

Strange, didn't happen here. but I don't browse the web while debugging.
Perhaps Ogre still works even windows not active, therefore unloading renderables...
(A reproducable bug is a gold bug for me.)

Just hit the null renderable again (always when posting on one of the forums ). In both cases the x coordinate is negative, the client was inactive in the background. Sadly the reproduction steps aren't known yet. I had this thing sitting idle for four hours with no problem, then start to post saying that I'm unable to reproduce when I'm back in the debugger.

And the hill popping is back Now what is special about the place where it occurs, compared with where it doesn't occur? And maybe these two issues are related? Since in both cases I was testing the hill popping by running around an area which has been trouble before. I found an interesting spot, stopped and was going to write a short note before digging into this.

I noticed that there was a sphere a ways in front of me, I would step forward and be warped up to a sphere on a hill. I wasn't crossing any boundaries, but was moved relative to the tile boundaries.

Good news on a couple of fronts. I have a reproduction step which should work, and an explanation for the crash. The keyboard listener is still active when the application no longer has focus. Which explains why things would mess up when I hop over here to write a message.

Sooner or later I type something with the letter 'L' and the map is reloaded. There is a race condition between the loading and the render loop accessing the renderable. Sooner or later an access is attempted before the pointer is assigned.

So, run client in a windowed mode. Alt-Tab to another application (say Notepad). Start typing something with a lot of L's (Welch users have an advantage here).

Check out the image below, it might shed some light on what is going on with the hills. I notice that when I head in a certain direction the tile sizes start to get compressed. The image below (if it posts correctly) shows two spheres in close proximity to each other. I do *not* see any sudden jumps in (x,z) coords, but there is a warp across the tile. And the spheres keep coming sooner and sooner.

Also, note the free count on pages... And for anyone concerned about the fps, this is a debug build with antialiasing set to L2. It is not supposed to be running fast.

OK, time to post another observation. This time I'm trying to understand why the scale compression occurs, and I think I'm running into the height problem you mentioned (or something similar).

In getInterpolatedWorldHeight I see a few oddities. First, on the first call to this method some values are way out of whack (to use a technical term):


23:04:54: bottom_right_x=0.000000, bottom_right_z=-0.000000, top_right_x=0.000000, top_right_z=-92559631349317831000000000000000000000000000000000000000000000.000000


This is due to a renderable with the currentRenderLevel set to -1, which results in a verdata-spread of -2 billion, and a lod mask of 2 billion. OK, bad things follow.

Fortunately subsequent calls have better renderables (and I'll ignore that first renderable for now).

Now, in that last "no stitching" case we want to determine the coordinates of the edges of a tile (guessing here, even when I think something is obvious I need to triple check, more so if I'm going to post for everyone to read). Here is the nice diagram (thanks to whoever included it):


// no stitching
// TL-----TR 1.0
// | / |
// | / | .
// | / | .
// | / | . ^
// | / | |
// BL-----BR 0.0 z
// 1.0 ... 0.0
//
// < - x


OK, a bit odd to have the origin in the lower right, but I'm over it. Here are the values of bottom_right and top_right

bottom_right = (0, 0)
top_right = (16, 16)


On a diagonal? Looks like TR and TL are swapped.

Now we go to find the four heights around the point:


BR-y is height at 0,0
BL-y is height at 16,0
TR-y is height at 0, 16
TL-y is height at 16,16


A second error has cancelled out the first. But the wrong coordinates are used to perform the stitching later on.

If you agree with this, I think the fix should be something like calculating top_left_x and top_left_z in the code above 774 (various cases), then using that value for the heights of the four points. My confidence in this is a bit low at the moment, I'll try it in the morning but it is time for bed here. I haven't taken the time to go through the stitching code enough to know what it is trying to do, I'm just trying to make the values match the diagram.

thanks for reproducable step.
did manage to reproduce it here.
not sure how to fix it.

fixed the non existing renderable.
fixed the non -1 renderlevel.
fixed stitching part, even on corners..

var name and graph may now bit outdated... as this part of code did changed many times.
once I finishe the corner part I'll try to fix the name and ascii graph.

I don't understand clearly the " tile sizes start to get compressed." ?
if it's only on border tile, is it somehow stichting related ?

[edit]
fixed
fixed free page display.
If still bugs after this commit please report.
[/edit]

Perhaps I can clear up what I mean by compression.

Initially the spheres on the boundaries are about 2807 units apart (on my scale, similar to grand canyon).


TR (0, 21647, 2807) TL (2807, 21764, 2807)
BR (0, 21764, 0) BL (2807, 21882, 0)


I expect that the numbers will change as I move around the map, but the distances will stay fixed (tile size won't change).

Instead, later I see


TR (26666, 25058, -17719)
BR (26666, 25058, -17894) BL (26842, 25058, -17894)


The spheres are now about 176 units apart. Thus we have the rather improbable picture I posted above: two spheres next to each other.

I was looking into this when I ran into the stitching problem.

Without knowing what bottom_right and top_left etc really mean, it is impossible to debug that code. If you could clarify these terms I could proceed, all I know is that the values currently don't match their names.

I'm suspecting that the spacing for the initial page is incorrect, rather than the later pages. 176 matches the scaling for the tile (mOptions->scale.x)

I've pulled down 1.47 and 1.48 this morning, and the spacing between spheres keeps changing as one moves away from the origin. The hill popping is still occurring.

This will keep me busy for a while. I'm still trying to figure out what behaviours are intentional and which are problems.

Here's another fun picture, note the 5! spheres with non-uniform spacing.

The more I look at this code, the less I know about what is going on. A simple statement about what each method is trying to do would go a long way here. As much as I would like to fix the problem with terrain changing radically five feet in front of me, I'm beginning to believe that I'm going to need to live with this.

did you check terrain with Wireframe mode (hit "t" key)
to understand how stitching works ?

It's hard to explain but easy to vizalize.

About code :
- no stitching : find correct triangle and just interpolate position.
- sticthing : find which stitching (north,south-east, etc..) and then iterate over triangle in this stitch until point is in. Then get result of intersection of the triangle and the query Ray

What I'm seeing should not be a stitching problem, as it extends beyond tile edges. This looks like LOD being applied very close. The following images show wire-frame initially and after stepping forward a small amount.

what is your max pixel error in your config file ?
Did you try vertex morphing if not using an ATI card ?

MaxPixelError is 15
VertexProgramMorph is being used
LODMorphStart is 0.5

try lower LODMorphStart to 0.2
and if possible MaxPixelError to less than 8

Still have radical changes with these settings.

MaxPixelError 1 ?

if still radical changes... it may comes from heightmap complexity. perhaps a different tilesize pagesize could help.

MaxPixelError = 1 works fine as far as I can tell. I'm off to study this setting and see if there is some way to make this operate at a further distance. I wouldn't mind if a higher pixel error was used a page away, but not right on top of the camera.

Thanks for the suggestion.

While I'd like to help with this code, I am struck by the lack of comments in rather complicated routines. I run across a routine like calculateMinLevelDist2 and begin to wonder if it wouldn't be quicker for me to write a new scene manager.

I may just be dense. But anything LOD related seems to leave me wondering what is going on.

Anyhow, now that I have stable terrain with MaxPixelError=1 I think I'll return to texturing.

It's same as Ogre Terrain Scene manager.

Distance is measured from the centre of the tile. Basically the LOD will drop as soon as you won't be able to tell the difference according to the allowed pixel error.

Read the paper on which is based the LOD system : geomipmaps

Thanks for the paper!!! I have looked for such a paper for a couple of days without success, and now have the added benefit of knowing that this is the paper on which Ogre's LOD system was based.

I can say with some certainty that the hills which popped in represented errors of more than 15 pixels, so there is some work remaining in this area.