[HELP] PLSM2 performance & a few questions

Dids

28-11-2005 20:08:59

Hey all, and thanks for this astonishing add-on.

First of all, I am using the latest PLSM2, and I'm having trouble finding a "balance" in the settings in the mymapfile.cfg file. With balance, I mean that I don't want too much/none-at-all preloading/loading when I move around the map, so at the moment I'm preloading a lot at startup (atleast for now this has cut down loading when I free "roam" around the world). My config file is as follows:

GroupName=PLSM2

LandScapeFileName=RvWorld

LandScapeExtension=raw

Width=16

Height=16

PageSize=257

TileSize=65

Data2DFormat=HeightField

MaxValue=5000

MinValue=0

ScaleX=2000

ScaleY=2000

ScaleZ=2000

Deformable=yes

VertexCompression=yes

VertexProgramMorph=yes

LODMorphStart=1

MaxPixelError=5

VertexNormals=no

TextureExtension=png

TextureFormat=BaseTexture

VertexColors=no

CoverageVertexColor=no

BaseVertexColor=no

BaseVertexShadow=no

BaseVertexInstantColor=no

MaterialHeight1=33

MaterialHeight2=66

SplatFilename0=splatting_grass.jpg

SplatFilename1=splatting_sand.jpg

SplatFilename2=splatting_rock.jpg

SplatFilename3=splatting_snow.jpg

VisibleRenderables=20

DistanceLOD=5.0

MaxRenderLevel=1

NumRenderablesLoading=200

MaxAdjacentPages=25

MaxPreloadedPages=50

ChangeFactor=2.0

CameraThreshold=5

MaxNumRenderables=500

IncrementRenderables=500

MaxNumTiles=500

IncrementTiles=500

HorizonVisibilityComputing=no

Secondly, is there a way to (this might be an ogre/plsm2 question, not sure) to show an image (static image) when preloading (at startup), since at the moment it is annoying to wait for even a few minutes for the preloading to stop (although this prevents the actualy gameplay loading), so atleast I could watch a loading screen while it's preloading, hehe.

And of course, if there is a way to preload/load when moving around the map, and *not* using too much of CPU/GPU, so that the game wouldn't lag (atleast not to the "casual eye"), since that is certainly not what I want.

My last question might be this: Since I'm using the latest PLSM2 from the cvs, I seem to be only capable of building the debug DLL, but not the release. Oh and I'm also using Ogre 1.0.6 (SDK & Source). The actual error is as follows (using VC7.1, and I'm average-good with compiler/include/header setup):

OgrePagingLandScapeSceneManager.obj : error LNK2001: unresolved external symbol "__declspec(dllimport) protected: virtual class Ogre::RenderQueue * __thiscall Ogre::SceneManager::getRenderQueue(void)" (__imp_?getRenderQueue@SceneManager@Ogre@@MAEPAVRenderQueue@2@XZ)

Linking error, and an add one too. The ogremain_d & ogremain libs that come with the PLSM2 work fine with PLSM2 I think, but the DLL I get using those to compile, doesn't work with the latest Ogre DLL's. Hope that made any sense. I'm just saying that Ogre 1.0.6 main DLL's (debug & release) seem to be incompatible with PLSM2 (release, works fine with debug), so I'll state once more that I'm using Ogre's OgreMain(_d) libs to compile.

Alright, I'll just wait for responses, and sit tightly next to my computer.

Again, thanks for this great plugin.

- Dids

tuan kuranes

29-11-2005 08:34:17

MaxAdjacentPages=25 and MaxPreloadedPages=50 is a very high value.
should rather be MaxAdjacentPages=2 and MaxPreloadedPages=3
you can also make page bigger (pagesize=513)

using the listener mechanism as demonstrated in the demo is the way to do the preload thing. you make the porgress bar move on pageloaded event and load is done on terrainready event.

plsm2 from CVS is to be used with Ogre CVS HEAD (dagon), not the SDK.

Dids

29-11-2005 23:00:23

Thank you Tuan, that helped more than enough :wink:

One more thing though, since I've tried to set the pagesize to 513 several times before, but I get an error about the raw size of the map it seems. I'm sure you know how to solve this, and it might even be just a small thing to fix.

I'm guessing it's comparing the pagesize to the height/width of the map, or, the actual size of it (in bytes).

Thank you again.

Jon

30-11-2005 06:03:35

Have you ran your map through the mapsplitter?

I'm assuming that you haven't (otherwise I can't imagine why PLSM would give you grief about the page size). In case this is helpful to anyone:

PLSM wants to see maps split into pages which are (2^N)+1 squares. It also wants to see the individual pages named as they come out of the map splitter.

So, you can take a large map, say 1024x1024 and split it into four 513x513 pages (which overlap).

Each page will then be split into tiles when the page is loaded (nothing you need to worry about, as long as (page_size-1) is divisible by (tile_size-1).

tuan kuranes

30-11-2005 14:18:31

Using mapsplitter you should try to generate png instead (wilk be 16 bits png), and using raw as mapsplitter input it needs height and widht to be specified.

Dids

30-11-2005 16:09:04

Ah right, that would explain it. I have my splitted map files as 256x256 raw (+ the png alpha, coverage and such), so I guess I need to run the mapsplitter again, but set it to split it in 512x512 dimension.

Thanks guys, as always, this has been really helpful

One thing concerning PLSM2 is that I do get crashes (in debug and release), and most of them come if I haven't waited for about 10 seconds when I start the application. Some errors appear only when I'm exploring the terrain and going straight for a minute or so. I'm sure this is either just me, or a small bug somewhere that hasn't been solved yet, but I'd just like to know if there's any way to solve it, since that kind of unstability in a big terrain isn't nice when this project gets released.

Thanks guys!

- Dids

tuan kuranes

30-11-2005 16:24:01

SDK version has some bugs that are corrected in CVS version.
But SDK version should soon be updated.

Jon

30-11-2005 16:36:52

I haven't seen any crashes on either the SDK version, or CVS. A stack trace would be helpful.

Do these happen on any of the standard terrain files (eg grand canyon), or just your own data?

Dids

30-11-2005 18:46:25

Well, I haven't yet tested any other maps other than my own. I created my map file via MapEditor, since I tried using MapSplitter to create from a 8193x8193 heightmap (not sure if it's in 32-bit, since Photoshop can't open it), and MapSplitter crashed when it was processing the heightmap (the texturemap was succesfully splitted, which was 8192x8192).

It might be that the map I created via the MapEditor is the one causing the crashes when running the application, but I'm not really getting all the settings needed for MapSplitter, even after reading the Wiki and default .cfg's etc.

I'm also sorry if I'm causing too much trouble

The only thing I want to do, is to create a big (8000+ x 8000+) heightmap, with image-based texture. That's all I'm concerned at the moment.

I was using a gen.cfg file which looks like this:

GroupName=PLSM2

LandScapeFileName=RvBig

LandScapeExportExtension=png

TextureExtension=png

TextureStretchFactor=2

Paged=yes

ScaleX=2

ScaleY=2

ScaleZ=2

OutDirectory=Tmp\LandScapeName

PageSize=513

TileSize=65

Data2DFormat=HeightFieldTC

MaxValue=3000

MinValue=0

ColorMapName=RvBigTexture.png

ColorMapSplit=yes

LitColorMapSplit=no

HeightMap=yes

InfiniteMap=no

HeightMapBlurFactor=0.0f

Equalize=no

MiniMap=yes

MiniMapWidth=512

MiniMapHeight=512

NormalMap=no

HeightNormalMap=no

ColorMapGenerated=no

LitColorMapGenerated=no

MaterialHeight2=50

AvgColorsExists=yes       

#Sand

MaterialColor0.r=1.0f

MaterialColor0.g=1.0f

MaterialColor0.b=0.0f

#Grass

MaterialColor1.r=0.13f

MaterialColor1.g=0.545f

MaterialColor1.b=0.13f

#Rock

MaterialColor2.r=0.411764f

MaterialColor2.g=0.411764f

MaterialColor2.b=0.411764f

#Snow

MaterialColor3.r=1.0f

MaterialColor3.g=0.98f

MaterialColor3.b=0.98f

SplatFilename0=splatting_sand.jpg

SplatFilename1=splatting_grass.jpg

SplatFilename2=splatting_rock.jpg

SplatFilename3=splatting_snow.jpg

BaseMap=yes

LitBaseMap=no

CoverageMap=yes

AlphaMaps=yes

AlphaSplatRGBAMaps=no

AlphaSplatLightMaps=no

RGBMaps=no

LightMap=yes

ShadowMap=no

Sunx=0.0f

Suny=0.88f

Sunz=0.47f

Ambient=0.5f

Diffuse=0.5f

Blur=0.0f

HorizonMap=yes

ElevationMap=no

ZHorizon=yes

PVSMap=no

Jon

30-11-2005 18:58:40

What is the indication that mapsplitter crashes? It would be nice to have a symptom and work backwards to a cause.

Dids

30-11-2005 19:17:12

Well, I'm not entirely sure what you mean with "indication", but after it loads all the resources, then start splitting the texture map (there's a line saying "RvBigTexture..png", so two dots, for some reason), and after it's succesfully splitted the texture, it says "RvBig.png" (just one dot this time), and after it's been splitting the heightmap (for a few seconds tops), it gives the basic Windows XP error message (about how sorry they are that the app crashed), and no new lines are in the MapSplitter output. Also tried to debug (via the error message, there's this "Correct errors" button) and I got this:

Unhandled exception at 0x00439241 in MapSplitter.exe: 0xC0000005: Access violation reading location 0x00c80000.

The weirdest thing here is that it actually created the RvBig.0.0->15.15 png files, along with the same amount of textures (256 files of both, so 512 including heightmap & textures, splitted). Hope this helps atleast a little bit.

Although, as I said, I'm not sure if the heightmap (the big one, non-splitted) is 32-bit.

EDIT: Hmm, also, after I tried using these maps (the splitted height maps were white btw), I got an error likes this:
Function: PagingLandScapeData2D_HeightField::_load
Description: Error: Image is not a gray image.(1 byte, 8 bits).

And I do know what that means I guess. It's not an 8-16-bit gray image, right? I'm also wondering why the heightmaps are completely white, since normally if they're flat, they're black. And I tried using Data2DFormat=HeightFieldN in the RvBig.cfg file, but that seems to need differently named/completely different heightmap files?

Jon

30-11-2005 19:31:50

Thanks for the information. I wanted to make sure I understood what you meant by crash, as people tend to use it to describe a lot of undesirable things.

An interesting situation. I missed the part about a 32-bit heightmap. But I wouldn't worry about it too much. DevIL is used to process the png, and I believe MapEditor was written with PLSM in mind. I also expect that MapSplitter would complain if the input file was in a odd format (I'm not going to look at the source for MapSplitter right now).

The part about Photoshop not opening the document is surprising, as Photoshop should be able to open just about anything. Can you make this heightmap available for inspection?

I wonder if a memory allocation failed somewhere. Do you think you have enough RAM/page space on this machine?

Jon

30-11-2005 19:44:20

EDIT: Hmm, also, after I tried using these maps (the splitted height maps were white btw), I got an error likes this:
Function: PagingLandScapeData2D_HeightField::_load
Description: Error: Image is not a gray image.(1 byte, 8 bits).

Well this would qualify as a complaint about the file format, although the access violation which follows would indicate a bug in MapSplitter.

My recollection of MapSplitter is that it wants either an 8-bit PNG or a 16-bit Raw, but I assumed that MapEditor would generate a compatible file.

If you can open this PNG in an editor and change the format you should be able to continue.

Re colors of heightmaps: which is high altitude, black is low altitude. A flat region may be either. However if the unsplit map wasn't in a supported format I wouldn't trust the content of the split files.

Dids

30-11-2005 19:45:12

First of all, yes, I've got 1gb of 400mhz DDR, so shouldn't be a problem.
Also, I was scrolling through the HeightmapName.cfg file, and I found an interesting part saying this:

# 8bits, grayscale images. (png, jpg, etc) or 16 bits (raw)

Data2DFormat=HeightField

# 32bits, RGBA images. (png, jpg, etc) RGB is normal information, in alpha we store heightmap

#Data2DFormat=HeightFieldN



#

#           The HeightField TC is a compresed format and need to know the value range from where the

#               heightfield file was compressed

MaxValue=3000

MinValue=0

# # 8bits, grayscale images. (png, jpg, etc) or 16 bits (raw)

#Data2DFormat=HeightFieldTC

# 32bits, RGBA images. (png, jpg, etc) RGB is normal information, in alpha we store heightmap

#Data2DFormat=HeightFieldNTC

From what I understand, I should use HeightFieldN or HeightFieldNTC (you can see what the default is, since it's the one uncommented, and two of those seem to be for 32-bit images). I tried using both of those, and in the log file it tells me this:

PLSM2 : Cannot find map named RvBig.HN.0.0.png

It might be that if the heightmap is 32-bit, it needs to export it with a different name (HN added I presume). I can link you to the original (and quite big) heightmap if you wish.

Jon

30-11-2005 19:52:37

Looking at a .gen.cfg file, the only supported loaders are HeightField and HeightFieldTC.

If mapsplitter didn't create the pages, I don't think changing the settings in the .cfg file is going to help. Your split files are useless until MapSplitter runs to completion.

Dids

30-11-2005 19:55:10

Ahh, you're right. Alright, so this only leaves me with one problem (I think so atleast): I need to convert the whole heightmap from 32-bit, to 16-bit (also make it greyscale, right?). Weird thing is, that ImageReady opens the png file without problems, but Photoshop doesn't. I'll keep trying to find a way to convert it, and see if converting will solve this for good.

Dids

30-11-2005 21:06:46

Hmm, now this is weird. I rebooted, and ran Photoshop. I even made a new image (Grayscale, 8-bit), and Window's file properties (right click -> properties) tells me the bit amount is 32. The problem hasn't gone away.

I wonder, is there an "easy" way for me to create a big heightmap (flat) with png extension?

Dids

30-11-2005 23:03:31

Sheesh, this isn't happening. I'm having the worst bad dream ever.

I tried making a grayscale, 16-bit, png file, in Photoshop. 8192x8192 dimension. I read/changed my .gen.cfg file a few times. I launched MapSplitter. It splitted & exported them (and it crashed again). MapEditor still complained that the image wasn't "grayscale, 8-bit" (I read the wiki, and it says that 16-bit png's are supported). So, I tried the map with my own application. It ran without errors, although, I couldn't see the ground, and when I moved forward, everything started to disappear (first they turned into transparent & gray things, and lost a few polygons I'd say), then, everything disappeared (if I'd go back, they all would "come back). Okay, so now I knew something was really *really* wrong. I ran about, just for a few minutes, and I got the same error as with the MapEditor generated map.

The "a bit rarer" error message I get (after that it crashes), is an "assertion failed" error. Also says something like "Expression: mPageNode", so there's a problem somewhere (yeah, I think it's my map/map-bits, but a map generated with MapEditor shouldn't crash the app).

Okay, at this point I started thinking if my own application is showing this error message? Nope, since PagingLandScape Demo shows this error when I run through my MapEditor-generated map:

Okay, so, I tested the Photoshop-created heightmap (Photoshop claimed it was 16-bit, grayscale, windows claimed it was 32 bit. MapEditor said it can't read it, because it's not 8-bit grayscale, but my app and PagingLandScape accepted it just fine, just didn't show anything, and I had graphical bugs), and this is a screenie of the window (if the PLS Demo texts help anyone):

Okay guys, I hope to get this solved (and that's why I keep posting, doh!), so thanks for everyones time.

- Dids

Jon

01-12-2005 00:11:30

I believe that the only 16-bit formats which may be used are raw files. Create an 8-bit png and see if that works.

Dids

01-12-2005 00:34:11

I checked the wiki, and it said that 16-bit grayscale png's are supported (I might be wrong, but that's what I read). I tried creating an 8-bit grayscale png. The size was 8192x8192 (not 8193x8193 as I tried a while ago). I'll link my .gen.cfg file and .cfg file, if that helps a bit. I still get the error in mapsplitter, and the terrain doesn't load (no errors though, atleast not in the logs).

I just noticed that my newly created heightmap is only around 60-70kb in size. I'll include a link to that aswell.

http://didsdesigns.com/ogre/ <- The three files.