Falagard
03-10-2006 20:08:41
Tuan,
I think we've discussed this before, but I'm going to bring it up again because I'm concerned about the PLM2 and want to know if there's any way I can bend it to my will
First, I'll explain my motivations: I want the page size of the terrain to be fairly small, say 128, so that very high complexity materials with per pixel lighting, normal mapping per splat texture, etc. can be used, but so that the material LOD can ensure that anything further than 128 will switch to a simpler material.
I want terrain that extends very far into the distance, perhaps up to 10 kilometers, and loads quickly.
I want to use as little memory as possible.
I want a fairly high resolution terrain for pages that are close to the camera, say 2 meter resolution (a vertex every 2 meters) but possibly even 1 meter.
Sounds crazy, I know.
With those insane requirements, the current implementation of PLM2 has some issues:
1. If I break up the terrain into 128x128 pages but can view let's say 5 kilometers of terrain at once, there will be lots of distant pages visible and each is a different render operation so it isn't batching these very well.
2. Each page stores its vertex data at the highest resolution as well as a copy of that data for the height data in an additional array stored with the page. In my mind, distant pages that are at a low resolution don't need to load the terrain at a high resolution (create the vertices, store the 1 meter resolution terrain data) until it gets much closer to the camera.
3. The loading time for all that data will take a long time, but would be much faster if it only loaded what was needed.
4. Really distant pages don't need the relatively high resolution splatting textures, or even the merged base maps at the resolution they're at, they could be using much lower resolution textures.
I'm thinking of using the PLM2 for terrain while editing, but creating a custom solution while in game that does something like the following:
I take the pages at 128x128 that were created in the PLM2 and perform a process on them to start merging them into groups of four, but scaling them back to 128x128 again. So for example, I would take four pages of height data (as well as splatting alphas, normals, each with their own images) and merge them into a 256x256 map then scale it back to 128x128. This 128x128 height data would be used to build terrain at 1/2 resolution of what it originally was, as well as used for splatting, normal mapping, etc. So for example four pages that are a certain distance camera will no longer render as four separate pages, they'll render as a single page with lower res mesh, splatting textures, etc. I'd then take those new 128x128 textures that represent four pages, and merge those four into a new 256x256 and scale it back to 128x128. This could keep on going for many levels of detail.
Then, when determining visibility, a certain radius of pages would be the high resolution original pages, then another radius could be the lower resolution groups of 4, and then another radius could be the lower resolution groups of 8, etc. Terrain geometry would be created from the lower resolution data, which would mean distant pages wouldn't even have vertices created for the higher res until the camera got close enough.
Here is an image of what I'm talking about:
What you see here is the smaller circle represents the radius at which high resolution tiles should be rendered. Any pages within the radius are loaded and rendered as high res. Since everything has to be grouped in groups of 4, the light blue pages are collateral (as in.. they have no choice, collateral damage) in that they must be high res to fill out that set of four pages.
The second larger circle represents the next LOD, with the darker green pages intersecting the circle, and the darker blue being the collateral pages.
You could imagine a further circle, and the darkest green lies in that circle. This could keep going for many levels of detail as necessary.
This means that I can manually control the distance at which each level of detail changes as well, by changing the concentric circle radii.
There are obvious problems, such as where the two different levels of detail join, the vertices would have to match so there aren't any seams. This could be done by making the higher resolution page (such as the light green) have its vertices match with the lower resolution page along the border edges where it will meet the lower resolution page. The vertices along these edges will basically have to match up with the lower resolution vertices - not really stitching though. There will be an assumption that the LODs can't jump more than one level of detail at a time, so it only needs to match up with the next lowest resolution page and only along the border.
I'm envisioning brute force rendering for now, with no additional LOD morphing, but perhaps some form of morphing could actually be done later. This will mean popping between LODs, but with higher end 3d cards I could brute force the terrain out quite far so only the really distant pages will pop and then it might not be so noticeable. Lower end cards will just have to deal with popping at a closer range.
Theoretically, it'd be possible to render an entire planet this way especially if the grid is in 3d instead of 2d, since when you're out in space it'd only have to load what was necessary at that far distance, then dynamically load new data as you get closer. I'm not saying that's what I want to do, but it gives you an idea of the possibility of this technique.
So I'm bringing this up to get your thoughts on the matter. I'd love to see this in the PLM2 code base, but I'm guessing it really is a huge change to what's going on currently and will not be possible.
As I said, my current plan is to use PLM2 while editing and then run a batch conversion process on it when I'm finished with the terrain to convert it into the different implementation. I think it will actually generate mesh files which will be used instead of storing the height data. This means that I could theoretically also support caves, overhangs, and extra complex terrain if I wanted to since I could output to a mesh format such as .obj and load it into a 3d program and edit it, then export it back out. Obviously there would be a problem that if I edited the highest resolution page, I'd have to subsequently edit lower resolution pages to make them the same shape, or find a way to merge actual meshes instead of just the height data, which is quite a bit more complex.
So... thoughts?
Clay
I think we've discussed this before, but I'm going to bring it up again because I'm concerned about the PLM2 and want to know if there's any way I can bend it to my will
First, I'll explain my motivations: I want the page size of the terrain to be fairly small, say 128, so that very high complexity materials with per pixel lighting, normal mapping per splat texture, etc. can be used, but so that the material LOD can ensure that anything further than 128 will switch to a simpler material.
I want terrain that extends very far into the distance, perhaps up to 10 kilometers, and loads quickly.
I want to use as little memory as possible.
I want a fairly high resolution terrain for pages that are close to the camera, say 2 meter resolution (a vertex every 2 meters) but possibly even 1 meter.
Sounds crazy, I know.
With those insane requirements, the current implementation of PLM2 has some issues:
1. If I break up the terrain into 128x128 pages but can view let's say 5 kilometers of terrain at once, there will be lots of distant pages visible and each is a different render operation so it isn't batching these very well.
2. Each page stores its vertex data at the highest resolution as well as a copy of that data for the height data in an additional array stored with the page. In my mind, distant pages that are at a low resolution don't need to load the terrain at a high resolution (create the vertices, store the 1 meter resolution terrain data) until it gets much closer to the camera.
3. The loading time for all that data will take a long time, but would be much faster if it only loaded what was needed.
4. Really distant pages don't need the relatively high resolution splatting textures, or even the merged base maps at the resolution they're at, they could be using much lower resolution textures.
I'm thinking of using the PLM2 for terrain while editing, but creating a custom solution while in game that does something like the following:
I take the pages at 128x128 that were created in the PLM2 and perform a process on them to start merging them into groups of four, but scaling them back to 128x128 again. So for example, I would take four pages of height data (as well as splatting alphas, normals, each with their own images) and merge them into a 256x256 map then scale it back to 128x128. This 128x128 height data would be used to build terrain at 1/2 resolution of what it originally was, as well as used for splatting, normal mapping, etc. So for example four pages that are a certain distance camera will no longer render as four separate pages, they'll render as a single page with lower res mesh, splatting textures, etc. I'd then take those new 128x128 textures that represent four pages, and merge those four into a new 256x256 and scale it back to 128x128. This could keep on going for many levels of detail.
Then, when determining visibility, a certain radius of pages would be the high resolution original pages, then another radius could be the lower resolution groups of 4, and then another radius could be the lower resolution groups of 8, etc. Terrain geometry would be created from the lower resolution data, which would mean distant pages wouldn't even have vertices created for the higher res until the camera got close enough.
Here is an image of what I'm talking about:
What you see here is the smaller circle represents the radius at which high resolution tiles should be rendered. Any pages within the radius are loaded and rendered as high res. Since everything has to be grouped in groups of 4, the light blue pages are collateral (as in.. they have no choice, collateral damage) in that they must be high res to fill out that set of four pages.
The second larger circle represents the next LOD, with the darker green pages intersecting the circle, and the darker blue being the collateral pages.
You could imagine a further circle, and the darkest green lies in that circle. This could keep going for many levels of detail as necessary.
This means that I can manually control the distance at which each level of detail changes as well, by changing the concentric circle radii.
There are obvious problems, such as where the two different levels of detail join, the vertices would have to match so there aren't any seams. This could be done by making the higher resolution page (such as the light green) have its vertices match with the lower resolution page along the border edges where it will meet the lower resolution page. The vertices along these edges will basically have to match up with the lower resolution vertices - not really stitching though. There will be an assumption that the LODs can't jump more than one level of detail at a time, so it only needs to match up with the next lowest resolution page and only along the border.
I'm envisioning brute force rendering for now, with no additional LOD morphing, but perhaps some form of morphing could actually be done later. This will mean popping between LODs, but with higher end 3d cards I could brute force the terrain out quite far so only the really distant pages will pop and then it might not be so noticeable. Lower end cards will just have to deal with popping at a closer range.
Theoretically, it'd be possible to render an entire planet this way especially if the grid is in 3d instead of 2d, since when you're out in space it'd only have to load what was necessary at that far distance, then dynamically load new data as you get closer. I'm not saying that's what I want to do, but it gives you an idea of the possibility of this technique.
So I'm bringing this up to get your thoughts on the matter. I'd love to see this in the PLM2 code base, but I'm guessing it really is a huge change to what's going on currently and will not be possible.
As I said, my current plan is to use PLM2 while editing and then run a batch conversion process on it when I'm finished with the terrain to convert it into the different implementation. I think it will actually generate mesh files which will be used instead of storing the height data. This means that I could theoretically also support caves, overhangs, and extra complex terrain if I wanted to since I could output to a mesh format such as .obj and load it into a 3d program and edit it, then export it back out. Obviously there would be a problem that if I edited the highest resolution page, I'd have to subsequently edit lower resolution pages to make them the same shape, or find a way to merge actual meshes instead of just the height data, which is quite a bit more complex.
So... thoughts?
Clay
Is it possible to *not* store the height data in memory in addition to the vertex and index buffers if we're not going to be using it for editing or query purposes, or is it absolutely required by the PLM rendering implementation?