ROAM Planet rendering

A place to show off your latest screenshots and for people to comment on them. Only start a new thread here if you have some nice images to show off!
Post Reply
User avatar
DavlexDesign
Orc
Posts: 400
Joined: Thu Apr 23, 2009 7:23 am
Location: Australia
x 19
Contact:

Re: ROAM Planet rendering

Post by DavlexDesign »

ultramedia wrote:lol, limited shader skills, simple math... if you say so
You would be surprised mate, The only place I use sin and cos in this thing is for the sphere routine for the atmosphere and clouds, and OH, I use a acos and atan for Lat Long calculations, that's it, the rest is multiply, divide, plus an minus. I have to admit though, that my vector math has improved some what since I started tackling this thing, I have a pretty good understanding of cross product and dot product stuff, all those simple things, but nothing over the top.

In shader, I do use sin and cos for the wave synthesis stuff, and if I want to enable the logarythmic depth buffer, log is in there too, but that wasn't my mind thinking that one up, that was a rip from alterra's site, so I can't take any credit for that.

Alex
palaslet
Halfling
Posts: 61
Joined: Tue May 25, 2010 7:52 am
Location: Fredrikstad, Norway

Re: ROAM Planet rendering

Post by palaslet »

DavlexDesign wrote:stick with simple math, seems to produce similar results, but at a lesser GPU cycle penalty
I thought modern GPU's had pretty effective matrix calculation, so if you could just do most calculations via matrixes you would get the best performance boost. Isn't that whats makes PhysX so good?

Am I way of now?
"It seems that perfection is reached not when there is nothing left to add, but when there is nothing left to take away"
User avatar
DavlexDesign
Orc
Posts: 400
Joined: Thu Apr 23, 2009 7:23 am
Location: Australia
x 19
Contact:

Re: ROAM Planet rendering

Post by DavlexDesign »

G'day palaslet,
DavlexDesign wrote:I thought modern GPU's had pretty effective matrix calculation, so if you could just do most calculations via matrixes you would get the best performance boost. Isn't that whats makes PhysX so good?
You could be right there, and I could be way off, but the proof is in the pudding in my eyes, removing the matrix calculations for the TBN matrix made the shader cheaper to run by a fair amount, and my accuracy over larger numbers increased hugely, I do know that matrix math can introduce allot of floating point rounding errors when dealing with reasonable sized numbers, so maybe it is only worth considering when dealing with numbers like mine, to alleviate the problems some what. Anyway, I could be talking through my arse too, so don't take this as gospel, but it seems to make sense to me, and like I said before, the proof is in the pudding, and the pudding is looking pretty sweet right now.

Alex
User avatar
DavlexDesign
Orc
Posts: 400
Joined: Thu Apr 23, 2009 7:23 am
Location: Australia
x 19
Contact:

Re: ROAM Planet rendering

Post by DavlexDesign »

G'day guys,

You know how I was talking about "The Proof is in the Pudding", well ....
A shot with the TBN Matrix stuff removed and the Tangent Space normal to world space normal conversion replacing it.
A shot with the TBN Matrix stuff removed and the Tangent Space normal to world space normal conversion replacing it.
And another more like the shot in the previous post.
And another more like the shot in the previous post.
If you take a look at these shots, they have the TBN Matrix stuff removed from the terrain shader, and replaced with the Tangent Space normal to World Space normal conversion stuff in its place, take a look at the fps jump, compare those numbers with some of the previous shots, makes a huge difference. I would say that is pretty much validating what I was saying "The Proof is in the Pudding", and it's getting sweeter all the time.

Alex
palaslet
Halfling
Posts: 61
Joined: Tue May 25, 2010 7:52 am
Location: Fredrikstad, Norway

Re: ROAM Planet rendering

Post by palaslet »

DavlexDesign wrote:The Proof is in the Pudding
Indeed.

I of course had it all wrong. After a quick refresh of my GPGPU knowledge I now know that PhysX uses the GPU because physics calculation is highly compatible with the Stream Processing concept. The parallel nature of the GPU makes the physics calculation much faster than on a CPU. Thats why! (GPGPU also has better support for matrix calculation than CPU)

So...
Simple math is... eh... simple.

Even for the GPU... :) (who would have guessed...)

Thanks for proving me wrong; or else I would have learned nothing.
"It seems that perfection is reached not when there is nothing left to add, but when there is nothing left to take away"
User avatar
DavlexDesign
Orc
Posts: 400
Joined: Thu Apr 23, 2009 7:23 am
Location: Australia
x 19
Contact:

Re: ROAM Planet rendering

Post by DavlexDesign »

G'day palaslet,

Something that surprised me too, is thinking this morning about the results I got last night with this experiment, when I took the TBN matrix out of the shader, I removed it from the vertex shader, so that calculation only gets used twice per vertex, once to get the Tangent Light direction and once for the Tangent Eye Direction, where my tangent to world calculation is done in the pixel shader, once for every pixel calculated, so, how efficient can the matrix calculations be ?

I know the pixel processing pipeline has more pipes available to it, and that may be a reason for the difference too, I wonder if putting the matrix calculations in the pixel shader would make a difference to the numbers ? Food for thought I feel.

Alex
palaslet
Halfling
Posts: 61
Joined: Tue May 25, 2010 7:52 am
Location: Fredrikstad, Norway

Re: ROAM Planet rendering

Post by palaslet »

Alex,

Have you tested with higher resolution? (Should cause more load on the pixel shader?)

Also, have you done the same tests on your sons computer?
Seems like his GPU is quite a bit faster that yours, and might also have better performance on matrix calculations :)

But, you also have the issue with floating point rounding as you pointed out earlier. Might be that you cannot do this by matrix because of the scale of your project.
"It seems that perfection is reached not when there is nothing left to add, but when there is nothing left to take away"
palaslet
Halfling
Posts: 61
Joined: Tue May 25, 2010 7:52 am
Location: Fredrikstad, Norway

Re: ROAM Planet rendering

Post by palaslet »

Hi, its me again...

Wondering about the height map for this thing. What are the parameters? Lat/Long or euclidean coordinates of the original sphere surface point?

I'm working on a height map that creates continents instead of the uniform land distribution you get with most fractal algorithms.
"It seems that perfection is reached not when there is nothing left to add, but when there is nothing left to take away"
User avatar
DavlexDesign
Orc
Posts: 400
Joined: Thu Apr 23, 2009 7:23 am
Location: Australia
x 19
Contact:

Re: ROAM Planet rendering

Post by DavlexDesign »

G;day palaslet,

Good idea on the hires test, so here it is ....
Using the TBN Matrix in the vertex shader.
Using the TBN Matrix in the vertex shader.
Using the Tangent To World Space routines.
Using the Tangent To World Space routines.
Both these shots where taken on my machine, as I thought that would be a better test, as I haven't got a rip snorter of a graphics card.
They are at 1680 x 1050 res (as high as my monitor goes), so close enough to 1080p I suppose.

The results there show that even with the load on the pixel shader, they are pretty comparable, and I also have allot more going on in the water shader than in the old one (first shot).
Anyway, it's not really that bad, the pixel shader routine, very few instructions ....

Here, try it out yourself if you can be bothered. (It's in cg by the way, and nothing special)

Code: Select all

//Converts a (normalized) World Space Normal to a Tangent Space Normal
float3 fromWorldToTangent(float3 wnormal)
{
   float3 cy        = cross(wnormal, float3(0.0, 1.0, 0.0));
   float3 cz        = cross(wnormal, float3(0.0, 0.0, 1.0));
   float  ly        = length(cy);
   float  lz        = length(cz);
   // As long as your normals are good, then this will be the tangent to the current normal
   float3 tangent   = lz > ly ? (cz) : (cy);
   return tangent;
}

//Converts a normal map fetched Tangent Space normal to world space normal
float3 fromTan2World(float3 wnormal, float3 tsnormal)
{
   float3 tangent  = fromWorldToTangent(wnormal);
   float3 binormal = cross(tangent, wnormal);
   return tangent * tsnormal.x + binormal * tsnormal.y + wnormal * tsnormal.z;
}
If any of you can come up with any improvements on this, I would love to see it, my math isn't that great and would apreciate any help I can get.

EDIT: Sorry guys, looking at those comparison pictures, I noticed I was out in my calculations, the revised snippets are in place now, and New Picture using the revised snippets.

Alex
User avatar
DavlexDesign
Orc
Posts: 400
Joined: Thu Apr 23, 2009 7:23 am
Location: Australia
x 19
Contact:

Re: ROAM Planet rendering

Post by DavlexDesign »

G'day guys,

Just diking around again, but getting the lighting added in, instead of just sky reflection, it's using lighting plus specular as well.....
Pretty much full Sunlight.
Pretty much full Sunlight.
Sun coming down.
Sun coming down.
Starting to get there.
User avatar
DavlexDesign
Orc
Posts: 400
Joined: Thu Apr 23, 2009 7:23 am
Location: Australia
x 19
Contact:

Re: ROAM Planet rendering

Post by DavlexDesign »

And again !!!!,
Down to sunset pos.
Down to sunset pos.
And at dusk.
And at dusk.
You can start to see more of the atmosphere coloring affecting the highlights more.
Still a fair way to go on this, but it's coming along.

Alex
palaslet
Halfling
Posts: 61
Joined: Tue May 25, 2010 7:52 am
Location: Fredrikstad, Norway

Re: ROAM Planet rendering

Post by palaslet »

DavlexDesign wrote: Here, try it out yourself if you can be bothered. (It's in cg by the way, and nothing special)

Code: Select all

//Converts a (normalized) World Space Normal to a Tangent Space Normal
float3 fromWorldToTangent(float3 wnormal)
{
   float3 cy        = cross(wnormal, float3(0.0, 1.0, 0.0));
   float3 cz        = cross(wnormal, float3(0.0, 0.0, 1.0));
   float  ly        = length(cy);
   float  lz        = length(cz);
   // As long as your normals are good, then this will be the tangent to the current normal
   float3 tangent   = lz > ly ? (cz) : (cy);
   return tangent;
}

//Converts a normal map fetched Tangent Space normal to world space normal
float3 fromTan2World(float3 wnormal, float3 tsnormal)
{
   float3 tangent  = fromWorldToTangent(wnormal);
   float3 binormal = cross(tangent, wnormal);
   return tangent * tsnormal.x + binormal * tsnormal.y + wnormal * tsnormal.z;
}
If any of you can come up with any improvements on this, I would love to see it, my math isn't that great and would apreciate any help I can get.
Try this:

Code: Select all

//Converts a (normalized) World Space Normal to a Tangent Space Normal
float3 fromWorldToTangent(float3 wnormal)
{
   float3 cy        = float3(0-wnormal.z, 0, wnormal.x);    //cross(wnormal, float3(0.0, 1.0, 0.0));
   float3 cz        = float3(wnormal.y, 0-wnormal.x, 0);    //cross(wnormal, float3(0.0, 0.0, 1.0));
   float ly         = cy.x*cy.x + cy.z*cy.z;                //length(cy);
   float lz         = cz.x*cz.x + cz.y*cz.y;                //length(cz);

   // As long as your normals are good, then this will be the tangent to the current normal
   float3 tangent   = lz > ly ? (cz) : (cy);
   return tangent;
}

//Converts a normal map fetched Tangent Space normal to world space normal
float3 fromTan2World(float3 wnormal, float3 tsnormal)
{
   float3 tangent  = fromWorldToTangent(wnormal);
   float3 binormal = cross(tangent, wnormal);
   return tangent * tsnormal.x + binormal * tsnormal.y + wnormal * tsnormal.z;
}
Computing cross product is heavy on the processor. At least when you know much of the answer because you input a fixed vector.

I ran a simulation of the cross product calculation on my CPU (not GPU), and got about 300% performance boost with my solution.

EDIT:
Also, calculating the length of a vector implies sqrt... If the only purpose is comparing lengths, we don't need to do sqrt. Also we know some of the components will be zero based on the previous calculations of the cross product so we can skip them in the calculation.
So I updated my proposal with a faster length calculation.

Testing both cross product and length calculation together on my CPU showed an impressive 350% performance gain.

EDIT2:
My CG is a bit rusty, so I tried to correct my syntax. Still don't know if its right, but you'll probably understand it anyway.
"It seems that perfection is reached not when there is nothing left to add, but when there is nothing left to take away"
User avatar
DavlexDesign
Orc
Posts: 400
Joined: Thu Apr 23, 2009 7:23 am
Location: Australia
x 19
Contact:

Re: ROAM Planet rendering

Post by DavlexDesign »

G'day palaslet,

Thanks for the feedback, and thanks for your nice little math logic ....
Without the modification from palaslet.
Without the modification from palaslet.
With the modification from palaslet.
With the modification from palaslet.
I ran the test by creating a new fragment shader that was an exact copy of the other original one, but, I changed the Tangent to World routine only so it would be a fair test, also I ran the sim at 1680 x 1050 resolution for maximum impact. I toggled the terrain shader off, and the clouds, and only left the sky (it doesn't use those routines at all), and the water, so there is no difference between the two, other than the colors, and that's it.

From the test you can see that palaslets' mod gave a slight improvement (THANK'S MATE !!!), any improvement is a bonus in my opinion.

Alex
palaslet
Halfling
Posts: 61
Joined: Tue May 25, 2010 7:52 am
Location: Fredrikstad, Norway

Re: ROAM Planet rendering

Post by palaslet »

Alex,

I'd expect more of an improvement really... That only goes to show how optimized the operations on the GPU are :)

I guess you got other stuff going on between frames as well, so the actual gain won't show as much.

Why the change of color?
"It seems that perfection is reached not when there is nothing left to add, but when there is nothing left to take away"
LordBaal
Halfling
Posts: 55
Joined: Thu Mar 31, 2011 9:43 pm
x 2

Re: ROAM Planet rendering

Post by LordBaal »

Holly cow that was some serious progress right there. Good work mr. palaslet and alex!
palaslet
Halfling
Posts: 61
Joined: Tue May 25, 2010 7:52 am
Location: Fredrikstad, Norway

Re: ROAM Planet rendering

Post by palaslet »

Here's some eye candy on the subject of world generation:

[youtube]nizHTdGUWkM[/youtube]

And when you have been impressed, you can read the details:
  • Framerate was capped at 30 fps
    It shows an area of 1 km x 1 km
    It has seven LOD
    It took 12 hours to generate in three VoxelFarm nodes
    The generated mesh dataset is 100 MB
Yes it is voxel and takes longer to load than an tape based Commodore64 game, but it sure looks impressive
"It seems that perfection is reached not when there is nothing left to add, but when there is nothing left to take away"
User avatar
DavlexDesign
Orc
Posts: 400
Joined: Thu Apr 23, 2009 7:23 am
Location: Australia
x 19
Contact:

Re: ROAM Planet rendering

Post by DavlexDesign »

G'day palaslet,
palaslet wrote:Why the change of color?
I already had a second material setup, so when I hit the "o" key, I could change the planet into a Mars like color scheme utilizing the Color Bias crap I have set up, so then it's easy to setup new planets sharing the same material, so I can have baron desert planets, rocky Arizona desert looking topography and stuff, Lush green forest tundra stuff, Alpine mountains with lush green foothills, all the while, it's just using one common material / shader base, set a couple of parameters and it's completely different.

I'm raving again aren't I ? Sorry :oops:

To answer your question, I just used the "o" key to change the material to the Mars colored planet, and made that planet use the new Tangent to world routine, so I could flip between the two versions without changing the camera location and rotation, gives a much more accurate result, rendering the same scene.
LordBaal wrote:Holly cow that was some serious progress right there. Good work mr. palaslet and alex!
Thanks mate, and yes, palaslet is a great help, beautiful reference pictures and even a bit of math (he is proving to be a real asset) and with all those magnificent pictures he's putting up, I want to go visit him over there, I'm a bit of a travel buff.

Hey palaslet, that video is pretty cool, 100 mb for that data set is pretty good, looks pretty detailed and stuff, are the grass and trees voxels as well ?
If that is the case, we are looking at an almost infinite detail type scenario.

As nice as it looks, it wouldn't be practical for what I'm trying to achieve, I'm trying to get similar looking terrain, when I enable my layered destructible terrain routine (still a work in progress, but close) that is. And the foliage, well that's close too, but I'm trying to create this stuff on the fly, not in a huge data store, that isn't where I am at with this thing, and 30fps won't cut it for me, I want this thing running true 3D as well, to really show off the scales this thing can achieve.

Anyway, that's my plan.

Alex
palaslet
Halfling
Posts: 61
Joined: Tue May 25, 2010 7:52 am
Location: Fredrikstad, Norway

Re: ROAM Planet rendering

Post by palaslet »

Hi alex,
DavlexDesign wrote: Hey palaslet, that video is pretty cool, 100 mb for that data set is pretty good, looks pretty detailed and stuff, are the grass and trees voxels as well ?
If that is the case, we are looking at an almost infinite detail type scenario.

As nice as it looks, it wouldn't be practical for what I'm trying to achieve, I'm trying to get similar looking terrain, when I enable my layered destructible terrain routine (still a work in progress, but close) that is. And the foliage, well that's close too, but I'm trying to create this stuff on the fly, not in a huge data store, that isn't where I am at with this thing, and 30fps won't cut it for me, I want this thing running true 3D as well, to really show off the scales this thing can achieve.

Anyway, that's my plan.
I know, as I said, it was all eye candy. 12 hour processing time on a modern computer should be able to put out something like this.

I think the trees are voxels, but the grass actually looks like billboards. I don't really think voxel engines are the way to go in game development (Yes, I have played minecraft, but it's not detailed enough to compare with). Just imagine the computing power needed to animate grass and trees during wind... Your approach seems much more adaptable, and that is what's needed in game development.

You are welcome to Norway by the way :)
"It seems that perfection is reached not when there is nothing left to add, but when there is nothing left to take away"
User avatar
DavlexDesign
Orc
Posts: 400
Joined: Thu Apr 23, 2009 7:23 am
Location: Australia
x 19
Contact:

Re: ROAM Planet rendering

Post by DavlexDesign »

G'day palaslet,
palaslet wrote:You are welcome to Norway by the way
I might take you up on that mate, love Europe, everywhere looks like a post card.

Anyway, I've been toying around again, and getting the cloud data in again, and fiddling with the shader a bit, seeing if I can get it pretty close to what I want, before I actually turn it into true volumes.
Here's what it looks like so far ....
Looking from far out in space.
Looking from far out in space.
Coming in to a closer orbit.
Coming in to a closer orbit.
And finally, on the ground.
And finally, on the ground.
I don't know what you guys thing, but it's not looking too bad from my end here, just a little more tweaking, and I think it will not take much to get these clouds looking really nice.
I've yet to include them in the reflection calculations, but that will come, I want to get all the data pretty right, before I tackle that side of things.

What do you guys think ?

Alex
User avatar
DavlexDesign
Orc
Posts: 400
Joined: Thu Apr 23, 2009 7:23 am
Location: Australia
x 19
Contact:

Re: ROAM Planet rendering

Post by DavlexDesign »

G'day guys,

Here's a shot from that vantage point in previous posts .....
We've seen this view before.
We've seen this view before.
I'm really liking the play of light with the atmosphere and cloud blending.

Alex
LordBaal
Halfling
Posts: 55
Joined: Thu Mar 31, 2011 9:43 pm
x 2

Re: ROAM Planet rendering

Post by LordBaal »

http://www.ogre3d.org/forums/download/file.php?id=4112
Unless you open and maximize the image, this looks more like a picture than a generated graphic. I say bravo!

http://www.ogre3d.org/forums/download/file.php?id=4114
http://www.ogre3d.org/forums/download/file.php?id=4113
This two are great too. It's noticeable that theres work to be done but the overall... what would be the word... volume, distribution, size... I don't know, seems correct.
http://www.ogre3d.org/forums/download/file.php?id=4115
This one, again, looks more like a picture, aka mission accomplished.
User avatar
DavlexDesign
Orc
Posts: 400
Joined: Thu Apr 23, 2009 7:23 am
Location: Australia
x 19
Contact:

Re: ROAM Planet rendering

Post by DavlexDesign »

G'day LordBaal,

Your planet is looking pretty nice at the moment, and thanks for the nice words, always gives me a boost when I hear nice things.
Won't be long, and your planet will be able to be flown around and viewed by all, Got to do a bit more work though, adding in the foliage and stuff, so you can roam around, and actually really see the scale of this thing.
These little pics really don't do it justice, don't forget, this planet is a 1:1 with our moon, so when you get to ground, the stuff you see is really far away. Those clouds are 100s to 1000s of kms across, all that kind of thing. Can't wait to upload the new demo for you guys, but I want to make sure it is pretty right first, I don't want to be laughed at.

Alex
LordBaal
Halfling
Posts: 55
Joined: Thu Mar 31, 2011 9:43 pm
x 2

Re: ROAM Planet rendering

Post by LordBaal »

Don't think anyone here, or any other place would dare laugh at your work.
User avatar
DavlexDesign
Orc
Posts: 400
Joined: Thu Apr 23, 2009 7:23 am
Location: Australia
x 19
Contact:

Re: ROAM Planet rendering

Post by DavlexDesign »

G'day guys,

Talk about touchy, I was wondering why I had such a large frame hit with the clouds on, so I looked a bit deeper, and found I was looking at just 1 register incorrectly, instead of looking at a z component of a vector, I was looking at a y component, at that screwed my early out of this thing, must have been too late last night....
Showing the speed improvement even out here.
Showing the speed improvement even out here.
Revised shot of the previous post picture. (note the frame rate change).
Revised shot of the previous post picture. (note the frame rate change).
I am a little happier now, that was bugging me in my sleep last night.

Alex
User avatar
DavlexDesign
Orc
Posts: 400
Joined: Thu Apr 23, 2009 7:23 am
Location: Australia
x 19
Contact:

Re: ROAM Planet rendering

Post by DavlexDesign »

G'day guys,

Here's an interesting one for you, I moved all the texture fetches up higher in the fragment shader, and left all the calculations after the texture fetches, and that seemed to make an improvement also ...
Note the frame rate, around 5 fps improvement.
Note the frame rate, around 5 fps improvement.
It must be something to do with the concurrency in the GPU or something, I'm not sure, but I will experiment further, mainly for the terrain and water shader and stuff, to see if I'm dreaming or not.
Have any of you guys found the same thing ?

Alex
Post Reply