HOW TO: Integrate Hydrax v0.4 with Caelum

Netskate

23-10-2008 12:31:17

When you go to integrate these two libraries you can encounter a series of problems.

This little how to should help you to solve these problems in the right way and should give you some "tricks" to avoid them.

This thread will deal only about the ProjectedGrid module (also if some problems are related to both hydrax module).

First of all you had to disable Caelum fog and be sure that setfarclipdistance of your camera is something like 9999*6 (if this setting is 0 you can't see the water).

You had to be sure that hydrax is updated AFTER any camera movement, else the projectedgrid infinite plane stay always a frame back from the camera and it

can't follow the camera correctly.

After you are sure you haven't problem with this you can go forward.

One of the most common problem is a black row in the horizon where hydrax infinite plane meet caelum, something like this:



You have two choice:
1. Waiting for the new hydrax v0.5 version that must have a method like: Hydrax::RttManager::addDisabledClipRenderQueue(...);

2. For now, edit hydrax source. The problem is that Caelum objects are rendered in different rendere queues, as you can see in the

CaelumPrerequisites.h:

// Render group for caelum stuff
// It's best to have them all together
enum CaelumRenderQueueGroupId
{
CAELUM_RENDER_QUEUE_STARFIELD = Ogre::RENDER_QUEUE_SKIES_EARLY + 0,
CAELUM_RENDER_QUEUE_MOON_BACKGROUND = Ogre::RENDER_QUEUE_SKIES_EARLY + 1,
CAELUM_RENDER_QUEUE_SKYDOME = Ogre::RENDER_QUEUE_SKIES_EARLY + 2,
CAELUM_RENDER_QUEUE_MOON = Ogre::RENDER_QUEUE_SKIES_EARLY + 3,
CAELUM_RENDER_QUEUE_SUN = Ogre::RENDER_QUEUE_SKIES_EARLY + 4,
CAELUM_RENDER_QUEUE_CLOUDS = Ogre::RENDER_QUEUE_SKIES_EARLY + 5,
CAELUM_RENDER_QUEUE_GROUND_FOG = Ogre::RENDER_QUEUE_SKIES_EARLY + 6,
};


This mean that Caelum objects avoid the disableCustomNearClipPlane that Hydrax disable in order to prevent these kind of artefacts.

So, you need to edit hydrax source to add the Caelum render queues which must be disabled.

Go to this file RttManager.cpp (row 497):

and substitute this:

void RttManager::CReflectionListener::CReflectionQueueListener::renderQueueStarted(Ogre::uint8 queueGroupId, const Ogre::String &invocation, bool

&skipThisInvocation)
{
if ((queueGroupId == Ogre::RENDER_QUEUE_SKIES_EARLY || queueGroupId == Ogre::RENDER_QUEUE_SKIES_LATE)
&& mActive)
{
mRttManager->mHydrax->getCamera()->disableCustomNearClipPlane();
Ogre::Root::getSingleton().getRenderSystem()->_setProjectionMatrix(mRttManager->mHydrax->getCamera()->getProjectionMatrixRS());
}
}

void RttManager::CReflectionListener::CReflectionQueueListener::renderQueueEnded(Ogre::uint8 queueGroupId, const Ogre::String &invocation, bool

&skipThisInvocation)
{
if ((queueGroupId == Ogre::RENDER_QUEUE_SKIES_EARLY || queueGroupId == Ogre::RENDER_QUEUE_SKIES_LATE)
&& mActive)
{
mRttManager->mHydrax->getCamera()->enableCustomNearClipPlane(mRttManager->mPlanes[RTT_REFLECTION]);
Ogre::Root::getSingleton().getRenderSystem()->_setProjectionMatrix(mRttManager->mHydrax->getCamera()->getProjectionMatrixRS());
}
}

with this:
void RttManager::CReflectionListener::CReflectionQueueListener::renderQueueStarted(Ogre::uint8 queueGroupId, const Ogre::String &invocation, bool

&skipThisInvocation)
{
if ((queueGroupId == Ogre::RENDER_QUEUE_SKIES_EARLY || queueGroupId == Ogre::RENDER_QUEUE_SKIES_LATE || queueGroupId ==

(Ogre::RENDER_QUEUE_SKIES_EARLY + 2))
&& mActive)
{
mRttManager->mHydrax->getCamera()->disableCustomNearClipPlane();
Ogre::Root::getSingleton().getRenderSystem()->_setProjectionMatrix(mRttManager->mHydrax->getCamera()->getProjectionMatrixRS());
}
}

void RttManager::CReflectionListener::CReflectionQueueListener::renderQueueEnded(Ogre::uint8 queueGroupId, const Ogre::String &invocation, bool

&skipThisInvocation)
{
if ((queueGroupId == Ogre::RENDER_QUEUE_SKIES_EARLY || queueGroupId == Ogre::RENDER_QUEUE_SKIES_LATE || queueGroupId ==

(Ogre::RENDER_QUEUE_SKIES_EARLY + 2))
&& mActive)
{
mRttManager->mHydrax->getCamera()->enableCustomNearClipPlane(mRttManager->mPlanes[RTT_REFLECTION]);
Ogre::Root::getSingleton().getRenderSystem()->_setProjectionMatrix(mRttManager->mHydrax->getCamera()->getProjectionMatrixRS());
}
}


now the black row in the horizon should be disappear :)

Another problem you can encounter is that sun, moon and/or other Caelum objects cause artifacts when fall into water. So, because they aren't ogre entities

to avoid this you need to add a specified hydrax depth technique to ALL caelum material objects.

This is the problem:


Add this HydraxDepthTechnique to ALL caelum objects materials, to avoid it:
technique HydraxDepth
{
scheme HydraxDepth
pass
{
lighting off
texture_unit
{
colour_op_ex modulate src_manual src_current 0 0 0
}
}
}


Overcome this annoying problems, you can go forward to tweak hydrax sun's reflects with caelum sun's position, colour, etc.

What we want to do?
We want making sun's reflects move over hydrax water plane, and change its colour in base of the current time (day, night etc). We also want change water

colour to make it more dark when night fall.

To do this, the first thing we need is the sun position. There are two way to calculate it.

1. use this function directly from nature demo (you can find it in the showcase forum):
Ogre::Vector3 sunPosition = camera->getDerivedPosition();
sunPosition -= mCaelumSystem->getSun()->getLightDirection() * 80000;

However, in my opinion, this function is not very accurate.

2. it's what I suggest, edit Caelum source adding a getter method for the abstract class BaseSkyLight scene node. In this way you can know the exact

position of every object that extend the BaseSkyLight class, as Sun and Moon :), it's very simple.

In SkyLight.h (around row 103) add this prototype:
/// get current scene node
Ogre::SceneNode *getSceneNode() const;


in SkyLight.cpp (around row 97) add this method definition:
Ogre::SceneNode* BaseSkyLight::getSceneNode() const{
return mNode;
}


Recompile Caelum library, exchanging the old dll in your project with the newest and rebuild your project.

Now all you had to do to get sun or moon position is write a simple function like this:
Ogre::Vector3 getSunPosition(){
return caelumManager->getSun()->getSceneNode()->getPosition();
}


More clearly and more accurated, in my opinion.

So now that we can know tha sun position we need to set some hydrax parameters every frame (before calling Hydrax::update();).

They are:

Hydrax::setSunArea();
Hydrax::setWaterColor();
Hydrax::setSunPosition();
Hydrax::setSunColor();


setSunPosition:
Use one the method descibed above.

setSunArea:
This is a value you can edit or not, I prefer to edit it making the sun area more big when getSunPosition().y is around 0.0
and more small when getSunPosition().y is around his max value (midday), to do this I use a simple linear interpolation.
Note: more small is the value of the sunArea and more big will be it.

setSunColor:
I set this value with "caelumManager->getSun()->getBodyColour()" value.

setWaterColor:
To set this value refer to this thread: http://www.ogre3d.org/phpBB2addons/viewtopic.php?t=8420

in the end, this are the result I have achieved:

http://www.youtube.com/watch?v=TzPpZ1HK-FY

I hope you can do the same, or better and sharing your result with us.

ps: sorry for my bad english, any correction is appreciated.
ps2: thanks to xavyiy for his patience and the help he gave me.

---- EDIT -----
I implement another tweak to avoid that sun reflects is shown on water when sky is cloudy.

There are many solution to do this, but this is the fastest and easiest
solution I have found.

The easiest way was to decrement the alpha value of sun colour in base of the increment of the cloud, this would have made sun reflects fading to trasparent/opaque.
But how xavy told, all the colour in hydrax are contained into a Vector3 instead of a ColourValue to optimize shader calculation (float3 is more fast than float4).

So here my method:
First of all you had to calculate you water "darkest" colour (the colour you would use at sun 0.0 heigth). Call it sunTransparentColour.

After you have done this, when sky is cloudy (choices a value of cloud cover to define it cloudy, for example cloudCover=0.5) instead of set sun colour position as described before, you can set the sun colour in this way:

mHydrax->setSunColor(getColourLinearInterpolation(0.0,hmHydrax->getSunColor(), 1.0, sunTransparentColour, mCaelum->getCloudCover()));

0.0 and 1.0 are the range of possible cloud cover (as defined in caelum).
getsuncolour is the actual sun color on the water.
suntrasparent colour is the colour calculated before.
getcloudcover is the current cloud cover of caelum.

here the code of the getColourLinearInterpolation:
Ogre::Vector3 GameSupport::getColourLinearInterpolation(Ogre::Real Xa,Ogre::Vector3 Ya,Ogre::Real Xb,Ogre::Vector3 Yb,Ogre::Real X){
const Ogre::Real k = Xa - Xb;
Ogre::Vector3 temp;

temp.x = ((((X-Xb)/k)*Ya.x)-(((X-Xa)/k)*Yb.x));
temp.y = ((((X-Xb)/k)*Ya.y)-(((X-Xa)/k)*Yb.y));
temp.z = ((((X-Xb)/k)*Ya.z)-(((X-Xa)/k)*Yb.z));

return temp;
}

Xavyiy

23-10-2008 17:40:01

Excelent explanation, and awesome the final Hydrax/Caelum integration video.

Xavi

Netskate

23-10-2008 18:52:44

wouldn't have been possible without you.

thanks

Caphalor

23-10-2008 20:39:37

Really great work! I had the same problems and they are solved now. :)
But I also have another problem with Hydrax/Caelum concerning underwater rendering.

In addition to the artifacts, the movement of the water and caustic is really strange and all in all not smooth. The problem only appears underwater when using Caelum (it works with a static skybox).

Xavyiy

24-10-2008 08:33:24

Add the HydraxDepth techinque to all Caelum materials ;)

Caphalor

24-10-2008 10:44:15

Thanks for the answer, but I did that already and it didn't solve the issue. :(

Netskate, does it work for you? You don't show underwater rendering your video...

Netskate

24-10-2008 11:26:33

I don't use underwater effects into my game.

After have seen your problem I tried to add them, and here how they looks:



seems to be another strange problem in the horizon with a blank line, and sun doesn't look as should be.

But I haven't set hydrax to disable customfarclipplane for all caelum render queues, can you try this?

ps: the water movement seems to be fine for me in my demo.

Caphalor

24-10-2008 11:57:24

But I haven't set hydrax to disable customfarclipplane for all caelum render queues, can you try this?
That doesn't help. :(

Xavyiy

24-10-2008 11:59:52

@Netskate
Post some debug screenshots of hydrax depth map underwater.

Custom near clip plane isn't involved in this problem(It's only for reflections). The fact is that caelum components seems not to be cliped by the near custom clip plane, so I think the problem is related with Caelum more than with Hydrax, I'll look into it as soon as I finish the new Radial grid module.

Xavi

Netskate

24-10-2008 12:33:45

here:

Xavyiy

24-10-2008 13:07:59

Okey, I know what's happening.
The fact is that Caelums components aren't clipped by the custom near clip plane(Only when camera reflection isn't actived, so... it must be an Ogre bug.., I will post it in Developer ogre forum tonight.), but Ogre should be care of this.. so Or it's an Ogre's bug, or Caelum components have some option for not be clippling.
Can you post a post explaining the problem in Caelum's thread?

Xavi

bharling

30-10-2008 10:18:13

Fantastic video, any chance of a demo?

Netskate

30-10-2008 13:08:34

you can download the demo at this url:

http://www.netskate.site90.net/download ... Hydrax.rar

Caphalor

30-10-2008 13:39:52

Any progress concerning the caelum underwater problem? *bump*

Netskate

30-10-2008 14:05:42

Okey, I know what's happening.
The fact is that Caelums components aren't clipped by the custom near clip plane(Only when camera reflection isn't actived, so... it must be an Ogre bug.., I will post it in Developer ogre forum tonight.), but Ogre should be care of this.. so Or it's an Ogre's bug, or Caelum components have some option for not be clippling.
Can you post a post explaining the problem in Caelum's thread?

Xavi


I suggest you to post in the Caelum forum and explain your problem.

Caphalor

30-10-2008 15:58:51

I thought Xavyiy spoke to you, my fault. I will post it.

cdleonard

31-10-2008 01:30:14

This is very confusing. From what I understand Hydrax uses a projection matrix hack to clip geometry while the camera is submerged. This sort of conflicts with Caelum which wants to always be visible.

Caelum's skydome is automatically resized to fall somewhere between the camera's near and far clip radiuses. It does not try very hard to be close to the far clip distance. Maybe that causes the underwater problem?

You could try to modify Caelum::CameraBoundElement to place the skydome closer to the far clip plane; maybe will prevent any unpleasant slicing?

But the problem is that you're trying to slice something which tries to be infinitely far away. Why exactly do you need to do this?

Xavyiy

01-11-2008 11:39:48

Hi cdleonard,

The fact is that Caelum geometry is not clipped by the camera's custom near clip plane, but yes if reflection is enabled(In the ogre camera), so I think it shouldn't work like this, maybe a bug in Ogre 1.6?

Hydrax doesn't uses any projection matrix hack to clip the geometry when is submerged, the only hack is done for the reflection RTT(That needs to render the sky and clip the other geometry in the scene), with this:

void RttManager::CReflectionListener::CReflectionQueueListener::renderQueueStarted(Ogre::uint8 queueGroupId, const Ogre::String &invocation, bool &skipThisInvocation)
{
if (mRttManager->_isRenderQueueInList(mRttManager->mDisableReflectionCustomNearCliplPlaneRenderQueues, static_cast<Ogre::RenderQueueGroupID>(queueGroupId)) && mActive)
{
mRttManager->mHydrax->getCamera()->disableCustomNearClipPlane();
Ogre::Root::getSingleton().getRenderSystem()->_setProjectionMatrix(mRttManager->mHydrax->getCamera()->getProjectionMatrixRS());
}
}

void RttManager::CReflectionListener::CReflectionQueueListener::renderQueueEnded(Ogre::uint8 queueGroupId, const Ogre::String &invocation, bool &skipThisInvocation)
{
if (mRttManager->_isRenderQueueInList(mRttManager->mDisableReflectionCustomNearCliplPlaneRenderQueues, static_cast<Ogre::RenderQueueGroupID>(queueGroupId)) && mActive)
{
mRttManager->mHydrax->getCamera()->enableCustomNearClipPlane(mRttManager->mPlanes[RTT_REFLECTION]);
Ogre::Root::getSingleton().getRenderSystem()->_setProjectionMatrix(mRttManager->mHydrax->getCamera()->getProjectionMatrixRS());
}
}


How hard could be add to Caelum the feature of not being rendered if in the viewport sky is disable? (Ogre::Viewport::setSkiesEnabled(bool))
This must solve all these integration problems with Hydrax.

Xavi

cdleonard

01-11-2008 17:39:26

Ok; let me see if I understand you. You use the projection clip plane hack for reflection above the water; and that works.

What exactly are you doing when submerged? Completely disabling the sky with the standard clip plane? That will not work.

How hard could be add to Caelum the feature of not being rendered if in the viewport sky is disable? (Ogre::Viewport::setSkiesEnabled(bool))
This must solve all these integration problems with Hydrax.


I guess it could be done but it seems a bit messy. setSkiesEnable is for Ogre's built-in skyboxes and I'd rather keep Caelum completely separate.

If you want to disable Caelum completely you can use visibility flags. CaelumSystem::forceSubcomponentVisibilityFlags(0) should completely disable Caelum rendering for a while. You can also set a custom flag for Caelum and then enable it per-viewport. Is this useful?

Dutchie

01-11-2008 20:40:51

I have the integration totally working, but how exactly do you do the water color changing acording to the ambient? because light blue water at night is not really good :P

Netskate

01-11-2008 22:57:32

I have the integration totally working, but how exactly do you do the water color changing acording to the ambient? because light blue water at night is not really good :P

I'm very happy this "how to" can help you.

In the first post I link this thread, speaking about changing the water colour.
http://www.ogre3d.org/phpBB2addons/viewtopic.php?t=8420

If you can tell me what you don't understand, maybe I can explain it to you, or maybe you can find a better method to do this.

Dutchie

02-11-2008 16:09:36

well i didn't really understand the part "for this I make an interpolation from max a min value of brightest and darkness" and couldnt find a really good way to convert the colors. But i have another thing for it now. its pretty simple but doesn't really get the result like you have, but it is good enough for my game. still have to tweak the colors a bit to create it a bit more blue.

Caelum::LongReal mJulian = mCaelumSystem->getUniversalClock()->getJulianDay();
Ogre::ColourValue mCol = mCaelumSystem->getSunLightColour(mJulian, mCaelumSystem->getSunDirection(mJulian));
mHydrax->setWaterColor(Ogre::Vector3(mCol.r, mCol.g, mCol.b+0.1));


it just takes the sunlight color and adds a bit blue to it and give it to hydrax.

i tried to put a vid of it on youtube, but it doesn't want to convert...

DarkHorizon

03-11-2008 02:18:02

well i didn't really understand the part "for this I make an interpolation from max a min value of brightest and darkness" and couldnt find a really good way to convert the colors. But i have another thing for it now. its pretty simple but doesn't really get the result like you have, but it is good enough for my game. still have to tweak the colors a bit to create it a bit more blue.

Caelum::LongReal mJulian = mCaelumSystem->getUniversalClock()->getJulianDay();
Ogre::ColourValue mCol = mCaelumSystem->getSunLightColour(mJulian, mCaelumSystem->getSunDirection(mJulian));
mHydrax->setWaterColor(Ogre::Vector3(mCol.r, mCol.g, mCol.b+0.1));


it just takes the sunlight color and adds a bit blue to it and give it to hydrax.

i tried to put a vid of it on youtube, but it doesn't want to convert...


Using a water color of:

mHydrax->setWaterColor(Ogre::Vector3(mCol.r - 0.3, mCol.g - 0.2, mCol.b));

...results in a more realistic water color during daytime, fades better at sunrise/sunset, and still prefers the blue component. It's not as accurate as NetSkate's, but is far faster to implement for its "good enough-ness".

Netskate

03-11-2008 11:37:51

Could be another good solution, can you upload a little video to see this?

However I'll try to explain my method.

So, the idea is simple. When sun is at its max heigth (think to midday) we want the water colour more bright, else when sun is at y=0.0 (when it sets) we want the water colour more dark. So our water colour depends directly from sun heigth position.

In my method I simply used a linear interpolation to calculate water colour at a gived Y (heigth) position.

Maybe a little example can clear your dubt.

The lowest sun position is 0.0
The highest sun position is 30120.1 (I calculated it printing on log sun position at every frame, and this is the highest value).

So, at 30120.1 water colour has to be it's original color, at 0.0 it has to be (for example) original color brightness / 15.0 (we decrease the brightness starting from original color, decreasing only the L value, this means you have already converted your colour to the hsl model).

Once you have done this, you can make a linear interpolation like this:
Ogre::Real lum = getLinearInterpolation(0.0, OriginalBrightness/15, 30120.1, OriginalBrightness, getSunPosition().y);

to obtain the new value of brightness at each frame in base of the sun position.

here the prototype of the getlinearinterpolation method:
Ogre::Real getLinearInterpolation(Ogre::Real Xa,Ogre::Real Ya,Ogre::Real Xb,Ogre::Real Yb,Ogre::Real X); // Interpolate the X value by linear interpolation

I hope this can help you.

Netskate

03-11-2008 12:03:59

I added another trick in the first post :)

Nodrev

01-12-2008 21:46:58

edit Caelum source adding a getter method for the abstract class BaseSkyLight scene node.
There's a solution to get the sun scenenode without modify Caelum:
ogreSunSceneNode = ogreSceneManager->getBillboardSet("CaelumSpriteSun")->getParentSceneNode();
and for moon's too, but it's a little more messy:
ogreMoonSceneNode = ogreSceneManager->getBillboardSet("Caelum/Moon/MoonBB/" + Ogre::StringConverter::toString((size_t)ogreCaelumSystem->getMoon(), 1))->getParentSceneNode();

Netskate

01-12-2008 21:49:43

thanks I'll update the first post :)

Iceman1540

03-12-2008 14:16:43

Hey NetSkate, very nice info in this thread it has helped alot.
On the first page you said to disable the fog in Caelum, but I need fog in my app I tried using Ogre fog with no luck, any ideas.
Thanks in advance.

Netskate

03-12-2008 14:21:26

thanks ^^

if you enable caelum fog you have problem with infinite projected grid.

the problem is that when you look at ocean you can't see the horizon because fog cover it, you could try to change fog colour (and change it for every time of the day) but when you look at terrain, maybe not look so good.

could you change fog colour related to what camera is looking at?

Iceman1540

04-12-2008 01:58:43

I need the fog to be everywhere not just the ground, Ogre fog can do this but I can't get it to work.

mSceneMgr->setFog(FOG_EXP, ColourValue::White, 0.0002);

It works with Hydrax but not both.

Zonder

04-12-2008 18:49:52

I need the fog to be everywhere not just the ground, Ogre fog can do this but I can't get it to work.

mSceneMgr->setFog(FOG_EXP, ColourValue::White, 0.0002);

It works with Hydrax but not both.


New here but surely you don't need calem to render when fog is on as you don't see the horizon anyway.

lonwolf

14-12-2008 15:06:08



http://www.netskate.site90.net/download ... Hydrax.rar
the link is dead can you upload it somewhere else?

Netskate

14-12-2008 15:20:18

you can download the updated version at this link:

http://www.mediafire.com/?byqxzdqmilw

Arcanor

20-02-2009 16:27:46

you can download the updated version at this link:

http://www.mediafire.com/?byqxzdqmilw


This demo looks amazingly good. Would you consider releasing the source code for the demo? I think it would be very instructional.

taxhax

09-05-2009 21:11:35

hello there,
i am a student from germany and i try to intergrate hydrax 0.5 with caelum 0.4. i have 2 problems that i cannot solve on my own.

My first problem is with the sunset



i have add this code to all materials but it doesn't work at all

technique HydraxDepth
{
scheme HydraxDepth
pass
{
lighting off
texture_unit
{
colour_op_ex modulate src_manual src_current 0 0 0
}
}
}



my second problem is with hydrax itself



sorry my english is not so good. any ideas to solve these problems ?

Hackslash

11-05-2009 20:07:47

This should be only a problem of hydrax depth. Though u said that you've added hydrax depth to all materials i think u have forgotten to add it in sun.material at "CaelumSphereSun" AND "CaelumSpriteSun"?

For your second question i have no solution but somewhere in this forum this problem was already discussed i think.

taxhax

11-05-2009 21:30:01

thank you for answering my questions, i have solved all my problems with your help.

now i want to make my application more realistic. do you have any idea how i can manage the light in caelum so that shadows and lights for example at the palms work correctly. previously i have disabled all lights from hydrax and i am only working with caelum lights.

didito

27-08-2009 23:06:45

thanks for the howto! i have a few more questions though.


Another problem you can encounter is that sun, moon and/or other Caelum objects cause artifacts when fall into water. So, because they aren't ogre entities
to avoid this you need to add a specified hydrax depth technique to ALL caelum material objects.

Add this HydraxDepthTechnique to ALL caelum objects materials, to avoid it:


technique HydraxDepth
{
scheme HydraxDepth
pass
{
lighting off
texture_unit
{
colour_op_ex modulate src_manual src_current 0 0 0
}
}
}



could you elaborate more on what you mean with that. what are caelum OBJECT materials???
i mean as far as i can see it worked for me by putting it into sun.material and moon.material.
should i put it anywhere else (materials files: starfield, skydome, depthcomposer, groundfog, layered clouds, pointstarfield, precipiation)???
does it matter if i put the technique before or after the "Default" technique (i've read somewhere that this order does matter in ogre material scripts)???
besides that, do i have to add any c++ sourcecode to the app (there were some posts about depths and techniques that are beyond my knowledge)???
could you also please explain what this addon does with the rendering??? why does it solve this specific rendering artefacts???
and finally does all of that apply to the newest version of hydrax (v0.5) and caelum (v0.5) as well???

any info/help welcome! thanks in advance!