New Features in the oven...

Hey everyone,

I don't add a whole lot of new features very often to PCZSM (not because it doesn't need any, but more because I'm too busy with other stuff). But, I thought I'd let you know what's coming in future updates to PCZSM:

1) Portals will become MovableObjects. This is mainly to allow for more efficient updating through dirty tracking. Initial work was done by lf3thn4d (Thank you!) and I've found a few bugs that I've been working to stomp out. This will be especially helpful for those of you who have huge scenes with tons (hundreds to thousands) of portals.

2) I'm working on Anti-portals right now, but it's going a little slow because I'm under heavy load on other parts of my current game project. I want this to go into the next major point revision of Ogre (1.7 or 1. but it might not make it in time. Antiportals will first be limited to planar quads. Later on (maybe) I'll add convex meshes or boxes.

Chaster

Just to clarify, a portal is like a window to another scene, and an anti-portal is a wall blocking view within a scene? If this is the case, maybe occluder would be a more appropriate name. Also, will anti-portals be Movable Objects also?

Just to clarify, a portal is like a window to another scene, and an anti-portal is a wall blocking view within a scene? If this is the case, maybe occluder would be a more appropriate name.

Yes, that's essentially correct. I was going to call them occluders, but in implementation, they will be an option on a portal, so they don't get to have a new name. ;p

Also, will anti-portals be Movable Objects also?

Since they will just be a switch you flip on a given portal, yes.

Chaster

Portals are what connect zones together right? So if occluding is a property of a portal, does that mean the camera won't see past the portal, but could walk through it and end up in another zone?

My current plan is to have anti-portals act just like occluders alone. The camera will not be able to use an anti-portal to go to another zone. They will just be occluders.

You're probably wondering why I don't just make a separate class for anti-portals and call them occluders (instead of making anti-portals a type of portal). The reasons are three-fold:

1) most of the code for the occluder would be straight from the portals.
2) Occluders and portals are closely related for scene traversal/light propagation. In fact, I need to do a sort on portals and anti-portals (occluders) and having them in the same class greatly simplifies the code and makes it more efficient.
3) Making the anti-portals a type of portal allows users to simply re-use any portal editing/placement code they have and just add a single call to create an anti-portal.

I could make anti-portals inherit from portals, but I judged this to be not a good way to go in this particular situation. I would have to re-jigger a lot of code to objectify the relationship between cameras/lights/portals to make it worthwhile, and honestly, it's just not worth my (very limited) time right now.

Chaster

Thanks for the information. I was going to suggest the option of creating a "PortalBase" class, and deriving the Portal and Occluder classes, but it sounds like it would be more work than you have time to develop.

Eh, that might be worthwhile. Available time is very, very sparse in my crazy world, but that would be a more OO approach... I'll take a look at the ramifications..

Chaster

Oh. PCZSM has a forum of it's own now. Yay. Onto antiportal/occluder stuff. I'm rather interested to get it in. Maybe I'll be able to help up. I'll give a hoot when I find some time to have a look.

Chaster: Would be good if you could point me to the right direction of the portal scene traversal/light propagation code. I'm not very familiar with those code. When I look into that, I'll also attempt to split up the algo into a PortalBase class.

Lf3THn4D,

PCZSceneManager::_updateSceneGraph( Camera * cam ) is a good place to start. It's the main "loop" so to speak of PCZSceneManager.

Eric

Ok, I'm giving a look into this right now.

I'll try to split up generic part of the portal methods into the base class first. Though PortalBase sounds kinda off but I can't seem to think of any other name for it. So I'll stick to PortalBase for the moment.

Ok, I have a good idea how to add the occluder/antiportal feature in. However, there's a few things I need to verify.
1. Should occluder/antiportal only occlude portals or should it also occlude scene objects? - if it only does portal, then antiportal is a more proper name.
2. Should all occluder in a zone be used for occlusion or only a fixed number of occluder nearest to the camera be used?
3. AABB and Sphere portal type doesn't make sense with occluder. However, if AABB can be translated to a Box that can have orientation, it would help alot. I've given some thoughts on this one. Instead of building 6 portal planes, the Box occluder should be able to cut down culling testing with a minimum of 8 clipping plane and maximum of 9. Compared this with building 6 portal planes, 6 portal planes would mean minimum of 10 clipping plane and maximum of 15 (each portal plane takes up 5 clipping plane - we can only view max 3 planes at once; 3 * 5 = 15).
4. Being that now portal is a movable object, can we safely assume that all portal must reside in a scene node? If that's the case, we could cut down culling code since we already check for scene node visibility. From there we should be able to find out if scene node has portal and traverse through it. This will improve performance a bit i think. However, it will probably screw up the order of front to back rendering of the zones? I dunno about this one or if it's even a problem.

Anyways, for 3, it makes it quite difficult to split portal stuffs in to the base as most of it doesn't even make sense for occluder.

So how should I proceed?

1. I really think antiportals should also occlude objects and not just other portals. One of huge uses of occluders is to cut out pieces in the scene. We'll just have to see how much this actually helps (if the processing of occlusion outweighs the savings).

Yeah.. I think so too. However, occlusion testing is actually quite expensive. For it to work, each object will have to be tested with every visible occluder. Which means a lot more processing compared to just plain portal traversal. That is also why I suggested 2. That way, if we only limit to say first 3 nearest occluder, it will cut down the number of tests needed.

Anyways, while I'm at this, I'll like to suggest another new feature: Manual Zone exclusion. Given zone A <-> zone B <-> zone C. We know that when camera is in zone A, there's no way it will see zone C as there's objects placed around in a way zone C will never be visible. Under this circumstance, if zone A can have an exclusion list that excludes zone C, we will not need to use occluder to hide other zones from rendering. This will help for cases where it would take too much occluders to fully occlude a portal into another zone. I got this idea from the unreal legacy wiki (http://wiki.beyondunreal.com/Legacy:Map ... _rendering). So it's not my idea at all.

This idea should be easy to implement. Of course, we could even move it into the portal part for more flexibility.

Another feature I wanted to suggest is for the antiportal to support a list of zones it will ignore testing with. That way, we could build buildings with interiors but have it as an occluder in itself to block whatever that is behind the building. At the same time, the interiors can be resided in another zone and be rendered correctly. Got this idea from the same wiki page as well. It's more clear cut there. I think this will be a good feature to have.

Start simple, with just Antiportals instead of full on occluders.

Chaster

Alright, i'll start with splitting the portal to portal base. Then move on to just antiportal vs portal. Once that is done, we'll see how we go from there.

Looking at it, antiportal should basically be reverse of portal. Which means it's just a matter of flipping the testing code. So antiportal could even be used as a culling AABB or sphere. Though what use that is I dunno. But it'll work as it should. Personally, imho, I found AABB and Sphere portal rather pointless. Can't seem to think of a good use case for it. For the most part, scene objects are already doing their own AABB check.

That's basically the gist of it, but I'll add the following tips:

You'll need to sort the portals and antiportals by distance from camera when doing the check for visibility. Currently, the code does not do this.

Also, portals currently add only those clip planes which are visible when the portal is "added" to the PCZFrustum. Visibility checks with simple portals just require a check to see if the BV of the node in question is outside of any SINGLE clipping plane. This is insufficient for antiportals. You will need to check to see if an object is completely within all 5 planes of the antiportal (4 planes created by the sides of the portal quad and the camera viewpoint plus the plane of the quad itself) to verify that a portal (or object) is completely occluded by the antiportal.

Take a look at the "isAntiportal" flag I added to the Portal class. It was just a start, but it should provide clues in the code where you will need to make modifications for handling antiportals.

Eric

You'll need to sort the portals and antiportals by distance from camera when doing the check for visibility. Currently, the code does not do this.

Yes, I did thought of that too. Though I was thinking if we add an additional plane as you described, there should be no need to sort. Unless you want to do antiportal vs antiportal check and remove checking of hidden antiportal. The question is if it's faster to sort or just do all antiportal checks. I guess sorting will help when we have many antiportals. In a way, sorting should be pretty quick and cheap when we have few of them lying around anyways. So yeah.. sorting it is.


Also, portals currently add only those clip planes which are visible when the portal is "added" to the PCZFrustum. Visibility checks with simple portals just require a check to see if the BV of the node in question is outside of any SINGLE clipping plane. This is insufficient for antiportals. You will need to check to see if an object is completely within all 5 planes of the antiportal (4 planes created by the sides of the portal quad and the camera viewpoint plus the plane of the quad itself) to verify that a portal (or object) is completely occluded by the antiportal.

From what I see, all we need to do is just do the inverse of what portal does. Right now for portal, it test to see if a BV is outside any single clipping plane it's hidden. So for antiportal, we can still leave it the way it is, and when it says it's hidden, we say it's visible. This of course assumes that majority of the testing will fall in the visible category where the worst case would be when everything falls inside the antiportal clipping planes.

One thing though. I noticed when adding portal culling planes, the planes are culled off when it lies outside of any extra culling planes. This wouldn't be an issue as long as we don't stack antiportal culling. So i think it's fine. Just thought I'll mention it.

Btw, I found that in your portal checking code, you check every point of the portal for every culling plane. I think that could be optimized with an early out if you just check if any single point lies outside of any single culling plane. Should I fix that and submit a patch?

From what I see, all we need to do is just do the inverse of what portal does. Right now for portal, it test to see if a BV is outside any single clipping plane it's hidden. So for antiportal, we can still leave it the way it is, and when it says it's hidden, we say it's visible. This of course assumes that majority of the testing will fall in the visible category where the worst case would be when everything falls inside the antiportal clipping planes.

One thing though. I noticed when adding portal culling planes, the planes are culled off when it lies outside of any extra culling planes. This wouldn't be an issue as long as we don't stack antiportal culling. So i think it's fine. Just thought I'll mention it.


Having multiple antiportals behind each other is definitely a possibility, and probably a common one if people use "privacy walls" in front of doorways as antiportals. So I think we'll need to account for stacks of antiportals.


Btw, I found that in your portal checking code, you check every point of the portal for every culling plane. I think that could be optimized with an early out if you just check if any single point lies outside of any single culling plane. Should I fix that and submit a patch?

That's not sufficient culling condition. You can have a portal with all corners outside of different culling planes, but the portal is still visible. For example, if you have two portals, a big one and a small one right in line with each other and you are looking through the small one at the big one. The small portal's culling planes would be added to the frustum, and the big one's points would all be outside of one (or more) of the culling planes, but the big portal is still visible - it just covers the entire culling frustum created by the small one - no single plane culls the entire big portal.

Chaster

Having multiple antiportals behind each other is definitely a possibility, and probably a common one if people use "privacy walls" in front of doorways as antiportals. So I think we'll need to account for stacks of antiportals.

Yes, but stacking antiportals doesn't seem to make sense for me. For basic portals it works since it's all about subset of convex culling region. But antiportal is about Union of culling regions which means a lot of concave cases which wouldn't work with the frustum culling test. So the only way is to test them separately. And to cut down the test case, we can sort the antiportals and test them against each other in the front to back order first. Then we omit those that are fully occluded by the nearer antiportal.


That's not sufficient culling condition. You can have a portal with all corners outside of different culling planes, but the portal is still visible. For example, if you have two portals, a big one and a small one right in line with each other and you are looking through the small one at the big one. The small portal's culling planes would be added to the frustum, and the big one's points would all be outside of one (or more) of the culling planes, but the big portal is still visible - it just covers the entire culling frustum created by the small one - no single plane culls the entire big portal.


Maybe I didn't make myself clear. What I meant was like this:
// For each active culling plane, see if all portal points are on the negative
// side. If so, the portal is not visible
pit = mActiveCullingPlanes.begin();
while ( pit != mActiveCullingPlanes.end() )
{
PCPlane * plane = *pit;
// set the visible flag to false
visible_flag = false;
// we have to check each corner of the portal
for (int corner = 0; corner < 4; corner++)
{
Plane::Side side =plane->getSide(portal->getDerivedCorner(corner));
if (side != Plane::NEGATIVE_SIDE)
{
visible_flag = true;
break; <------------------ Add this to skip checking other points as there isn't any need anymore.
}
}
// if the visible_flag is still false, then this plane
// culled all the portal points
if (visible_flag == false)
{
// ALL corners on negative side therefore out of view
return false;
}
pit++;
}


Having multiple antiportals behind each other is definitely a possibility, and probably a common one if people use "privacy walls" in front of doorways as antiportals. So I think we'll need to account for stacks of antiportals.

Yes, but stacking antiportals doesn't seem to make sense for me. For basic portals it works since it's all about subset of convex culling region. But antiportal is about Union of culling regions which means a lot of concave cases which wouldn't work with the frustum culling test. So the only way is to test them separately. And to cut down the test case, we can sort the antiportals and test them against each other in the front to back order first. Then we omit those that are fully occluded by the nearer antiportal.

Ah, I see. I wasn't sure what you meant by "stacking". I agree, the antiportals need to be handled separately (keeping all of their culling planes grouped together).


That's not sufficient culling condition. You can have a portal with all corners outside of different culling planes, but the portal is still visible. For example, if you have two portals, a big one and a small one right in line with each other and you are looking through the small one at the big one. The small portal's culling planes would be added to the frustum, and the big one's points would all be outside of one (or more) of the culling planes, but the big portal is still visible - it just covers the entire culling frustum created by the small one - no single plane culls the entire big portal.


Maybe I didn't make myself clear. What I meant was like this:
// For each active culling plane, see if all portal points are on the negative
// side. If so, the portal is not visible
pit = mActiveCullingPlanes.begin();
while ( pit != mActiveCullingPlanes.end() )
{
PCPlane * plane = *pit;
// set the visible flag to false
visible_flag = false;
// we have to check each corner of the portal
for (int corner = 0; corner < 4; corner++)
{
Plane::Side side =plane->getSide(portal->getDerivedCorner(corner));
if (side != Plane::NEGATIVE_SIDE)
{
visible_flag = true;
break; <------------------ Add this to skip checking other points as there isn't any need anymore.
}
}
// if the visible_flag is still false, then this plane
// culled all the portal points
if (visible_flag == false)
{
// ALL corners on negative side therefore out of view
return false;
}
pit++;
}


Ah, again, I see. You meant early out because the portal is known to be visible. I thought you meant early out because the portal is culled by a single plane. My bad. If that's all it is, don't bother with the patch, I'll add the early out break.

Chaster

Alrights, I've been quiet for quite a while. Just to give you guys a head up. I'm halfway working on adding antiportal. Due to the Chinese New Year holiday week here I didn't get much job done. However I've managed to get at least this much:
- Created PortalBase class and moved generic stuffs over.
- Made Portal subclass PortalBase.
- Undo checks on antiportal stuff in Zone methods. (we should really keep a separate list for antiportal)
- Modified open() close() method to setEnable() method. (I think this sounds more sane as you don't 'Open' 'Close' antiportal)
- Made some methods in PortalBase const to allow access when they are const cast. (Mainly those derived values)
- Added PortalFactory to allow easy type compare between Portal and AntiPortal classes through the TypeFlags.
- Made PCZSceneManager assign PortalFactory as creator when instantiating Portal. (so that it will be able to access the right TypeFlags)
- Added PortalFactory registration and unregistration when plugin start and end.

That should be about it. I hope I haven't missed anything.

I've tested these to work pretty much with no issue. Now I'll start working on actually getting antiportal in.

P.S. Do tell if I'm changing some stuffs the bad way. I'll gladly amend it as best as I can.

Those all sound like reasonable changes to me. I'll wait until I see the code though to pass judgement. Carry on though, I appreciate your efforts!

Chaster

Alright I'll update more about what I've done:
- Created AntiPortal class.
- Added AntiPortalFactory class.
- Added additional methods for creation and deletion of AntiPortal class in PCZSceneManager.
- Added methods to add / remove AntiPortal in PCZone class.
- Moved methods that add /remove Portal from DefaultZone and OctreeZone to PCZone. (They are doing the same thing. This way will be easier to manage. Produces smaller code too.)
- Optimized updatePortalsZoneData() to avoid checking if both portal did not move. (Chaster: You commented that part to fix the bug where I only checked the second portal. I've made it to check both now which should fix the bug and also do less checking.)

This patch is starting to feel very big.
My next move will be to add AntiPortal traversal to other zones in updatePortalsZoneData().
Then I'll move on to the real deal of AntiPortal culling. Then I should be done.. I think.
(obviously after that will be adding an AntiPortal to the sample project.)

Could rendering efficiency be improved by making the portals static (i.e. unmovable)?

-Fish

Yes, that would help a lot if you have many static portals and few moving portals. But it's not a rendering efficiency optimization though. That's more like portal updating efficiency. But I'm not going to do that in this patch. Maybe next time. For now, I just want to get AntiPortal up.

Update:
- Added AntiPortal traversal through portal code.
- Reordered some conditionals in updatePortalsZoneData() for better early out case. (Test pointer compare before intersection check)
- Removed list clearing from PCZone destructor. (They are not needed.)

[edit]
Let me apologise before hand for my rant next. IMHO, but the PCZSM code isn't very tidy. I'm seeing many indentations everywhere that are mixed up with spaces and tabs. And variable declaration doesn't seem to have any standard. Sometimes it's BlahBlah& var, sometimes it's BlahBlah & var and other time it's BlahBlah &var. It gets on my nerves a little. Also I see many single character variables that doesn't really describe what that variable stands for. Once in a while I get this itch to fix everything up. Really sorry about this rant, Chaster. You did an awesome job for this Scene Manager. Infact I'm harvesting the goodness of this in my game. It's just the coding standard leaves something to be desired. I'm suggesting to have a code tidy move after I get AntiPortal in.

Maybe I'm too much of a perfectionist. But I think it's good if we can try to fix it up to at least follow the Ogre coding convention. I'll feel better at contributing too.
[/edit]

[edit]
Let me apologise before hand for my rant next. IMHO, but the PCZSM code isn't very tidy. I'm seeing many indentations everywhere that are mixed up with spaces and tabs. And variable declaration doesn't seem to have any standard. Sometimes it's BlahBlah& var, sometimes it's BlahBlah & var and other time it's BlahBlah &var. It gets on my nerves a little. Also I see many single character variables that doesn't really describe what that variable stands for. Once in a while I get this itch to fix everything up. Really sorry about this rant, Chaster. You did an awesome job for this Scene Manager. Infact I'm harvesting the goodness of this in my game. It's just the coding standard leaves something to be desired. I'm suggesting to have a code tidy move after I get AntiPortal in.

Maybe I'm too much of a perfectionist. But I think it's good if we can try to fix it up to at least follow the Ogre coding convention. I'll feel better at contributing too.
[/edit]

LOL, sorry about the coding style Lf3THn4D. I am guilty as charged with regards to naming variables and such. However, the indentations are NOT my fault. I am very consistent with indentation, but apparently, some others who have touched the code are not (especially when using other editors). Feel free to make the formatting more "consistent" if it makes you feel better. Everyone (including myself) will appreciate it.

Chaster

Finally! I've got it working. Yay!

Changes as of the above few posts and bellow:
- Made changes to PCZCamera and PCZFrustum classes to be aware of portal and antiportal differences.
- Added methods isFullyVisible() to PCZFrustum for antiportal occlusion test.
- Made changes to scene traversal to deal with AntiPortal culling (sort by distance, then do cull test before portal traverse)
- Made portal classes invalidate it's derived values when attached to scene node. (we should probably also deprecate the setNode() method. It's non standard and not needed anymore)

And here's the patch:
https://sourceforge.net/tracker/index.p ... tid=302997

P.S. Sorry for the huge changes. Hope I didn't screw anything up.
P.S.S. Now that I'm done, I'll be doing code tidying once in a while when I find the time. And also maybe changing PortalList stuffs to use std::vector for faster access and lower memory consumption. (std::list is a doubly link list which makes it real bad since it keeps an additional two pointers to it's neighbors on each element.)

Woohoo! Excellent, I will take a look asap and run some tests. If everything works as hoped, antiportals will go into my next commit. But everyone needs to be patient, it takes me time to test... Big Thanks lf3thn4d!

Hey Chaster,
I made a mistake in my code which caused NULL pointer access violation. I've submitted a fixed patch for it in the tracker.

Thanks Lf3thn4d. I apologize to those who are waiting on this cool new feature. I've been very swamped with work (now I have two completely unrelated projects going simultaneously... crazy!) so I have not been able to integrate the antiportals yet. I hope to get to it asap though.

Chaster

Hey Chaster,
Sorry to bug you but when will we get anitportals in? I'm itching to add Zone<->Zone traversal filter and also code clean ups.

Yeah, sorry about the long delay. I've been slammed with work and lots of bugs (which have proven tough to track down), and since we use the PCZSM in our project, I have been kinda slow with the patches for the last couple months. I'll make a concerted effort to get it in this week though. I want the antiportal stuff too... Just burning the candle at both ends a bit much you know...

Chatser

I've noticed a strange bug which seems to crop up only in the PCZTestApp. The interior rooms are not culled when the camera is outside the buildings. At least, they don't seem to be culled. I don't know how long this bug has been in there either because I went back to the version before the anti-portal patch, and the bug still seems to be there. Lf3th4d, if you can take a look as well, perhaps you can see what's going on... One of us should be able to track down the problem.. I'm kind of overwhelmed right now with deadlines...

Chaster

Yes, I know about that problem. It's the test app's bug. Has already been there even before antiportal changes. I believe the patch I submitted already covers this bug. Changes are in the patch file. I've just tested on my side again just to be sure and it's working without any issues.

The bug happens because interior scenenodes was the child of exterior scenenode. Hence it will never get culled since exterior scenenode is visible when you are outside.

I found this problem too when I was testing the antiportal with the test app. Took me quite a while to figure it out.

Hm... That is not the same as original behavior. I assume you corrected this by parenting both rooms and exterior to a common "building" parent node instead of parenting everything to the exterior...?

Chaster

Gah! I hate Tortoise SVN's patch utility... It only *partially* patched some of my files... Grrr.... (i.e. some of your patches didn't go in...).. Time for another full recompile...

Hey Chaster, sorry for the late reply. My ISP gone nuts yet again. sigh.... whole day without internet is a pain in the ass. Yes, that's what I did. Just create a 'root' node where the exterior and interior nodes is the children.

Btw, I found a bug in the PCZSceneManager::_findVisibleObjects(). It's getting the frame count from Root::getSingleton().getNextFrameNumber(). This is bad because when we need to render the scene multiple times, I.E. rendering scene within the compositor passes, nothing will get rendered in that compositor pass. I think we should revert to the commented line which just increment the counter.

Hmm... But there was a very good reason (which I can't remember right now) why I commented that line and went with getNextFrameNumber...

Chaster

Oh.. hmmm.. okay, how about moving the frame counter thing into the renderScene method? Or is that even worst? As it is, I can't get my refraction compositor to work properly due to this. Switching to the previous method helped there. Though like you said, that might have caused some other problem I'm not aware of. I'll try and dig through to see if I can spot it.

I vaguely recall that it might have been something to do with lighting...

Ah... I think I see it. Lights are updated with a frame count check. Hmmm... It's using it to check against the homezone's frame count. That's rather confusing. What's that for?

I am going by memory (away from my work computer right now) but I think it's because the recursive nature of the PCZSM. Without the check, the lights get updated multiple times in the same frame which caused problems I think...

Splitting discussion to a new thread: viewtopic.php?f=22&t=9591&start=0