The Boost issue (again)

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: The Boost issue (again)

Post by sinbad »

Wolfmanfx wrote:Why not using openMP instead of boost thread (openMP is a standard not add dep only a compiler switch)
Mostly because OpenMP is designed for some very specific scenarios, that don't really cover what we do. It's great for high-level splitting of work packages, but it's really mostly useful for data parallelism, or very high level process parallelism. Put it this way, it's great for business batch jobs and scientific processing, but it's not really flexible enough for what we need.
xavier wrote:It's truly enormous. If you somehow can limit yourself to things that are truly header-only, it's less onerous, but eventually you really want to use something that requires a lib build, and then the interdependencies start mounting (they do even if you are just "header only", because Boost code is incestuous).
You've clearly been burned in ways I never have. :) I use rather small areas of Boost (threads, signals, bind, function, shared_ptr etc) on a regular basis and have absolutely no issues with it. Sure, every few months you have to pull down a new version and leave your machine building it over a lunchtime - actually you don't even have to do that if want to try to selectively build it, and most of the bits I use don't have libraries anyway, but to be honest I don't bother - I go to lunch and it's done when I get back, then I never worry about it again. Sure there are cross-references in the headers (bind to function etc) but so what? I never need to tease them apart anyway since I have the whole thing sitting there to be referenced as I need. This isn't unusual - The STL, Windows Platform SDK, DirectX and many others are all interwoven with themselves too, but you don't try to pull those apart either ;) If you treat Boost just as any other SDK, rather than something to be dissected, it's easy to use.
User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
x 56
Contact:

Re: The Boost issue (again)

Post by Klaim »

xavier> If it's only about installation and building (on windows), the installers really help as you select exactly what you want, wich lib, wich mode (debug/release), static/dynamic, wich compiler - already compiled for you. Not sure that's what you mean exactly?
There are other decisions made recently regarding the project that have made Ogre less attractive to me personally, but none of those are germane to this topic, other than to say that this (Boost) is not a decision I could support.
May I ask witch one?
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Re: The Boost issue (again)

Post by xavier »

sinbad wrote:The STL, Windows Platform SDK, DirectX and many others are all interwoven with themselves too, but you don't try to pull those apart either ;) If you treat Boost just as any other SDK, rather than something to be dissected, it's easy to use.
I guess my point is that I also don't have to go get and build those, either (well, I have to get DirectX, true).

Like I said, some of the decisions made lately may make sense for the Ogre project, but they don't for me personally.
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.
User avatar
cdleonard
Goblin
Posts: 266
Joined: Thu May 31, 2007 9:45 am

Re: The Boost issue (again)

Post by cdleonard »

sinbad wrote:This isn't unusual - The STL, Windows Platform SDK, DirectX and many others are all interwoven with themselves too, but you don't try to pull those apart either ;) If you treat Boost just as any other SDK, rather than something to be dissected, it's easy to use.
Those examples are widely used in the entire industry and have very well-defined deployment mechanisms; which are still not fool-proof. Every now and then issues pop up about msvc redistributables and d3dx9_XX.dll. Ogre can't just start indiscriminately pulling code from the "boost SDK". Deployment-wise every individual boost dll is a separate dependency you have to deal with when building a setup kit for the end user. Pulling in a new dll requires updates in setup scripts.

Ogre should only use a certain well-defined subset of boost; preferably one without lots of dlls. One additional advantage of TR1 would be that microsoft's implementation is part of microsoft redistributables.
User avatar
PolyVox
OGRE Contributor
OGRE Contributor
Posts: 1316
Joined: Tue Nov 21, 2006 11:28 am
Location: Groningen, The Netherlands
x 18
Contact:

Re: The Boost issue (again)

Post by PolyVox »

Tricky one... I can understand people objections to the extra dependancies but I can also see the appeal. Leaving aside the compile/install issues, it would also be important to make sure that Ogre finds Boost easily. Every time I get a clean copy of Ogre I have to tell it where the DirectX SDK is as it looks in the wrong place by default. It's a small step, but slightly annoying... maybe this improves with CMake.

I guess the options are (in order of decreasing desiribility):

1) Make Ogre dependant on only those parts of boost which will be in TR1. An option to switch between boost and compiler vendor version would then be nice. Not too much to worry about here - Ogre would basically just be moving to more modern C++.

2) Make Ogre dependant only on those parts of boost which are header only. Copy these headers into the Ogre source tree (I believe this is actually the appraoch recommended by Boost anyway). Ignore whatever other versions of boost are on the system - we just use our own. Keeps building nice and simple.

3) Make Ogre dependant on more complex libraries which aren't header only. The main candidate seems to be threads. I know nothing about how Ogre uses thread internally, but in my own project I am instead planning to provide 'hooks' such that the user can use whichever threading library they wish, and I avoid being dependant on one. Just a thought.
User avatar
steven
Gnoll
Posts: 657
Joined: Mon Feb 28, 2005 1:53 pm
Location: Australia - Canberra (ex - Switzerland - Geneva)
Contact:

Re: The Boost issue (again)

Post by steven »

In our project we moved away from boost::thread, boost::bind and especially boost::function.

The resulting bytecode was big, difficult to debug (especially with multi-threaded dlls) on all platforms/compilers, the implementation is not optimal and -some times ago- boost::thread didn't even have a read-write mutex! The lack of RWMutex was the reason we dumped boost.

So we are now using:
  • fastdelegate which is a simple .h and is the most optimal way possible to do binding!
  • For the threading & atomic operations we use Poco and/or TBB. We made a simple wrapper around them so users can choose the implementation they prefer. When tr1 is available it will be very easy to use it also as fallback.
Take a look at our thread wrapper - it works perfectly on Win and Linux (no Mac users yet).

IMO Poco is features rich even more than boost but more important it was designed as a whole not the hit-and-miss of boost.
User avatar
Kojack
OGRE Moderator
OGRE Moderator
Posts: 7157
Joined: Sun Jan 25, 2004 7:35 am
Location: Brisbane, Australia
x 534

Re: The Boost issue (again)

Post by Kojack »

I haven't actually used it yet, but I just grabbed Poco and built the foundation. It comes with solutions for several compilers (including vc7.1-9.0) and a cmake project.
Poco is designed for modularity and the different subsystems are easy to isolate. Ignoring the xml (sax2 and dom) module, networking module, utilities module (command line and config file handling, and server apps) and unit testing module, the foundation module is the interesting one.

Foundation compiles to a single 6MB static lib (non static versions are an option too). It contains:
- platform abstraction (data types, big/little endian, os info, system debugger interface)
- memory management (smart pointer, shared pointer with reference counting garbage collection, singleton template)
- string utils (using std::string)
- error handling (exceptions and asserts with special debugger support)
- streams (stream based encoding and decoding of base64, hex and zip data)
- threading (threads, mutexes, events, locks, semaphores, thread pool management, scoped locks, timers, activities (background tasks), active methods (asynchronous tasks)
- datetime (microsecond timestamps and stopwatch, time zones, calendars)
- filesystem (cross platform file system stuff, directory iterator, temp file classes, file finding class)
- logging (console, file, unix syslog, windows event log, remote log, cascaded channels, filters, formatters)
- processes (synch between processes with global mutexes and events, shared memory, interprocess communication)
- shared libs (platform independent class to make plugin writing easier)
- notifications (type safe sending of notification objects both inter and intra thread, notification queue)
- events (synchronous and asynchronous events, similar to events in C#)
- crypt (md2, md4, md5, sha-1 message digests)
- text (converting between utf-8, utf-16, ascii, etc)
- regular expressions (perl compatible)
- uri (uniform resource identifiers)
- uuid (time, name and random based uuid generation)
- cache (generic caching framework)

They claim that Poco is designed to work well on small embedded environments, it's already being used on windows ce and qnx neutrino.

It might not provide as many random things as boost (octonian maths, graph path finding, complete c++ parser, etc), but it might be worth a look to see if it offers what ogre needs compared to boost (at the very least, there's no way their filesystem class can be as horrendous as the boost one, the only component of boost I dislike). Oh, and it's released under the Boost license. :)

(As I said at the top, I've never used it myself)
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: The Boost issue (again)

Post by sinbad »

Well, as i happens I've had a rethink for the bit I originally wanted to use boost::bind for - I was originally going to use a property system to hit 2 birds with one stone - abstraction of serialisation and editor-friendliness - but regarding the former I started to hit some edge cases with dealing with evolving version support (I'm planning for data chunks to change & evolve over time, as .mesh has), and in respect of the latter I started to become unhappy that I was forcing support of reflection information (which is not free) in all cases - wheras what I would like to do is provide the facility to expose properties for editors, but have the ability to omit it when it's not needed.

So, it needs to be a bolt-on anyway, which means the core functionality has to operate without it, which means I don't need it yet ;) I'm going to commit my property support classes anyway, but as an optional component; CMake will only build it when Boost is already detected. For now, I'll 'park' the issue there.

I'm interested in Steven and Kojack's comments (thanks guys). I should properly evaluate the alternatives here when I come back to this - I'll commit the Boost-referencing versions anyway since they're not forcing you to have Boost available anyway (so no change), they're just a 'stake in the ground' and I'm not necessarily ruling out the alternatives. I just want to move on for now!
User avatar
nikki
Old One
Posts: 2730
Joined: Sat Sep 17, 2005 10:08 am
Location: San Francisco
x 13
Contact:

Re: The Boost issue (again)

Post by nikki »

For binding, I'd suggest FastDelegate. It's really small, fast, and simple, and is a single header file.

I have nothing against boost, I in fact love the thing, and I use boost.python, boost.any and boost.serialisation quite a bit. :) I'd prefer header-only libraries as Ogre dependencies though. Are there any specific libraries you're thinking of using that aren't header-only?
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: The Boost issue (again)

Post by Assaf Raman »

boost\thread is not header only.
Watch out for my OGRE related tweets here.
User avatar
Praetor
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 3335
Joined: Tue Jun 21, 2005 8:26 pm
Location: Rochester, New York, US
x 3
Contact:

Re: The Boost issue (again)

Post by Praetor »

Honestly, lately I've been moving away from boost::bind and boost::function and general callbacks and function pointer wrappers back to more raw listeners. It is more verbose obviously, but I find the added explicitness is actually a good thing (purely my preference of course). Managing the different interfaces can be annoying, but boost::bind can be confusing and has caused me difficult to debug situations. I find I'm much more concerned lately with generically representing data than generically representing functions.

Overall, I can't say I'm "against" boost. I personally use Poco now because of its good support for a few things outside of boost, but I can't say I'm against these large, general frameworks on principle. I will definitely agree with xavier about the problem of trying to extract portions of boost. Both good and bad is boost's large amounts of self-referencing. If the only new incentive is to get boost::bind then I'd say it isn't worth it. If there's more that we want to pull in, them maybe it is worth it.
Game Development, Engine Development, Porting
http://www.darkwindmedia.com
User avatar
PolyVox
OGRE Contributor
OGRE Contributor
Posts: 1316
Joined: Tue Nov 21, 2006 11:28 am
Location: Groningen, The Netherlands
x 18
Contact:

Re: The Boost issue (again)

Post by PolyVox »

I have no opinion on what is better out of Boost and Poco (having only used Boost), but one thing I would say in favour of Boost is that parts of it are included in the next version of C++. By limiting Ogre to these parts it's can be viewed as a 'temporary dependancy', that can be removed again in the future.

Also, according to Wikipedia, threading support (based upon Boost::Thread) is actually included in C++0x.

So, is there anything we need from Boost which isn't included in C++0x?
Trigve
Kobold
Posts: 25
Joined: Sun Dec 17, 2006 10:20 pm
Location: Slovakia
Contact:

Re: The Boost issue (again)

Post by Trigve »

I think that adding boost as dependency isn't bad thing but it should be done carefully. As boost isn't (always) ABI/API compatible with previous version I suggest embedding the boost inside the Ogre sources (like nedmalloc as it was suggested). It can be easily done with BCP. And I think it will be better not to use boost namespace directly but use some aliases in Ogre namespace. With this I think clients can use other boost version without confilcts.

Anyway I think that all tr1 stuff will be merged back to std namespace once c++0x standard will be finished.

Trigve
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: The Boost issue (again)

Post by sinbad »

PolyVox wrote:Also, according to Wikipedia, threading support (based upon Boost::Thread) is actually included in C++0x.

So, is there anything we need from Boost which isn't included in C++0x?
Probably not. I've personally enjoyed using boost::signal on other projects but so far we haven't had a need for it; I was thinking of using for generic exposure of custom (subclass) events but have backtracked on that for now.

Regarding boost::bind and similar (including fastdelegate) vs explicit listener classes - I tend to go back and forth on this issue depending on the context. Listeners are more explicit, and that's good - I'm happy with how we use them in the core right now. There are cases though where a more flexible callback approach is hugely useful, particularly when you have rich callback interfaces and/or where the callback interfaces can be extended significantly (e.g. widget libraries). Forcing these specialised callback interfaces to be new classes is awkward in that case, since it involves lots of downcasting, and the alternative is to use some generic event (like RenderSystem::Listener) which encodes the event as a String or something like that, which is less efficient because you need a big if/else statement to figure out which you received. I think there's actually room for both approaches, but it depends how rich your event system is. I've worked on projects which simply would not have worked with explicit listeners and boost::bind / boost::signal was a godsend, but that doesn't mean you should reach for it in simpler cases. Ogre is still probably within the bounds that an explicit listener setup is still workable.
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Re: The Boost issue (again)

Post by jacmoe »

Kojack wrote:Poco is designed for modularity and the different subsystems are easy to isolate.
Thanks a lot Kojack! :D

I am not a fan of Boost - it's syntax is weird - I like POCO much better.
Besides, POCO is designed for embedded systems and is meant to be lean.
And the syntax is very clean and concise - very Ogre like. :)

I read about OGE using POCO a while back, and that's why I checked it out - thanks Steven! :)
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.
User avatar
_tommo_
Gnoll
Posts: 677
Joined: Tue Sep 19, 2006 6:09 pm
x 5
Contact:

Re: The Boost issue (again)

Post by _tommo_ »

I really don't like Boost... and i tried to like it at least two times.

Often it looks like it's made only to show that they can push C++ to the limit, so that you over-engineer situations where a cleaner and simpler (while more verbose) approach could have made everything simpler... as Klaim said and as i myself saw with Lua.

Also, i feel like that much of the added functionality that boost provides just doesn't belong to C++... if you don't want to deal with pointers, you want to have dynamic typing, etc... you could as well use another language, often it's very simple :roll:
For example in my game, i have Lua callbacks that are called by name string: maximum flexibility, and in C++ a call is 4 lines.
You can even mix or call them from a vector, a thing that boost can't surely do as cleanly as a script language.

Apart from this boost is a nigthmare to build and use, and for anyone under the "expert" level its syntax is utter chinese... so that you really don't understand what you are using.
It's a library based on C++ tricks in many ways, imho.

So i don't see it fit to Ogre: it could deal a terrible code bloat, should the user be forced to learn it while learning Ogre.
It could be a viable idea only if it pops up "on demand", so that you can use it only when you want... it would be ugly to have common functions use boost types, for example.
OverMindGames Blog
IndieVault.it: Il nuovo portale italiano su Game Dev & Indie Games
User avatar
nullsquared
Old One
Posts: 3245
Joined: Tue Apr 24, 2007 8:23 pm
Location: NY, NY, USA
x 11

Re: The Boost issue (again)

Post by nullsquared »

_tommo_ wrote: For example in my game, i have Lua callbacks that are called by name string: maximum flexibility, and in C++ a call is 4 lines.
You can even mix or call them from a vector, a thing that boost can't surely do as cleanly as a script language.
My GUI uses boost::function as a generic widget event callback. My Lua interface allows scripts to register arbitrary Lua functions (global/local, named or not) as widget callbacks.

What's the catch? The GUI system knows absolutely nothing of Lua. So how can you possibly do this?

Code: Select all

aButton:update({
    onClick = function(caller)
        print("clicked on " .. caller:name())
    end
})
Well, you couldn't, if it wasn't for boost and luabind ;) (and luabind uses boost, anyway)

Yes, boost does have some awkward syntax here and there. But some of the awkward functionality it offers can't live with a cleaner syntax. Considering many parts of boost are going into the next C++ standard, I'd say they're more than well written and tested ;)

@ Kojack: How is boost's filesystem class "horrendous"? It's one of the most useful, cleanest parts of boost I've ever used :|
User avatar
_tommo_
Gnoll
Posts: 677
Joined: Tue Sep 19, 2006 6:09 pm
x 5
Contact:

Re: The Boost issue (again)

Post by _tommo_ »

Of course I'm not saying that Boost isn't tested and well-written :wink:
It's only that often it's way too obscure and heavy to be integrated in a framework with a diverse target such as Ogre.
As for your example, it demonstrates well my opinion, because without studying it i can't even say if it's written in Lua, C++, or a mix of both.
It's surely more complicated that a single method that takes a string, to me...
you always trade more functionality for more complexity, and i think boost often is not convenient...
OverMindGames Blog
IndieVault.it: Il nuovo portale italiano su Game Dev & Indie Games
warmi
Gnoll
Posts: 674
Joined: Sun May 27, 2007 3:56 am
Location: USA

Re: The Boost issue (again)

Post by warmi »

_tommo_ wrote:if you don't want to deal with pointers, you want to have dynamic typing, etc... you could as well use another language, often it's very simple :roll:
It is not that people don't want to deal with pointers - pointers are good but once you go past simple programs , you often end up with scenarios where pure pointers are simply not good enough ... you end up with massive ownership problems which require very messy solutions if you are just working with raw pointers.

In a situation like that ,a lightweight smart pointer like for instance boost::intrusive_ptr<> is godsend.
Same goes for their intrusive containers which, while somewhat more limited , are a heap- friendly version of standard std containers.

Yeah, some of their stuff is just plain nightmarish in terms of syntax (string_algo comes to mind) but overall I found their stuff to be useful.
Shareth
Gnoblar
Posts: 13
Joined: Tue Aug 21, 2007 7:29 am

Re: The Boost issue (again)

Post by Shareth »

It might be very interesting to see how many people are actually using boost or any other similar library in their projects. As a very general library, IMO boost is used by a very wide range of projects. If that is the case, adding it as a dependency over any other library is a nobrainer. But adding something that is used rarely or known by very few people is very questionable. For example, if boost is used by more than 50% of projects and some other library is used by say 25%, there should be no question what library to pick (there are other things to consider of course, but I'm estimating the quality and other factors to be the same).

On the other note, we use boost a lot in our project. Considering parts of it are going into C++ standard, for me personally the answer to the question 'Should Ogre use boost?' is obvious. I never liked that Ogre is using its own SharedPtr for example. There is really no reason for that.
User avatar
_tommo_
Gnoll
Posts: 677
Joined: Tue Sep 19, 2006 6:09 pm
x 5
Contact:

Re: The Boost issue (again)

Post by _tommo_ »

Doh looks like i will have to make a third attempt at boost :lol:
OverMindGames Blog
IndieVault.it: Il nuovo portale italiano su Game Dev & Indie Games
User avatar
Noman
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 714
Joined: Mon Jan 31, 2005 7:21 pm
Location: Israel
x 2
Contact:

Re: The Boost issue (again)

Post by Noman »

I agree with Shareth. IMO, the biggest advantage boost has over any other generic library (like POCO) is that its common. Poco's forums are pretty quiet. Bigger projects are more likely to survive over time, and while having the source code does allow you to maintain it yourself in case the project stops being updated, do we want to?
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: The Boost issue (again)

Post by sinbad »

The interesting thing to me about POCO is that its threading support seems to be far better than Boost's. It exposes thread priorities (a major missing element of boost::thread), thread pools, and has really nice 'waiting for result' wrappers and other higher-level concepts. For example, see ActiveMethod - a simple way of making an implementation run in parallel, using a thread from a thread pool if you want, with a return type which automatically manages the parallelised result. Compared to what boost::thread offers, that's awesome - it would significantly simplify threaded work compared to what you'd have to write with Boost.

Since I've shelved the general dependence on Boost for now, I'm going to give POCO a run through for the threading support soon, because it looks much more usable and powerful than Boost. Sure, it may have a smaller community, but if it works, I'm open to it.
CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
x 58
Contact:

Re: The Boost issue (again)

Post by CABAListic »

If it's merely about threading, then what about Intel's Threading Building Blocks? It does not seem to have an equivalent for ActiveMethod, but it has some interesting support for parallelising tasks, plus it brings atomic operations and some concurrent containers which could be very valuable for performance. And it's an even smaller dependency than POCO.
I have used neither personally, but if boost::thread is going to be replaced (which is indeed rather basic), it can't hurt to have looked at all alternatives.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: The Boost issue (again)

Post by sinbad »

TBB is GPL (dual-license commercial) and therefore unusable. It's a silly license for a component as small as that, and a textbook bait-and-switch IMO. I'd be happy to use it on a commercial project, but it's not something you can realistically use on an open source project unless you're already GPL.

[edit]Oops, didn't notice the runtime exception. I take it back. Still, it's a bit disingenuous, particularly as they offer a commercial license and don't give clear guidance as to when you would need to buy it (the inference is for commercial support on stable releases - but they don't appear to actually say that). Doesn't make me that comfortable personally. Anyone got more clarification on this? If it makes me nervous, I imagine it might be a turn-off for others too. Yes, I know libstd++ has the same license but a) they don't sell a commercial version and b) we're not actually directly dependent on that.
Post Reply