Split Ogre in OgreCore and Ogre3d

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
User avatar
nec
Gremlin
Posts: 175
Joined: Fri Sep 29, 2006 5:56 pm
Location: Rosny Sous Bois, France
x 11
Contact:

Split Ogre in OgreCore and Ogre3d

Post by nec »

Hello,

Ogre at start was a 3d engine but the need to have working sample applications forced the Ogre programmers to implement transversal code like memory management, file management, types libraries....
All this transversal code is the foundation that every multiplatform project needs but each multipltaform project implement its own "system abstraction".

May be a split of ogre into :
-OgreCore : transversal code (memory management, file access, thread, time, ...)
-Ogre3d : all the 3D relative code (based on OgreCore)
would be interresting.

This could allow new projects, to use this system abstraction without having to link to all the 3d part of Ogre. Then we will be able to see new projects like an OgreSound, OgreAI, ...
This would be helpful for the futur of Ogre or all the ecosystem around it.

In fact the need of a portable set of system function is common to all multiplatform projects.

This system abstraction should be
- as simple as possible
- overridable
User avatar
nec
Gremlin
Posts: 175
Joined: Fri Sep 29, 2006 5:56 pm
Location: Rosny Sous Bois, France
x 11
Contact:

Re: Split Ogre in OgreCore and Ogre3d

Post by nec »

sorry i missed that papercut are tiny changes.
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Re: Split Ogre in OgreCore and Ogre3d

Post by jacmoe »

:)

Interesting idea, though.
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.
User avatar
xavier
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 9481
Joined: Fri Feb 18, 2005 2:03 am
Location: Dublin, CA, US
x 22

Re: Split Ogre in OgreCore and Ogre3d

Post by xavier »

And one that I have suggested more than once. ;)

I still like the DX and GL approaches -- gl, glu and glut have increasing levels of abstraction/power as you go outward from the core. I still think that Ogre could benefit from such an architectural shift.

[moved topic from Papercuts to Dev Talk]
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.
User avatar
Gohla
Gnoblar
Posts: 10
Joined: Mon Jun 09, 2008 9:43 pm

Re: Split Ogre in OgreCore and Ogre3d

Post by Gohla »

I like this idea! In my libraries that implement the low level functionality I don't really want to depend on Ogre, but it would be nice to have some of its functionality like the resource system and math classes (Node, Vector3, etc) there. Currently I have to copy over the math classes into my library and convert between those and Ogre's math classes which is a pain.
User avatar
Zonder
Ogre Magi
Posts: 1168
Joined: Mon Aug 04, 2008 7:51 pm
Location: Manchester - England
x 73

Re: Split Ogre in OgreCore and Ogre3d

Post by Zonder »

I have to agree as well it is an interesting idea those thing really do have no place in ogre3d other than as a dependancy :)
There are 10 types of people in the world: Those who understand binary, and those who don't...
User avatar
boyamer
Orc
Posts: 459
Joined: Sat Jan 24, 2009 11:16 am
Location: Italy
x 6

Re: Split Ogre in OgreCore and Ogre3d

Post by boyamer »

I like this idea too... :)
reptor
Ogre Magi
Posts: 1120
Joined: Wed Nov 15, 2006 7:41 pm
Location: Finland
x 5

Re: Split Ogre in OgreCore and Ogre3d

Post by reptor »

How can file system access be 'core' to a 3D engine? It should be the other way around. The 3D engine is the core and code that feeds it stuff from a file system, or from elsewhere, is an optional 'utility' component which users of the 3D engine should be able to replace with another component with minimum hassle. There could be an optional file system component next to other components. OGRE would then talk to that component's interface when wanting to load something. People could then hopefully more easily use their own systems for data storage and loading.

This is my view on this topic. I've always thought OGRE is a nice engine, but it's architecture on these things is backwards. Because of this, many people end up basing their programs on top of OGRE, when I think of OGRE only as a component of my program. There is a clear conflict here. It's like the OGRE project is saying this is only a rendering engine but then it has added loads of stuff which has not got much to do with rendering. It is understandable because there must be working samples but I think it could be organised better. Thankfully OGRE is open source! So anybody can change this... but if someone starts working on it there should be commonly agreed goals so more people can join to help.
User avatar
Brocan
Orc
Posts: 441
Joined: Tue Aug 01, 2006 1:43 am
Location: Spain!!
x 8

Re: Split Ogre in OgreCore and Ogre3d

Post by Brocan »

reptor wrote:How can file system access be 'core' to a 3D engine? It should be the other way around. The 3D engine is the core and code that feeds it stuff from a file system, or from elsewhere, is an optional 'utility' component which users of the 3D engine should be able to replace with another component with minimum hassle. There could be an optional file system component next to other components. OGRE would then talk to that component's interface when wanting to load something. People could then hopefully more easily use their own systems for data storage and loading.

This is my view on this topic. I've always thought OGRE is a nice engine, but it's architecture on these things is backwards. Because of this, many people end up basing their programs on top of OGRE, when I think of OGRE only as a component of my program. There is a clear conflict here. It's like the OGRE project is saying this is only a rendering engine but then it has added loads of stuff which has not got much to do with rendering. It is understandable because there must be working samples but I think it could be organised better. Thankfully OGRE is open source! So anybody can change this... but if someone starts working on it there should be commonly agreed goals so more people can join to help.
I'm currently implementing a small game engine based in plugins, and I've the same "problem". Use the ogre resource system or implement my own? or maybe use my own to feed the ogre's one? (two resource systems? redudant...). Use my own core vector / quaternion / (other math structure) classes, or use ogre's ones? (breaking the engine core encapsulation, because ogre need to link to the core...), etc...

I like the idea of split it, tools, math, memory management, profiling... as "OgreTools" and 3D engine as "OgreRenderer" or something like that :D
CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 2903
Joined: Thu Jan 18, 2007 2:48 pm
x 58
Contact:

Re: Split Ogre in OgreCore and Ogre3d

Post by CABAListic »

Well, technically the resource system is already pluggable - if you don't like the default backends (file systems, zip files), you can supply your own. I have repeatedly used PhysFS as the backend, which works really well.

So, how exactly would you want to split the resource system? Just move the default backends out of the core? Or remove the entire resource system? But then how does Ogre get hold of a mesh to render? How do you want to deal with resource dependencies, e.g. texture names which appear in materials etc.? Would you require that all resources are manually preloaded and registered with Ogre? Or would you want a messaging/callback system for that?
User avatar
madmarx
OGRE Expert User
OGRE Expert User
Posts: 1671
Joined: Mon Jan 21, 2008 10:26 pm
x 50

Re: Split Ogre in OgreCore and Ogre3d

Post by madmarx »

OgreCore : transversal code (memory management, file access, thread, time, ...)
-Ogre3d : all the 3D relative code (based on OgreCore)
IMO, the application is supposed to do the system stuff (file access/thread/timer) and there are very good libraries to do that (boost, qt, poco, dlib). In my apps, Ogre3D is only used for 3D relative code, and I don't why would someone wants to use ogre timer (as example) to manage a game logic... Even maths calculations are done outside Ogre, which allow me to use big floats libraries when needed (e.g simulation of planetory systems).

Here are 2 opposite elements, and the common solution :

1/ Such OgreCore would be a waste of effort, because there are already plenty of libraries to do that, in better. I prefer to know the Ogre3D team can work on something related to 3d, than on something that we can already find out there, tested by more people.
2/ But on the other side, we can't of course ask Ogre to 'select' one existing lib and do its life with it. This would either bind Ogre completely to 1 library, or for the developers to develop an horrible #ifndef web to select the correct sub-system element (ex: select between a dlib.timer or a poco.timer) if such elements do not share the same interface ...
3/ The traditionnal answer in C++ to such problem is in a system concept : an interface to all these 'general/system' calls is created, and then, an implementation using one of the 'big' library respect this concept. To avoid virtual calls, 1 typedef is used to select the corresponding big library implementation. This would also allow developers for consoles to 'glue' their systems very easily.

Ideally the team only provide the concepts for "these general need classes" to the community, which provides concepts implementations (this also force the concepts to be clean and documented), and so the team can work more on the 3d-specific features (both 3D concepts and 3D implementations in that case) for which they are more talented than us.

I am curious to know if I am completely wrong, missing the point, or not?

Best,

Pierre

EDIT : "the community, which provides concepts implementations " => I mean I would gladly participate in that.
Tutorials + Ogre searchable API + more for Ogre1.7 : http://sourceforge.net/projects/so3dtools/
Corresponding thread : http://www.ogre3d.org/forums/viewtopic. ... 93&start=0
User avatar
Kojack
OGRE Moderator
OGRE Moderator
Posts: 7157
Joined: Sun Jan 25, 2004 7:35 am
Location: Brisbane, Australia
x 534

Re: Split Ogre in OgreCore and Ogre3d

Post by Kojack »

Ogre is a hybrid of two approaches. It's fairly modular with factories, plugins and components. But ogremain itself is a massive library which includes things other engines split off into foundation or support libs (especially physics engines: physx, bullet and havok all have separate foundation libraries).

People have wanted the maths code split off for ages. There are other parts such as logging, datastreams, etc which aren't directly 3d related.

Wild Magic (the source for some of ogre's maths code) did this when they went from version 4 to 5. They divided maths and core into separate libraries.
(On the other hand they got rid of the separate render system libs and merged them all into one, but that's another story)
User avatar
_tommo_
Gnoll
Posts: 677
Joined: Tue Sep 19, 2006 6:09 pm
x 5
Contact:

Re: Split Ogre in OgreCore and Ogre3d

Post by _tommo_ »

I think that the overall idea is good, but i'm with reptor when he says that at the "Core" we should have the 3D and not the other way around.

To me the Ogre Core should be just the thing that grabs data sets from RAM and displays them on your screen using the API of your choice.
So no file access, no scripts (everything should be doable with plain C++, with scripting as an optional "library" component ), no scene graphs, no scene managers.
Only geometry, textures, lights and lots of shaders, everything thread safe when needed.
Ideally this core would be the perfect fit for platform that require a lightweight engine like IOS and Android.

Then there's the "library", that implements more abstract concerns using OgreCore, like the scene graph or the scripting.
The Math library would be another component, but not one that is optional, as the 3D core itself needs to use its classes... so it wouldn't change much from the current situation.
OverMindGames Blog
IndieVault.it: Il nuovo portale italiano su Game Dev & Indie Games
User avatar
jacmoe
OGRE Retired Moderator
OGRE Retired Moderator
Posts: 20570
Joined: Thu Jan 22, 2004 10:13 am
Location: Denmark
x 179
Contact:

Re: Split Ogre in OgreCore and Ogre3d

Post by jacmoe »

Maybe something for Ogre 1.9 ?
As a preparation for the scenemanager rewamp of Ogre 2.0 ? :)
That would make sense..
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, fueled by Passion.
OgreAddons - the Ogre code suppository.
User avatar
madmarx
OGRE Expert User
OGRE Expert User
Posts: 1671
Joined: Mon Jan 21, 2008 10:26 pm
x 50

Re: Split Ogre in OgreCore and Ogre3d

Post by madmarx »

To me the Ogre Core should be just the thing that grabs data sets from RAM and displays them on your screen using the API of your choice.
I agree, but I don't want to put pressure on the great team !
Tutorials + Ogre searchable API + more for Ogre1.7 : http://sourceforge.net/projects/so3dtools/
Corresponding thread : http://www.ogre3d.org/forums/viewtopic. ... 93&start=0
User avatar
Gohla
Gnoblar
Posts: 10
Joined: Mon Jun 09, 2008 9:43 pm

Re: Split Ogre in OgreCore and Ogre3d

Post by Gohla »

madmarx wrote: 3/ The traditionnal answer in C++ to such problem is in a system concept : an interface to all these 'general/system' calls is created, and then, an implementation using one of the 'big' library respect this concept. To avoid virtual calls, 1 typedef is used to select the corresponding big library implementation. This would also allow developers for consoles to 'glue' their systems very easily.
This would be the way to do it, let ogre depend on interfaces, and let the application implement these interfaces and probably provide some default implementations in a separate library.

Might this change also be related to the OGRE_THREAD_SUPPORT == 3 change, since I see that people had their own resource/workqueue implementations there?
User avatar
Zonder
Ogre Magi
Posts: 1168
Joined: Mon Aug 04, 2008 7:51 pm
Location: Manchester - England
x 73

Re: Split Ogre in OgreCore and Ogre3d

Post by Zonder »

Gohla wrote:
madmarx wrote: 3/ The traditionnal answer in C++ to such problem is in a system concept : an interface to all these 'general/system' calls is created, and then, an implementation using one of the 'big' library respect this concept. To avoid virtual calls, 1 typedef is used to select the corresponding big library implementation. This would also allow developers for consoles to 'glue' their systems very easily.
This would be the way to do it, let ogre depend on interfaces, and let the application implement these interfaces and probably provide some default implementations in a separate library.

Might this change also be related to the OGRE_THREAD_SUPPORT == 3 change, since I see that people had their own resource/workqueue implementations there?
Yes this does make sense. OgreTools is more appropriate for the existing implementations of non 3d related things. Like many have said ogre shouldn't be the foundation of the app. I was the one who requested the renderoneframe method years ago as I disagreed with ogre been the main loop of my app :)
There are 10 types of people in the world: Those who understand binary, and those who don't...
conallmmcg
Kobold
Posts: 29
Joined: Fri Jan 02, 2009 1:50 am

Re: Split Ogre in OgreCore and Ogre3d

Post by conallmmcg »

so from what i gather you are looking to split all the rendering specific stuff of Ogre into ogrecore and all the non-renderingstuff/optional stuff into ogretools?
ogretools being stuff that ogrecore may require to perform certain functionality i.e filesystem, multithreading, memory management etc?
people can write/integrate their own code to handle this functionality but ogretools would be an official implementation provided so that ogre is at least usable to most users right off the bat and as a way of showing how to write/integrate your own stuff into ogrecore.

if this is the case could certain things that would not normally officially be a part of the Ogre itself but would be really helpful, be provided in ogretools.
for example from what i can see the last year or two i've been on here there is no standard scene format, there is dotscene but isn't there a few different implementations of this format each with some differences?

it would be good if there was a standard scene format / loader ( obviously not catering to everyone ) that can do the basics of loading a scene and maybe even support custom data or at least be extended to do so, that works with the stock scenemanger(s) etc, as an implementation of a scene format that you can use as a base or starting point to writing your own.

this would for easier tutorials and starting off with ogre, meaning you can create your scene and instantly use it with most of the example/tutorial code without having to implement one of the other formats or your own ( maybe an editor like Ogitor could implement this)

i'm guessing that there isn't an official/standard format currently as the Ogre team don't want to force a certain style/format upon users?
but if what this split is doing what i gather it is, wouldn't this fit in the ogretools category?
stuff most people would usually need , an implementation is given for you but is optional and you are free to create your own implementation of feature/functionality x.

The ogretools code would give you the necessary code/tools to get up and running and working with Ogre as quickly and easily as possible?
This is what the name ogretools says to me.

anyway it was just an idea of mine, and gave an example of what i meant by stuff that usually wouldn't fit into the main Ogre3d/ogrecore library but could be in ogretools, if you disagree completly then a simple no would suffice lol and i shall leave it at that.
User avatar
_tommo_
Gnoll
Posts: 677
Joined: Tue Sep 19, 2006 6:09 pm
x 5
Contact:

Re: Split Ogre in OgreCore and Ogre3d

Post by _tommo_ »

conallmmcg wrote: ogretools being stuff that ogrecore may require to perform certain functionality i.e filesystem, multithreading, memory management etc?
No; ogrecore would be built as if ogretools didn't exist, so that you can use ogrecore without even bothering to download the tools, in the case you don't need any of them.
For the rest, i agree on the fact that having a project that contains the abstractions could lead to better standards for the content pipeline:
it is optional, so if you don't like some decisions you can as well not use it ;)
So the team could endorse a scene format, an editor, and many other decisions that are now left to the community as the current vertical structure would require them to be enforced on any Ogre user.
For example I hate generic scene formats, so i'm glad Ogre doesn't support any of them, but i see that for many others not having that isn't even an option.
So a splitting of concerns would satisfy both the "lowlevel" (myself :mrgreen: ) and "game engine" users, the first having a trimmed-to-the-bone software, the second having a more encompassing pipeline.

My two cents :D
OverMindGames Blog
IndieVault.it: Il nuovo portale italiano su Game Dev & Indie Games
conallmmcg
Kobold
Posts: 29
Joined: Fri Jan 02, 2009 1:50 am

Re: Split Ogre in OgreCore and Ogre3d

Post by conallmmcg »

exactley! :) _Tommo_ pretty much summarised where i was headed for. it satisfies both/most sides of Ogre users

basically satisfying "low-level" users, who get access to the rendering code they want without the extra, and pick what they want from ogretools (if anything) and code the rest themselves to suit themselves.

then for the newer-intermediate users getting used to Ogre, can dive straight in and learn and get results on screen with the least amount of code, then once they learn more then can implement their own library for feature x or their own scene format etc

this is what i mean by stuff that would not usually enter into the main Ogre3d/ogrecore library, stuff the more advanced users wouldn't be happy with in the main lib but the new users would find very helpful/essential for starting off.
User avatar
Zonder
Ogre Magi
Posts: 1168
Joined: Mon Aug 04, 2008 7:51 pm
Location: Manchester - England
x 73

Re: Split Ogre in OgreCore and Ogre3d

Post by Zonder »

Ogre tools would have some stuff that most people would never reimplement like the memory management so most people would end up with both anyway. Another thing would be in ogre tools I think OgreRoot which auto inherits all ogre tool implementations. So as not to break ogre in all projects in a huge way.
There are 10 types of people in the world: Those who understand binary, and those who don't...
User avatar
liberator
Greenskin
Posts: 117
Joined: Mon Jan 24, 2005 2:27 pm
Location: Sillicon Valley, California
Contact:

Re: Split Ogre in OgreCore and Ogre3d

Post by liberator »

Gohla wrote:
madmarx wrote: 3/ The traditionnal answer in C++ to such problem is in a system concept : an interface to all these 'general/system' calls is created, and then, an implementation using one of the 'big' library respect this concept. To avoid virtual calls, 1 typedef is used to select the corresponding big library implementation. This would also allow developers for consoles to 'glue' their systems very easily.
This would be the way to do it, let ogre depend on interfaces, and let the application implement these interfaces and probably provide some default implementations in a separate library.

Might this change also be related to the OGRE_THREAD_SUPPORT == 3 change, since I see that people had their own resource/workqueue implementations there?
I agree that this would be the way to do it if the library is split.
I have my own foundation libraries (as many people do) and as such I could then leave out the suggested util library to shrink down the module size.
mukik182
Halfling
Posts: 68
Joined: Mon Nov 09, 2009 10:30 pm
Location: Barcelona, Spain
x 1

Re: Split Ogre in OgreCore and Ogre3d

Post by mukik182 »

I've been only a short time around the community and I have already seen some suggestions similar to this one that seemed to end just as a suggestion. I'm not proficient enough to propose a design so I would ask you to create a wiki page and start a real discussion/proposal on how should it be designed so the coommunity can help the team.

I'm thinking of the project as a building in construction, with the team as the architects and "controllers" (I don't know the english word for building workers chiefs) telling users how and where to put the bricks and solving and pointing the tricky questions and the community (and I'm counting myself) as workers. I say this because such a change surely will require a lot of work, and I think, maybe, much of the work will be trivial (to rearrange files, rewrite CMake...) leaving to the team the interface design and some implementations.

I'm not sure and I'm probably too naïve, but if I could help with this little things, I would (but I usually I don't find myself knowledgeable enough to).
User avatar
Klaim
Old One
Posts: 2565
Joined: Sun Sep 11, 2005 1:04 am
Location: Paris, France
x 56
Contact:

Re: Split Ogre in OgreCore and Ogre3d

Post by Klaim »

I think design by commitee is bad for this kind of thing. A better one would be to propose an implementation. If someone have time and will to do it, it's easy to setup a repo and let people follow the changes, maybe participate. If someone else wants too, he can too create a clone and let people see it. Then when proposition (if there are more than one) are considered finished, the team pull the one that seem to "solve the problem".
User avatar
madmarx
OGRE Expert User
OGRE Expert User
Posts: 1671
Joined: Mon Jan 21, 2008 10:26 pm
x 50

Re: Split Ogre in OgreCore and Ogre3d

Post by madmarx »

I agree with Klaim.
I have already seen some suggestions similar to this one that seemed to end just as a suggestion
Yes, but this is not tragic either. Some propositions come back regularly. So it is interesting to see what users are interested in anyway, and if we share the same point of view or not.
After that comes 2 things (if I compare to the regular 'null renderer request') that explain that things often finish just as suggestions :
1/ it simply takes time to do things (and even more for things like that)
2/ maybe there are other priorities right now.

just my 2 cent.
Tutorials + Ogre searchable API + more for Ogre1.7 : http://sourceforge.net/projects/so3dtools/
Corresponding thread : http://www.ogre3d.org/forums/viewtopic. ... 93&start=0
Post Reply