Split Ogre in OgreCore and Ogre3d

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.

Split Ogre in OgreCore and Ogre3d

Postby nec » Mon Jan 17, 2011 4:48 pm

Hello,

Ogre at start was a 3d engine but the need to have working sample applications forced the Ogre programmers to implement transversal code like memory management, file management, types libraries....
All this transversal code is the foundation that every multiplatform project needs but each multipltaform project implement its own "system abstraction".

May be a split of ogre into :
-OgreCore : transversal code (memory management, file access, thread, time, ...)
-Ogre3d : all the 3D relative code (based on OgreCore)
would be interresting.

This could allow new projects, to use this system abstraction without having to link to all the 3d part of Ogre. Then we will be able to see new projects like an OgreSound, OgreAI, ...
This would be helpful for the futur of Ogre or all the ecosystem around it.

In fact the need of a portable set of system function is common to all multiplatform projects.

This system abstraction should be
- as simple as possible
- overridable
User avatar
nec
Gremlin
 
Posts: 175
Kudos: 11
Joined: 29 Sep 2006
Location: Rosny Sous Bois, France

Re: Split Ogre in OgreCore and Ogre3d

Postby nec » Mon Jan 17, 2011 5:42 pm

sorry i missed that papercut are tiny changes.
User avatar
nec
Gremlin
 
Posts: 175
Kudos: 11
Joined: 29 Sep 2006
Location: Rosny Sous Bois, France

Re: Split Ogre in OgreCore and Ogre3d

Postby jacmoe » Tue Jan 18, 2011 1:37 am

:)

Interesting idea, though.
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, Fueled by Passion.
Ogre AppWizards - Ogre project wizards for VC 8-10, Code::Blocks and KDevelop.
OgreAssimpConverter - command-line to convert models to Ogre format.
TwOgreGUI - wrapper for AntTweakBar GUI library.
I accept donations | Me on Google+
User avatar
jacmoe
OGRE Moderator
OGRE Moderator
 
Posts: 21024
Kudos: 162
Joined: 22 Jan 2004
Location: Denmark

Re: Split Ogre in OgreCore and Ogre3d

Postby xavier » Tue Jan 18, 2011 3:49 am

And one that I have suggested more than once. ;)

I still like the DX and GL approaches -- gl, glu and glut have increasing levels of abstraction/power as you go outward from the core. I still think that Ogre could benefit from such an architectural shift.

[moved topic from Papercuts to Dev Talk]
Do you need help? What have you tried?

Image

Angels can fly because they take themselves lightly.
User avatar
xavier
OGRE Moderator
OGRE Moderator
 
Posts: 9481
Kudos: 22
Joined: 18 Feb 2005
Location: Dublin, CA, US

Re: Split Ogre in OgreCore and Ogre3d

Postby Gohla » Tue Jan 18, 2011 9:16 am

I like this idea! In my libraries that implement the low level functionality I don't really want to depend on Ogre, but it would be nice to have some of its functionality like the resource system and math classes (Node, Vector3, etc) there. Currently I have to copy over the math classes into my library and convert between those and Ogre's math classes which is a pain.
User avatar
Gohla
Gnoblar
 
Posts: 10
Kudos: 0
Joined: 09 Jun 2008

Re: Split Ogre in OgreCore and Ogre3d

Postby Zonder » Tue Jan 18, 2011 2:50 pm

I have to agree as well it is an interesting idea those thing really do have no place in ogre3d other than as a dependancy :)
There are 10 types of people in the world: Those who understand binary, and those who don't...
My Blog - http://www.dreamwalkerblog.com
User avatar
Zonder
Bugbear
 
Posts: 834
Kudos: 19
Joined: 04 Aug 2008
Location: Manchester - England

Re: Split Ogre in OgreCore and Ogre3d

Postby boyamer » Tue Jan 18, 2011 3:53 pm

I like this idea too... :)
User avatar
boyamer
Orc
 
Posts: 443
Kudos: 5
Joined: 24 Jan 2009
Location: Italy

Re: Split Ogre in OgreCore and Ogre3d

Postby reptor » Tue Jan 18, 2011 6:59 pm

How can file system access be 'core' to a 3D engine? It should be the other way around. The 3D engine is the core and code that feeds it stuff from a file system, or from elsewhere, is an optional 'utility' component which users of the 3D engine should be able to replace with another component with minimum hassle. There could be an optional file system component next to other components. OGRE would then talk to that component's interface when wanting to load something. People could then hopefully more easily use their own systems for data storage and loading.

This is my view on this topic. I've always thought OGRE is a nice engine, but it's architecture on these things is backwards. Because of this, many people end up basing their programs on top of OGRE, when I think of OGRE only as a component of my program. There is a clear conflict here. It's like the OGRE project is saying this is only a rendering engine but then it has added loads of stuff which has not got much to do with rendering. It is understandable because there must be working samples but I think it could be organised better. Thankfully OGRE is open source! So anybody can change this... but if someone starts working on it there should be commonly agreed goals so more people can join to help.
reptor
Ogre Magi
 
Posts: 1120
Kudos: 4
Joined: 15 Nov 2006
Location: Finland

Re: Split Ogre in OgreCore and Ogre3d

Postby Brocan » Tue Jan 18, 2011 7:24 pm

reptor wrote:How can file system access be 'core' to a 3D engine? It should be the other way around. The 3D engine is the core and code that feeds it stuff from a file system, or from elsewhere, is an optional 'utility' component which users of the 3D engine should be able to replace with another component with minimum hassle. There could be an optional file system component next to other components. OGRE would then talk to that component's interface when wanting to load something. People could then hopefully more easily use their own systems for data storage and loading.

This is my view on this topic. I've always thought OGRE is a nice engine, but it's architecture on these things is backwards. Because of this, many people end up basing their programs on top of OGRE, when I think of OGRE only as a component of my program. There is a clear conflict here. It's like the OGRE project is saying this is only a rendering engine but then it has added loads of stuff which has not got much to do with rendering. It is understandable because there must be working samples but I think it could be organised better. Thankfully OGRE is open source! So anybody can change this... but if someone starts working on it there should be commonly agreed goals so more people can join to help.


I'm currently implementing a small game engine based in plugins, and I've the same "problem". Use the ogre resource system or implement my own? or maybe use my own to feed the ogre's one? (two resource systems? redudant...). Use my own core vector / quaternion / (other math structure) classes, or use ogre's ones? (breaking the engine core encapsulation, because ogre need to link to the core...), etc...

I like the idea of split it, tools, math, memory management, profiling... as "OgreTools" and 3D engine as "OgreRenderer" or something like that :D
User avatar
Brocan
Orc
 
Posts: 441
Kudos: 9
Joined: 01 Aug 2006
Location: Spain!!

Re: Split Ogre in OgreCore and Ogre3d

Postby CABAListic » Tue Jan 18, 2011 7:32 pm

Well, technically the resource system is already pluggable - if you don't like the default backends (file systems, zip files), you can supply your own. I have repeatedly used PhysFS as the backend, which works really well.

So, how exactly would you want to split the resource system? Just move the default backends out of the core? Or remove the entire resource system? But then how does Ogre get hold of a mesh to render? How do you want to deal with resource dependencies, e.g. texture names which appear in materials etc.? Would you require that all resources are manually preloaded and registered with Ogre? Or would you want a messaging/callback system for that?
CABAListic
OGRE Retired Team Member
OGRE Retired Team Member
 
Posts: 2902
Kudos: 59
Joined: 18 Jan 2007

Re: Split Ogre in OgreCore and Ogre3d

Postby madmarx » Tue Jan 18, 2011 10:02 pm

OgreCore : transversal code (memory management, file access, thread, time, ...)
-Ogre3d : all the 3D relative code (based on OgreCore)


IMO, the application is supposed to do the system stuff (file access/thread/timer) and there are very good libraries to do that (boost, qt, poco, dlib). In my apps, Ogre3D is only used for 3D relative code, and I don't why would someone wants to use ogre timer (as example) to manage a game logic... Even maths calculations are done outside Ogre, which allow me to use big floats libraries when needed (e.g simulation of planetory systems).

Here are 2 opposite elements, and the common solution :

1/ Such OgreCore would be a waste of effort, because there are already plenty of libraries to do that, in better. I prefer to know the Ogre3D team can work on something related to 3d, than on something that we can already find out there, tested by more people.
2/ But on the other side, we can't of course ask Ogre to 'select' one existing lib and do its life with it. This would either bind Ogre completely to 1 library, or for the developers to develop an horrible #ifndef web to select the correct sub-system element (ex: select between a dlib.timer or a poco.timer) if such elements do not share the same interface ...
3/ The traditionnal answer in C++ to such problem is in a system concept : an interface to all these 'general/system' calls is created, and then, an implementation using one of the 'big' library respect this concept. To avoid virtual calls, 1 typedef is used to select the corresponding big library implementation. This would also allow developers for consoles to 'glue' their systems very easily.

Ideally the team only provide the concepts for "these general need classes" to the community, which provides concepts implementations (this also force the concepts to be clean and documented), and so the team can work more on the 3d-specific features (both 3D concepts and 3D implementations in that case) for which they are more talented than us.

I am curious to know if I am completely wrong, missing the point, or not?

Best,

Pierre

EDIT : "the community, which provides concepts implementations " => I mean I would gladly participate in that.
Tutorials + Ogre searchable API + more for Ogre1.7 : http://sourceforge.net/projects/so3dtools/
Corresponding thread : viewtopic.php?f=1&t=57693&start=0
User avatar
madmarx
OGRE Expert User
OGRE Expert User
 
Posts: 1647
Kudos: 43
Joined: 21 Jan 2008

Re: Split Ogre in OgreCore and Ogre3d

Postby Kojack » Tue Jan 18, 2011 10:22 pm

Ogre is a hybrid of two approaches. It's fairly modular with factories, plugins and components. But ogremain itself is a massive library which includes things other engines split off into foundation or support libs (especially physics engines: physx, bullet and havok all have separate foundation libraries).

People have wanted the maths code split off for ages. There are other parts such as logging, datastreams, etc which aren't directly 3d related.

Wild Magic (the source for some of ogre's maths code) did this when they went from version 4 to 5. They divided maths and core into separate libraries.
(On the other hand they got rid of the separate render system libs and merged them all into one, but that's another story)
User avatar
Kojack
OGRE Moderator
OGRE Moderator
 
Posts: 6520
Kudos: 420
Joined: 25 Jan 2004
Location: Brisbane, Australia

Re: Split Ogre in OgreCore and Ogre3d

Postby _tommo_ » Wed Jan 19, 2011 1:19 am

I think that the overall idea is good, but i'm with reptor when he says that at the "Core" we should have the 3D and not the other way around.

To me the Ogre Core should be just the thing that grabs data sets from RAM and displays them on your screen using the API of your choice.
So no file access, no scripts (everything should be doable with plain C++, with scripting as an optional "library" component ), no scene graphs, no scene managers.
Only geometry, textures, lights and lots of shaders, everything thread safe when needed.
Ideally this core would be the perfect fit for platform that require a lightweight engine like IOS and Android.

Then there's the "library", that implements more abstract concerns using OgreCore, like the scene graph or the scripting.
The Math library would be another component, but not one that is optional, as the 3D core itself needs to use its classes... so it wouldn't change much from the current situation.
OverMindGames Blog
IndieVault.it: Il nuovo portale italiano su Game Dev & Indie Games
User avatar
_tommo_
Gnoll
 
Posts: 677
Kudos: 3
Joined: 19 Sep 2006

Re: Split Ogre in OgreCore and Ogre3d

Postby jacmoe » Wed Jan 19, 2011 1:29 am

Maybe something for Ogre 1.9 ?
As a preparation for the scenemanager rewamp of Ogre 2.0 ? :)
That would make sense..
/* Less noise. More signal. */
Ogitor Scenebuilder - powered by Ogre, presented by Qt, Fueled by Passion.
Ogre AppWizards - Ogre project wizards for VC 8-10, Code::Blocks and KDevelop.
OgreAssimpConverter - command-line to convert models to Ogre format.
TwOgreGUI - wrapper for AntTweakBar GUI library.
I accept donations | Me on Google+
User avatar
jacmoe
OGRE Moderator
OGRE Moderator
 
Posts: 21024
Kudos: 162
Joined: 22 Jan 2004
Location: Denmark

Re: Split Ogre in OgreCore and Ogre3d

Postby madmarx » Wed Jan 19, 2011 1:51 am

To me the Ogre Core should be just the thing that grabs data sets from RAM and displays them on your screen using the API of your choice.

I agree, but I don't want to put pressure on the great team !
Tutorials + Ogre searchable API + more for Ogre1.7 : http://sourceforge.net/projects/so3dtools/
Corresponding thread : viewtopic.php?f=1&t=57693&start=0
User avatar
madmarx
OGRE Expert User
OGRE Expert User
 
Posts: 1647
Kudos: 43
Joined: 21 Jan 2008

Re: Split Ogre in OgreCore and Ogre3d

Postby Gohla » Wed Jan 19, 2011 7:26 am

madmarx wrote:3/ The traditionnal answer in C++ to such problem is in a system concept : an interface to all these 'general/system' calls is created, and then, an implementation using one of the 'big' library respect this concept. To avoid virtual calls, 1 typedef is used to select the corresponding big library implementation. This would also allow developers for consoles to 'glue' their systems very easily.
This would be the way to do it, let ogre depend on interfaces, and let the application implement these interfaces and probably provide some default implementations in a separate library.

Might this change also be related to the OGRE_THREAD_SUPPORT == 3 change, since I see that people had their own resource/workqueue implementations there?
User avatar
Gohla
Gnoblar
 
Posts: 10
Kudos: 0
Joined: 09 Jun 2008

Re: Split Ogre in OgreCore and Ogre3d

Postby Zonder » Wed Jan 19, 2011 10:19 am

Gohla wrote:
madmarx wrote:3/ The traditionnal answer in C++ to such problem is in a system concept : an interface to all these 'general/system' calls is created, and then, an implementation using one of the 'big' library respect this concept. To avoid virtual calls, 1 typedef is used to select the corresponding big library implementation. This would also allow developers for consoles to 'glue' their systems very easily.
This would be the way to do it, let ogre depend on interfaces, and let the application implement these interfaces and probably provide some default implementations in a separate library.

Might this change also be related to the OGRE_THREAD_SUPPORT == 3 change, since I see that people had their own resource/workqueue implementations there?


Yes this does make sense. OgreTools is more appropriate for the existing implementations of non 3d related things. Like many have said ogre shouldn't be the foundation of the app. I was the one who requested the renderoneframe method years ago as I disagreed with ogre been the main loop of my app :)
There are 10 types of people in the world: Those who understand binary, and those who don't...
My Blog - http://www.dreamwalkerblog.com
User avatar
Zonder
Bugbear
 
Posts: 834
Kudos: 19
Joined: 04 Aug 2008
Location: Manchester - England

Re: Split Ogre in OgreCore and Ogre3d

Postby conallmmcg » Wed Jan 19, 2011 8:14 pm

so from what i gather you are looking to split all the rendering specific stuff of Ogre into ogrecore and all the non-renderingstuff/optional stuff into ogretools?
ogretools being stuff that ogrecore may require to perform certain functionality i.e filesystem, multithreading, memory management etc?
people can write/integrate their own code to handle this functionality but ogretools would be an official implementation provided so that ogre is at least usable to most users right off the bat and as a way of showing how to write/integrate your own stuff into ogrecore.

if this is the case could certain things that would not normally officially be a part of the Ogre itself but would be really helpful, be provided in ogretools.
for example from what i can see the last year or two i've been on here there is no standard scene format, there is dotscene but isn't there a few different implementations of this format each with some differences?

it would be good if there was a standard scene format / loader ( obviously not catering to everyone ) that can do the basics of loading a scene and maybe even support custom data or at least be extended to do so, that works with the stock scenemanger(s) etc, as an implementation of a scene format that you can use as a base or starting point to writing your own.

this would for easier tutorials and starting off with ogre, meaning you can create your scene and instantly use it with most of the example/tutorial code without having to implement one of the other formats or your own ( maybe an editor like Ogitor could implement this)

i'm guessing that there isn't an official/standard format currently as the Ogre team don't want to force a certain style/format upon users?
but if what this split is doing what i gather it is, wouldn't this fit in the ogretools category?
stuff most people would usually need , an implementation is given for you but is optional and you are free to create your own implementation of feature/functionality x.

The ogretools code would give you the necessary code/tools to get up and running and working with Ogre as quickly and easily as possible?
This is what the name ogretools says to me.

anyway it was just an idea of mine, and gave an example of what i meant by stuff that usually wouldn't fit into the main Ogre3d/ogrecore library but could be in ogretools, if you disagree completly then a simple no would suffice lol and i shall leave it at that.
conallmmcg
Kobold
 
Posts: 29
Kudos: 0
Joined: 02 Jan 2009

Re: Split Ogre in OgreCore and Ogre3d

Postby _tommo_ » Thu Jan 20, 2011 12:02 am

conallmmcg wrote:ogretools being stuff that ogrecore may require to perform certain functionality i.e filesystem, multithreading, memory management etc?


No; ogrecore would be built as if ogretools didn't exist, so that you can use ogrecore without even bothering to download the tools, in the case you don't need any of them.
For the rest, i agree on the fact that having a project that contains the abstractions could lead to better standards for the content pipeline:
it is optional, so if you don't like some decisions you can as well not use it ;)
So the team could endorse a scene format, an editor, and many other decisions that are now left to the community as the current vertical structure would require them to be enforced on any Ogre user.
For example I hate generic scene formats, so i'm glad Ogre doesn't support any of them, but i see that for many others not having that isn't even an option.
So a splitting of concerns would satisfy both the "lowlevel" (myself :mrgreen: ) and "game engine" users, the first having a trimmed-to-the-bone software, the second having a more encompassing pipeline.

My two cents :D
OverMindGames Blog
IndieVault.it: Il nuovo portale italiano su Game Dev & Indie Games
User avatar
_tommo_
Gnoll
 
Posts: 677
Kudos: 3
Joined: 19 Sep 2006

Re: Split Ogre in OgreCore and Ogre3d

Postby conallmmcg » Thu Jan 20, 2011 12:44 am

exactley! :) _Tommo_ pretty much summarised where i was headed for. it satisfies both/most sides of Ogre users

basically satisfying "low-level" users, who get access to the rendering code they want without the extra, and pick what they want from ogretools (if anything) and code the rest themselves to suit themselves.

then for the newer-intermediate users getting used to Ogre, can dive straight in and learn and get results on screen with the least amount of code, then once they learn more then can implement their own library for feature x or their own scene format etc

this is what i mean by stuff that would not usually enter into the main Ogre3d/ogrecore library, stuff the more advanced users wouldn't be happy with in the main lib but the new users would find very helpful/essential for starting off.
conallmmcg
Kobold
 
Posts: 29
Kudos: 0
Joined: 02 Jan 2009

Re: Split Ogre in OgreCore and Ogre3d

Postby Zonder » Thu Jan 20, 2011 9:54 am

Ogre tools would have some stuff that most people would never reimplement like the memory management so most people would end up with both anyway. Another thing would be in ogre tools I think OgreRoot which auto inherits all ogre tool implementations. So as not to break ogre in all projects in a huge way.
There are 10 types of people in the world: Those who understand binary, and those who don't...
My Blog - http://www.dreamwalkerblog.com
User avatar
Zonder
Bugbear
 
Posts: 834
Kudos: 19
Joined: 04 Aug 2008
Location: Manchester - England

Re: Split Ogre in OgreCore and Ogre3d

Postby liberator » Sat Jan 22, 2011 12:49 am

Gohla wrote:
madmarx wrote:3/ The traditionnal answer in C++ to such problem is in a system concept : an interface to all these 'general/system' calls is created, and then, an implementation using one of the 'big' library respect this concept. To avoid virtual calls, 1 typedef is used to select the corresponding big library implementation. This would also allow developers for consoles to 'glue' their systems very easily.
This would be the way to do it, let ogre depend on interfaces, and let the application implement these interfaces and probably provide some default implementations in a separate library.

Might this change also be related to the OGRE_THREAD_SUPPORT == 3 change, since I see that people had their own resource/workqueue implementations there?


I agree that this would be the way to do it if the library is split.
I have my own foundation libraries (as many people do) and as such I could then leave out the suggested util library to shrink down the module size.
User avatar
liberator
Greenskin
 
Posts: 117
Kudos: 0
Joined: 24 Jan 2005
Location: Sillicon Valley, California

Re: Split Ogre in OgreCore and Ogre3d

Postby mukik182 » Wed Jan 26, 2011 5:37 pm

I've been only a short time around the community and I have already seen some suggestions similar to this one that seemed to end just as a suggestion. I'm not proficient enough to propose a design so I would ask you to create a wiki page and start a real discussion/proposal on how should it be designed so the coommunity can help the team.

I'm thinking of the project as a building in construction, with the team as the architects and "controllers" (I don't know the english word for building workers chiefs) telling users how and where to put the bricks and solving and pointing the tricky questions and the community (and I'm counting myself) as workers. I say this because such a change surely will require a lot of work, and I think, maybe, much of the work will be trivial (to rearrange files, rewrite CMake...) leaving to the team the interface design and some implementations.

I'm not sure and I'm probably too naïve, but if I could help with this little things, I would (but I usually I don't find myself knowledgeable enough to).
mukik182
Halfling
 
Posts: 68
Kudos: 1
Joined: 09 Nov 2009
Location: Barcelona, Spain

Re: Split Ogre in OgreCore and Ogre3d

Postby Klaim » Wed Jan 26, 2011 11:10 pm

I think design by commitee is bad for this kind of thing. A better one would be to propose an implementation. If someone have time and will to do it, it's easy to setup a repo and let people follow the changes, maybe participate. If someone else wants too, he can too create a clone and let people see it. Then when proposition (if there are more than one) are considered finished, the team pull the one that seem to "solve the problem".
User avatar
Klaim
Old One
 
Posts: 2419
Kudos: 57
Joined: 11 Sep 2005
Location: Lille, France

Re: Split Ogre in OgreCore and Ogre3d

Postby madmarx » Wed Jan 26, 2011 11:52 pm

I agree with Klaim.

I have already seen some suggestions similar to this one that seemed to end just as a suggestion

Yes, but this is not tragic either. Some propositions come back regularly. So it is interesting to see what users are interested in anyway, and if we share the same point of view or not.
After that comes 2 things (if I compare to the regular 'null renderer request') that explain that things often finish just as suggestions :
1/ it simply takes time to do things (and even more for things like that)
2/ maybe there are other priorities right now.

just my 2 cent.
Tutorials + Ogre searchable API + more for Ogre1.7 : http://sourceforge.net/projects/so3dtools/
Corresponding thread : viewtopic.php?f=1&t=57693&start=0
User avatar
madmarx
OGRE Expert User
OGRE Expert User
 
Posts: 1647
Kudos: 43
Joined: 21 Jan 2008

Next

Return to Developer talk

Who is online

Users browsing this forum: Yahoo [Bot] and 2 guests