Build Process Suggestions

So I managed to get Mogre for Ogre 1.6.3 to build this evening from the svn sources, but it's not a pretty process. It seems that there are a number of things that could be done make the process easier and less time consuming. I'm going to offer up a couple of suggestions here and hope for some feedback/commentary from others who have managed to get this thing to build.

1. It seems like a really good idea to split mogre.dll into two pieces, mogre_core.dll that contains only the CLR objects and mogre.dll that contains everything else. Then mogre_core.dll becomes very stable and not dependent on the ogre libraries. Ogre could then link to mogre_core instead of mogre and we could avoid the double ogre build process.

2. It seems that a little work with the autowrap tool could yield some big benefits in usability. A configuration dialog for directories would be an excellent start. We could also have it launch build.bat and validate that the results looked somewhat reasonable before continuing.

3. This one is a lot of work, but I seriously wonder if the whole technique of having the ogre object maintain a manged object is worth the misery. I wonder if we could instead simply keep a hash table of ogre object pointers and their associated managed objects that mogre had already seen. Then when a function returns an ogre object it could be looked up in the hash table rather than asked for it's managed object. It would be a little slower, but it would allow mogre to run against the regular ogre builds without the patching and rebuilding process.

Any thoughts?

Hank

Hi Hank,

Thanks for your suggestions, comments to follow..

1. It seems like a really good idea to split mogre.dll into two pieces, mogre_core.dll that contains only the CLR objects and mogre.dll that contains everything else. Then mogre_core.dll becomes very stable and not dependent on the ogre libraries. Ogre could then link to mogre_core instead of mogre and we could avoid the double ogre build process.
I don't quite understand what you mean mogre_core.dll that contains only the CLR objects, can you elaborate ?

The situation is this:
-There is mogre.dll and ogremain.dll
-mogre.dll needs to link to ogremain.dll because the managed objects call the native ogre methods.
-ogremain.dll needs to link to mogre.dll because the native ogre objects call functions into mogre.dll to create the managed objects.

We may have a mogre_core.dll that contains the "create managed objects" functions, but this doesn't break the cyclic dependency, it just creates a cyclic dependency between 3 dlls; I mean instead of having
mogre <-> ogremain
we will have

mogre_core <-
/ \
-> mogre -> ogremain


The ideal way to solve this dependency mess is to unite mogre.dll and ogremain.dll and have only one .dll that contains both native and managed code.
This is doable but, unfortunately, when I tried it the linker wasn't "up to the task", it died after consuming all the memory in the system (2 GB)

2. It seems that a little work with the autowrap tool could yield some big benefits in usability. A configuration dialog for directories would be an excellent start. We could also have it launch build.bat and validate that the results looked somewhat reasonable before continuing.
Improving autowrap would be great, any patches on that area are more than welcome!

3. This one is a lot of work, but I seriously wonder if the whole technique of having the ogre object maintain a manged object is worth the misery. I wonder if we could instead simply keep a hash table of ogre object pointers and their associated managed objects that mogre had already seen. Then when a function returns an ogre object it could be looked up in the hash table rather than asked for it's managed object. It would be a little slower, but it would allow mogre to run against the regular ogre builds without the patching and rebuilding process.
There's more to the current mogre<->ogre connection besides associating an ogre object with a mogre one.
Since the ogre object actually creates the mogre one, it can create the appropriate subclass object (i.e, an ogre Camera will create a mogre Camera object).

And the hash table scheme is "fragile" since it doesn't handle destruction of objects. Currently, when an ogre object gets destroyed it releases the mogre one and allows it to be garbage collected. Using a hash table we wouldn't know when an ogre object is destroyed so that we no longer keep an association with a dead object.
It is possible that a new ogre object gets the same address of another ogre object that was destroyed, and now we have an association that is wrong.

Bekas,

One the first point on about a mogre_core.dll:

If I pull the mogre.dll out of my link I get unresolved externals that fall in three categories:

1) CLRHandle.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) void __cdecl Mogre::Implementation::FreeCLRHandle(void *)

This one could easily be moved out to a core library that is included in both ogre and mogre.

2)CLRObject.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) void __cdecl Mogre::Implementation::FreeCLRObject(void *)

As could this one.

3>CLRObject.obj : error LNK2019: unresolved external symbol "__declspec(dllimport) void __cdecl Mogre::_Init_CLRObject_OGRECLASSNAME(class CLRObject *)

This is the tricky one. We get a bunch of these errors, one per Mogre/Ogre class that is exposed to public, and because it's per class we would have troubles moving into a core library. In effect, these lines indicate ogre is trying to link to one function for every class in mogre.

What we need here is an Init function in the core library that takes a class name and a CLRObject. Then, to keep the core library from having a dependency on mogre, the core library would export a registration method that took a class name and a delegate. Registration would store the delegate in a hash table with class name as the key. Then when core init is called it would simply look up the requested class name in the hash table and execute the returned delegate on the offered CLRObject. Either a static init function that registered all of the delegates from mogre would need to be called before anything else could occur, or mogre.Main could do it in its constructor, or a static class constructor could be added to Mogre.Main, etc.

In this way both mogre and ogre are dependent on the core dll, but there is no circle, which makes building much, much less painful. It will cost a small amount of performance because each init picks up a hash table lookup. However, even that could be worked around by replacing the core init function with a core getinitdelegate function. Then the ogre object could get the delegate and cache it, only resulting in one hash table lookup per class, not one per object.

There are alternatives to continuing with a cyclical dependency. I'm just suggesting we work one out and implement it.

Hank

Hey Hank,

Keep in mind that, no matter how painful the build process is, it's a one-time thing; you build the DLLs and that's it, your .NET code use the DLLs and doesn't have to deal with the build process at all.
Therefore, making the build easier by introducing any runtime complexity or overhead is unacceptable.

The way to go, as I said, is to have only one DLL, which will simplify the build process and deployment too. If you'd like to help, try looking into moving all the stuff from mogre.dll into the ogremain.dll, and ending up with a DLL that contains the native ogre code and the managed wrappers.

I've also succeeded building Mogre. It was a long way though.

Easy one-step building is essential for getting feedback. I did not read the AutoWrapper code, but the build process can be made simple using any amount of scripting that does the patching, wrapping, modifies files and calls msbuild. Building each time against an up-to-date SVN Ogre source (automatic 'svn update') would also be an option then. Raw scripting means batch files or Python (better); build tools include SCons and NAnt (although to build from sln, msbuild must be called by these quite manually). Any preferences?

Samples must also be built every time, both for testing purposes and for providing working reference code. I therefore encourage developers who do the release to add samples to each package.

Below is the build process used by me. The Wiki requires some modifications, basically steps 3. and 4. must be added from the following list. I would happily edit the Wiki, I lack an account, and do not know how to set one up. Would it not be better to allow people to edit the Wiki more freely?

1. copy ogre folder containing the Ogre source over the location
./Mogre/Ogre
So that
./Mogre/Ogre/OgreMain/include
contains Ogre.h .
There may be some patching issues, so you may want to use SVN checkouts only. Checkout from
https://svn.ogre3d.org/svnroot/ogre/branches/v1-6
or
https://svn.ogre3d.org/svnroot/ogre/trunk
It is the version from the first link that was used by other people.

2. copy Dependencies to
./Mogre/Ogre/Dependencies

3. apply patches to Ogre.
For TortoiseSVN copy the patches
./Mogre/Ogre Patches/*.patch
to
./Mogre/Ogre
and select "Apply Patch..." from the patch context menu. Then browse to ./Mogre/Ogre if necessary and select "Patch All" from context menu. Do so for each patch.

4. copy the files CLR* from
./Mogre/Mogre/Ogre
to
./Mogre/Ogre/OgreMain/include
./Mogre/Ogre/OgreMain/src
CLR*.h go to include, CLR*.cpp go to src.

5. go to
./Mogre/cpp2java
and run
doxygen build

6. Compile and run ./Mogre/AutoWrap.sln
Make sure that ./Mogre/Mogre/include/auto
contains non-empty CLRObjects.inc

7. Change
./Mogre/Mogre/Ogre/CLRConfig.h
./Mogre/Ogre/OgreMain/include/CLRConfig.h
the line
LINK_TO_MOGRE 0
Also, remove Mogre.lib and Mogre_d.lib
from Release and Debug projects respectively in the solution
./Mogre/Ogre/Ogre_vc9.sln

8. build Debug/Release for
./Mogre/Ogre/Ogre_vc9.sln
NOTE: building patched Ogre is needed to build Mogre
NOTE: building of the static libraries may fail, build only Debug and Release, not DebugStaricLib not ReleaseStaticLib

9. Build Debug/Release for
./Mogre/Mogre_vc9.sln
NOTE: On errors, check if proper non-empty CLRObjects.inc gets included.

10. Change
./Mogre/Mogre/Ogre/CLRConfig.h
./Mogre/Ogre/OgreMain/include/CLRConfig.h
the line
LINK_TO_MOGRE 1
(I am not sure if You are to change both)
Also, add Mogre.lib and Mogre_d.lib
from Release and Debug projects respectively in the solution
./Mogre/Ogre/Ogre_vc9.sln

11. build Debug/Release for
./Mogre/Ogre/Ogre_vc9.sln

MOIS

1. build OIS
2. run cpp2java/doxygen build
3. build and run AutoWrap
4. build MOIS

Slightly off-topic: I did not manage to run the samples. After copying the media, samples and dlls to the directory structure like that of OgreSDK I get the following error in every sample (so probably at resource loading stage):
OGRE EXCEPTION(7:InternalErrorException): ../../media/packs/OgreCore.zip - error whilst opening archive: Unable to read zip file. in ZipArchive::checkZzipError at ..\src\OgreZip.cpp (line 259)
This is also the case for mcladen's build http://www.ogre3d.org/addonforums/viewtopic.php?f=8&t=10930.

The Wiki requires some modifications, basically steps 3. and 4. must be added from the following list. I would happily edit the Wiki, I lack an account, and do not know how to set one up.

you can use a ogre forum account to log in (don't know if account from add-ons forum work though). it's the same user name and password.

I think as bekas said making one DLL is best but if there is a memory issue with the linker that does put a stop to it unless you have sufficient ram.

thanks kelebron, I'm having a bit of difficulty building, your notes above will be helpful. I'll update the wiki if I can get your process to work for me.

My suggestion for easing the pain of the build process, the process can be made fairly simple using a script. This is my contribution, but note that it does not work yet, I get errors that I can't work out how to resolve. Edit: errors now resolved, wiki updated, thanks to GantZ' tips.