Future of MOGRE

Hello everybody,

Bekas, I saw the posts about you leaving and I was wondering about a few things:

-first of, if you'll post at least an idea of where you intend to take the MOGRE project in relation to mono
-secondly, if you could post a list of the things that are pending in for the "next release" (things that you haven't quite finished working on)

And thanks for such a great project, Radu.

After various ideas, "research" and testing, I've settled on this plan:

Replace the C++/CLI automatic wrapper with an IL assembly one.

I've come to the conclusion that pure managed code (using down-to-the-metal IL and P/Invoke) can directly use C++ objects, call C++ virtual methods, subclass C++ virtual methods etc.
In fact, IL assembly provides more flexibility than C++/CLI. With C++/CLI, instead of focusing on the wrapping code, I usually struggle with the C++ syntax and parser (with "make sure you get the include files in the right order", "put out a proxy class in order to access protected methods/fields" and various other headaches that make things a bit messy).

Also, since I need full information on C++ classes (i.e. how the fields are mapped in memory), I'm going to use the internal syntax tree representation of GCC's C++ parser to get it.

So, in short, I intend to replace [doxygen / C++/CLI] with [GCC C++ parser / IL assembly].

Are you going to custom hack gcc, or switch from doxygen's xml output to gccxml's? If you are doing that may I suggest Pygccxml. Its a python library that allows very easy examination of the C++ types that gccxml has parsed.

Also, what is it the doxygen's xml output doesn't provide? It looked so much cleaner than the mess that gccxml produces.

Full disclosure, Py++ used by Python-Ogre uses gccxml and pygccxml.

Are you going to custom hack gcc, or switch from doxygen's xml output to gccxml's?
Custom hack. Nothing too fancy, just getting the missing info through the -fdump-translation-unit and parsing the result. The output has the same flat structure as gccxml but it's smaller and I get templates and everything else there is. It's not xml of course but making a parser for -fdump-translation-unit is trivial.

Also, what is it the doxygen's xml output doesn't provide?
Size of classes, memory locations/sizes for individual fields, virtual table for functions etc.
[edit] Also you get to use a 100% working C++ parser. Doxygen, for example, can't handle preprocessor directives.. [/edit]

Also, what is it the doxygen's xml output doesn't provide? It looked so much cleaner than the mess that gccxml produces.
I've come across a promising library for C++ parsing (called MetaC++), that is using GCC (as is gccxml), it provides a C++ interface for parsing the info, and, optionally, it can produce a xml output that has even better form than doxygen.

Home: http://www-user.uni-bremen.de/~strasser/metacpp/
SVN: svn://piskernig.at/metacpp/trunk

But I couldn't get it to compile (tried SVN). If you take a look at it and manage to get it working, let me know.

If I have time I will give it a go, but don't you think what you are doing is a little to compiler dependent? You will be limited to whatever version of GCC you decided to hack and will need to keep updating it and possibly support multiple versions of GCC.

The reason gccxml doesn't provide the information you want is because it would not be of most use because it changes so much. If you are willing to go this close to the metal it would make more sense to finish the GCC CLI backend that way you would get an equivalent to what you already have on windows.

Why can't take the approach Swig and Py++ take, use wrapper classes? It worked for OgreDotNet, they just lost there maintainer too.

don't you think what you are doing is a little to compiler dependent?
Yep, I initially thought that MSVC and GCC would have a lot of differences, but after some reading and lots of testing I couldn't find differences in the object layout between the two (for data members locations, multiple inheritance, even virtual inheritance).
This may seem strange but if you consider that due to C compatibility the data members are placed in declaration order, optimized alignment padding is processor-specific, and there can't be a lot of optimized ways to implement C++ inheritance, it makes sense that they use the same C++ object layout.

Why can't take the approach Swig and Py++ take, use wrapper classes?
For performance reasons mostly (i.e. instead of switching to unmanaged code in order to access a field, I'll be able to access it with pure managed code) and flexibility (i.e. it will be way easier to override virtual C++ methods)

Just out of sheer curiosity why do you need the internal representation of the c++ objects?

I'm asking this because I've been doing some reading lately and aside from some string issues that might arise there was no indication that such data might be needed.

The thing witch struck me in as particular was the fact that the article mentioned that GTK# was wrapped without marshaling, specifically so it could use advanced OO techniques (such and inheritance) and to allow native GTK applications to consume .net specific functionality (draw on a widget with System.Drawing).

I just stumbled on this one and haven't had time to look into it more seriously, but at first glance this might seem like the holy grail we were looking for (no marshaling => no performance hit).

The paragraph that mentioned this can be found here.

Just out of sheer curiosity why do you need the internal representation of the c++ objects?
Here's an example:
Suppose that there's this C++ class:
class Native
{
public:
int x,
int y;
};
When you make a wrapper class for it, here's what C++/CLI "kind of" does (C#):

[StructLayout(LayoutKind.Sequential, Size = 8)]
struct Native
{
}

class NativeWrapper
{
private Native* _nativeptr;

public int X
{
get
{
byte* ptr = (byte*)_nativeptr;
return *(int*)ptr;
}
set
{
byte* ptr = (byte*)_nativeptr;
*(int*)ptr = value;
}
}

public int Y
{
get
{
byte* ptr = (byte*)_nativeptr;
ptr += 4;
return *(int*)ptr;
}
set
{
byte* ptr = (byte*)_nativeptr;
ptr += 4;
*(int*)ptr = value;
}
}
}

I'd like to be able to do something like this:

[StructLayout(LayoutKind.Explicit, Size = 8)]
struct Native
{
[FieldOffset(0)]
public int x;
[FieldOffset(4)]
public int y;
}

class NativeWrapper
{
private Native* _nativeptr;

public int X
{
get
{
return _nativeptr->x;
}
set
{
_nativeptr->x = value;
}
}

public int Y
{
get
{
return _nativeptr->y;
}
set
{
_nativeptr->y = value;
}
}
}

For both cases you need to know how the fields of Native class are mapped on memory.

The other way would be to use C wrapper functions and call them with P/Invoke:

C++:
class Native
{
public:
int x,
int y;
};

int Native_getX(Native* ptr)
{
return ptr->x;
}

void Native_setX(Native* ptr, int val)
{
ptr->x = val;
}

int Native_getY(Native* ptr)
{
return ptr->y;
}

void Native_setY(Native* ptr, int val)
{
ptr->y = val;
}

C#:
class NativeWrapper
{
private void* _nativeptr;

[DllImport("library")]
private extern int Native_getX(void* ptr);
[DllImport("library")]
private extern void Native_setX(void* ptr, int val);
[DllImport("library")]
private extern int Native_getY(void* ptr);
[DllImport("library")]
private extern void Native_setY(void* ptr, int val);

public int X
{
get
{
return Native_getX(_nativeptr);
}
set
{
Native_setX(_nativeptr, value);
}
}

public int Y
{
get
{
return Native_getY(_nativeptr);
}
set
{
Native_setY(_nativeptr, value);
}
}
}

The latter is less performant because of the P/Invokes.

The thing witch struck me in as particular was the fact that the article mentioned that GTK# was wrapped without marshaling, specifically so it could use advanced OO techniques (such and inheritance) and to allow native GTK applications to consume .net specific functionality (draw on a widget with System.Drawing).
Where is the article ?

I just stumbled on this one and haven't had time to look into it more seriously, but at first glance this might seem like the holy grail we were looking for (no marshaling => no performance hit).
There is still the performance hit of the managed->unmanaged switch. And C++/CLI does no marshalling by itself, any required marshalling needs to be manually specified. For example, when a Mogre method gets a .NET string, there's code to convert it to an Ogre string, so there's still marshalling involved, there's no way around it.

Sorry about not posting the link. Must have gotten lost on the way.
Here it is.

I took the time to look at this thing a bit more, but I haven't had time to actually test this on a piece of code. I took a look at the source code of the bindings of the GTK# libraries and they look very similar to what I remember MOGRE to look like (skipping the c# - c++ diffrences).

As I have to do some binding myself I'll try to do it using the methods mentioned there, trying to use as little marshaling as possible. However I don't expect to have any results until next week, since I'm kinda swamped at work (laying some more optic fiber) and the computers I'm working on are kind of... crippled by Alzheimer (polite way of saying my bureaucratic bosses won't by more RAM for them).