OgreXMLConverter and big xml file

What it says on the tin: a place to discuss proposed new features.
Post Reply
claustre
Goblin
Posts: 278
Joined: Thu Mar 23, 2006 10:35 am
Location: Toulouse, France
Contact:

OgreXMLConverter and big xml file

Post by claustre »

I work with big mesh files (up to 1 million polygons) imported from different file format. For simplicity I convert them to xml first then apply the converter to get binary mesh file.

The converter is very memory consuming as it loads the xml file entirely in memory before converting. It should be fine to load/unload it incrementally while building the geometry buffers...
User avatar
Game_Ender
Ogre Magi
Posts: 1269
Joined: Wed May 25, 2005 2:31 am
Location: Rockville, MD, USA

Post by Game_Ender »

I believe it currently uses tinyxml which is a dom based parser, that by its very nature always loads everything into memory. You would have to rewrite the the tool to use what I believe is called a SAX based parser.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

For very large meshes you would be better not to go via XML at all - can you not write your converter to go direct to binary via MeshSerializer instead of writing XML?
claustre
Goblin
Posts: 278
Joined: Thu Mar 23, 2006 10:35 am
Location: Toulouse, France
Contact:

Post by claustre »

sinbad wrote:For very large meshes you would be better not to go via XML at all - can you not write your converter to go direct to binary via MeshSerializer instead of writing XML?
Of course I can, but it is far simple to go with XML :
- my converter does not have to depend on Ogre
- my converter does not have to manage the evolution of the mesh file format (transfered to the OgreConverter)
- my converter has a small and comprehensive source code (even for a person that don't know Ogre)

What I should do is to copy the OgreConverter code as a base, you know, reinventing the wheel ;-)
claustre
Goblin
Posts: 278
Joined: Thu Mar 23, 2006 10:35 am
Location: Toulouse, France
Contact:

Post by claustre »

Just for information :

I have converted a 1,000,000 vertices mesh to XML, which result in a 200 MB file (not as big). The converter crashes because requiring more than 2 GB of memory !
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Post by sinbad »

Mostly this is because TinyXML builds a DOM. We'd need to use SAX to deal with something this large I bet - not a quick change.
Post Reply