Custom background loading of textures/materials..

Problems building or running the engine, queries about how to use features etc.
Post Reply
User avatar
syedhs
Silver Sponsor
Silver Sponsor
Posts: 2703
Joined: Mon Aug 29, 2005 3:24 pm
Location: Kuala Lumpur, Malaysia
x 51

Custom background loading of textures/materials..

Post by syedhs »

The titles says it all. I have read quite a log of articles, and I think I am going toward the route Ogre compile with no thread capabilities. But I am going to create thread myself and load textures/ populate materials from there. Any ideas/clues/links?
A willow deeply scarred, somebody's broken heart
And a washed-out dream
They follow the pattern of the wind, ya' see
Cause they got no place to be
That's why I'm starting with me
AusSkiller
Gremlin
Posts: 158
Joined: Wed Nov 28, 2012 1:38 am
x 13

Re: Custom background loading of textures/materials..

Post by AusSkiller »

syedhs wrote:The titles says it all. I have read quite a log of articles, and I think I am going toward the route Ogre compile with no thread capabilities. But I am going to create thread myself and load textures/ populate materials from there. Any ideas/clues/links?
That doesn't work, all communication with the GPU needs to be done from the same thread and you need to communicate with the GPU to create/load a texture for it to use. Whatever thread you create Ogre in is the thread you need to load textures in, even when compiling with threading enabled that's still mostly the case, except that reading the file off disk and doing any processing it needs (like converting it to the appropriate texture format) is all done on the CPU so that can be done in another thread, but it still has to be sent to the GPU in the main thread. Fortunately most of the loading time is usually from reading the file off disk and processing it, just sending the data to the GPU doesn't take too long so if you only send one texture per frame and are doing the rest in another thread you can still get reasonable frame rates. The best option is to compile Ogre with threading support and use the background loading features it provides because that's pretty much what it does.
User avatar
syedhs
Silver Sponsor
Silver Sponsor
Posts: 2703
Joined: Mon Aug 29, 2005 3:24 pm
Location: Kuala Lumpur, Malaysia
x 51

Re: Custom background loading of textures/materials..

Post by syedhs »

AusSkiller wrote: That doesn't work, all communication with the GPU needs to be done from the same thread and you need to communicate with the GPU to create/load a texture for it to use.
Yes I know.. and I am going to load a lot of textures and page them in and out as necessary.
What I intend to do in thread is to load textures, and pass them to Ogre to do all the low level resource stuff and of course, upload them to GPU. I intend to minimze all FPS stutter due to texture paging. The code is already doing texture load one per 5 frame to ensure FPS as smooth as possible (but still not that smooth).
The best option is to compile Ogre with threading support and use the background loading features it provides because that's pretty much what it does.
I didn't purse this as I want to have complete control of file loading (which include file caching etc). And as stated in my post, I read a lot of post regarding this and one of them mention Sinbad's intention toward OGRE_THREAD_SUPPORT=3. In this mode, resource is load better and faster too. I have done many multithread projects in the past, so what I am looking for is the way to load texture files myself and then, send them to Ogre resource manager.
A willow deeply scarred, somebody's broken heart
And a washed-out dream
They follow the pattern of the wind, ya' see
Cause they got no place to be
That's why I'm starting with me
AusSkiller
Gremlin
Posts: 158
Joined: Wed Nov 28, 2012 1:38 am
x 13

Re: Custom background loading of textures/materials..

Post by AusSkiller »

Ahh, so you want to do something more akin to texture streaming rather than just asynchronous loading, I haven't tried any texture streaming in Ogre so I'm not really sure what the best way to go about that would be :(.
User avatar
syedhs
Silver Sponsor
Silver Sponsor
Posts: 2703
Joined: Mon Aug 29, 2005 3:24 pm
Location: Kuala Lumpur, Malaysia
x 51

Re: Custom background loading of textures/materials..

Post by syedhs »

I think I have got the pseudocode right.. so I am posting my pseudocode just in case if others have got better opinion...

Issues: I want to background load materials (eventually textures) so that when texture paged in, it will not cause slowdown or stutter because the main process has to wait for textures to be loaded (synchronous load).

My solution (not tested yet, but should be working):-
1) Main thread request a material to be loaded. This is not done using MaterialManager::getSingleton().load(...) because we dont want to Ogre to actually load the material now (more to that later). But this is done using application's own code.
2) Code will figure out what textures are to be loaded from the material. Here, if textures are not loaded yet, then this texture name (string) will be put into queue.
3) This queue will be asynchronously loaded - separate thread is needed as we also need to decode the texture (save some cpu off the main process).
4) Once a texture (or several) has been loaded, and TextureManager has been made aware of this resource (using ManualLoader), then only we execute MaterialManager::getSingleton().load(<material>)
A willow deeply scarred, somebody's broken heart
And a washed-out dream
They follow the pattern of the wind, ya' see
Cause they got no place to be
That's why I'm starting with me
Post Reply