2 Multithreading questions

Anything and everything that's related to OGRE or the wider graphics field that doesn't fit into the other forums.
Post Reply
chincillya
Halfling
Posts: 94
Joined: Sun Dec 19, 2004 7:12 pm
Location: Finland
x 3
Contact:

2 Multithreading questions

Post by chincillya »

1. Okay, I made a thread earlier about this ( http://www.ogre3d.org/forums/viewtopic.php?f=1&t=66516 ) but I got no answers, so I guess I phrased myself poorly. My question is this: Can I create resource groups and load resources in a background thread created by myself (ie. not using Ogre's built-in background loading system) ? I would not alter the scene graph in the loading thread, only create resources and load them. If this is not possible, is there some good tutorial on how to load stuff in the background using Ogre's own threading system?

2. My app is currently single threaded. However, Ogre seems to create a bunch of threads itself. As soon as I call createRenderWindow() on the Ogre root my thread count jumps from 1 to 21 (!), according to the windows task manager. What is Ogre using 20 threads for? I am currently using the 1.7.2 prebuilt SDK.

Thanks!
"Black holes are the result of God dividing by zero."

http://www.sebastian-ahlman.fi/
User avatar
Jabberwocky
OGRE Moderator
OGRE Moderator
Posts: 2819
Joined: Mon Mar 05, 2007 11:17 pm
Location: Canada
x 218
Contact:

Re: 2 Multithreading questions

Post by Jabberwocky »

chincillya wrote: Can I create resource groups and load resources in a background thread created by myself
I think you should try to use Ogre's built in functionality for background resource loading. No point reinventing the wheel.
chincillya wrote:is there some good tutorial on how to load stuff in the background using Ogre's own threading system?
Unfortunately, I was very surprised at how difficult it was to find a good example of using Ogre's background resource loading.
Still, I'd give it a shot if I were you.

You have the documentation.

This person seems to have gotten it working after some initial trouble.

There's also some code in the ogre file PlayPen.cpp (search for "ResourceBackgroundQueue"), but I see that this code has been commented out, and I'm not sure why, so maybe there's some issues with it. Ogre's use of threads has evolved between each recent major release, which adds to the confusion, and might be why that code is commented out.

Inside OgreConfig.h, you'll see this:

Code: Select all

/** Support for multithreading, there are 3 options

OGRE_THREAD_SUPPORT = 0
	No support for threading.		
OGRE_THREAD_SUPPORT = 1
	Thread support for background loading, by both loading and constructing resources
	in a background thread. Resource management and SharedPtr handling becomes
	thread-safe, and resources may be completely loaded in the background. 
	The places where threading is available are clearly
	marked, you should assume state is NOT thread safe unless otherwise
	stated in relation to this flag.
OGRE_THREAD_SUPPORT = 2
	Thread support for background resource preparation. This means that resource
	data can streamed into memory in the background, but the final resource
	construction (including RenderSystem dependencies) is still done in the primary
	thread. Has a lower synchronisation primitive overhead than full threading
	while still allowing the major blocking aspects of resource management (I/O)
	to be done in the background.
*/
If you're building from source, your value of OGRE_THREAD_SUPPORT will be set in OgreBuildSettings.h
I'm not sure how OGRE_THREAD_SUPPORT = 1 vs OGRE_THREAD_SUPPORT = 2 will change the code needed for background resource loading (if at all), but you may want to try them both.
Image
chincillya
Halfling
Posts: 94
Joined: Sun Dec 19, 2004 7:12 pm
Location: Finland
x 3
Contact:

Re: 2 Multithreading questions

Post by chincillya »

Thanks for info.

I actually read that post already, but as the OP never seemed to get the app to work without crashing I did not think he had done it the right way. The reason I am "reinventing the wheel" in this case is that I need multithreading for other subsystems as well, and I'd like to route all parallel processing through my own system.

Anyone know anything about the 20 threads?
"Black holes are the result of God dividing by zero."

http://www.sebastian-ahlman.fi/
User avatar
Jabberwocky
OGRE Moderator
OGRE Moderator
Posts: 2819
Joined: Mon Mar 05, 2007 11:17 pm
Location: Canada
x 218
Contact:

Re: 2 Multithreading questions

Post by Jabberwocky »

but as the OP never seemed to get the app to work without crashing
It looks to me like he did - did you read the [EDIT] at the bottom of the last post?
But yeah, it doesn't seem like a rock solid example regardless.
The reason I am "reinventing the wheel" in this case is that I need multithreading for other subsystems as well, and I'd like to route all parallel processing through my own system.
Understood.
You may want to look at Ogre's WorkQueue class as a potential base for your multithreading. Although I understand if you want to work completely with your own code instead.
Anyone know anything about the 20 threads?
I think Ogre only explicitly spawns threads for either terrain loading (if you're running a terrain sample), or background resource loading, if you've specifically told Ogre to do so.
Although Ogre links to all sorts of other libraries, so maybe that's what you're seeing.
Image
chincillya
Halfling
Posts: 94
Joined: Sun Dec 19, 2004 7:12 pm
Location: Finland
x 3
Contact:

Re: 2 Multithreading questions

Post by chincillya »

Jabberwocky wrote:
It looks to me like he did - did you read the [EDIT] at the bottom of the last post?
But yeah, it doesn't seem like a rock solid example regardless.
Hmm, I have to read through that again....
Jabberwocky wrote:
Understood.
You may want to look at Ogre's WorkQueue class as a potential base for your multithreading. Although I understand if you want to work completely with your own code instead.
Everyone seems to favour the "Task model" aka. "Job model" for multithreading these days, mainly because it scales to N threads, so I though I would go with that model as well. What I'd really like to do is create a loading job (a function that does all the loading for a game level, loads a given resource group for Ogre, creates all the sound resources, runs level scripts etc.) while the main thread renders an animated loading screen, continues to process network messages and so on. I know I could probably do the same thing with Ogre's background loading systems, but I really don't want to. I'm just now beginning to like the idea of having a multithreaded engine, and I want to have all the threading related code in one system that I really understand :) .
Jabberwocky wrote:
I think Ogre only explicitly spawns threads for either terrain loading (if you're running a terrain sample), or background resource loading, if you've specifically told Ogre to do so.
Although Ogre links to all sorts of other libraries, so maybe that's what you're seeing.
I am using the Ogre terrain system, but it is createRenderWindow() which makes all those extra threads appear. I have not even created any Ogre::TerrainGlobalOptions object at that point yet. This is not that important anyway, I am just curious about what Ogre does with all those threads.
"Black holes are the result of God dividing by zero."

http://www.sebastian-ahlman.fi/
User avatar
Jabberwocky
OGRE Moderator
OGRE Moderator
Posts: 2819
Joined: Mon Mar 05, 2007 11:17 pm
Location: Canada
x 218
Contact:

Re: 2 Multithreading questions

Post by Jabberwocky »

I want to have all the threading related code in one system that I really understand.
Fair enough.
I doubt you'll find much help on how to (re)implement background resource loading, because it's not something most people would do. But since it is already implemented in Ogre, you'll have the source code of a working example to look at. Just start at by looking at the source for Ogre::ResourceBackgroundQueue, and see where it takes you.

I don't know how hard it's going to be. But I do know that Ogre's background resource loading involves a bunch of locks (mutexes?) of some sort. I'm not sure if you can build on top of this functionality, or if you're actually going to need to get your hands dirty with Ogre code to implement your own background resource loading (hopefully not). But familiarizing yourself with how the existing implementation works is definitely the right approach. You really should understand that code very well anyway, so that once you're dealing with your own threads, you understand what's happening inside Ogre under the hood, and can deal with any bugs or problems along the way.
Image
chincillya
Halfling
Posts: 94
Joined: Sun Dec 19, 2004 7:12 pm
Location: Finland
x 3
Contact:

Re: 2 Multithreading questions

Post by chincillya »

I briefly looked over the code that is run when the render window is created, and it seems that the WorkQueue is initialized at that point, probably what creates all those extra threads. Just weird that it shows 20 threads instead of 8 (I was running it on an Intel Core i7 with 8 HW threads).

One thing that came to my mind was that, if I create an own scheduler and N number of work threads (where N is the number of HW threads on the target machine), and Ogre ALSO creates N number of work threads, can this be a problem? Then I would have one main thread + 16 worker threads. This sounds like a lot to me but maybe it's not, what do you think? Is there some significant overhead for having this many threads created? What about idle threads? I guess the Ogre work threads are put to sleep when not needed. Do these require any prosessing power?
"Black holes are the result of God dividing by zero."

http://www.sebastian-ahlman.fi/
User avatar
Jabberwocky
OGRE Moderator
OGRE Moderator
Posts: 2819
Joined: Mon Mar 05, 2007 11:17 pm
Location: Canada
x 218
Contact:

Re: 2 Multithreading questions

Post by Jabberwocky »

I'm totally not an expert on multithreading. But an idle thread should have no significant impact on your computer's performance.
Look at all the processes running on your computer. Most of them are 0% CPU and very low memory usage. A thread is more lightweight than a process.
Image
User avatar
Zonder
Ogre Magi
Posts: 1168
Joined: Mon Aug 04, 2008 7:51 pm
Location: Manchester - England
x 73

Re: 2 Multithreading questions

Post by Zonder »

Ignore the thread count it is irelevent unless they are active also most system are running thousands of threads the OS is designed to handle this.

You should also not try and force threads to certain CPUs as this can degrade performance let the OS thread scheduler handle this it's what it's designed to do. All you should care about it you need somthing running in the background so start it in the background and forget about it as the OS will know what CPU is under less load and place the thread there.

The reason for precreating the threads is it can be slow, so having them premade will speed up launching background tasks (hence having a scheduler) you should create more threads though if your background tasks use up all threads. Your background tasks should also be short not long running
There are 10 types of people in the world: Those who understand binary, and those who don't...
chincillya
Halfling
Posts: 94
Joined: Sun Dec 19, 2004 7:12 pm
Location: Finland
x 3
Contact:

Re: 2 Multithreading questions

Post by chincillya »

Zonder wrote: You should also not try and force threads to certain CPUs as this can degrade performance let the OS thread scheduler handle this it's what it's designed to do. All you should care about it you need somthing running in the background so start it in the background and forget about it as the OS will know what CPU is under less load and place the thread there.

The reason for precreating the threads is it can be slow, so having them premade will speed up launching background tasks (hence having a scheduler) you should create more threads though if your background tasks use up all threads. Your background tasks should also be short not long running
Yeah, this pretty much plays into what I already have in mind. I plan on starting as many threads as there are cores and assign jobs to them based on job priority and dependencies. I don't know why you mention forcing threads to certain CPUs. This architecture does not do that, right?

And yes, the jobs should be very short tasks. Most will probably be short enough so that a large amount of them can be run during one frame. The excpetion being the loading jobs, which can take several seconds to complete.
"Black holes are the result of God dividing by zero."

http://www.sebastian-ahlman.fi/
User avatar
Zonder
Ogre Magi
Posts: 1168
Joined: Mon Aug 04, 2008 7:51 pm
Location: Manchester - England
x 73

Re: 2 Multithreading questions

Post by Zonder »

several seconds is fine for a job I always think 10 is too much. It is still somthing you have to judge for yourself though if you run only 1 job that take 5 mins and lots of small jobs that will probably work well. Best thing to do is always test with a single core system if you have a single core with hyper threading turn that off in the bios.

I just mentioned forcing threads to a cpu as a warning didn't explain well :)
There are 10 types of people in the world: Those who understand binary, and those who don't...
Jusas
Halfling
Posts: 91
Joined: Mon May 14, 2007 9:14 am
x 8

Re: 2 Multithreading questions

Post by Jusas »

The background loading isn't too difficult, I think what makes it feel difficult is the lack of examples. I'm using several threads myself and I can tell you that Ogre's own background threads doesn't really mess with yours, it's very straightforward to use. Use your own methods to create and manage your own engine's threads but let the Ogre resource loading threads do their assigned job. I've implemented it on several projects and to help you get on the right track here's an example with some code samples:

My application in short:
  • 1. The app starts, Ogre gets initialized
  • 2. We are given a scene file to load and told where all the resources lie
  • 3. We load the scene file, read which meshes we need to load and we begin the background loading of those meshes and their materials
  • --- In the meanwhile we render a pretty, animated loading screen ---
  • 4. When the loading is complete, construct the scene
  • 5. Scene constructed, now show it
1) You know the drill...

2) Initialize the proper resource locations. For example:

Code: Select all

ResourceGroupManager* resman = ResourceGroupManager::getSingletonPtr();
resman->addResourceLocation(path, "FileSystem", "General");
resman->initialiseResourceGroup("General");
This parses the material scripts and declares the resources it finds.

3) Now we've parsed a scene file and have a list of meshes to load. To monitor the progress we need our loading class to implement the ResourceBackgroundQueue::Listener interface, ie.

Code: Select all

class SceneLoader : public Ogre::ResourceBackgroundQueue::Listener
{
	...
	virtual void operationCompleted(Ogre::BackgroundProcessTicket ticket, const Ogre::BackgroundProcessResult& result);
	...
};
Every time our resource gets loaded, operationCompleted gets called. It gets called from the Ogre thread, worth mentioning.

So... we have created a new thread to handle the loading and leave Ogre to render stuff. Now from our new thread we push our resources to the background queue to get loaded, for example:

Code: Select all

unsigned long ticket = ResourceBackgroundQueue::getSingletonPtr()->load("Mesh", filename,
                ResourceGroupManager::DEFAULT_RESOURCE_GROUP_NAME, false, 0, 0, this);
m_tickets[ticket] = filename;  // store all tickets...
Now that all the resources are in queue, we wait. Whenever a resource gets loaded, operationCompleted lets us know and we'll know what resource just got loaded. We keep record of the tickets and when all the tickets have been processed the loading is complete.

4) Now that the resources are loaded we can build our entities and scene hierarchy. This can also be done from the background thread, that's what I do since building all the entities takes time. However I can't do everything in this thread as the whole Ogre isn't thread safe; (somebody please correct me if I'm wrong but) I recall it isn't safe to do these things from another thread: creating SceneNodes, cameras, lights and attaching entities to nodes or creating static geometry isn't safe so I leave them to be done in the Ogre thread.

So what I do is:
- Create skeletal animations (loading thread)
- Create entities (loading thread)
- Create scene nodes, lights and cameras and attach the entities to the proper scene nodes, create static geometry (Ogre thread)

I hope this helped. I may have missed something so if you've got questions please ask away.
chincillya
Halfling
Posts: 94
Joined: Sun Dec 19, 2004 7:12 pm
Location: Finland
x 3
Contact:

Re: 2 Multithreading questions

Post by chincillya »

Wow, thanks a lot, this will surely help me get my app together. I'm currently still working on the thread scheduler for the other non-ogre threads, but now I have something to go after when tieing it to Ogre. :)
"Black holes are the result of God dividing by zero."

http://www.sebastian-ahlman.fi/
User avatar
Meraviglioso
Deactivated User
Posts: 1
Joined: Tue Sep 20, 2011 11:56 am

Re: 2 Multithreading questions

Post by Meraviglioso »

Jusas, you did a great job. Kudos from me too
CapnJJ
Gnoblar
Posts: 6
Joined: Tue Mar 09, 2010 7:48 am
x 1

Re: 2 Multithreading questions

Post by CapnJJ »

@Jusas:
Just jumping in with my two cents here...
fantastic explanation/mini-tutorial!

Thank you very much for providing it!

Adding my kudos for the post!

And thanks to the OP for the thread!

Cheers,
CapnJJ
zfx
Gnoblar
Posts: 6
Joined: Thu Sep 23, 2010 4:34 pm

Re: 2 Multithreading questions

Post by zfx »

Hi Jusas!

Thanks for the step-by-step explanation. At first, it seemed that it had all I needed to implement my own background loading, but now that I started to do it, a few questions have come up.

You say:
"So... we have created a new thread to handle the loading and leave Ogre to render stuff. Now from our new thread we push our resources to the background queue to get loaded, for example:"

I thought that by using Ogre::ResourceBackgroundQueue we leave the loading thread creation to Ogre. Or do you mean a general loading thread, which itself uses Ogre::ResourceBackgroundQueue which creates another loading thread from within our own loading thread? So in the end we have 2 threads, the first created by us, and the second created by Ogre when using Ogre::ResourceBackgroundQueue inside our own thread? Is this correct?

I have further questions, but some may be irrelevant, depending on the answer to my first question.
Of course, anyone can feel free to give me an answer or suggestion... I decided not to make yet another "Background resource loading" topic, as this post seems to go into the right direction and maybe after some additional info it will become something like a tutorial (I hope). It looks like Ogre::ResourceBackgroundQueue had a lot of work invested in it and it is a real pity that there is no proper explanation and simple example of how to use it as intended.
For scene loading I am using OgreMax, but for the beginning I want to get a better idea of how this is supposed to work.

I want to say also that I spent a couple of days searching the forum, looking at the source code and step-by-step debugging into Ogre::ResourceBackgroundQueue. At first i thought things get clearer, but from some moment on things got more and more confusing and contradicting. So any help is GREATLY appreciated :)

Thanks in advance to everyone!
Jusas
Halfling
Posts: 91
Joined: Mon May 14, 2007 9:14 am
x 8

Re: 2 Multithreading questions

Post by Jusas »

You say:
"So... we have created a new thread to handle the loading and leave Ogre to render stuff. Now from our new thread we push our resources to the background queue to get loaded, for example:"

I thought that by using Ogre::ResourceBackgroundQueue we leave the loading thread creation to Ogre. Or do you mean a general loading thread, which itself uses Ogre::ResourceBackgroundQueue which creates another loading thread from within our own loading thread? So in the end we have 2 threads, the first created by us, and the second created by Ogre when using Ogre::ResourceBackgroundQueue inside our own thread? Is this correct?
I should have been clearer, the threads we're looking at are:
1. Ogre thread - the thread that created Ogre::Root and handles the rendering. Usually the "main" thread of your application.
2. Ogre background loading thread - created automatically by Ogre to load resources in the background.
3. Our own application's loading thread - in my case its job is to handle the whole loading process. It reads the scene file contents, grabs the list of meshes that need to be loaded, tells Ogre to load them (ResourceBackgroundQueue::getSingletonPtr()->load(...)) and goes to "sleep", awaiting a signal that all the required resources have been loaded and then wakes up again to do the finishing touches, such as creating entities, setting up custom data etc. and schedules the creation of scenenodes etc. to the Ogre thread.

I'll see if I have some time during the weekend to write a complete tutorial to the wiki with sources. It looks like people could have use for it.
progmars
Halfling
Posts: 60
Joined: Fri Jun 17, 2005 10:47 am

Re: 2 Multithreading questions

Post by progmars »

Thanks, Jusas, your last post had almost all the missing bits of info I was looking for.
Waiting for a full wiki article, if you'll get to it.

Multithreading in games is complicated, so it is best if you can desing some solid, rock-stable task scheduler, so you later can throw any task at it and be sure that there will be no data corruption nor race conditions between threads.

I recently tried Microsft Visual Studio 2010 and C++ asynchronous agents with thread-safe buffers. I was using them to process frames from a webcamera, the overall experience was good, pretty simple and straightforward to use. And now I am curious, would async agents be good to use also in a game engine's task scheduler?
But that's too Microsoft specific, I guess :lol:
Jusas
Halfling
Posts: 91
Joined: Mon May 14, 2007 9:14 am
x 8

Re: 2 Multithreading questions

Post by Jusas »

I've got the code done already, just need to write an article. However I still can't get it working properly with OpenGL - I asked for solutions on the forums but received no answers. I was hoping to get it sorted before posting a wiki article but it doesn't look too good. I'm very much puzzled since it's not doing anything out of the ordinary, just using the standard Ogre ResourceBackgroundQueue to load the meshes. I guess there are still issues with OpenGL and background loading and I just can't figure them out.

Oh well, perhaps somebody will figure it out after the article is posted. I can hopefully write it this week.
Uldall
Gnoblar
Posts: 13
Joined: Fri Mar 25, 2011 5:10 pm

Re: 2 Multithreading questions

Post by Uldall »

Hi Jusas,

Did you ever get time to write the article? And did you resolve the problems with OpenGL?

Best,
Uldall
Post Reply