Texture streaming

Threads related to Google Summer of Code
Post Reply
Wyk3d
Gnoblar
Posts: 2
Joined: Sat Mar 13, 2010 2:54 pm

Texture streaming

Post by Wyk3d »

Hi!

I still have plenty of researching to do on the subject but here's what I got so far:

The benefits of streaming textures are twofold. Frist, the initial loading of a scene only needs to last until the necessary mip levels are loaded and the rest can be streamed in later. And second, the memory footprint of textures can be reduced (allowing a higher level of detail per frame). There's an important point to make at least in the OpenGL case here. As far as I know if the available memory in the GPU is exceeded OpenGL allows continuing to load textures intro normal memory and the driver can handle swapping it to VRAM latter. To properly estimate how much texture streaming will help I still need to find more details on how good it is at doing that. Also, in general with modern GPU having truckloads of RAM, I could use some help evaluating the actual impact this project would have on the community, how many people would really benefit from it.

Streaming involves dynamic loading and unloading. The efficiency of the approach depends on how well the times are chosen to perform those operations. With calculating the exact time to load a particular mipmap being fairly complex, an approximation based on distance and orientation to polygons mapped with the texture can be used. To get around doing it for every polygon a more rough estimate can be obtained for a particular texture by taking the shortest distance to the bounding box of an object mapped with it. If the camera is moving towards such an object, enough time should be left to do the load by the time it gets close enough. So the user should be allowed to set a speed factor depending on the maximum speed of the camera. If I understood correctly, blending through shaders will be needed when the new texture fails to load fast enough and then that will help reduce the popping effect ?

Based on remarks from "Pro Ogre Programming" and on the fact that WorkQueue::addRequest is only callled from terrain/paging code it is possible to put a new request in thread pool from the application side to manually load a texture but the loading code in Texture uploads all mipmap levels to the hardware buffers. The main loading loop will need to be refactored to allow loading specific levels separately and then separate requests could be added to the work queue for them. This introduces several thread safety concerns, the mip levels are looped over in several other places. Thread safety seems like an issue even without separating mipmaps, there doesn't seem to be any guarding against unloading the texture while it is being loaded for example.

To be continued .. well, maybe next year of if i find more time.

Any feedback, suggestions or advice quite welcome :).
Last edited by Wyk3d on Sun Mar 21, 2010 6:25 pm, edited 2 times in total.
Wyk3d
Gnoblar
Posts: 2
Joined: Sat Mar 13, 2010 2:54 pm

Re: Texture streaming

Post by Wyk3d »

After bumping into this thread (who would have thought the search feature actually works :D) I thought I should mention that I was thinking about not necessarily loading the top N mip levels so much as the right set of mip levels. So I figured if the object is far enough away and is small enough you could save memory heavily by skipping not the lower but the higher detail levels. If the approximation is good enough then in the case of the streamed initial loading version, the textures shouldn't look like "arse" (quote from sinbad ;) ) when first displayed but only if the camera moves too fast in the direction of textures that need mip levels not yet loaded and that other hand could be fixed by the speed factor and some blending. Of course getting the formula just right will probably be the last things I will tackle as there's plenty of other things to do up to that point. In the design I will definitely want to explicitly allow multiple strategies for this, to decouple it as much as possible.

Also that thread seems to partially answer my "community impact" problem above :).
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: Texture streaming

Post by sinbad »

Thanks for the thoughts.

Yes, mostly we're interested in streaming in smaller mip levels first, actually into a smaller copy of the texture (to save RAM) then upgrading it to the full sized, full mip stack texture later as and when needed. It's actually possible that there may be a use for multiple stages in this transition rather than just low/high.

Being able to have only high mip levels present isn't that useful, since even close up, polygons at glancing angles will reference small mips for efficiency, since skipping across large areas in the texture when rasterising (which is what happens at glancing angles) makes the texture cache useless, so it's better to sample from the smaller mip in these cases.

Even with GPUs having larger VRAM these days, it's never, ever enough because artists just want to put bigger textures in it ;) So streaming is useful regardless of the base specification of the machine.

The tricky part with any kind of streaming is knowing when to ask for data to be pre-fetched. Inherently you need to combine all the requests on a given texture by all objects and evaluate what the closest requirement is, combined with a buffer allowance which will depend on how fast cameras (notice this is plural) are moving. Viewport size is also a factor, since higher resolutions need higher detail mips.The Paging system in 1.7 might be a decent place to start with this, at least to hook into what cameras are being tracked, or maybe just as a guide to the style.
User avatar
DavlexDesign
Orc
Posts: 400
Joined: Thu Apr 23, 2009 7:23 am
Location: Australia
x 19
Contact:

Re: Texture streaming

Post by DavlexDesign »

G'day guys,
Just throwing in an idea here, but maybe the "Mega Texture idea" may throw in some light on the accessing and selection of mipmaps maybe.
The GDal library http://www.gdal.org has some nice systems for what you may be looking for, and it is open source too.
It seems to be able to handle amazing amounts of data quite efficiently.

Alex
User avatar
ModelTheMasses
Gnoblar
Posts: 8
Joined: Thu Mar 10, 2011 7:59 pm

Re: Texture streaming

Post by ModelTheMasses »

DavlexDesign wrote:G'day guys,
Just throwing in an idea here, but maybe the "Mega Texture idea" may throw in some light on the accessing and selection of mipmaps maybe.
The GDal library http://www.gdal.org has some nice systems for what you may be looking for, and it is open source too.
It seems to be able to handle amazing amounts of data quite efficiently.

Alex
Davlex,

Have you begun working in 1.8 yet with GDAL? I was really comfortable in 1.7.3, but the 1.8 updates have me a bit flustered at this point. I just wanted to check in with you and see how often you put the source code to use...Last time I ran a major upgrade some of my registries got a little bit messed up and for some reason I lost all of my mail contacts. Luckily I was able to run scanpst.exe, but either way thank you for your time you really helped me troubleshoot how I am going to animate my model and convert the video with a YouTube downloader.
Post Reply