Question regarding HLMS microcode cache
-
- Greenskin
- Posts: 145
- Joined: Fri Jun 12, 2015 6:53 pm
- Location: Florianopolis, Brazil
- x 17
Question regarding HLMS microcode cache
My app have 1 directional light, but since it has day/night cycle, this light keeps changing it's intensity, going from 0,0,0 to almost white in a full day cycle.
I noticed that everytime I get close to a city, if it's a different clock time than the previous time I was there, it recompiles the shaders (freezes for a few seconds).
What I thought is that maybe the HLMS hash is equal but the shader is different if it's night or day, then if I was in the city during the day, the microcode stored one set of shaders (with the light on), and then, when I go there at night, it detects that it needs to recompile the shaders, but then overwrite the day shaders with the night ones, since they share the same hash (assuming they do...) Then, if I ever go back to the city during the day it'll need to recompile again the day shader, since it dumped the previous one(got overwritten), even though it should be stored somewhere in the cache.
I DID notice that different shaders share the same filename, for instance, the shader 538607746PixelShader_ps.hlsl may have some content now, but if I backup this file, and come back to it after playing a little, the shader content will be different than before, I did this and compared the backep up file with the current one, and they were completely different shaders, one had normals, the other had different UVs, etc.
Does any of this make sense?
I noticed that everytime I get close to a city, if it's a different clock time than the previous time I was there, it recompiles the shaders (freezes for a few seconds).
What I thought is that maybe the HLMS hash is equal but the shader is different if it's night or day, then if I was in the city during the day, the microcode stored one set of shaders (with the light on), and then, when I go there at night, it detects that it needs to recompile the shaders, but then overwrite the day shaders with the night ones, since they share the same hash (assuming they do...) Then, if I ever go back to the city during the day it'll need to recompile again the day shader, since it dumped the previous one(got overwritten), even though it should be stored somewhere in the cache.
I DID notice that different shaders share the same filename, for instance, the shader 538607746PixelShader_ps.hlsl may have some content now, but if I backup this file, and come back to it after playing a little, the shader content will be different than before, I did this and compared the backep up file with the current one, and they were completely different shaders, one had normals, the other had different UVs, etc.
Does any of this make sense?
-
- OGRE Contributor
- Posts: 226
- Joined: Thu Oct 14, 2010 12:30 pm
- x 56
Re: Question regarding HLMS microcode cache
Are you absolutely sure that the only thing that you're changing is just the light's colour? Can you post the different shaders that have the same filename?
-
- Greenskin
- Posts: 145
- Joined: Fri Jun 12, 2015 6:53 pm
- Location: Florianopolis, Brazil
- x 17
Re: Question regarding HLMS microcode cache
I'm changing a lot of things, but I couldn't think of anything that could cause this.
I'm changing the Sky shader, Fog shader, light color, light power scale, light direction (it emulates a sun movement).
I'm NOT: changing shadow setup, adding more lights, removing lights (on top of zeroing out the main light color), changing any mesh related stuff or textures.
Backup #1:
https://www.dropbox.com/s/wswk3lzj8mygz ... p.zip?dl=0
Backup #2:
https://www.dropbox.com/s/l23yiusohqtj1 ... 2.zip?dl=0
Thank you.
I'm changing the Sky shader, Fog shader, light color, light power scale, light direction (it emulates a sun movement).
I'm NOT: changing shadow setup, adding more lights, removing lights (on top of zeroing out the main light color), changing any mesh related stuff or textures.
Sure thing, I've picked only the files that have the same name but different content from two runs:Hotshot5000 wrote:Can you post the different shaders that have the same filename?
Backup #1:
https://www.dropbox.com/s/wswk3lzj8mygz ... p.zip?dl=0
Backup #2:
https://www.dropbox.com/s/l23yiusohqtj1 ... 2.zip?dl=0
Thank you.
- dark_sylinc
- OGRE Team Member
- Posts: 5296
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1278
- Contact:
Re: Question regarding HLMS microcode cache
Hi!
The names of the shader may vary per runs, that is to be expected. However the microcode cache hashes the contents of the generated shader and its macro preprocessor options; therefore even if the filenames are different, the cache will still work.
The reason has to be somewhere else, possibly:
To debug what's going on place breakpoints at GpuProgramManager::getMicrocodeFromCache at see what's going on.
The names of the shader may vary per runs, that is to be expected. However the microcode cache hashes the contents of the generated shader and its macro preprocessor options; therefore even if the filenames are different, the cache will still work.
The reason has to be somewhere else, possibly:
- The shader loading / saving routine is not set correctly (e.g. overwriting previous caches, corrupting it, etc). Things to look out for: Accidentally calling GpuProgramManager::clearMicrocodeCache, never calling loadMicrocodeCache, not calling saveMicrocodeCache at shutdown (thus cache is not updated), calls to removeMicrocodeFromCache are being made, two processes trying to access the cache with write access at the same time, thus the second process overwrites the cache or even corrupts it(*).
- You're using very different settings for some reason between each run, thus you're encountering something new that wasn't in the cache.
To debug what's going on place breakpoints at GpuProgramManager::getMicrocodeFromCache at see what's going on.
-
- Greenskin
- Posts: 145
- Joined: Fri Jun 12, 2015 6:53 pm
- Location: Florianopolis, Brazil
- x 17
Re: Question regarding HLMS microcode cache
This is what I needed to know, so the filenames aren't important, the cache is.dark_sylinc wrote:The names of the shader may vary per runs, that is to be expected. However the microcode cache hashes the contents of the generated shader and its macro preprocessor options; therefore even if the filenames are different, the cache will still work.
I'll try to sort this out then.
Let me ask you another question: Should I distribute the cache to my players? Will they use the same D3D11 cache even if they have a different machine (GPU, etc)?
- dark_sylinc
- OGRE Team Member
- Posts: 5296
- Joined: Sat Jul 21, 2007 4:55 pm
- Location: Buenos Aires, Argentina
- x 1278
- Contact:
Re: Question regarding HLMS microcode cache
For D3D11 the cache can be distributed to other players with different CPU/GPUs and it will work as intended.