DirectX 11 render system - work-in-progress

Discussion area about developing or extending OGRE, adding plugins for it or building applications on it. No newbie questions please, use the Help forum for that.
Post Reply
User avatar
stealth977
Gnoll
Posts: 638
Joined: Mon Dec 15, 2008 6:14 pm
Location: Istanbul, Turkey
x 42

Re: DirectX 10 render system - work-in-progress

Post by stealth977 »

I can see why supporting the same functions on both systems are very important, its part of being an engine, but;
* If OpenGL falls back as it is the case now, will that also mean that OGRE will fall back with it just to meet the criteria to have all functions work on both systems?
* What if OpenGL and Directx parts their ways, implementing stuff in their own way which can not be simulated with a combination of refactored small functions?
* When will the actual switching to DirectX10-11 will happen? At some point DX9 may become obselete and what will OGRE do when it happens and if OpenGL and DX10-11 cant be unified anymore (or at least cant be unified at the time of happening)?

Would it be wise to have a separate OGRE Core that is designed to work on DX10-11 and add OpenGL support to it when OpenGL is ready? Because graphics is a technology sensitive part of programming, it is not so wise to fall behind too much in a race.

ismail,
Ismail TARIM
Ogitor - Ogre Scene Editor
WWW:http://www.ogitor.org
Repository: https://bitbucket.org/ogitor
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: DirectX 10 render system - work-in-progress

Post by Assaf Raman »

My main problem with d3d10 only features is that d3d10 is only supported on vista – and I want to support XP also. Moreover – most of the users have hardware that doesn't support d3d10, so – basically – d3d10 is still too bleeding edge for me to actually commercially use.
Watch out for my OGRE related tweets here.
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: DirectX 10 render system - work-in-progress

Post by sinbad »

My take on this is that you don't want to add any mandatory elements that rely on Dx10/11 etc, but it's ok to add 'enhanced' functionality on these platforms, such as the geometry shaders etc. This is the same strategy we've had for ages - we still support fixed function while adding shader support etc. I don't want to change that - some people suggest dropping fixed-function, but I don't think it's necessary or desirable, since it locks you out of some platforms, even new ones like the iPhone. I think it's entirely doable to support a wide range of hardware like this.

However, the issue is with priorities. I've personally found that you can do a hell of a lot with just SM2 or even 3, and in practice you want to concentrate your efforts on getting the best quality out of what the vast majority of people can use. That still isn't DirectX10; that may start changing if Windows 7 is more popular than Vista, but it's still held back by a relatively small hardcore of people that have Dx10 machines that actually deserve to be classed as Dx10. As such, Dx10/11 remain a 'nice, but niche' option for the moment. Most of my personal effort goes into things that are more universal, and that's not likely to change until I have a specific reason to major on Dx10/11.

Far too many enthusiasts label technologies 'dead' way before their time - people were declaring Dx9 dead years ago and they've been wrong every year since then. Lots of technophiles like to only work with the latest & greatest, but anyone designing an engine around a Dx10/11 prerequisite is nuts. The real world has a much larger range of software & hardware - even Crytek realised they'd painted themselves into a Dx10 corner and that they had to cater for a much wider audience than they were doing previously. If you have the time to add Dx10-specific features then great, but they should be considered an optional extra, the most important thing is to refine the experience for the other 80% of users. Even players on Steam are less than 30% Dx10-capable, and they're a self-selecting bunch that are generally more up to date than the general public (as indicated by the fact that only 4% of them have Intel chipsets).
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: DirectX 10 render system - work-in-progress

Post by Assaf Raman »

I fixed the assert and added WRAP driver support. I also added the "normal" software driver so you can compare.
Get the latest code from the SVN, compile and you will have a new render system option named "Driver type" with 3 options - Hardware, Software and Warp.
Tell me if it works for you.
Watch out for my OGRE related tweets here.
User avatar
Red5_StandingBy
Greenskin
Posts: 109
Joined: Mon Jun 26, 2006 1:06 pm
Location: Turkey

Re: DirectX 10 render system - work-in-progress

Post by Red5_StandingBy »

Hi,

In my previous company, we were very enthusiastic about using both DX9 and DX10 in our game engine for our next game. I had to quit just as we started to get some good results in terms of performance and also being able to manage the content pipeline for both paths. Putting aside geometry shaders and other DX10 only features, it turned out that setting up a lighting-shadowing system that works for both renderers was doable with little changes to Ogre core.

I like to encourage people working on DX9 apps to consider (or reconsider) making an attempt at supporting DX10 at a basic level too. It will not be a big effort. I think it is needed to create some kind of momentum in DX10-11 direction.
User avatar
boyamer
Orc
Posts: 459
Joined: Sat Jan 24, 2009 11:16 am
Location: Italy
x 6

Re: DirectX 10 render system - work-in-progress

Post by boyamer »

The source code is in the trunk?
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: DirectX 10 render system - work-in-progress

Post by Assaf Raman »

Yes.
Watch out for my OGRE related tweets here.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: DirectX 10 render system - work-in-progress

Post by Assaf Raman »

Seems that d3d11 is out!
http://www.microsoft.com/downloads/deta ... laylang=en
The August 2009 DirectX SDK contains the first official release of the DirectX developer resources for Direct3D 11, DXGI 1.1, Direct2D, and DirectWrite. Developers can now publish and distribute Direct3D 11 applications and games that leverage all of the software and hardware features of DirectX 11 in Windows 7 and Windows Vista.
I am waiting for the commit of Nir's RT Shader System Component before starting to work on it.
Watch out for my OGRE related tweets here.
KAV
Gnoblar
Posts: 9
Joined: Thu May 29, 2008 2:40 pm

Re: DirectX 10 render system - work-in-progress

Post by KAV »

Is Microsoft releasing Direct3D APIs faster than our team can handle to implement them for Ogre3D?
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: DirectX 10 render system - work-in-progress

Post by Assaf Raman »

KAV wrote:Is Microsoft releasing Direct3D APIs faster than our team can handle to implement them for Ogre3D?
Do you have any project or anything of the sort that actually need any of the new API?
Watch out for my OGRE related tweets here.
User avatar
joew
Greenskin
Posts: 113
Joined: Fri Nov 03, 2006 6:03 pm

Re: DirectX 10 render system - work-in-progress

Post by joew »

Considering the userbase for D3D10 (I'm not even going to mention 11 :lol: ) in my opinion Ogre is actually ahead of time implementing this.
User avatar
Azgur
Goblin
Posts: 264
Joined: Thu Aug 21, 2008 4:48 pm

Re: DirectX 10 render system - work-in-progress

Post by Azgur »

I am wondering though, from what I've understood DX11 is backwards compatible.
Is it strange to think this render system might end up being evolved into a DX11 render system and dropping DX10? (dropping as in, not being compiled against dx10, but against dx11, but still keep dx10 hardware in mind)
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: DirectX 10 render system - work-in-progress

Post by Assaf Raman »

D3d11 also supports d3d9 hardware.
And yes - I think we will drop d3d10 support after we get the d3d11 render system going - no one uses is so - I guess no one will miss it if the d3d11 render system will support the same API.
Watch out for my OGRE related tweets here.
KAV
Gnoblar
Posts: 9
Joined: Thu May 29, 2008 2:40 pm

Re: DirectX 10 render system - work-in-progress

Post by KAV »

Assaf Raman wrote:Do you have any project or anything of the sort that actually need any of the new API?
Will Ogre3D be ready the day the d3d10 / d3d11 features become relevant?
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: DirectX 10 render system - work-in-progress

Post by Assaf Raman »

No, but then we will have a reason to start working on being ready.
On a side note I ported the d3d10 code to d3d11 and was able to compile the code in about 2 hours, d3d11 is almost the same as d3d10 – they just added better thread support and a few new shader types.
Watch out for my OGRE related tweets here.
User avatar
Lee04
Minaton
Posts: 945
Joined: Mon Jul 05, 2004 4:06 pm
Location: Sweden
x 1

Re: DirectX 10 render system - work-in-progress

Post by Lee04 »

Can we perhaps rename the thread DirectX11 - work in progress now.
So newbees don't think we are left behind here at the Ogre project.



.
Ph.D. student in game development
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: DirectX 10 render system - work-in-progress

Post by Assaf Raman »

When (or if) I will start the work - I will create a new thread.
Watch out for my OGRE related tweets here.
User avatar
Assaf Raman
OGRE Team Member
OGRE Team Member
Posts: 3092
Joined: Tue Apr 11, 2006 3:58 pm
Location: TLV, Israel
x 76

Re: DirectX 11 render system - work-in-progress

Post by Assaf Raman »

I changed the topic title from DirectX 10 to DirectX 11.

As you may see from the trunk – I created a d3d11 render system – base on the d3d10 render system code.
D3d11 is supported on the same OS as d3d10 and has support also for d3d9 class hardware.
Moving to d3d11 doesn't limit any project that I can see.
The team hasn't decided yet if the d3d10 will be removed – but this seems to be the direction.
I added RTSS support for the d3d11 render system by modifying both the render system and the RTSS. If you want to run the RTSS with the render system using cg – update the dependencies to cg 2.2 (h, lib and dll).
There are still many unresolved issues that you can see when running the samples – like render to texture, cube map, texturing issues and more.
The same problems exist also in the d3d10 render system.

My focus if I will work on this topic will be:
1. Make the RTSS to be the only shader generator.
2. Make all the sample work correctly.
3. Add support for dynamic linkage to the shader generated by the RTTS.
4. Multi window support.
5. Multi device support.

I don't have and deadline, this is a hobby.

Anyone that wants to help is welcome.
Watch out for my OGRE related tweets here.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: DirectX 11 render system - work-in-progress

Post by dark_sylinc »

OK. Assaf and I had a chat, and this is what came out from DX10/11:

By design, there are some issues that are holding back D3D10/11 support (NO! I'm not talking about fixed function)

I've been seeing that one big deal now are the resource usage/access flags from Textures. For instance, this is preventing many samples to work because this is an issue with shadow textures (the ShadowTextureManager creates a 1x1 texture filled with 1.0 in case a shader has more shadow texture bounds to the material than the current specified shadow textures through the SceneManager).
So, any sample using shadow textures (pretty standard by now) isn't running.

Let's see the differences:
  • In D3D9, textures can be created with flags that closely relate TextureUsage flags (dynamic vs static, write & read access, etc)

    In D3D10, textures with CPU access can not be used normally in the GPU. That is, you can't lock a buffer in a frame, write contents from the CPU, and expect to see it rendered being used as a texture.
When a texture is created with CPU access, it's contents have then to be copied to a GPU-only access texture using CopyResource().
This could be emulated in D3D10 RS by creating two textures (one w/ CPU access, another w/ GPU access), when D3D10Texture::lock() is called, the CPU texture gets locked, then in D3D10Texture::unlock() we copy the resource. Problem solved... not quite.

This would mean that every texture should be created in pairs, using this technique. Awful. It wastes memory and processing time. And internally this probably means lots of locks.

The only way to write to a GPU-only-access texture from CPU, is when it is initialized.
In Ogre, there's such no interface to do that. Currently this is handled by creating the texture, and then locking it's contents only once.

If the texture is going to be created to be only locked once, then the D3D10 flag "D3D10_USAGE_IMMUTABLE" is the best suited for it. Probably this flag suits most textures loaded from file.

So, problem #1: Ogre lacks a interface to define the contents at initialization
Basically I'm saying there should be a createTexture() function that accepts a buffer as the last parameter, which will be the contents that will be uploaded to the GPU.
On D3D10, "immutable" flag is used and the buffer is supplied. On D3D9 and OGL, this behavior is emulated by creating the buffer with similar flags as STATIC_WRITE_ONLY and then using a lock to upload the contents.

So, problem #2: current TextureUsage flags are not enough
There's no flag that hints the RenderSystem the resource won't ever be accessed by the CPU again (besides initialization). The closest match is TU_STATIC_WRITE_ONLY, but this flag actually guarantees the CPU can gain write access again at any point. We need something like

Code: Select all

TU_STATIC_IMMUTABLE or TU_STATIC_NO_ACCESS
that hints the RS the immutable flag is desired (which gives the fastest performance). Other RS can use to tell it wants the flag combination that maximize performance while restricting CPU access as most as possible (i.e. most likely static write only in D3D9)
Locking an Ogre texture created with TU_STATIC_IMMUTABLE should raise a Log error (or exception?) in all RenderSystems to catch portability issues (i.e. code works fine in D3D9 & OGL, but crashes in D3D11)

I think this new flag should be used for creation of 2D textures by default.


Now, D3D10/11 specific flag combinations:
With TU_STATIC_NO_ACCESS leaving the responsibility to get maximum performance for the developer, we can now take care of the other flags:
It doesn't seem that D3D10/11 makes much distinction between "Dynamic" and "Static".
  • When texture is TU_STATIC_WRITE_ONLY, use D3D10_USAGE_DEFAULT; but paired with a D3D10_USAGE_DYNAMIC. When the lock is called, write contents (using Map()) to the Dynamic testure. When unlocking, the texture is copied to the Default one, using CopyResource

    When texture is TU_DYNAMIC_WRITE_ONLY; I'm a bit confused by Direct3D documentation. It seems it could be possible to just use a texture using D3D10_USAGE_DYNAMIC. If this isn't possible, then the only alternative would be to do the same as TU_STATIC_WRITE

    When there is a READ flag (whether static or dynamic), use D3D10_USAGE_DEFAULT, paired with another texture using D3D10_USAGE_STAGING. This would act as intermediate in the same way. When the lock is made, the contents from the main texture are copied to the intermediate one; then after unlocking the contents are copied from the intermediate to the main one.
So, long story short the important changes are:
* Create an Ogre interface that allows specifying the initial contents of the texture.
* Create a new TextureUsage flag
* Change many portions of Ogre to use this new flag and interface.

For example in the case of the 1x1 Shadow Texture not working in D3D10/11, it could be easily solved by submitting the contents when creating, instead of creating, then locking.

What do you guys think?
Cheers
Dark Sylinc

Edit:
There's a chance this applies to other resources too (i.e. Vertex & index buffers)
User avatar
Lee04
Minaton
Posts: 945
Joined: Mon Jul 05, 2004 4:06 pm
Location: Sweden
x 1

Re: DirectX 11 render system - work-in-progress

Post by Lee04 »

Sounds good
But more research and perhaps asking people that knows DirectX9/10/11 API use and intended use really well.
If someone from Microsoft's DirectX team or ATI/NVidia/Intel would pop by the Ogre forums now and again and check up on the DirectX11 thread and code in progress would be good.
And talk to us about Ogre DirectX9/10/11 specific problems.
Ph.D. student in game development
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: DirectX 11 render system - work-in-progress

Post by dark_sylinc »

Lee04 wrote:Sounds good
But more research and perhaps asking people that knows DirectX9/10/11 API use and intended use really well.
If someone from Microsoft's DirectX team or ATI/NVidia/Intel would pop by the Ogre forums now and again and check up on the DirectX11 thread and code in progress would be good.
And talk to us about Ogre DirectX9/10/11 specific problems.
There's a lot about that already

I'm out of words in that phrase, here's more.

The big problem is that nobody is seriously caring about DX10 due to it's Vista/7 nature. May be Win7 will change this.
Furthermore, there's some truth about what you say regarding lack of experience. When I was fixing a bug yesterday, the official documentation was a bit ambiguous so I googled it. There was so little information about it! Where DX9, when you google something, you will find someone who has already got the problem and fixed.
And no one is dedicating full time on the plugin just because.. well, like I said very few people is caring.
User avatar
Lee04
Minaton
Posts: 945
Joined: Mon Jul 05, 2004 4:06 pm
Location: Sweden
x 1

Re: DirectX 11 render system - work-in-progress

Post by Lee04 »

There was so little information about it!
That is what I am saying. We need to go to the source the people how have been dreaming up the API's Drivers and hardware.
Don't trust what other people online are saying unless it's testable. We need to go to the source and talk to them between four eyes.
and get a propper understanding.
Ph.D. student in game development
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: DirectX 11 render system - work-in-progress

Post by sinbad »

For the record, I'm completely OK with adding new interfaces and usage modes to better support Dx10/11's styles. The keys are:
  1. To not break existing Dx9/GL implementations
  2. To retain a unified interface across all render systems. If that means deprecating some usage models which are not supported in Dx10/11, that's ok so long as there the new route which works on Dx9/GL. And, there's a 'hierarchy' of interfaces here - the most important ones to keep unified and consistent are those the user is most concerned with, that being the resource interfaces and scene object interfaces. The render system interface itself (and lower-level aspects of the resource management that the user doesn't see) is for advanced users and therefore more change here is tolerable, again so long as all the other APIs are kept up to date and working.
  3. To make it clear where divergent behaviour or performance will occur based on the choice of interface on each rendersystem.
In the case of dynamic textures which are written from the CPU, Dx10/11 is only different on an API basis, it's not really different in substance - in Dx9 when you have a dynamic texture writable from the CPU, you're actually writing to driver memory and the driver is synchronising it with the GPU copy; basically the same thing as you do in Dx10 manually with CopyResource. Unfortunately Microsoft decided to force people to implement such relaying themselves, presumably to 'teach' them what actually happens under the surface or to make their own code simpler at the expense of the application; even though most people already fully understood the issues. It's change for change's sake in my opinion, but that's par for the course with MS.

Dx10/11 is still in my 'interesting, but I have better things to do' pile, where it's sat for about 2-3 years now; which is also where it's been for a very large number of people outside of research and GPU labs. If I had a specific need for it I'd be all over it, but I still don't - literally next to no-one I deal with asks about it, and those that do aren't that bothered when I say it's not done yet. Win7 may change this over time, we'll see. But actually I think the landscape has shifted and people are more interested getting Dx9-class content to devices like phones and tablets than stretching high-end PC cards. Anyone who has a more pressing need for it is welcome to help out.
User avatar
dark_sylinc
OGRE Team Member
OGRE Team Member
Posts: 5296
Joined: Sat Jul 21, 2007 4:55 pm
Location: Buenos Aires, Argentina
x 1278
Contact:

Re: DirectX 11 render system - work-in-progress

Post by dark_sylinc »

sinbad wrote:For the record, I'm completely OK with adding new interfaces and usage modes to better support Dx10/11's styles. The keys are: (...)
Cool. Funny how we think similar. So I guess you agree with the suggestions I've made.
sinbad wrote: basically the same thing as you do in Dx10 manually with CopyResource. Unfortunately Microsoft decided to force people to implement such relaying themselves
Thanks! I've imagined at some point this could've been what was happening, that Dx10 was just moving to us what Dx9 driver was doing.
While this increases the code size on app side, it allows more flexibility and avoids the driver from "keep guessing" what we intended to do. Cool for me.

So now everything is much clearer now for me. And now I understand:
  • Always use the D3D10_USAGE_DEFAULT flag
    Presence of WRITE flag, absence of READ flag means we can use the intermediate buffer w/ D3D10_USAGE_DYNAMIC
    Presence of READ flag, means we need to use the intermediate buffer w/ D3D10_USAGE_STAGE

    Presence of DYNAMIC flag means the intermediate buffer should be created along w/ the main one, and destroyed when the main one is release. Needs more VRAM but is perfect for frequent locking. Ideally the contents should be updated to the main buffer before rendering, not after unlock()
    Presence of STATIC flag means the intermediate buffer should be created when locking, and released when unlocking. This helps saving VRAM. Contents would be updated to the main buffer when unlocking.
Doing this should leave Dx10 in a compatible state w/ D3D9 & OGL applications regarding textures.

And Dx10 now has a more powerful flag to rule them all (D3D10_USAGE_IMMUTABLE). This one's the best for many Ogre usages (i.e. automatic texture loading from files) but is the hardest to achieve because it needs code changes in Ogre:
  • * Add flag to Ogre (easy)
    * Add interface to specify the buffer contents on creation (easy)
    * Handle the new flag in existing RSs (OGL & D3D9); treat them as STATIC_WRITE, and cause an exception when trying to lock, warning it is possible to lock in D3D9, but it would break other RenderSystems. (slightly moderate)
    * Rewrite many Ogre routines scattered everywhere to use this new flag (moderate to hard)
    * Advanced Users may need to change their manual creation routines to adapt to the new flags
    * See how this rules apply to other resources (i.e. vertex buffers) as well (easy to moderate)
sinbad wrote:
Dx10/11 is still in my 'interesting, but I have better things to do' pile
Same here

So, anything who agrees/doesn't agree with what I just said?
User avatar
sinbad
OGRE Retired Team Member
OGRE Retired Team Member
Posts: 19269
Joined: Sun Oct 06, 2002 11:19 pm
Location: Guernsey, Channel Islands
x 66
Contact:

Re: DirectX 11 render system - work-in-progress

Post by sinbad »

dark_sylinc wrote:Presence of WRITE flag, absence of READ flag means we can use the intermediate buffer w/ D3D10_USAGE_DYNAMIC
Presence of READ flag, means we need to use the intermediate buffer w/ D3D10_USAGE_STAGE
In principle yes, but actually what might be a good idea is to break the permanent 1:1 relationship between the GPU copy and the CPU-accessible version in both read and write modes. Given the same size & format for example, there's no reason why multiple textures couldn't be updated via one intermediate buffer, or why a read operation couldn't re-use an existing temporary surface used for all textures of the same dimensions and format. This could lead to more efficient staging of content which is presumably the idea behind the change.

As for the D3D10_USAGE_IMMUTABLE flag, the key will be to defer creation of resources until the last possible moment (ie when data is available), which isn't that hard in principle except that the texture loading routines currently use a common loading path which assumes a create surfaces/populate data sequence. This could either be changed throughout to allow for unified implementations of create/populate, or the Dx11 implementation could simply override the common routines more fundamentally to change that assumed sequence. The former is better, but more complicated.
Post Reply