Ogre / Oculus Rift DK2 Progress

[Kojack]

Hmm, DK2 is a lot more annoying to support than the DK1 was.

The latest rift sdk has two modes: Direct and Extended.
In Extended mode, the DK2 shows up as a 1080 x 1920 monitor in windows. Yes, that's 1080x1920, it's sideways because the DK2 screen is a portrait mobile screen mounted sideways. So you need to either rotate the screen in the windows display config or use a rotated projection in Ogre.
In Direct mode, the DK2 isn't listed as a monitor. Windows and ogre can't see it. Instead, you need to use the rift sdk to attach the DK2 to your app.

Direct mode is the preferred method, since it hides all the multimonitor stuff (starting a game on the wrong screen, rotated display, etc) from end users.

How does Direct mode work? Before creating directx or opengl devices, you need to init the rift sdk. This loads in 7 common rendering libraries (dxgi.dll, d3d9.dll, d3d11.dll, dxgidebug.dll, d3d10core.dll, d3d10.dll, opengl32.dll) and hijacks their creation methods like Direct3DCreate9Ex to redirect them to OVRDisplayRT32.dll (the "render shim"). It also hijacks kernel32.dll and some others to redirect methods like LoadLibrary and GetModuleHandleEx. OVRDisplayRT32.dll contains the code to do the direct mode stuff.

This is where the first problem happens. Direct mode is currently faulty on OpenGL. That's well known. But it seems it's also incompatible with DirectX 9 in Ogre. Trying to attach to the DK2 fails, you just get a normal monitor view and the rift stays in standby mode. DirectX 11 works fine, but you have to remove DX9 from your plugins.cfg file because just having it there (even if you don't select it) will break direct mode for all other render systems (this is because Ogre inits every render system before you select one in the config dialog, when that dialog pops up there are already dx9, dx11 and gl renderers running at the same time).
Ok, that's not too bad. Ogre 2.0 is going to drop DX9 anyway and I'm working with 2.0 now.

Next problem: Direct mode doesn't go direct to the rift (yeah, it's name is a little misleading). You need to create a window on your desktop (full screen or windowed, doesn't matter). The render shim does something to get the contents of the window's swap chain to the rift display. It appears to be doing a copy, rotate and scale (more on why I think this later). On it's own, this isn't a problem as such. It's confusing that direct to rift mode doesn't go direct to the rift but you really render to a monitor like normal and it gets copied, but it works. Some devs are moving the window way off screen so it can't be seen. I've hidden it with SW_HIDE, that works but you lose the app from the task bar.
No, the issue is that the user's monitor may have a lower resolution than the DK2. My monitors are 1680x1050. In extended mode the rift shows up as a 1080x1920 (or 1920x1080 if rotated) screen, and ogre is happy. But in direct mode you need to create a 1920x1080 window. That's not possible on windows typically, you can't make windows bigger than the desktop resolution (apparently there's some ways to trick it though).

On my system, Ogre doesn't list 1920x1080 because it can't see a direct mode DK2. I can force the res to 1920x1080 in windowed, but Ogre detects windows that are too big and clamps them. In fullscreen it throws an exception, only listed resolutions can be used. If I take a screenshot with fraps (the rift's render shim corrupts Ogre's screenshot ability), the result is 1080x1920. It's clear that it isn't upscaling my window (clamped to 1680x1050) on the DK2, fraps wouldn't see that. Plus the screenshot shows that it's a resized image (pixel thick lines are now slightly blury).

The swap chain doesn't have to be the same res as the window. That's what the oculus sdk demos do, they run the window at half the res of the swap chain. 960x590 window with 1920x1080 swap chain. So does Ogre allow this? Sadly no. It seems even if you make a custom window, the swap chain always reads the window's actual resolution and uses that. I can't bypass the size calculation, since I still need _createSwapChain() but it's protected.

So far the only way for 1680x1050 monitor owners to get maximum quality (1 to 1 pixel ratio between render and rift) in direct mode is to hack the ogre dx11 renderer to hard code 1920x1080 as the swap chain resolution. I've tested it, the only downside I've found so far is that it's unacceptable.

That's where I've reached so far. Maybe Ogre needs another miscParam for window creation that overrides the swap chain resolution. But that would mean all previous versions of Ogre are out of luck. I can't see any way around it with the current (1.9 or 2.0) ogre. Unless I just reimplement all the protected window and swap chain init stuff myself. Icky.

Anybody got suggestions? I think another miscParam is the only neat way and direct rift support will be limited to future ogre changesets. Current ogre versions can still use extended mode for full rift resolution. Are there any other use cases for different window and swap chain resolutions (to make this more enticing to the Ogre team)? I guess you could have a small panel in an app that saves at much higher resolution.
Oh, I should mention that I have no idea how this works with dx9 or gl, I've only tested this in dx11.

(At least head tracking is still easy. The new IR camera thing and position tracking is very simple to add, got that working in minutes).

[avengre]

I don't know that it would be a concern if previous versions of Ogre would be out of luck, those versions aren't typically updated anyway. I'd say going forward a setting in cmake to enable/disable the bounds of window creation would be okay.


Keep up the good work Kojack, I've been secretly stalking your progress over the past year.

[Kojack]

One thing that makes debugging this tricky is that all the new stuff like the camera support and the direct mode implementation are hidden from us. The sdk is open source, but what it now does for the new functionality is use network rpc calls to talk to a closed source service. So I can trace into the sdk function that attaches the rift and the app for direct mode, but all that function does is set up a network packet and send it. So I know the sdk is failing to start the rift when I use DX9, but I can't debug why.
This also means any wrapper I write for ogre will require winsock 2 as a dependency, because the oculus sdk uses it and is a static library (so ws2_32.lib is needed to resolve dependencies).

Next up will be understanding the new compositor in 2.0.

[dark_sylinc]

Interesting.

Btw., you can create a window larger than the desktop (Windows).
At WindowEventUtilities::_WndProc, locate this line:

Code: Select all

case WM_GETMINMAXINFO:
        // Prevent the window from going smaller than some minimu size
        ((MINMAXINFO*)lParam)->ptMinTrackSize.x = 100;
        ((MINMAXINFO*)lParam)->ptMinTrackSize.y = 100;
        break;

And set:

Code: Select all

((MINMAXINFO*)lParam)->ptMaxSize.x = 4096;
((MINMAXINFO*)lParam)->ptMaxSize.y = 4096;
((MINMAXINFO*)lParam)->ptMaxTrackSize.x = 4096;
((MINMAXINFO*)lParam)->ptMaxTrackSize.y = 4096;

Where 4096x4096 is the hardcoded limit in this example (note that this can lead to weird behaviors, like maximizing suddenly going 4096x4096).
The actual max limit depends on the GPU (amount of vram, maximum RTT size, etc).

As for Compositors, in our private branch we have added a feature to ease the Compositor rendering to two different regions (without having to manually duplicate the compo nodes with different viewport region parameters); this feature is meant for stereo rendering and multiplayer split screen. Eventually it will be integrated into the public branch. Probably this is most useful for extended rendering.
Let me know if you're interested.

As for the swap chains, I can't really comment without seeing the code.
From what you've been saying it looks like "Direct" method may be the recommended method but doesn't look like it

Cheers
Matias

[Kojack]

Resizing the window bigger than the screen probably isn't good for full screen mode (haven't tried). It would work for windowed (at least direct mode is always fullscreen on the rift itself, it's just the desktop that is windowed) but I don't trust windows overlapping multiple monitors to run at full speed (maybe it's ok, but in the past I've seen framerate of ogre apps massively drop when the window straddles two monitors).

I've got it working so far. I've added two new miscParams to the window creation code: swapChainWidth and swapChainHeight. Very simple change to the dx11 renderer. I think it's a bit like the contentScalingFactor miscParam for ios 4 (except not floats, which seem icky for resolution calculations).
Now I can do this:

Code: Select all

	if(m_root->showConfigDialog())
	{
		m_root->getRenderSystem()->setConfigOption("Video Mode", "192 x 108 @ 32-bit colour");
		m_window = m_root->initialise(false, m_appName);
		Ogre::NameValuePairList miscParams;
		miscParams["colourDepth"] = "32";
		miscParams["vsync"] = "true";
		miscParams["swapChainWidth"] = "1920";
		miscParams["swapChainHeight"] = "1080";
		m_window = m_root->createRenderWindow(m_appName, 192, 108, false, &miscParams);
	}

That makes a 192x108 window with a 1920x1080 swap chain that's rift compatible. The tiny window shows a down sampled version of the scene.
(Default is still to use the window resolution, so it won't affect normal people)
I'll submit a pull request for the team to take a look at after I find out how dx9, opengl and gles can achieve similar results. (Although I'd much rather stick to dx11. 1 platform is easier to support)
For the time being though it's enough for me to resume testing later bits (like compositors) with a modified ogre.

Edit: The setConfigOption probably isn't needed now, I do everything it did below. I just haven't removed it yet.

[cybereality]

Great news! Glad you are making progress with this.

[avengre]

Would you be willing to post a .zip of your files you're using atm? The other thread is quite long, and confusing as to what people are doing... I'm wanting to use my DK2 in my ogre project as well (And can post a version of it for people to try out I suppose), but seeing a framework of implementation is so much more friendly than trudging it alone!

[Kojack]

Yay, slight progress.

I've partially gotten my head around the new compositor system in 2.0. Here's my current result:



What that is showing is:
- compositor workspace that does a red background and 2 cameras rendered into either half of a manually created render texture
- a second scene manager containing just a quad manual object with the render texture applied to it and a blue background

It's sideways because that's what the DK2 is actually given by the sdk's direct mode.

Of course using the second scene manager with a manual object that's just a quad is a waste, but that's needed for when I change the quad to the distortion mesh.
Probably not the best way to do things, but the new compositor is a bit confusing for me at the moment.

I accidentally wiped my code that adds custom swap chain support, I'll have to write that again. At the moment I'm running at 1680x1050 with upscaling to 1920x1080 being done by the oculus sdk. Not good, but all I can do with the way ogre works at the moment. I need to either modify ogre's source like I did previously, or tell people ogre can't render to the rift at native resolution if their monitor isn't at least the same. Or just buy a new monitor and pretend this isn't an issue.

[Germanunkol]

This is a great thread - thanks for the write-up so far! It has already saved me from a lot of trouble.

I'd be greatly interested in a demo as well, or, maybe even better, a wiki page "Oculus Support in Ogre" or similar.
Keep it up!

[lygyue]

nice work. I 'm working with this.

[Kojack]

Wow, been a while.
But I'm finally back on this and have progress.
I now have Ogre 2.0 working with the DK2 in direct mode with directx 11 and the Oculus 0.4.4 sdk.
Everything is working correctly, at least the bits I've implemented so far. But the code is a mess of experimental hacking and using a private framework. I need to start turning it into an actual releasable package (in other words a complete rewrite, plus a demo that uses standard ogre).

At the moment it's not rendering at full quality (at least for me), because as I mentioned above Ogre lacks manual swap chain size control and I'm developing on a 1680x1050 monitor. So I'm rendering at 2364 x 1461 (optimal res chosen by the sdk for post distortion quality), which is down scaled by ogre in the distortion stage to 1680x1050, which is then up scaled by the rift direct mode driver to 1920x1080. Umm, yeah, not the best way to go.
I think my next step should be to find out if opengl and dx9 have a similar concept and make a patch for Ogre.

I still need to look into the whole timewarp thing, I haven't tried it out with ogre yet.

10:17am, time for bed. (Spent all night fighting with this thing to get it working)

[cybereality]

I now have Ogre 2.0 working with the DK2 in direct mode with directx 11 and the Oculus 0.4.4 sdk.

That's great news!

Hope you can sort out the resolution stuff as it makes a pretty big difference.

[Kojack]

It's definitely possible in dx11. It just depends on if opengl supports it too. (dx9 is less important, since it's being dropped anyway).

Hmm, I'm looking into the dx9 lockup with the rift driver active. It seems to be FSAA related, the oculus replacement for IDirect3D9::CheckDeviceMultiSampleType is returning D3DERR_NOTAVAILABLE for every requested fsaa mode (even no fsaa), making ogre go into an infinite loop of checking types since it can't find a safe value. It's also stopping the config dialog fsaa list from being populated. IDirect3D9::CheckDeviceMultiSampleType called with a type of none and quality of 0 succeeds when the oculus driver isn't loaded, but fails if it is.
If I skip the fsaa check (just force it to none) then the oculus driver makes D3DXCheckTextureRequirements fail for a simple 512x512 r8g8b8 texture (the ninja's jpg).

I can't really go any deeper than that since the oculus code for that area is secret. If I bypass the fsaa check and put no textures in the scene, I can get ogre running in dx9, but the rift doesn't turn on. I'll stick to dx11 for now, it works.

[Kojack]

I hacked up a quick tuscany loader using RapidXML.

[cybereality]

Hello, virtual world!

[Germanunkol]

It's definitely possible in dx11. It just depends on if opengl supports it too. (dx9 is less important, since it's being dropped anyway).

Ah, I really hope you get this working in opengl. My current method using the extended mode is just a placeholder until you've done your magic

Thanks for the update (looking great!), and good luck with the next steps!

[Kojack]

I was going to try, but my test program was only displaying a blank magenta window in opengl (gl3+ renderer). I couldn't get anything else rendering. I eventually tried just the ogre 2.0 sample browser, it's also only magenta in gl3+. I guess 2.0 (taken from the repo about 3 weeks ago) doesn't like gl3+ on my pc.

[Kojack]

I grabbed the latest Ogre 2.0 source and built it, the GL3+ render system is still dead (just a magenta screen, if I terminate it an exception appears about running out of memory in a vertex buffer or something).
I swapped to the old gl renderer instead.
Yay, Direct Rift mode working in GL! Well, only the left eye, but it's actually turning on the rift and rendering it correctly.
Now why the hell is the right eye missing?

[Kojack]

Right eye fixed. Turns out it was a bug in the DX11 renderer's handling of viewports that made OpenGL break.
You see, when I started playing around with ogre 2.0 stuff for oculus, I tried to understand viewport compositors by changing values. I changed the width of the viewport for clearing the render target to 0.5. But still got a full screen clear. I thought I must be using it wrong and forgot about it. But the gl render system followed it correctly and was only clearing the z buffer for half the viewport. (I'll do some investigating and jog a bug).

So both eyes working. I've got DX11 and GL working in Direct Rift mode. GL3+ seems to be broken on the ogre side, and DX9 is broken on the oculus side, fixing the former isn't worth it since Matias probably has a newer version anyway and fixing the latter is impossible from my end (no source code).

Opengl render: (It's monochrome because I haven't ported the shader I used for the house to GL yet, unrelated to oculus support)

[Germanunkol]

Opengl render:

Yay!

Funny though, you also get the blueish border on the edge of the screen as in my extended mode implementation. So it really isn't my fault...

It's really showing up in my setup, though, you notice it when wearing the rift. Kind of annoying, and oculus won't reply to my thread.

Anyways, I'm looking forward to giving this a spin.

[Zonder]

and oculus won't reply to my thread.

There out partying with all the money they got

[cybereality]

@Germanunkol: Sorry, I must have missed your thread somehow. Can you send me a PM on the Oculus forum and next week I'll see if I can take a look.

[Kojack]

I guess I don't have to worry about getting DX9 working, Oculus are depreciating it in the next sdk anyway. Which fits well with my decision to focus on Ogre 2.0 (which is also dropping DX9) at the moment.

Anybody seen a good demo environment? I need something like the Oculus tuscany demo house, really nice looking but in an Ogre exportable format (like blender) and not insanely detailed. I keep checking Blendswap and similar places, but most stuff on there isn't suited to real time engines.

[Germanunkol]

I remember Dexsoft for making great models, and supplying them in multiple formats. They sell packs which are intended for real-time applications.
I don't know if you're willing to spend money, and it might be difficult to supply this with a free demo, though. But maybe they're willing to cooperate?
http://3dmodels-textures.com/

[duststorm]

Anybody seen a good demo environment?

Perhaps any of these would serve well: https://3dwarehouse.sketchup.com/collec ... 1efde62bf9

You'd have to check the license or ask the author, though. And they would have to be converted to Ogre assets.