Adding OpenVR/Vive support to my project. (1.10 DX11)

A place for users of OGRE to discuss ideas and experiences of utilitising OGRE in their games / demos / applications.
User avatar
mkultra333
Gold Sponsor
Gold Sponsor
Posts: 1894
Joined: Sun Mar 08, 2009 5:25 am
x 114

Re: Adding OpenVR/Vive support to my project. (1.10 DX11)

Post by mkultra333 »

Haven't posted anything here in a while.

I've put aside synchronization for the moment. I did find one interesting method, very simple, that had good results though. I simply ignored all the timing constructs from the OpenVR api, since they all seem flaky and never quite worked the way they were supposed to. Instead, I just set up a timer that ensured every game frame takes at least 16 milliseconds before calling for the WaitGetPoses update. Provided the engine and rendering part were under about 10ms, this resulted in a smooth and acceptable 45 FPS. A horrible waste of CPU and GPU time though. Still, better than any of my synchronization attempts, and much better than no synchronization at all.

Some other weird thing... it shouldn't matter, but things seem oddly sensitive to what my shaders are doing. In particular, WaitGetPoses seems to hate my SSAO shader. Even though the shader is quite fast, if I have it enabled the WaitGetPoses performance goes to hell, often taking over 22.222ms. There's not really any excuse for that, that's missing 2 HMD vsyncs, at most I should only ever miss 1 HMD vsync.

At the moment I'm focused on optimizing the game engine to cut down the time for the engine and rendering. If I manage to get it under 6ms I should be able to maintain 90 FPS on the Vive, otherwise 45 FPS will do fine. So some major restructuring is going on to get the game logic fully threaded and the rendering as tight as possible. Good progress so far.
"In theory there is no difference between practice and theory. In practice, there is." - Psychology Textbook.
User avatar
mkultra333
Gold Sponsor
Gold Sponsor
Posts: 1894
Joined: Sun Mar 08, 2009 5:25 am
x 114

Re: Adding OpenVR/Vive support to my project. (1.10 DX11)

Post by mkultra333 »

So I more or less completed the restructure. Improved batching (although I was already pretty good in that area) and more importantly, fully threaded the game engine versus the renderer. This shaves another 1-3 ms off my render time. I had it down to about 3 to 5 ms for a complete game update of 2 full resolution frames.

I thought that would be enough. Nope.

It seems that there are frequent random spikes of 1-2 ms or so in the frame times. These knock on and cause the Vive to keep entering "reprojection" mode, and basically add a judder to things as it jumps back and forward between using reprojection or not. In hunting for the spikes, I ended up removing everything from the game except rendering a black screen. Didn't make any difference. And they only seem to be there in VR mode.

Could be my graphics card, GTX 970 have some dodgy memory configuration.
Could be the graphics card drivers.
Could be the Vive.
Could be OpenVR.
Could be Ogre 1.10.
Maybe it is in my code and somehow I overlooked it when I removed virtually everything my code does.

At any rate, it sucks, and I can't fix it. Hmm, what plan am I up to now? Plan A, Plan B, Plan C... Think I must be up to Plan W by now, still a few plans left. Will have to hack and cut and make a gutted renderer only for the Vive that takes less than about 3.5 ms. So much fun.
"In theory there is no difference between practice and theory. In practice, there is." - Psychology Textbook.
User avatar
mkultra333
Gold Sponsor
Gold Sponsor
Posts: 1894
Joined: Sun Mar 08, 2009 5:25 am
x 114

Re: Adding OpenVR/Vive support to my project. (1.10 DX11)

Post by mkultra333 »

Here's an image demonstrating the spike problem. This is not my game, this is the test demo I created earlier. It does almost nothing, just puts you in a cube and draws the cube (I did one that also removed the Gui that draws the text data, got the same result.)

So there's minimal shader work, minimal textures, no physics or game code, and a scene that consists of a cube, a few rectangles, and the controller models supplied by the API. It's running as fast as WaitGetPoses will allow. Underneath is Steam's own frame timer diagnostic screen.

Huge spikes appear with excessive regularity, triggering the reprojection mode and causing a horrible judder in the user experience. So what is the cause? Could be anything, from drivers to cards to the Vive to Ogre to some subtle problem in my code. Though with this level of minimal code from myself, it's hard to understand.

Image

However, it doesn't happen in the hellovr_opengl.exe sample program. :/

Edit: Partly solved the above by moving RenderWnd->update() to before updating the hmd. Unfortunately this only reduced the spikes, it didn't eliminate them, and I already have renderwnd before hmd update in my main project.
"In theory there is no difference between practice and theory. In practice, there is." - Psychology Textbook.
User avatar
mkultra333
Gold Sponsor
Gold Sponsor
Posts: 1894
Joined: Sun Mar 08, 2009 5:25 am
x 114

Re: Adding OpenVR/Vive support to my project. (1.10 DX11)

Post by mkultra333 »

The fast, cut down renderer is going well. Unfortunately, it underlines the point that the problem is more the random spikes than the time it takes to render the typical frame. I could get the frame time down to zero and those spikes and bumps would still mess everything up.

Fortunately there is a last ditch option, and it works pretty well.

In the SteamVR control panel, under the "Developer" section, there is a check box called "Enable Always-On Reprojection". It comes with a warning that this is for slower applications, and lowers positional prediction accuracy. It doesn't hurt the frame rate, things still run at 90Hz. When I activate this, it basically gives an extra 2ms of render time, plus for whatever reason the spikes on the GPU disappear. The performance is much, much better, and I haven't noticed any difference in the positional accuracy at all.

So from this point on I'm going to target acceptable performance using the "Enable Always-On Reprojection" option and my fast renderer. This is for the lowest end system that the Vive is intended for, a 3.3 GHz gpu and a GTX970, the system I have. People with better systems can try without the always-on reprojection and the normal renderer if they want.
"In theory there is no difference between practice and theory. In practice, there is." - Psychology Textbook.
User avatar
mkultra333
Gold Sponsor
Gold Sponsor
Posts: 1894
Joined: Sun Mar 08, 2009 5:25 am
x 114

Re: Adding OpenVR/Vive support to my project. (1.10 DX11)

Post by mkultra333 »

Did a couple of level runs on my game, freshly booted computer, most diagnostics turned off to save time. Using my new fast renderer.

First run was with "Enable Always-On Reprojection" activated. Went very well. Hardly missed a frame, maintained 90 Hz. VR was smooth, didn't notice any tracking inaccurcy.
Second run was without "Enable Always-On Reprojection". Went pretty well. A few missed frames here and there, but mostly held up. More or less as smooth as most the VR games I've played so far, just the occasional hitch.

So I'm happy with the new fast renderer. The fast renderer removes every effect that isn't absolutely essential, and then optimizes whatever is. For instance:

- There's no HDR, SSAO or motion blur and no fancy glass distortion/lighting/colouring effects. Plus some other FX dropped.
- Some effects that were done as alpha blends in separate passes are now done as hard edged alpha tests and added to the MRT render.
- The MRT render, which is used for the deferred shading, used to be 4 RGBA surfaces, here it is 3 RGB surfaces which is almost a 50% reduction in memory/render bandwidth (goodbye separate specular colour!)
- Moving deferred point lights, glass, billboards and other "plasma" effects are rendered at 70% downsample, making them render roughly twice as fast. Stationary point lights are still full resolution.
- Shadow casting spotlights are now unfiltered simple depth shadows instead of filtered VSM shadows... not gonna lie, they sometimes look pretty ugly, but are mostly acceptable.

And probably the biggest change is I added dynamic scaling of the render resolution. This means that if a frame is taking too long, it can drop the screen resolution to speed things up. This drop in resolution runs down the entire render chain, from the MRT to the fx and lighting. It can happen on a per-eye basis, so for instance if the left eye has rendered, and for some reason it took a little long, the right eye can render the scene at half/quarter the normal resolution. Or both eyes can render at a lower resolution if there was some CPU spike leading up to the rendering. I've found this works pretty well. At 90 Hz, it's very hard to spot the occasional low res frame, especially if it's only in one eye. Obviously there are limits, it can look bad if a scene is spiking badly and the whole scene is flickering between low and high resolution, but that's pretty rare. Mostly it does a good job of smoothing out the occasional spike.

Overall it looks good. Not as pretty as the normal renderer, but then the fact that you're in VR makes up for that. I can probably add back a bit of graphical sugar later, but for now this will do.

(Side Note: When not in VR and using the fast renderer, my game can typically maintain 240 fps.)
"In theory there is no difference between practice and theory. In practice, there is." - Psychology Textbook.
User avatar
mkultra333
Gold Sponsor
Gold Sponsor
Posts: 1894
Joined: Sun Mar 08, 2009 5:25 am
x 114

Re: Adding OpenVR/Vive support to my project. (1.10 DX11)

Post by mkultra333 »

Another test. Now that my frame rate is very fast, I've found the "Allow reprojection" setting does more harm than good. This option, under "Settings/Performance" in the SteamVR app, is supposed to help programs that are running slow, but in my program it mostly just causes horrible frame rate judders as it over-reacts to the occasional spike. Doing a run without "Allow reprojection", things are very smooth. In the heat of the action a missed frame here or there mostly goes unnoticed.

So things are looking up at this point. I'll recommend to the user that they disable "Allow reprojection" under "Settings/Performance," and that they will also benefit from allowing "Enable Always-On Reprojection" as long as they don't have any tracking issues.
"In theory there is no difference between practice and theory. In practice, there is." - Psychology Textbook.
User avatar
mkultra333
Gold Sponsor
Gold Sponsor
Posts: 1894
Joined: Sun Mar 08, 2009 5:25 am
x 114

Re: Adding OpenVR/Vive support to my project. (1.10 DX11)

Post by mkultra333 »

Someone PMed me to ask if I thought Ogre 1.10 and OpenVR was a workable proposition. So here's my thoughts on getting the best of it. Disclaimers are that I'm no expert, I'm learning this as I go, and that my experiences might be particular to my setup and coding skills. I'm using Windows 7, VS2013, a GTX 970 graphics card, Intel i7 3.4Ghz and 16 gig ram. I'd describe my coding skill/style as "cowboy."

I've found Ogre 1.10, with or without VR, to be very stable and dependable. Haven't had any problems with its DX11 support, except one minor issue with a missing type of surface that has since been fixed.

Setup of OpenVR was really easy once I had the work of others to go from at the beginning of this thread. Adding the basic code to get the Vive running is no challenge. The Test example I gave does the job (I've since noticed it needs a couple of tweaks: Put the RenderWnd update before drawing to the Vive for less spikes, and you might need some more code to detect the controllers if they're already turned on when you start the app.)

This is where things got difficult. Getting good performance from the Vive was hard. I figured I'd have close to 11 ms to do my engine update and draw two frames, since the Vive updates at 90Hz. That was not the case. The Vive requires a 3ms "warm up" per frame, but even then you don't have 8 ms. In practice I've found that unless I get everything rendered in under about 4.5 ms directly after WaitGetPoses() has returned, I can't maintain 90Hz. And if you don't maintain 90Hz on the Vive, things look awful. The frames judder and stall as their compensatory "reprojection" mode kicks on and off. It's horrible.

And even when I do maintain 4.5ms frames or less, there are still lots of spikes. But I've since found two SteamVR settings that fix that. If your frame times are really good, it pays to turn off "Allow reprojection" to stop the judder. And if you also tick "Enable Always-On Reprojection" you get even better performance and even an extra millisecond or two to render. I know, it sounds like those two should cancel each other out, but they don't. Enable Always-On Reprojection comes with a warning that it might lower tracking accuracy, but I've not noticed any problem. But my game now runs fine with or without it, provided I disable "Allow reprojection."

Don't bother trying to use the OpenVR timing functions to synchronize your rendering. They are worse than useless and a total waste of effort. Just focus on rendering as fast as possible as soon as WaitGetPoses() returns so that you can call WaitGetPoses() again.

If you've played a few VR games you'll probably notice by now that they often have relatively simple graphics compared to a standard AAA title. At first I thought that was just because they were indie games, but now I think it's because you really need to render fast if you want to get good Vive performance.

So use your 4.5 ms wisely. Make sure your engine is threaded so no time is wasted on enemy AI and game logic, all that needs to happen in parallel with the graphics rendering. And steamline you graphics rendering. Strip it back to the bare essentials, and then test carefully to see what FX you can add back without losing 90 Hz. The Vive seems to weirdly hate some shaders, for instance it hates my SSAO shader, if I use that performance plummets even though it works fine without the Vive. So you have to test.

Another tweak I suggest is variable render resolution. This is where you drop the render resolution if frames are taking too long. For instance, if the left eye has been rendered at high resolution, and has taken a bit too long, render the right eye at a lower resolution this frame to speed things up. Or render both frames at a lower resolution this frame if there was some other delay or stall somewhere. This required altering viewport render areas on the fly and passing that information on to the shaders as well, but isn't too difficult. I've found it can compensate for some spikes but not others. Still a good optimization for maintaining 90 Hz though.

At this point I'm pretty happy with the performance I'm getting from the Vive, most the problems that plagued me early on have been worked around or fixed. At the moment I'm working on redoing my gui and menus to work in VR.
"In theory there is no difference between practice and theory. In practice, there is." - Psychology Textbook.
OGREHEAD
Goblin
Posts: 260
Joined: Tue Feb 02, 2010 6:25 pm
x 2

Re: Adding OpenVR/Vive support to my project. (1.10 DX11)

Post by OGREHEAD »

I am trying to get OpenVR/Vive and Ogre connected, but I have difficulties getting the OpenVR to initialize.

I have steam and steamvr up and running.
I have downloaded latest openvr from github.
I have compiled both openvr and samples.

I would now like to start examples like hellovr_opengl or similar to see openvr is working.

But when I run hellovr_opengl I get an error saying VR_Init failed / unable to init vr runtime / vrclient shared lib not found / error102

I have tried different samples and different libopenvr_api.so locations and made a application with vr_init, but all crash with messages such as "hmd is not present".
I have tried with steam running and not running and nothing seems to have any influence.

Is it not suppose to work on linux?
What am I missing?
Post Reply