HoloLens integration with Ogre - Initialization problems

Problems building or running the engine, queries about how to use features etc.
Post Reply
dorin_dorin
Gnoblar
Posts: 2
Joined: Thu Aug 18, 2016 7:15 pm

HoloLens integration with Ogre - Initialization problems

Post by dorin_dorin »

What am I trying to achieve ?

I am trying to integrate Ogre with HoloLens. I want to use Ogre to create and render a simple scene in HoloLens, in the HolographicSpace


What was my approach ?

My approach was not to touch the Ogre code. I tried to initialize Ogre and than to get access to the D3D11Device using the getCustomAttribute function from Ogre. From there I would have had access to the Direct3D context.
After doing that my thought was to render the scene in a Texture2DArray (in Ogre terms some sort of render to texture technique). I didn't even reach this step, because there were some issues in the initialization process.


What is the problem ?

Here is the description of the problem that I have encountered when I have tried to initialize both Ogre and the HolographicSpace in HoloLens.

In the first scenario I have tried to initialize Ogre first.

In the second scenario I have tried to initialize HoloLens first.

The results are the same: It fails -> I get an exception and I will explain here why this is happening.

I will describe only the first scenario, the second is having the same issue.

Ogre needs a render window (also discussed here: http://www.ogre3d.org/forums/viewtopic.php?f=2&t=57406) when it’s being initialized.

So the call of "createRenderWindow" (declared in the class Ogre::Root) is necessary for the initialization of Ogre.

Note that this function needs an externalWindowsHandle and given that we are using Windows10 for the development (- HoloLens is running Windows 10), the window handle is actually of type CoreWindow.

Given that we are using Direct3D 11 as rendering system, at some point in the initialization process of Ogre, in the class D3D11RenderWindowCoreWindow, the following method is being called:

CreateSwapChainForCoreWindow (this method is part of the Direct3D API and is declared in the IDXGIFactoryN class)

After the initialization of Ogre is completed, the render window is successfully created and Direct3D is initialized. So far so good.

The next step is to initialize the HolographicSpace for HoloLens.

In the initialization process a call of CreateForCoreWindow (defined in the HolographicSpace) seems to be necessary. This method expects a single parameter of type CoreWindow.

Please be advised that we are using the same CoreWindow that we used to initialize Ogre.

At this point the application crashes and the reason is this:

"Calling this method permanently assigns the CoreApplicationView to be holographic. You may no longer call IDXGIFactory2::CreateSwapChainForCoreWindow after calling this method, or vice versa.".

This information is taken 1:1 from here:

https://msdn.microsoft.com/en-us/librar ... corewindow

So the problem is that we are not allowed to call CreateForCoreWindow and CreateSwapChainForCoreWindow on the same CoreWindow.


How can this problem be solved without touching the Ogre code ?

Short answer: I don't now.

Another approach that I thought about would be to initialize Ogre in a 2D window (a window that is not attached to a holographic space) and from that 2D window to initialize another window that is going to be attached to the HolographicSpace.
(In this case CreateForCoreWindow and CreateSwapChainForCoreWindow are called on different window handles)

Very probably this will allow the initialization process to go through, but the problem that I see is that once you switch to the HolographicSpace, the 2D window, where Ogre was initialized, is suspended by the operating system, and at this point I am not sure if the D3D11Device device can still be used to render the scene in a Texture2DArray that is bound to the HoloLens camera buffers, given that Ogre seem to be aware of the suspension (and in such case is calling internally the Trim() function, defined in the class IDXGIDevice. The call is in Ogre in the class D3D11RenderSystem). This function is freeing up resources.


Could anyone give some input on this issue? It would be great to get Ogre running on HoloLens.


Would it make sense to change Ogre code in such a way that it doesn't need a window for initialization ?


Thank you very much in advance,


Dorin Dorin
frostbyte
Orc Shaman
Posts: 737
Joined: Fri May 31, 2013 2:28 am
x 65

Re: HoloLens integration with Ogre - Initialization problems

Post by frostbyte »

just in case you did'nt try it already( not sure if this can help )
you don't realy need to use the first window( the one needed to initialize the render system )-
i always create a hidden 1x1 pixels borderless window and never touch it
regarding your second approach and holo-lense suspending all windows - i think you should check that before assuming all sort of assumptions

if you still don't find a way to integrate HoloLense-Ogre then twicking/tempering with ogre code is not that horrible solution,
after all being able to do so is one of using open-source biggest advantages
just be careful not to break other stuff( maybe use ogre-tests ) and keep some kind of diff/merge journal or better yet just fork Ogre...

i'll wait till price drops to 300$ maybe then i can help more on this topic (-:
goodLuck...
the woods are lovely dark and deep
but i have promises to keep
and miles to code before i sleep
and miles to code before i sleep..

coolest videos link( two minutes paper )...
https://www.youtube.com/user/keeroyz/videos
dorin_dorin
Gnoblar
Posts: 2
Joined: Thu Aug 18, 2016 7:15 pm

Re: HoloLens integration with Ogre - Initialization problems

Post by dorin_dorin »

frostbyte wrote: regarding your second approach and holo-lense suspending all windows - i think you should check that before assuming all sort of assumptions
Yes, I will give this approach a try and see how is going to behave.
frostbyte wrote: i'll wait till price drops to 300$ maybe then i can help more on this topic (-:
goodLuck...
Even if I have access to a HoloLens, I am using the HoloLens emulator for development.
Let me know if you want to give it a try with the emulator.
Maybe we can cooperate somehow in finding a solution.
frostbyte
Orc Shaman
Posts: 737
Joined: Fri May 31, 2013 2:28 am
x 65

Re: HoloLens integration with Ogre - Initialization problems

Post by frostbyte »

nice to know that you don't need the hardware( can use emulation )...
300$ for me is more of a green flag...
its not only for the lack of money( can get Meta for 1k ), but mainly for the lack of user-base/audiance...for now its just a prototype/first steps
even with occlus/vive i'm not that all in a hurry, occuls has reached about one million users( not a lot )...my guess is that most of them are devs...
i'm gonna wait until all this tech will mature a bit...
for sure in some-areas this tech needs to be pioneerd( medicial/industry ) but for gaming and entertaiment , large user-base is more important than innovation
have fun hololucinating (-:
the woods are lovely dark and deep
but i have promises to keep
and miles to code before i sleep
and miles to code before i sleep..

coolest videos link( two minutes paper )...
https://www.youtube.com/user/keeroyz/videos
VY_Holo
Gnoblar
Posts: 3
Joined: Tue Jun 20, 2017 4:13 pm

Re: HoloLens integration with Ogre - Initialization problems

Post by VY_Holo »

Hi dorin_dorin,

I'm also trying to integrate Ogre with Hololens. I've exactly the same problem than you. Did you manage it ? If yes, how did you proceed ?
Post Reply