Ogre 1.10 Mid-Term Report

With the 1.10.8 release, the 1.10 support cycle is approximately half way over, which is a good time to recap the features and switches that got introduced since the initial release.

 

Note that Ogre 1.10 is the only actively maintained release – all other branches have reached their end of life, so no bugfixes will be back-ported.
If you are still using Ogre 1.9 or even a previous release, you are encouraged to upgrade to 1.10.

 

But besides bugfixes, there were also some notable changes, which will be presented below.

Java component

Following the Python Component that got introduced with the initial 1.10 release, we now introduce the Java Component. Both of them share the same SWIG definitions, meaning they have the same API coverage.
This allows you to write pure Java applications that leverage Ogre – which is especially useful for Android development. Take a look on the updated AndroidJNI sample.

OpenGL improvements

The existing Vertex Array Object (VAO) implementation in GL3+ and GLES2 was so broken that VAOs had to be updated each frame – essentially disabling them. From 1.10.7 on VAOs work as expected and give a performance boost of about 10%.
Thanks to a contribution by Jean-Baptiste Griffo the GL3+ RenderSystem got a StateCache (accompanying GLES2 and GL). We took it as a starting point to further improve and unify the State Cache implementations and indeed could eliminate various bugs inside the GLES2 and GL caches.

Deprecation of node-less positioning

Traditionally Ogre allowed you to place Cameras and Lights in the scene without attaching them to a SceneNode. This resulted in three different positioning APIs for Cameras, Lights and SceneNodes with slight inconsistencies (Camera::move vs. SceneNode::translate) and one big one:
  • SceneNodes & Cameras look into (0,0,-1) by default
  • whereas Lights look into (0,0,1)
To make things consistent, using the Camera and Light API for positioning is deprecated now. If you want to move away from the origin, you have to attach to a SceneNode (same as you would have to with Ogre 2.x).
The only exception is Light::setDirection, which you should call with (0,0,-1) to make Lights behave like the other Nodes.
We cannot go ahead and change the default direction for 1.10 as it would break existing applications.

Compile time switches

Lets turn to some more fundamental changes that change the API and thus are hidden behind CMake switches.

OGRE_THREAD_SUPPORT == 3

To quote Steve, who initially implemented threading in Ogre:
OGRE_THREAD_SUPPORT==1 where resource management is fully threaded, was hopelessly naive. It required too many locks, and also that the rendersystem was multi-threaded. OGRE_THREAD_SUPPORT==2 improved that by not requiring that the rendersystem was threaded, and just doing disk I/O in the background, but still, the whole process is still driven within the Resource class, which means the locks are still in place on Resource and by association SharedPtr and a bunch of other classes too.
Therefore we introduce the new OGRE_THREAD_SUPPORT==3 option. Setting this, Ogre core objects – most notably Resource – are not thread-safe. However the DefaultWorkQueue is threaded. This allows the Terrain and MeshLOD Components to be multi-threaded without the locking overhead in OgreMain. See #454 for details.

OGRE_USE_STD11

Setting this option, the custom SharedPtr and AtomicScalar types become merely aliases for std::shared_ptr and std::atomic which are both more portable and higher performance alternatives.
Also std::thread is used as the default threading provider.

OGRE_NODE_STORAGE_LEGACY

Traditionally Ogre uses (hash-)maps to store everything by name, including sub-nodes and attached MovableObjects. While this gives you O(log(N)) lookup, iteration performance is terrible as maps are spread all over the memory and thus trash the CPU caches.
However with rendering the most common operation is iterating (for e.g culling) over all children while lookup is only done for changing the state. Furthermore high-frequency lookups can be easily avoided by just storing the returned pointer.
Setting OGRE_NODE_STORAGE_LEGACY=0 will use the cache friendly std::vector instead of maps. Naturally lookup by name becomes O(N) instead of O(log(N)) and the API changes. However this improves rendering performance by about 20% – bringing Ogre 1.x into 2.0 range when only a few nodes need updates. See #440 for details.

An outlook on Ogre 1.11

As the amount of possible configurations explodes with each switch we introduce, we are going to only allow one setting with Ogre 1.11 to keep things maintainable.
So rather think about the switches in terms of feature previews and not so much as options.

 

With Ogre 1.11 we will choose the following configuration:
  • Only OGRE_USE_STD11=ON will be supported. In other words Ogre will switch to C++11.
  • Only OGRE_NODE_STORAGE_LEGACY=OFF will be supported.
  • OGRE_THREAD_SUPPORT=3 will be the default. Only the std::thread provider will be supported.
  • OGRE_RESOURCEMANAGER_STRICT=ON will be the default.
  • Default Lights direction will be (0,0,-1).
  • Lots of the deprecated API will be dropped – better start looking at the deprecation warnings now.
  • Custom memory allocators will no longer be supported. You will be able to define C++11 template aliases to set them for STL containers though.

 

Ogre3D 1.10 released

We finally tagged the Ogre 1.10 branch as stable (tag: v1-10-4), making it the new current and recommended version. We would advise you to update wherever possible, to benefit from all the fixes and improvements that made their way into the new release.
This release represents more than 3 years of work from various contributors when compared to the previous 1.9 release. Notably, it includes the following three GSoC 2013 projects:

The ResourceManager redesign GSoC was superceded by the strict mode CMake switch.
Enabling the strict mode is highly recommended, however the default is legacy mode to ensure compatibility with existing projects.

Additionally it includes the changes from the git fork that got merged back into default. The fork formally did the 1.10.0 release, thats how we ended up with 1.10.4 here.

Currently we don’t have any published SDKs yet, since we still rely on team and community members to help with the building and packaging process and that takes some time of course. We will update the download page as they become available and also try to update the announcement thread in the forums.

Changes

For an detailed overview of the new features see the New And Noteworthy Document. Among the highlights are:

  • Python bindings as a component
  • vastly improved GL3+/ GLES2 renderers with GL3+ now being the recommended choice on *nix systems
  • Bites Component for rapid prototyping of applications
  • Emscripten platform targetsupporting Web Assembly & WebGL2
  • improved build system, automatically fetching all required dependencies.
  • A new HLMS Component implementing physically based shading
  • Unified Documentation: the API docs, the manual and some Wiki pages have been merged and are now managed with Doxygen. As a consequence, the Wiki is outdated when it comes to OGRE 1.10. If you find something particularly missing, feel free to submit an additional tutorial.

Despite the amount of new features OGRE 1.10 provides the smoothest upgrade experience between OGRE releases so far. See the API/ ABI change overview for OGRE 1.7 – 1.10 that is kindly provided by ABI-laboratory.
Note that some components are marked as [BETA]. This does not mean that they are likely to crash, but that we can not give any API stability guarantees for them right now. You should expect their API to change without a deprecation period while we we iron the warts out as the Component get more exposure.

In turn for the core components, our deprecation list has grown considerably. You can keep using these APIs for now, as we intend to support them until OGRE 1.11. Speaking of which; to make OGRE releases predictable, we will switch away to a feature based to a time based release model for the 1.x branch. This means that you can expect OGRE 1.11 in April 2018.

Contributors

We would like to thank everyone who helped us make this release possible. The following list shows a list of contributors for 1.10 based on the git logs:

Pavel Rojtberg, Eugene Golushkov, David Rogers, Jesse Johnson, Murat Sari, Lior Lahav, Peter Szücs, Philip Allgaier, Matías N. Goldberg, Sasu Robert, Assaf Raman, Jannik Heller, lunkhound, Juan Borda, Daniel K. O., Jan Drabner, Transporter, Ahmed Allam, Flavien Bridault, Lf3T-Hn4D, J. Edward Sanchez, Mattan Furst, Shenjoku, arkeon, nxgraphics, 0xc0deface, Alexpux, Caleb Bartholomew, Christian Refvik, Harald Reingruber, Stefania Pedrazzi, Unknown, ja0335, threax, Al Reay, Andrew Piper, BAntDit-PC\BAntDit, Bastian Köcher, David Reepmeyer, Inifinity, Joel Croteau, Tomáš Kováč, tangren, Arroy One, Charles Prévot, Chris Burns, Daniel Brall, Daniel Gouvêa, Jean-François Verdon, Joël Lamotte, Matthew Fletcher, Misha, Oleksiy Ivanov, Simon Willshire, al2950, xantares, Alex Peterson, Alexey Knyshev, Andrew Fenn, Arroy, Bruno Sanches, Carsten, Crashy, Dainius Vaiksnys, David Avedissian, Denis CARLUS, Francesco Guastella, Gauthier POGAM–LE MONTAGNER, Glenn Ramsey, Holger Frydrych, J W, James Chen, Joe Brown, Justin Grant, KoenMertens, KungFooMasta, Mako_energy, Marcin Juszkiewicz, Michaël Broutin, Osamu Takasugi, Richard Plangger, Robert Hildebrandt, Robin Norrman, Ryan Thoryk, SNiLD, Sam Hocevar, Steve Peters, TheOnlyJoey, Ziriax, emilie.harquel, mkultra333, philiplb

BTW: if you want to contribute to OGRE, you can now use github as well.

New team member: Pavel Rojtberg

The new year has just started and already we have good news for the Ogre community: Pavel Rojtberg officially joined the development team. Many of you will already be familiar with him and his work in his Ogre 1.10 fork. Many / most / all of his changes will step-by-step make their way into the official 1.10 branch for which Pavel took over the maintenance. Some of you might already have noticed that based on him having commit/merge privileges now in the repository and him already making use of those since a few days.

Since most of the team resources are currently locked up in Ogre 2.x development, Pavel’s support is most welcome to make sure that the current Ogre 1.x versions don’t get left behind.

So, welcome on board Pavel!

Game highlight: “Shadows: Heretic Kingdoms”

Once again, we want to highlight one of the many games based on Ogre3D that have come out recently. This time: “Shadows: Heretic Kingdoms“.

Shadows Heretic Kingdoms PackageAs usual we asked the team behind the game if they could share some insights into the Ogre3D usage and how the game was built in general, and the guys from bitComposer were kind enough to provide those:

  1. The game was developed on an older Ogre version, but continually upgraded to the most recent versions (currently 1.10). We are thrilled to get the game to Ogre 2.0 though as there are some limits which we are not able to overcome with the 1.x versions (mostly related to hyper-threading and slow scene management).
  2. Apart of a few bugs fixes we have not made any modifications to Ogre core. However, we are using some extensions of shader rendering pipeline. To be more specific, we have modified deferred shading with cubemaps projected lights, added off-screen particles, outlines glow and color correction to name the biggest changes.
  3. We are supporting both DirectX 9 and DirectX 11 renderers. DX11 was included because of the improper detection of graphics cards on some computer with multiple graphics cards with DX9 version.
  4. For 3D modelling, animation and export we are using Maya. Some outsources create their models in 3D Studio and we import those into Maya environment and export into the games.
  5. We have developed our own editor which we are using for scene assembling which includes ragdoll editors, lightning.
  6. For particle effects we are using Particle Universe with in-game editor for seamless implementation of particles to the game environments.
  7. For physics simulation in Shadows we have used the Bullet library (mostly for collisions and ragdolls).
  8. Scripting is done using LUA language. A lot of external files are XML formatted.
  9. For text content handling (there is a lot of text content as you can imagine) we are using our own proprietary tool Locedit which is also available for 3rd parties if someone may be interested in. It features multiple languages, Unicode, subtitler for movies, XLS export/import, voice check and direction, etc.
  10. Game content is encrypted and packed to improve loading times.
  11. User generated content is planned to be distributed using Steamworks.
  12. The entire toolset will be released for the community together with the game sometimes in the near future. We believe that Ogre community may benefit from our tools even for their own projects… It certainly will lack the polish and proper documentation of Unreal but it likely may be the most complex tool for Ogre and available for all Shadows owners without any additional fee.
  13. We are currently starting porting of Shadows to Ogre 2.0 together with the Ogre team, so I hope our work will contribute to early availability of complete 2.0 version of Ogre.

You can see more videos and screenshots on Steam where you of course can also purchase the game: Shadows Heretic Kingdoms @ Steam

 

Below a first look at the editor that was used for the game and is planned to be released to the public in the near future:

Review of 2014 & Outlook on 2015

First off: Happy New Year to everyone!

As the old year starts to slowly fade away in our rear view mirror, we want to look back on what 2014 had in store for Ogre3D and at the same time look ahead what 2015 might bring. So strap in for this ride back to the beginning of last year…

Review of 2014

It all started off with one key event early in 2014 when one well-known (at least in Ogre3D circles) Argentinian fellow joined the team: Matias Goldberg. In the last few months he was (and still is) the driving force behind the efforts to develop the revamped Ogre 2.0 version. Without him that would not have been possible in its current form…or more accurately: It would not have been possible…full stop.

The work on Ogre 2.0 was one of the cornerstones in last year’s efforts, but of course the rest of the team was not sitting around idly, but instead focused on Ogre 1.10. The DirectX11 and GL3+ render systems in particular received a lot of attention and will be among the highlights of this upcoming release. More on that later once we arrive back in the present time on our journey through the last 365 days.

Two other notable technical points are the addition of WebGL support to Ogre3D as well as AZDO work:

WebGL / Javascript (Emscripten): From Ogre version 1.10 on, we officially support WebGL. The respective code has already been pushed to the repository. An online demo can be viewed at http://coderabbit.at/webgl/viewer (with an downloadable offline copy being available on our Sourceforge site) and build instruction can be found under the following link: http://coderabbit.at/webgl/viewer/OgreEmscripten.txt.

AZDO: As part of his Ogre 2.0 efforts Matias has been working on optimizing Ogre3D from an AZDO (Almost Zero Driver Overhead) point of view. For now this only covers the GL3+ render system, but will be extended in the future. A dedicated post from Matias will follow, outlining the changes and improvements in more detail.

Of course all that work on the engine itself only makes sense if there is an active community and eco-system actually making use of it. Some projects to showcase are listed below:

So all in all it was another pretty good year for Ogre3D, but we got quite a few things to optimize and change for 2015…and that is where we are back in the present and “put the pedal to the metal” to jump-start into 2015 and all we have planned for it:

Outlook for 2015

One of the first things we want to finish is the go-live of the new Ogre3D website we have planned. This basically entails a face-lift and getting rid of the slightly annoying ad banners. So stay tuned for that.

Another major step will be the release of two new Ogre versions, namely Ogre 1.10 as well as the first Ogre 2.0 CTP (Community Technology Preview). Once we have set the final dates we will let you know via a blog post as well as where to find further details. This especially pertains to Ogre 2.0 for which we want to set up some dedicated sources of information to help with the upcoming transition once 2.x nears its first stable release.

At the same time, the regular development efforts will continue and as always we encourage everyone to chime in wherever possible, by providing feedback and input in the forums, reporting issues on our JIRA tracker and ideally also help solving those by submitting pull-requests for our official source code repository. Additionally, donations are always welcome as well to help managing the infrastructure costs.

This last point nicely leads to another thing we want to tackle this 2015. Go-live of our CI server (Continuous Integration) for automated testing of Ogre3D. Once this is ready to use, we will announce it here as well in a dedicated post.

We of course also hope to participate again in this year’s GSoC after we did not make the cut last year. The preparation for it will begin soon by opening up the discussion in our GSoC forum section regarding interesting potential projects ideas. If you have one, note it down and join in the upcoming idea discussion.

Lastly, we also plan to do another Ogre User Survey to get some insights into the user base along with its priorities and wishes and some information about the usage of our engine in general. As you already guessed: This will also be covered in a dedicated announcement once we survey is open. Prior to it, we will trigger a public discussion in the forums regarding the questions we want to ask. Feel free to join in if you have questions that might be of interested and that you would like to see included in the survey.

That’s about it in terms of what we have planned right now, but as usual we will have to see how things pan out. As usual let us know what you think in the forums…and then let’s get cracking.