SoC2013 Ogre 2.0 Revamp        

General Porting Notes:

  • Create static objects first.
  • Sort your creation order of Entities by frequency of creation & destruction: Entities that will be created & destroyed often should be added last. Entities that will never be destroyed should be created first, followed by Entities that will rarely be removed/destroyed.
  • Names are no longer unique (i.e. two SceneNodes can have the same name). To identify uniqueness, use getId()
  • Detaching an Entity from a Node will set its visibility to false. Attaching it to a Node will turn it on. Any user-defined value passed to setVisible() will not be preserved after attaching/detaching. Setting visibility to 'true' for an Entity that is not attached is not allowed and will crash the engine!

SCENE_DYNAMIC & SCENE_STATIC

Q: Do the changes mean that you can set a "static"-flag on any {LEX(Entity)}entity{LEX} and it automatically gets treated as static geometry and the batch count goes down when there are many static entities sharing the same material?

A: No and yes. On normal entities, "static" allows Ogre to avoid updating the {LEX()}SceneNode{LEX} transformation every frame (because it doesn't change) and the {LEX()}AABB{LEX} bounds from the Entity (because it doesn't change either).
This yields massive performance bump.
When using Instancing however, we're already batching everything together that has the same material, so it is indeed like Static Geometry, except that we cull per instance basis (which puts a bit more strain on {LEX()}CPU{LEX}, but allows for very fine grained frustum culling on the {LEX()}GPU{LEX}, giving it less work), and 2.0's culling code is several times faster than 1.9's.
When using normal entities, batch count won't go down when using the "static" flag. However it will improve performance compared to 1.9, because we're skipping the scene node transform & AABB update phases and that takes a lot of CPU time.
See also the doxygen comment (OgreCommon.h) in the source code.

GSoC

TODO:

First term

  • Integrate ArrayMath into Ogre core DONE
  • Create Unique ID system DONE
  • Replace unique name system
  • Refactor new Node & SceneNode class DONE
  • Implement basic {LEX()}SIMD{LEX} UpdateAllTransforms function (single threaded). DONE
  • Refactor Entity, SubEntity and it's base classes (MovableObject & Renderable). Keep each Entity & Subentity SoA memory (a cache of the world matrix + bounding sphere + aabb) contiguous organized by render queue ID DONE
  • Implement basic SIMD FrustumCulling function (single threaded). DONE
  • Restore Ogre::Bone using "OldNode" DONE

Second term

The schedule had to be reviewed. Shadow mapping was high priority and deeply broken. Thus the compositor started having more priority and I ended up making almost all of the Compositor that was scheduled for 2.1 in 2.0; the advantages are massive. The compositor was favoured over restoring billboards & particle FXs.
Additionally, bugs in shadow mapping from Ogre 1.x were fixed. Some of them where inherent to Ogre's 1.x design and would be hard to port them back.

Task Status
Restored Instancing (all techniques) DONE
Restored Billboards 20%
Restored Particle FX
Restored others (misc)
Restored shadow maps DONE
Removed old, deprecated functionality (stencil shadows, old instancing) DONE
Implemented Compositor Nodes DONE
Implemented Compositor Workspaces DONE
Implemented Compositor script's translator
Implemented Explicit FSAA/MSAA resolves DONE
Implemented Static & Dynamic entities DONE
Implemented hashed strings (IdString) DONE
Improved how forward lights are sent to GPU DONE
Added threading to Ogre DONE
Added Mesh Partitioner(1)


(1) The idea might be abandoned. The Render Queue needs a serious refactor and a better solution may be more viable.

Repository Fork
GSoC2013 Ogre 2.0 Forum Thread
Ogre 2.0: O(1) insertion, removal and contiguous memory iteration
Good bye AxisAlignedBox, Hello Aabb
Ogre 2.0 is up to 3x faster
Ogre 2.0 - HW Instancing up to 15x faster
Shadow mapping nightmare
Ogre 2.0 - Shadow Mapping
Changing how lights are bound in forward passes
View most resources offline: All Blogposts & slides (updated: 2013-09-10)
Ogre 2.0 Porting Manual DRAFT

Proposal Timeline

Bonding period (27 May – 17 June):

*Actively interact with my mentor and Ogre3D community to discuss final details of my intended implementation. Some details can elude me, for example regarding Android & iOS devices which are very important to Ogre's future./list

Official coding time starts (17 Jun – 2 Aug 2013)

First work to do is to implement the main code changes: Memory & Hierarchy management as well as transform math.

Stage 1 (17 Jun – 23 Jun)

Integrate ArrayMath into Ogre core. Implement unique ID system to replace unique name system. Create flexibility levels.

Stage 2 (24 Jun – 30 Jun)

Refactor new Node & SceneNode class. Remove unneeded virtuals, use a Transform class that holds all SoA (Structure of Arrays) pointers. Manage their memory allocation according to their parent-child relationship (handle attach and detachments). Implement basic SIMD UpdateAllTransforms function (single threaded).
Stage 3 (1 Jul – 14 Jul): Refactor Entity, SubEntity and it's base classes (MovableObject & Renderable). Keep each Entity & Subentity SoA memory (a cache of the world matrix + bounding sphere + aabb) contiguous organized by render queue ID. Implement basic SIMD FrustumCulling function (single threaded).

Stage 4 (15 Jul – 21 Jul)

Copy the original Ogre::Node and rename it to "Ogre::OldNode". Ogre::Bone will derive from this, as animations and skeletons aren't yet ready. Modify AutoParams to take the World Matrix from the new memory model.

Stage 5 (22 Jul – 2 Aug)

Refactor SceneManager's main _renderScene so that transform and animation is removed from the loop. Prepare everything to have a working dll that can run a basic demo.

Mid-term deadline (2 Aug)

At this point Ogre will be in compilable state. Many features will be broken and therefore disabled (Compositor, RibbonTrails, Billboards, InstanceManager, possibly ManualObject & SimpleRenderable) but the core system will be up and running, and be useful to test performance and find any hidden bottleneck. Most demos won't work (may not even compile) so a separate quick demo will be written.

After mid-term (2 Aug – 16 Sept 2013)

Past this point, the hard part has already been done, it's time to do the 10% (the details!)

Stage 6 (2 Aug – 8 Aug)

Restore broken stuff. RibbonTrails, Billboards, InstanceManager, ManualObject & SimpleRenderable.

Stage 7 (9 Aug – 18 Aug)

Implement a new null Compositor. Such as in the slides, it has to manage all RenderTargets, dispatch them appropiately with read-only thread safety, and manage the RenderQueues afterwards. It has also be able to request the SceneManager which RenderQueues are going to be used. A full blown compositor is scheduled for 2.1b and thus not part of this GSoC

Stage 8 (19 Aug – 25 Aug)

Write the Mesh partitioner. For multiple practical reasons, RenderQueue lives in MovableObject while it is mostly a Renderable property. So if a Mesh wants to have multiple sub meshes with different RenderQueues, it won't be possible (this is already a problem in 1.x, where a patch workarounded a solution). The Mesh Partitioner is just a helper that will split the submeshes into one Mesh per RenderQueue so that the user can spawn all of them at the same time and still get the desired results.

Stage 9 (26 Aug – 1 Sept)

Write a ThreadManager abstraction layer with a Windows implementation and thread FrustumCulling and UpdateAllTransforms.

Stage 10 (2 Aug – 16 Sept)

Fix any left over, fix bugs. If there's enough time, convert Bones to a more cache friendly memory layout and implement a proper UpdateAllAnimations.

Any remaining time will be used for overall testing, bug fixing, and implementing whatever is on the wiki that was not included here.

Suggested ‘pencils down’ (16 Sept)

Ogre 2.0 will be ready for testing and can get prepared for release. Take note that Ogre 2.1b will be close in features to that of current Ogre 1.8 branch; therefore 2.0 must be treated more of a "sneak peek" of what's coming. I was inspired by Blender's approach of redesigning it's 2.49 interface by releasing the "use at your own risk" 2.5x branch, and finally releasing stable software in branch 2.6x