Potential bug in hardware skinning and some suggestions

[genva]

Mesh::compileBoneAssignments compile bone assignments into blend index and weight buffers, and use VET_FLOAT1,VET_FLOAT2,VET_FLOAT3 or VET_FLOAT4 for VES_BLEND_WEIGHTS element base on numBlendWeightsPerVertex. But whether D3D or OpenGL, VET_FLOAT1 will expanded to (float, 0, 0, 1), VET_FLOAT2 will expanded to (float, float, 0, 1), and VET_FLOAT3 will expanded to (float, float, float, 1).

Now you see, issue occuring if someone write a general vertex program which dealing with four weights per vertex, but the supplied mesh numBlendWeightsPerVertex < 4. Because the lastest blend weight is 1 (hardware expanded describing above), and the lastest blend index is 0 (It's zero filled? may be, dependence by hardware vertex buffer implementation), all vertices will full weighted by first bone delta transform.

Yes, I known, this issues can be resolved by supply special case version vertex program for each mesh. But I want the computer do it for me, and if the suggestion below can be implement inside the Ogre core, why not?

This issues unnotice if someone write vertex program like this (copy from Ogre "Example_Basic.cg" routine hardwareSkinningFourWeights_vp):

Code: Select all

	// transform by indexed matrix
	float4 blendPos = float4(0,0,0,0);
	int i;
	for (i = 0; i < 4; ++i)
	{
		blendPos += float4(mul(worldMatrix3x4Array[blendIdx[i]], position).xyz, 1.0) * blendWgt[i];
	}
	// view / projection
	oPosition = mul(viewProjectionMatrix, blendPos);

because the output position will re-weighted by homogeneous division, and in most situations, the first bone is the root of the skeleton and its bone delta transform may be identify or "average" of all bone delta transform, thus, just slightly difference.

I come up against this issues because I want optimise the code above, like:

Code: Select all

	// transform by indexed matrix
	float3 blendPos = float3(0,0,0);
	int i;
	for (i = 0; i < 4; ++i)
	{
		blendPos += mul(worldMatrix3x4Array[blendIdx[i]], position) * blendWgt[i];
	}
	// view / projection
	oPosition = mul(viewProjectionMatrix, float4(blendPos, 1.0));

so, the lastest blend weight can not reduce by homogeneous division.

Suggestion below:

Extend vertex program attribute includes_skeletal_animation <bool> to includes_skeletal_animation <numBonesPerVertex> <maxBones>, <numBonesPerVertex> means how much bones-per-vertex this vertext program can dealing with, <maxBones> means how many world matrices can set to world_matrix_array_3x4. <maxBones> is need to improve vs_1_1 24 bones limitation support mention later.

Entity::reevaluateVertexProcessing will take care with <numBonesPerVertex> and <maxBones>, find the best fit (or anyone) technique to process skinning mesh, or auto fallback software skinning if any sub-entity except the vertex program capability for all techniques for it.

Furthermore, for each sub-entity, sorting out the bones that never used by the sub-entity, reorganise blend indices to "packed" form, so the <maxBones> for each individual sub-entity will drop down, make hardware skinning more usable. Some code to explain what I means:

SubEntity::getWorldTransforms will work like this:

Code: Select all

    for (i = 0; i < this->mNumActualUsedBones; ++i)
    {
        *xform = mParentEntity->mBoneMatrices[this->mBlendIndexToBoneIndexMap[i]];
        ++xform;
    }

Where mNumActualUsedBones is the number actual used bones for the sub-entity, and mBlendIndexToBoneIndexMap is an array use to map the blend indices in the VES_BLEND_INDICES element to the skeleton bone index. mBlendIndexToBoneIndexMap will construct by the modified version of Mesh::_rationaliseBoneAssignments and Mesh::compileBoneAssignments.

Sorry for my ugly English

[:wumpus:]

This all sounds very good, especially the part about only passing bones to a subentity that are actually used. This will make it possible to split up meshes so that you can have more than 24 bones on an entity with vs1_1 or arbfp. This will mean some changes to the mesh format and exporters (or- at least the OgreXMLConverter) though; for example, the submesh will have to contain a blendIndexToBoneIndexMap.

[genva]

The mesh format no need to changes, the mBlendIndexToBoneIndexMap can built up in runtime by Mesh::_rationaliseBoneAssignments and Mesh::compileBoneAssignments, I think there no more overload.

[jonnii]

about time genva was made an MVP... i mean honestly.

[sinbad]

about time genva was made an MVP... i mean honestly.

Check again - he already is

@Genva: sounds like a good idea. I don't think the <maxBones> is required though, since the number of bones that the shader can manage is entirely a product of the vertex shader version and the fact that 3 constants are required for each bone. Allowing a maxBones parameter could be seen as a bit misleading because the shader writer might put a value in there which is not guaranteed to be supported by the shader version so it could miss the need for a fallback if they chose vs_1_1 and too many constants - which might work on some cards but since the minimum is lower, would fail on others. I think it's better to just tell people to stick to the shader limits. Unless you mean that maxBones is only ever intended to reduce from the value allowed by the shader version?

Whatever the case, I think the <bool> version should remain as an option with implied default values for numBonesPerVertex / maxBones, for those who don't need the flexibility and just have a standard shader / modeller setup.

[jacmoe]

about time genva was made an MVP... i mean honestly.

Check again - he already is

About bloody time!

[jonnii]

i meant to say i'm glad he is! i noticed it with this post, i mentioned he should be one a while ago =)

[genva]

Unless you mean that maxBones is only ever intended to reduce from the value allowed by the shader version?

May be <maxBones> can determinate by render system capabilities and shader version, user need to supply a parameter to indicate how many constants already used, moreover, the render system capabilities should be provide number of constants vertex programs support for each shader version (not too bad, but may be mess up something elsewhere).

i meant to say i'm glad he is! i noticed it with this post, i mentioned he should be one a while ago =)

I'm noticed, thanks.

[sinbad]

Ok, that makes sense - so the max bones is actually the lesser of maxBones in the program definition, and the maximum number of bones you could achieve given the number of shader constants available for the profile.