[2008 - Accepted] Motion Synthesis based Character Animation

[tuan kuranes]

1. Rather than 'automatic rigging', at first and only if need, 'automatic fitting' would be enough, which means if a skeleton has exaclty the skelton that MOCAP animation uses, but with different names, user can provide a 'mapping' file that help MOCAP apply animation on correct bone.
But that can be left for later, just 'planned by design' could be enough.
'AutoRig' is still an option, but only once everything else done.

2 ogrecollada is just another format. as BVH/AMF are 'industry standard', and converter from one to another already exists, only supporting one fomat is enough.

only animation in .skeleton is replaced

Could rather be 'added', or in another skeleton.

downsample ...keyframing the original animation.

I would first import it totally without downsample or any form of simplification. One Frame of mocap => one Keyframe.

As a second step, with some control parameters do the downsample, smoothness, acceptable error level, and any other stuff.

Those methods of 'downsampling, keyframing, optimising, etc..' could be methods of Ogre::VertexAnimation like existing one 'Ogre::AnimationTrack::optimise()' already does.

That way it could benefits all other form of vertex animation.

[Wolfmanfx]

Hi westine,
A nice feature for me would be log messages when you make a commit

[westine]

Hi westine,
A nice feature for me would be log messages when you make a commit

Ok, I will do it when I commit.

[westine]

Now I can add mocap data in bvh mocap data format into Ogre's .skeleton file animation section, but there are some problems when the new added animation is been playing.

the cammand you use to add animation is as follows:

OgreXMLConverter.exe -addanimation rush.bvh jaiqua.skeleton.xml rush.skeleton.xml

rush.bvh is a bvh mocap file which contains the animation you need,
jaiqua.skeleton.xml is a original Ogre file, which is associated with a mesh file, the new merged rush.skeleton.xml contains the new animation.

The problem is that, the bvh has a different bone hierarchy with jaiqua.skeleton's.

bvh_bone id = "0" name = "hip" skeleton_bone id = "33" name ="Spineroot"
bvh_bone id = "1" name = "lhipjoint" skeleton_bone id = "27" name ="Llegroot"
bvh_bone id = "2" name = "lfemur" skeleton_bone id = "16" name ="Lthign"
bvh_bone id = "3" name = "ltibia" skeleton_bone id = "17" name ="Lshin"
bvh_bone id = "4" name = "lfoot" skeleton_bone id = "18" name ="lfoot"
bvh_bone id = "5" name = "ltoes" skeleton_bone id = "20" name ="Ltoe"
bvh_bone id = "6" name = "ltoes_End" skeleton_bone id = "" name =""
bvh_bone id = "7" name = "rhipjoint" skeleton_bone id = "32" name ="Rlegroot"
bvh_bone id = "8" name = "rfemur" skeleton_bone id = "14" name ="Rthigh"
bvh_bone id = "9" name = "rtibia" skeleton_bone id = "15" name ="Rshin"
bvh_bone id = "10" name = "rfoot" skeleton_bone id = "19" name ="Rfoot"
bvh_bone id = "11" name = "rtoes" skeleton_bone id = "21" name ="Rtoe"
bvh_bone id = "12" name = "rtoes_End" skeleton_bone id = "" name =""
bvh_bone id = "13" name = "lowerback" skeleton_bone id = "13" name ="Spine01"
bvh_bone id = "14" name = "upperback" skeleton_bone id = "12" name ="Spine02"
bvh_bone id = "15" name = "thorax" skeleton_bone id = "11" name ="Spine03"
bvh_bone id = "16" name = "lowerneck" skeleton_bone id = "34" name ="neckroot"
bvh_bone id = "17" name = "upperneck" skeleton_bone id = "1" name ="neck"
bvh_bone id = "18" name = "head" skeleton_bone id = "0" name ="head"
bvh_bone id = "19" name = "head_End" skeleton_bone id = "" name =""
bvh_bone id = "20" name = "lclavicle" skeleton_bone id = "26" name ="Lshoulderroot"
bvh_bone id = "21" name = "lhumerus" skeleton_bone id = "23" name ="Larmroot"
bvh_bone id = "22" name = "lradius" skeleton_bone id = "9" name ="Lforearm"
bvh_bone id = "23" name = "lwrist" skeleton_bone id = "25" name ="Lhandroot"
bvh_bone id = "24" name = "lhand" skeleton_bone id = "10" name ="Lhand"
bvh_bone id = "25" name = "lfingers" skeleton_bone id = "22" name ="Lfingers"
bvh_bone id = "26" name = "lfingers_End" skeleton_bone id = "" name =""
bvh_bone id = "27" name = "lthumb" skeleton_bone id = "" name =""
bvh_bone id = "28" name = "lthumb_End" skeleton_bone id = "" name =""
bvh_bone id = "29" name = "rclavicle" skeleton_bone id = "31" name ="Rshoulderroot"
bvh_bone id = "30" name = "rhumerus" skeleton_bone id = "28" name ="Rarmroot"
bvh_bone id = "31" name = "rradius" skeleton_bone id = "3" name ="Rforearm"
bvh_bone id = "32" name = "rwrist" skeleton_bone id = "30" name ="Rhandroot"
bvh_bone id = "33" name = "rhand" skeleton_bone id = "4" name ="Rhand"
bvh_bone id = "34" name = "rfingers" skeleton_bone id = "5" name ="Rfingers"
bvh_bone id = "35" name = "rfingers_End" skeleton_bone id = "" name =""
bvh_bone id = "36" name = "rthumb" skeleton_bone id = "" name =""
bvh_bone id = "37" name = "rthumb_End" skeleton_bone id = "" name =""

this is the video http://www.youtube.com/watch?v=MKzpOE4HzOc

[tuan kuranes]

Three case I guess ?


1) If it's just a bone naming problem, then user would just have to provide a config 'mapbones.cfg' file

Code: Select all

hip=Spineroot
lhipjoint=Llegroot
...
[code]

then cmd line would become
OgreXMLConverter.exe -addanimation rush.bvh jaiqua.skeleton.xml rush.skeleton.xml mapbones.cfg

So for now, user has to provide its mapping config file.

2) If some bones miss, then again mutliple case, either start-bones, end-bones or middle bones. Start bones cannot be discarded, but end-bones can. Middle bones perhaps can have their position either discarded or 'summed' with neighbour bones ? (using config file)

discard can be explicit
[code]
rthumb=0
rthumb_End=0
ltoes_End=0
...
[code]

or implicit, if bone is not listed in the config file.

3) If it's a bone approximation fitting problem, where target bones doesn't fit at all. Then approach is much more complex,  '[url=http://www.postech.ac.kr/dept/cse/vr/yvr2004/yvr2005/MotionLoD_YVR2005_mod.ppt]Skeleton simplification[/url]',  '[url=http://www.mit.edu/~ibaran/autorig/code.html]mesh autorig[/url]' or even latest amazing '[url=http://visgraph.cse.ust.hk/projects/skeleton/]skeleton extraction[/url]' may help there, but anyway, those are out of gsoc scope imho.

[westine]

I use CeguiMeshviewer to view animation with the new added skeleton,
all the original animations are well played, but the rush ( which is new added) is not listed in the animation list.

And I have discovered the difference in character skeleton between bvh and OgreSkeleton,
as bvh is only skeletal animation, so the orientation of each joint is for the direction of its child bone's offset, and there is endeffector which only has offset but not orientation as on more child bone needs to extend from.
When it comes to the Ogre Skeleton, it is for mesh animation, so the position and orientation of each bone is for its own Entity node's local use.

All these concludes that more bone fitting work need to be done, and this probably fall in the 3rd case, but could be more simpler.

[westine]

I have been trying to bind a exact matching skeleton this week using Mayaã€

[sinbad]

Does it have to be a specific model, or can you use something like the Jaiqua model?

[braindx]

I wrote a tool for some animation research I was doing with Ogre this semester that might help if I understand your problem correctly...

I use Milkshape 3D to deal with importing and exporting animation and I've found the Valve SMD format easiest to work with. I wrote a tool that given a skeleton and animation in SMD format will export a new animation fit to the skeleton.

You provide a file that maps bones in the animation to bones in the skeleton. The requirement is that the bone structure in the animation must be be as or more complex than the bone structure in the mesh (i.e. every bone in the mesh must be matched to a bone in the animation).

The tool goes through the input animation and transforms the rotation of the bones to world space. It then uses the mapping file to remove unmapped bones, renames the mapped bones to the correct names for the mesh, transforms the rotations back to local space and outputs a new SMD file.

If this is useful, I can try to clean up the code a little bit and send it on over to you.

Additionally, a lot of professional mocap studios use FBX as their format of choice for saving mocap. They will not load properly with the importer provided with Milkshape but I've written one that is compatible if you need. If all your mocap data is in BVH format, Milkshape can convert to/from SMD easily.

[tuan kuranes]

every bone in the mesh must be matched to a bone in the animation

I think westine needs exactly that.

Westine, can you give either a sample bvh link or the bone hierarchy you needs?

Meanwhile, perhaps you should stick to a very simple animated mesh/skeleton, like this.

[westine]

Does it have to be a specific model, or can you use something like the Jaiqua model?

It should be an exactly matching model, because the skeleton in my bvh is not matching the skeleton in any model in the current Ogre.
I tried to match bvh skeleton to Jaiqua and ninja as possible as I can, but because the binding pose is not the same, then the bone weight assignments are different, resulting in angly animations.

[westine]

I wrote a tool that given a skeleton and animation in SMD format will export a new animation fit to the skeleton.

Does this animation use character model, I means skinning?

You provide a file that maps bones in the animation to bones in the skeleton. The requirement is that the bone structure in the animation must be be as or more complex than the bone structure in the mesh (i.e. every bone in the mesh must be matched to a bone in the animation).

Ogre dose not require every bone in a mesh must be matched to a bone with animation track. I have tried to just bind a small part of bones to animation track, this really works. But my trouble is the Binding Pose.

Additionally, a lot of professional mocap studios use FBX as their format of choice for saving mocap. They will not load properly with the importer provided with Milkshape but I've written one that is compatible if you need. If all your mocap data is in BVH format, Milkshape can convert to/from SMD easily.

All my mocap files are originally asf/amc downloaded from CMU Mocap Lab http://mocap.cs.cmu.edu, and I converted them to bvh for more compatibler use.

Your work is really valuable, now there is no automatically tool for skeleton and mesh binding, we can collaborate and contribute it to the community. Thanks so much.

[westine]

pushup


jump

I use simple but probably representable stick man mesh to play motions.
Thanks for braindx' timely and quick help.
An associate video
http://www.youtube.com/watch?v=fEu1n9Xa1g4

[braindx]

Does this animation use character model, I means skinning?

I'm not sure what you mean. The SMD format can be used to hold mesh data, but it can also be used to hold only animation data, like a bvh file.

Ogre dose not require every bone in a mesh must be matched to a bone with animation track. I have tried to just bind a small part of bones to animation track, this really works. But my trouble is the Binding Pose.

The binding pose is only an issue because the animation is based off of local rotations starting from the binding pose. The original binding pose doesn't matter because my tool moves everything to world space rotations first. Only the binding pose of the new skeleton that you pass in to it is used when saving the converted animations.

All my mocap files are originally asf/amc downloaded from CMU Mocap Lab http://mocap.cs.cmu.edu, and I converted them to bvh for more compatibler use.

These bvh files can be converted to SMD format using Milkshape 3D.

[Hel]

Hey guys

I'm quiet new with animation but have to do some face animation for my project.
The problem is I have captured a face using infrared capturing device with about 40 markers on the face(.bvh file) , now I want to map the markers to a face mesh ( generated by FaceGen ) . actually, the goal is to animate the face mesh by the captured data.

I tried to use Maya, Poser and some other softwares bit all seem very confusing, has anybody any idea how to do this mapping of the markers and doing the animation ? HELP I'M CLULESS


HH

[braindx]

This isn't really the right thread for a post like this Hel, but if you're looking for some direction I would look up tutorials on doing vertex animations using Maya. Typically, facial animation is done through vertex morphs rather than skeletal animation. You're going to need to map the markers in the mocap file to vertices in the face mesh. The easiest way to do that would probably be to write a Maya plugin or something like that. I don't know if you're going to find any easily accessible software to do what you want.

[westine]

Hey guys

I'm quiet new with animation but have to do some face animation for my project.
The problem is I have captured a face using infrared capturing device with about 40 markers on the face(.bvh file) , now I want to map the markers to a face mesh ( generated by FaceGen ) . actually, the goal is to animate the face mesh by the captured data.

I tried to use Maya, Poser and some other softwares bit all seem very confusing, has anybody any idea how to do this mapping of the markers and doing the animation ? HELP I'M CLULESS


HH

I have been struggling with binding bvh's control joints onto meshs for 2 weeks, although I haven't made any face animation, I should say:
1ã€

[braindx]

Hey Westine, just out of curiosity, what algorithm do you use to compare two frames or clips of animation when deciding whether or not you can build a transition between them in the graph?

[westine]

Hey Westine, just out of curiosity, what algorithm do you use to compare two frames or clips of animation when deciding whether or not you can build a transition between them in the graph?

I use

to evaluate the distance between frames.
In a small motion graph, all pairwise distances are calculated in a preprocessing step, then every frame has a number of successors within a low distance metric.
Reference to Charpter 4 of http://pages.cs.wisc.edu/~kovar/thesis/index.html for more detail, it's intuitive and easy to code.

Early demo demonstrates this method is nice in small diameter motion graph.
There are two main difficulties:
1ã€

[braindx]

Makes sense. I think the ability to restrict the motion set used when compiling the graph is definitely a good idea, maybe just the ability to "tag" animations with categories and limit what types of animations can transition to what other types of animations. If the distance threshold is small enough and the frame window is large enough, linear interpolation should be fine for motion blending. I'm not sure what you mean about IK though...

Also, just to clarify, this motion graph will be computed offline and then loaded at run-time, having only traversals calculated in real-time, right?

[Assaf Raman]

@westine: I recommend using http://pages.google.com as a site to upload images to - images you want use in the forum in posts..

[westine]

@westine: I recommend using http://pages.google.com as a site to upload images to - images you want use in the forum in posts..

Thanks for your recommendation, I have updated all images to google page.

[westine]

After fundamental functions of motion graph have been implemented, I made three demos on it.

They are:
1ã€

[westine]

Animation scripts are like :

Code: Select all

States: 7
0,jump
1,walk
2,rush
3,wonder
4,wheel
5,round
6,idle
Transitions: 8
0,'AnimationEnd',walk,1
1,'AnimationEnd',rush,2
2,'AnimationEnd',idle,6
6,'AnimationEnd',wheel,4
4,'AnimationEnd',wonder,3
3,'AnimationEnd',round,5
5,'AnimationEnd',round,5,
5,'Timesup=5',jump,0

Althoung this one doesn't cover the whole semantic, the snippet shows a static animation script.

"Static" means once you design the script, you can not intervene it except "quit".

The State and Transition number is now required, it just gives a help to count numbers in my code, but I think forcing users count number is not friendly. Why I leave this not improved is that motion script will be serialized in binary in OgreResourceManager in the next development step, it will be improved together.

The semantic of motion graph script is very simple,
there are a set of quadruples , which in state machine graph will be more
easy to understand and editor, it is the reason why GUI of script is indeed needed.
Quadruple =
{ SourceState , TriggerType , Action , DestState }

SourceState , DestState: stateNum
TriggerType: the triggerType this SourceState feeds
Action: more param with the trigger

TriggerType now supported:

Code: Select all

	enum TriggerType
		{
			NON_TRIGGER,
			ANIMATION_END,
			DIRECTION_CONTROL,
			SPEED_CHANGE,
			ACTION_IDLE,
			TIMESUP
		};

this is now coded in OgreMotionGraph.h
To extend it more flexibly, editor and external script language such as Python is a suggestion.

[westine]

The TriggerType is not only for scripted animation, they include both scripted and interactive Triggers.

First thing to be cleared is what " Trigger " is.

Trigger is like an interrupts in operating system.

Normally Motion graph is continuously taking time step and advancing through scripted animation, and in every time tick it peeks whether there are triggers.
If triggers exsit, process them, this is done in MotionGraph::ProcessTrigger() in
OgreMotionGraph.h
OgreMotionGraph.cpp.